Hacker News new | ask | show | jobs
by dqbd 1172 days ago
Author here, you are correct! The issue here is due to the fact that a single user-perceived character might span into multiple tokens. This should be fixed now.
1 comments
egorfine 1172 days ago
Hey. Thank you! However has the fix not been deployed yet? Still shows broken UTF-8.

> a single user-perceived character might span into multiple tokens

Is this the way it works as designed or is this a bug?

link