[threedaymonk]

> But sadly Lua is still missing Unicode support.

Whenever I hear someone complain about (lack of) Unicode support, my ears prick up. See, Unicode support is a many-headed beast, and almost no languages have very much of it, and most of what people naively think of as Unicode support (length, indexing into "characters", case conversion) doesn't really work when you take into account combining diacritics, ligatures, Turkish I, German S, etc.

I'm not especially familiar with Lua, but it transparently stores and compares UTF-8 strings, and there are even bindings for ICU. So what's missing from "Unicode support"? Script specifiers in regular expressions, perhaps? I'm asking out of genuine curiosity.

[est]

Those who fail to realize the importance of unicode support is the same group of people think handling strings with \0 terminator is OK.

But no, it's fucking not OK. Lacking basic unicode handling means everyone handles unicode strings in different ways, so various libraries clutter in basic string capabilities. Sometimes this really piss me off.

[cmccabe]

Um, UTF-8, a valid unicode encoding, handles strings with a null terminator just fine. Those who throw stones, should not live in glass houses.

Proper unicode support would include things like normalizing strings (Unicode has 4 different normal forms!) and testing for fuzzy equivalence. But last time I checked, the usual library for doing those things, libicu, was bigger than the whole Lua interpreter. So I can see why there isn't a lot of enthusiasm, especially if there isn't a real use case.

[est]

yes, packing icu with every lua distribution or standalone app is bad, and we don't even need to think about OS's Unicode capabilities. So every lang deserves to be suck at unicode handling. Because, you know, ICU is big.

[bitcracker]

> but it transparently stores and compares UTF-8 strings

Thanks for this good news! The last time I tried Lua it didn't work. UTF-8 is ok, full Unicode is not necessary.