[spyrja]

True. But then again, backward-compatibility isn't really such a hard to do with ASCII because the MSB is always zero. The problem I think is that the original motivation which ultimately lead to the complications we now see with UTF-8 was based on a desire to save a few bits here and there rather than create a straight-forward standard that was easy to parse. I am actually staring at 60+ lines of fairly pristine code I wrote a few years back that ostensibly passed all tests, only to find out that in fact it does not cover all corner cases. (Could have sworn I read the spec correctly, but apparently not!)

[eru]

You might like https://fsharpforfunandprofit.com/series/property-based-test...

[spyrja]

In this particular case it was simply a matter of not enough corner cases defined. I was however using property-based testing, doing things like reversing then un-reversing the UTF-8 strings, re-ordering code points, merging strings, etc for verification. The datasets were in a variety of languages (including emojis) and so I mistakenly thought I had covered all the bases.

But thank you for the link, it's turning out to be a very enjoyable read! There already seems to be a few things I could do better thanks to the article, besides the fact that it codifies a lot of interesting approaches one can take to improve testing in general.

[eru]

Nice! Glad to be of service.

Python, of all languages, probably has the best property based testing library out there with "hypothesis". I sometimes even use it to drive tests for my Haskell and OCaml and Rust code. The author of Hypothesis wrote a few nice articles about why his approach is better (and I agree), however I can't find them at the moment..