Hacker News new | ask | show | jobs
by PaulHoule 3504 days ago
Malformed data is a scalability problem. Unusual failure modes from coding problems to random bit flips become inevitable as the data volume approaches infinity.
3 comments

Agreed, but the payload from the article doesn't seem to have suffered from astral radiation. Rather, random attempts at quote-escaping by someone who doesn't understand what they're doing. Also notice the "nan" value -- JSON has no concept of NaN.
Yes, and that's a problem to be solved at the transport and storage layers, not the application layer.
But to be clear, error correction should be done at a level far lower than the parsing stage. It's usually a property of the storage medium or the firmware that accesses it.

If you have to correct for bit flips when you begin to read or parse data, it's too late.