[londons_explore]

JSON is an exchange format... sqlite is an exchange format... even protocol buffers are an exchange format...

CSV is only an exchange format if there is no user generated strings in the data... If there are, then you'll almost certainly screw up the encoding when someones name has a newline or comma or speech mark in it, or some obscure unicode etc. Even moreso if awk is part of your toolkit.

[benhoyt]

That may have been more true years ago, but now quoting is pretty well defined with RFC 4180, and most tools seem to produce and consume RFC 4180-compatible CSV (which properly handles commas, quotes, and even newlines in fields). That said, there still are too many non-standard or quirky CSV files out there.

[dspillett]

> and most tools seem to produce and consume RFC 4180-compatible CSV

Laughs in SSIS…

There are some significant tools (or common add-ins for them) that don't entirely respect RFC4180. Though I see few files that breach it these days, thre are tools that break with conforming files (looking at you, Excel, trying to be clever about anything isn't conclusively provable not to be a date).

Our clients use it all the time, to the point where we'd lose sales if we didn't support it, but CSV is far from a safe way to transport data IMO. Each time a new requirement to deal with CSV comes in I treat it as a custom format that may or may not be something like RFC4180.

[recursive]

I see lot of CSVs generated with LF instead of CRLF as line endings. Blindly following the RFC is still probably not advised.