[oh_sigh]

What if you're ingesting thousands or millions of small feeds? You might not have much control or desire to dictate format to your clients

[dmix]

Yeah not everyone, I’d even say the majority of people, are using software parsing libraries where they are in control of the input data format.

[dkersten]

For storing stuff yourself, sure, but as a web developer, most data I consume is JSON served by some third-party REST API and the format they serve me is definitely not under my control. Anecdotally, most developers I know or have spoken to are in similar situations for a large portion of their data-processing needs (at least, for stuff that's not in a database, although even in DB's, JSON is increasingly popular for a number of reasons).

Even for output, there is the common case where your clients expect JSON because its the de facto standard and is super accessible (every language has parsers for it), so you have little choice but to serve your data as JSON.

[captbaritone]

The readme specifies that it’s not optimized for reading a large number of small files.

[glangdale]

This would be an easy extension if you wanted to concatenate the files. The plumbing and API aren't there right now, but it isn't hard to see how to do it.