[spullara]

Running into a couple issues right out of the gate:

1) Needed to increase maximum_object_size 2) Unexpected yyjson tag in ValTypeToString

Couldn't find a reference anywhere to that error. Loads into Snowflake without a hitch - which is where I normally query large JSON files.

[jjwiseman]

Tried "select * from 'data.json' limit 10" on a 6.3 MB file (which feels relatively tiny…) and got the same `unexpected end of data. Try increasing "maximum_object_size"` error. (This is my very first attempt to use duckdb, so with respect I'm not invested enough to open an issue).

[mytherin]

Thanks for trying it out!

Could you perhaps open an issue [1] or share the file with us so we could investigate the problem?

[1] https://github.com/duckdb/duckdb/issues

[atombender]

I tried to do "select * from ... limit 1" from a 1.7GB JSON file (array of objects), and I had to increase maximum_object_size to 1GB to make it not throw an error. But DuckDB then consumed 8GB of RAM and sat there consuming 100% CPU (1 core) for ever — I killed it after about 10 minutes.

Meanwhile, doing the same with Jq ("jq '.[0]'") completed in 11 seconds and consumed about 2.8GB RAM.

I love DuckDB, but it does seem like something isn't right here.