Hacker News new | ask | show | jobs
by jarofgreen 1847 days ago
At work we work an a Python Library to do this, and much more:

PyPi: https://pypi.org/project/flattentool/

Source: https://github.com/OpenDataServices/flatten-tool

Docs: https://flatten-tool.readthedocs.io/en/latest/

It converts JSON to CSV and vice versa but also Spreadsheet files, XML ...

It has recently had some work to make it memory efficient for large files.

Work, BTW, is an Open Data Workers Co-op working on data and standards. We use this tool a lot directly, but also as a library in other tools. https://dataquality.threesixtygiving.org/ for instance - this is a website that checks data against the 360 Giving Data Standard [ https://www.threesixtygiving.org/ ].

1 comments

What are the advantages of this tool in comparison with e.g. pandas' json_normalize?
Flatten-tool has more options and functions than just that one pandas functions (But I haven't done a full comparison to all Pandas functions. I wasn't around when the tool was started so I can't say what analysis was done at the time.)

For instance I note with interest their examples on nested data and arrays. We have various different ways you can work with arrays, so you can design user-friendly spreadsheets as you want and still get JSON of the right structure out: https://flatten-tool.readthedocs.io/en/latest/examples/#one-... (Letting people work on data in user-friendly spreadsheets and converting it to JSON when they are done is one of the big use cases we have)