To be fair, GDPR stipulates only that it should be available in a common machine-readable format. It doesn’t require the most convenient format conceivable.
Also, CSV can’t easily handle nested objects. If the data model is even slightly more complex than a plain table, it doesn’t make much sense. I’d also argue that even if the source data is stored in an RDMS without exotic data types, a JSON with a nested object representation is probably going to be more friendly even to non-developers than multiple files with opaque foreign keys linking back and forth.
Only if you accept a potentially unlimited number of CSV files/sheets. Many forms of data aren't really easily normalizable to a limited number of flat tables without losing information.
Open it in any decent plaintext editor/viewer, it'll likely have support for 'prettyprinting' json, and it'll be readable.
If you want to 'do stuff' with the data, then JSON is a very (IMHO most, but your mileage may vary) reasonable format; unless that data really is just a single flat table, it would be hard to blame them from picking this format.
Also, CSV can’t easily handle nested objects. If the data model is even slightly more complex than a plain table, it doesn’t make much sense. I’d also argue that even if the source data is stored in an RDMS without exotic data types, a JSON with a nested object representation is probably going to be more friendly even to non-developers than multiple files with opaque foreign keys linking back and forth.