IMO the best file format for this set of requirements is SQLite. It's the world's most widely deployed database engine, the file format is simple and the spec is public domain. The entire database engine has been re-implemented in multiple languages. SQLite is a "recommended storage format"² (along with XML, JSON, and CSV) of the library of congress because of it's long term sustainability, among other factors³.