[mkhorton]

For crystallography specifically, there's ourselves (Materials Project), OQMD, AFLOW, Materials Cloud, JARVIS, and a number of more specific (but no less important) specialized databases. There are also a number of commercial offerings.

Best practices are incredibly difficult. We're trying to establish a common API currently (https://github.com/Materials-Consortia/optimade) that can be adopted by all database providers. How the data is stored behind the scenes is something that ends up being very specific to how the data is generated and what its applications are. We're definitely better as a community than we were ten years ago, but there's a lot of work to be done here.

In terms of scientific databases outside crystallography/materials science, Nature's Scientific Data is a good open-access journal to peruse: https://www.nature.com/sdata/