Hacker News new | ask | show | jobs
by fermigier 2250 days ago
Quick list of Python libraries that help with application configuration:

- Python application configuration -> https://github.com/edaniszewski/bison

- Configuration with env variables for Python -> https://github.com/hynek/environ_config

- Configuration library for python projects -> https://github.com/willkg/everett

- Strict separation of config from code -> https://github.com/henriquebastos/python-decouple

This is from my personal notes. See also: https://github.com/vinta/awesome-python#configuration

Anything else that we've missed ?

12 comments

Marshmallow. You can use its schema validation for any dict/json, which makes it a nice fit for validating json config files (which mitigates some of the json concerns from the article). Just immediately move the json.reads through a schema validate, build some classes around it for different config files.

marshmallow.readthedocs.io/en/

I use a similar library called Schema at https://pypi.org/project/schema/ I love the expressive nature of it. Just this week I was validating both yaml and json configuration data with it.

One thing missing from this article is to always use a proper external key value store for everything but the stage (dev test prod etc) and the connections to the kv store. Config files on disk and even in environment variables suck for anything but the most trivial platforms.

dataclass_json is also very useful for schema validation. It combines python's native dataclass objects with marshmallow's schema to provide additional functionalities simply through a @dataclass_json decorator on your dataclass.

https://lidatong.github.io/dataclasses-json/

I made the similar dataclasses_serialization library. It doesn't require a special decorator on your classes, and is extensible for custom classes, and custom serialization methods (JSON and BSON provided by default).

https://github.com/madman-bob/python-dataclasses-serializati...

I do something similar with dataclasses using the `dacite` (https://github.com/konradhalas/dacite) library as a constructor for python dicts to dataclasses with runtime type-checking.

Works really well with marshmallow for additional validation.

The two big ones in machine learning at least are Google's Gin:

https://github.com/google/gin-config

and Facebook's Hydra:

https://hydra.cc/

the name usage of "gin" is quite common, it seems:

https://github.com/gin-gonic/gin

This and the list of other libraries in responses is a great reason to avoid Python. Its nearly as bad as Javascript, how are you supposed to live with this.
There's also confuse[0], used by beets. I've found it to be a good way to handle configuration when the end-user is not a programmer.

[0] - https://github.com/beetbox/confuse

Dynaconf [1], though I think this article's approach in combination with pydantic is better.

[1] https://dynaconf.readthedocs.io/en/latest/

https://github.com/crdoconnor/strictyaml - typesafe YAML parser (solves the issues mentioned in 2 and 1).
Here's my tiny contribution to the field: https://github.com/berislavlopac/figga
And I also like the .env approach: https://github.com/theskumar/python-dotenv
Cerberus for schema validation.

https://docs.python-cerberus.org/en/stable/