[duerrp]

Thanks for your feedback. I see your point, but at this point I am kind of attached to the name - as a researcher many of my scripts are related to some kind of experiment. This also explains the NumPy and matplotlib dependencies. As for structlog, if I understand the documentation correctly, you could easily add structlog to your project on top of pyexperiment, right?

[sirclueless]

I want to voice my support for keeping the driving use case "experiments".

I'm coming from the perspective of a software engineer here. To a software engineer, a "program" is a collection of stateless routines and behavior. Data is external and separate, the same program should be able to process a wide range of data. "Reproducibility", as much as that matters, is having a tested system that responds in a predictable and reliable way to inputs, and data is one such input.

When I first worked extensively with a scientist on an experiment, I was shocked how much common wisdom from computer science was turned on its head. One is expected to load up a Matlab workspace with data and code all in the same file? Scripts irreversibly mutate data, and often run exactly once? How could one possibly keep track of such an environment? How does one fix bugs in a series of commands typed into an interactive prompt? Reproducibility to a scientist is a log of actions that could be repeated by another human, but the environments used often just dropped such things on the floor, to be caught only by the most diligent researcher with an unusually well-kept notebook.

I think there is definitely a happy medium somewhere. Reproducibility as a scientist understands it; interactivity in a way that makes sense to a scientist writing a one-off script. Program state stored easily so that the scientist doesn't feel lost every time they restart their environment, as I imagine they must do when editing python scripts in vim as a software engineer might. But all this in a world where scripts can be maintained and versioned and fixed without their hair catching fire.

[duerrp]

Thanks for the kind words. Until I left academia I used to work with matlab a lot, and pyexperiment is probably a result of trying to get that experience while making scripts that can easily be shared along with the data needed to run them.

E.g., the issue with irreversible mutation of data is addressed in pyexperiment with rotating state (and log) files. For example if you store the state of your experiment in one run, and then change it in the next, you will get a backup of the old state with a numerical extension (by default up to 5 backups are rotated). Moreover, pyexperiment by default comes with commands to display stored state and configuration options (though they still need to be improved), and both are stored in formats compatible with a host of other software (including matlab).

Btw., along the same lines, I love ipython notebooks, but the way I use them makes them very hard to share, and compared to plain python scripts, version control is a pain (even with the usual hacks to make diffs readable).