[getnormality]

> You have to be very disciplined to make a Jupyter notebook that is actually reproducible

This seems not necessarily very hard to me? All you have to do is keep yourself honest by actually trying to reproduce the results of the notebook when you're done:

1. Copy the notebook

2. Run from first cell in the copy

3. Check that the results are the same

4. If not the same, debug and repeat

What makes it hard is when the feedback loop is slow because the data is big. But not all data is big!

Another thing that might make it hard is if your execution is so chaotic that debugging is impossible because what you did and what you think you did bear no resemblance. But personally I wouldn't define rising above that state as incredible discipline. For people who suffer from that issue, I think the best help would be a command history similar to that provided by RStudio.

All that said, Marimo seems great and I agree notebooks are dangerous if their results are trusted equally as fully explicit processing pipelines.

[tastyminerals2]

Not very hard to you, however the reproducibility numbers tell a different story. Back in the days, when we were searching for some ML model implementations in the public repos and found ipynb files in it, we skipped the repo without delving into details. Within the company data engineer research notebooks were never allowed inside a repo. Experiment, yes, but rewrite it in plain python and push.

[getnormality]

A lot of people don't put away shopping carts, but the conclusion from that isn't that putting shopping carts away requires very high discipline. (Maybe if what is meant by "very high" is "not so low that everyone will do it", which is perhaps the point)

[exe34]

The few times I've made notebooks, I've tried to migrate code out of the notebook as soon as possible and then only import foo and run foo.bar() in the notebook. It helps to only have the top level config/layout in the notebook.

[cantdutchthis]

Fun detail, you can actually define functions in a marimo notebook and load them in another Python file if you want.

Needs a bit of extra config but tis a really nice feature.

https://docs.marimo.io/guides/reusing_functions/

[esafak]

Notebooks ought to have embedded metadata, like a pyproject.toml, to list the dependencies.

[kylebarron]

https://github.com/manzt/juv

[esafak]

Good. This is the part I'm talking about: https://peps.python.org/pep-0723/

[philsnow]

> This seems not necessarily very hard to me? All you have to do is keep yourself honest by actually trying to reproduce the results of the notebook when you're done

It's one thing when I'm relying on my own attention to detail to make sure all the intermediate results have been correctly recalculated, but it's entirely another when I have to rely on even trusted co-workers' attention to detail, much less randos on github. As a sibling comment points out, the "reproducibility crisis" numbers are very much not in favor of this approach being the right idea.

... Or you could work in a format that makes incorrect / out-of-date intermediate state impossible (or at least hard) to represent, which is (I believe) what marimo is an attempt at.