[KolenCh]

I don’t disagree anything you said. Jupytext can be a good tool to bridge some gap, where you pair ipynb to a py script and can then commit the py only (git-ignore all ipynb for your collaborators.)

Also, while many practices out there is questionable, in alternative scenarios where ipynb doesn’t exist, they might have been using something like matlab for example. Eg, in my field (physics), often time there are experimentalists doing some coding. Ipynb can be very enabling for them.

I think a piece of research should be broken down and worked by multiple people to improve the state of the project. Some scientists might be passing you the initial prototype in the form of a notebook, and some others should be refactoring to something more suitable for deployment and archival purpose. Properly funding these roles is important, and is lacking but improving (eg hiring RSE.)

In my field, the most prominent way when ipynb is shared a lot is for training. It’s a great application as that becomes literate programming. In this sense notebook is highly underused as literate programming still hasn’t got mainstream.

[spiralk]

I've looked into Jupytext, but ultimately decided to go with pure python. Most of the practical functionality can be replicated, but I do admit there isn't a easy single install tool or guide to replace notebooks at the moment.

I think the notebooks are a fine learning tool to introduce people to programming initially, but I'm afraid it doesn't allow for growth beyond a certain level. You have a good point about funding for those software roles. Perhaps this may not be as big of a concern if there were more software talent in these labs to handle the issues that arise.

[KolenCh]

In an ideal world that we control everything and/or don’t need to collaborate with others, then whatever tooling one use is actually not that important (and each can choose the best fitting their needs.) So Jupyter+Jupytext is useful in the context of collaboration, where you can’t control your collaborators but want something from them.

While in an ideal world scientists who write softwares should write professionally, the same goes for anything they do, including math and stats used in their research, writing and typesetting and generates publishing quality visualization… That rarely happens because of how the academic world is financed, and the incentives associated with it. I can certainly complain about that all days, but in short a researcher hired by a research university, especially with a tenured track position in the US, will not be successful to get such position, let alone getting tenured, if they had not focused their scarce resource of time to maximize their “research output” (publications, grant, etc.), where software engineering is not part of. (Sorry, sentence too complicated.)