Could this be the answer to "I don't like notebooks" [0]? Or does calling scripts from within a notebook still do horrible things to the managed state?
I've never thought this criticism was that relevant. Notebooks have been incredibly successful, and if it's possible to call your code in the wrong order and create a mess, it's probably a problem with your code not your coding environment. The more functional your code, the less of a problem jupyter notebooks are. They encourage better code not worse imo.
The "managed state" is important, though. For example I deal with cases where I'm doing some big expensive operations upfront (eg, processing a bunch of data coming in over a network share— ROS bag files, which are also bzipped, adding more upfront processing cost), and I'm interesting in tweaking the plots and analysis coming out the other end without having to re-run the processing each time.
Even as a relatively experienced developer, I've found it hard to reason about what exactly it is that Jupyter is doing under the hood; basically things will break in weird ways, and rerunning everything will mysteriously fix it, or I have to use hacks to force it to re-import certain modules in order to get things the way I expect.
Basically, it ends up being easier to just manage this myself, like doing the processing and pickling the result, then using other, standalone notebooks to load and render the pickle. But this shouldn't be necessary.
That sounds really weird. The only time I noticed such weirdness was when a colleague and me were simultaneously editing a notebook on Google Colab. The code was shared but not the state, super weird...
I think this partially matches the parts of the criticism about writing a book, and how the notebook is unsuitable - mainly thinking along similar lines to writing software tutorials pages that are fully testable.
After seeing that post I had a look around for something a little like this and found Pweave http://mpastell.com/pweave/ which seems to approach the problem from the inverse direction direction, but I think I'd rather have something like this where the code is marked up and runnable as normal code but transformable into formatted documentation.
This seems like a reasonable match, but it looks like it's explicitly tied to notebooks and I'm not completely sold on the overloading of plain comments for markup purposes. Still, probably a good step.