|
|
|
|
|
|
by eric24234
2909 days ago
|
|
|
Exactly. Anything that needs even very minimal human interaction like pressing a "y" key is not automated at all. It is very common to see in the corporate environment that scripts needs some editing in b/w to be done during its running with editor which can be easily automated using sed. |
|
|
This becomes especially egregious over month to year long experiments where I run the same experiment every day on end.
There was really no reason not to auto generate every possible plot, every possible analysis every time (and I cannot use ipython notebooks or things like that because it's many distributed things chained together with lots of scheduling).
The productivity gains have been enormous and are hard to overstate. I don't dread any experiment any more because even in a large complicated distributed setup, everything from initialising kerberos tickets to tons of config files, restarting services, running multiple experiments dependent on each other, and generating plots and summaries and committing them to a repo is one command. Anything that's analysed once is evaluated always.
I now almost look forward to setting up new experiments because of the pleasure I get from just chaining together calls from my control utilities.
All I have to do is pull on my laptop and I download a filter with all results pre-generated paper ready. I think a lot of people do this in experiments where everything is on a single machine, but I haven't seen it as excessive from other phd students doing complicated distributed stuff. There is always a lot of manual command line args passing, manually changing some config while instead of just creating dedicated scripts, etc.