Hacker News new | ask | show | jobs
by _fbpt 2870 days ago
>avoid linear scripting as an exploratory technique

What do you recommend instead for exploratory {data analysis? science?}

1 comments

The same thing you do for other types of development. Place separate units of logic into well modularized functions / classes / units of organization; factor out any aspects of config; add a makefile or other build scripts.

An experiment would most often be the creation or modification of a config file followed by just invoking a build command.

No cell-by-cell evaluation, no commenting things out to run differently, no magic constants or big sequences of plotting code sprinkled all over.

The program itself to explore data or fit a model might be an imperative program, but that doesn’t mean it should exist in a single large functional unit that receives modification through commenting things out, re-running a notebook cell to change parameters, etc.

While obviously there is a trade off regarding how much design effort to put in for an experiment, most often people are not putting any design into it, nowhere close to the boundary where the trade off matters at all. Basic things like organizing separate functions, putting constants into a simple config file, etc., cost almost nothing but drastically improve usability and clarity, so you should pretty much always believe those efforts are worth it from the beginning of starting a project.