[wiz21c]

pretty neat. But what I need the most right now is a way to visualize information. I work on computer fluid dynamics and there I have:

- super intricate algorithms with lots of variables running around

- lots of huge numpy arrays storing the bulk of the data

I find the current debuggers lacking here:

- showing numpy arrays in colours, easily changing color scales, etc. is not easy (you can do your own visualization, but then it's code); explore specific parts of the array in detail; link that to other arrays, etc.

- build watches panels where you can mix analysis and values extracted from code. I can't count the number of time I freeze the execution, export data to R, then do analysis there, then fix the code. It'd be nice to have that integrated in the IDE.

- have watches that record values as execution runs (I guess that's one of the stuff omniscient debugging does)

And while I'm at it:

- why can't I cut paste formulas, pictures, etc. in my code ?? (no Jupyter, you're ok for a nice paper like presentation but not for code with hundreds of thousands of lines)

[pizza]

What if Jupyter was ok for code with hundreds of thousands of lines? I suspect it might be.. I think nbdev ameliorates my major gripes with jupyter: it lets you easily maintain a bidirectional jupyter <-> (library as standard python files) mapping, while also giving you the excellent presentational flexibility of Jupyter. Plus a good looking docs site, for free. Since the code is exported as scripts, you can just `pip install -e .` and get all your IDE integration as normal.

[wiz21c]

I'll look at it.

My major gripe with jupyter is that it's hard to document classes because a class must be inside one code block and you can't mix in comments block...

(unless I didn't see some obvious thing)

[pizza]

Could you have special markdown cells that have mixed python code blocks and other inline media that gets converted to stripped python cells? You might be able to write an ipython ‘magic’ for something like that

[corysama]

Charles Simonyi's Intentional Software was doing that back in the days of Windows NT https://www.youtube.com/watch?v=tSnnfUj1XCQ But, they stayed insular and enterprise right up until Microsoft bought them back.

We mortals barely got to see or hear anything about it. I heard one podcast interview of a developer there who said they basically represent the code as an s-expression tree with a bunch of metadata on each node.

[vaughan]

Whoa this is incredible. The things you find on HN!

I've been researching these kind of things for ages...and this is the first time I find out about this.

Code as a database.

[heavenlyblue]

Good software development implies writing code that is easy to debug, including writing your own tools to debug your own niche-specific code!

[wiz21c]

Yeah sure but I'd like to sprinkle lots of little codes in many different places, ad hoc. This happens while I'm debugging issues 'cos I have to integrate information from various places. Note also that it's not like trying to figure a clearly visible bug: sometime sI have to wander through literally hundreds of data arrays just to find what's going wrong. Since I do that close to the code (modifying bits here and there to see how it influences things), it'd be nice to have a "data analysis debug context" very close to the code.

I'm sure there's a space here where one can provide tools. For example a drop in "record_this(object)" method integrated with the debugger so that I can look at the results, without the need to build a logging structure to do it myself.

[vaughan]

I had an idea to trace all usages of values. So you can see the entire history of how a value came to be. All the places it went. All the transformations too.

Also, I think code should be written so that it can be easily visualized. Nothing else! Code is for humans to read, not machines - that's what compilers are for.

Code instrumentation is usually for code coverage...but for debugging it can be great too.

[tracnar]

Indeed, on top of stacktraces it would be great to have 'value traces' (with functional programming I suppose those would be the same). Especially in testing, what's important are the inputs with the system under test.

[db48x]

Pernosco has that!

[vaughan]

I wish it was open-source.