[abdullahkhalids]

The same problem exists with spreadsheets. Should we get rid of excel (the single tool that literally runs half the world), and start manually writing markdown tables in text files?

The tool and the tool maker are supposed to serve the user. The user is not supposed to conform to the whims of the tool maker.

[ants_everywhere]

Since 94% of business spreadsheets contain errors [0], then probably yes we should get rid of or significantly improve spreadsheets.

Probably the solution is that things like Jupyter notebooks and spreadsheets should be views into some better source of truth rather than the source of truth themselves.

[0] https://phys.org/news/2024-08-business-spreadsheets-critical.... I remember a similar figure from studies a decade or so ago.

[glzone1]

The funny thing is I've seen folks try to deploy software to get rid of spreadsheets. It always ends badly to terribly.

Spreadsheets are the nonprogrammers programming / modeling tool in business.

It does presentation, data filtering / sorting, modeling and more.

No AI needed (and you can now plug AI in in some cases).

[ants_everywhere]

Sure, but here's the basic problem I think:

Suppose you have some formula that computes a financial metric for your company. Someone you've shared it with drunkenly fat-fingers the formula 3/4 of the way down a long row, and that causes all entries below it to recompute with the wrong formula. Unless the change is really drastic, you may never know it happened.

And this sort of mistake -- basically a typo or a bad mouse movement -- happens daily in every company in the world in some spreadsheet. Often people will notice the mistake, but not with probability 1.

Software engineers have mechanisms to guard against some of these mistakes, and even we have a hard time getting people to take code review or tests seriously. What is the guard in the spreadsheet world?

[anthk]

Have you seen the Excel disaster on genomics? Enough to ban Excel for anything serious except for accounting.

[ants_everywhere]

I had not heard of that, but it appears to be this

https://www.nature.com/articles/d41586-021-02211-4

[paddy_m]

Another issue is that jupyter, pandas, and polars don't take displaying tabular data seriously. Just have a better default table display widget. Look at ipydatagrid, perspective, or buckaroo (my project) for examples of how it could be done better.