[wiz21c]

I've been hearing the same complaint between Cobol and Java for years : it was simpler before, more efficient, etc.

Of course it was, but you were tied to one system (no application server), security was login/pw, database had no constraint, typing systems were ultra limited, everybody has its own way of writing batches (no Spring), business code was mixed with tons of technical code (no JPA).

Now, sure, if you glue some R, some SQL, etc. you can extract insights worht millions of dollars. But all of that exist just because we have digitalized all of the processes, data collection, etc. And the rise of data scientists will continue only if there are more stuff put in the databases, thanks to you plain, regular, normal programmers...

[philipodonnell]

> But all of that exist just because we have digitalized all of the processes, data collection, etc. And the rise of data scientists will continue only if there are more stuff put in the databases, thanks to you plain, regular, normal programmers...

IMHO there's far to little attention paid to how data might be valuable in an economic sense when storage strategies are being designed by database designers. I recently gave a talk at a developer conference and was really surprised at the level of pushback to adding more data elements or higher precision data "just in case it might be useful".

The preconception that you have to be maximally efficient with storage has led to huge quantities of valuable data being lost.

[bloomer]

The net value of nearly all data is negative. It’s a liability not an asset. Unless you are actually selling data to customers or solving a specific problem that requires collection of specific data, it’s just wasting the company money. You would never hear a manufacturer say “ let’s stock pile aluminum in case we need it”. But that line of thinking comes up all the time with respect to data.

[wiz21c]

>> The preconception that you have to be maximally efficient with storage has led to huge quantities of valuable data being lost.

I 100% agree. Many programmers are trained as if memory, CPU are finite resources. Although it's true, in many cases, that reality may be safely ignored, opening tons of opportunities (to store data, to develop faster because you don't optimize, etc.).

At a time where my phone has gigabytes of memory, I'm always surprised that some people ask me to put a limit on a text field. I understand the technicalities behind the question, but from a conceptual point of view, that's often pointless.