[bocytron]

It seems you're all missing the point here: it's not about storing useless data, it's about destroying the environment in the process. I understand you all want to keep everything, just in case, because it's cheap and you don't see the externalities. But there are externalities, and they are big.

[palata]

This.

We need to think about the data we need to store before we store it, only store the data that we need to store, and only store it for as long as needed.

It reminds me of CIs. It's now so easy to throw 40 jobs on GitHub actions that people don't think about them. I have been in a startup where people would debug in CI: they wouldn't have e.g. Windows on their machine (maybe they should have, given that their product was supposed to run there) and were fixing compilation issues by sending patches and patches to the CI. Every single time it would trigger the 40 jobs. Sometimes you could see a patch sent every 5 min for 3 days (where reproducing the issue locally would actually take 3s and not 5min). They did not even bother disabling the 39 uninteresting jobs.

For open source projects, it's just wasted energy, for private repos it was costing the company a lot. This was just malpractice. But nobody cared. The finance person would say "GitHub is expensive", the CEO that "well we need it" and the engineers that "I don't want that Windows crap on my computer", I suppose.

[andybak]

In which case we should be reading an article about how important it is to correctly price externalities. And that is not this article.

[palata]

Well that article clearly says "it's hurting the environment, and in my experience the vast majority of that data is useless".

Which I believe is not uninteresting, given the amount of answers here where people say "Is that data useless? I don't know, I could imagine that it's not, I think it's a hard problem". Well here we have one person saying "I have experience in that, and I can tell you that most of it is useless". Just a data point, but that's still interesting.