[austin-cheney]

The problem moving forward is a concept known as lost knowledge. The ancient Egyptians had steam powered robots, full industrialization, incredible libraries, and paved roads. Europe had none of that until 200 years ago even though they were in Egypt at the time.

If people cannot refactor the code they own the only option is to start from scratch every single time. Any perceived economic benefits assumed of AI cannot be realized if the common knowledge underlying it dies.

[keeda]

Hmm, I doubt that would be an issue for a couple of reasons:

1. We have infinitely more durable means of storing, indexing and retrieving knowledge (and now potentially even means of programmatically reasoning about that knowledge!) We can re-derive things from our records using first principles, as long as we have critical reasoning skills -- about the only skill we will need in the future, and ironically the one most at risk of becoming rarer as people increasingly outsource all thinking to LLMs.

2. We will always need and have engineers of the caliber in TFA. They derive their capabilities, in no small part, by having in-depth knowledge of the entire technology stack they work with. I'd say most of them can operate at the abstract architecture and algorithmic level down to low-level hardware bit-twiddling. If most engineers in the future are of that caliber (which I'd argue is now easier with LLM assistance for those so inclined) there is no chance we'd lose that knowledge.

[austin-cheney]

That missed the point in past history and misses the point now. The technology to better preserve human knowledge has been available for thousands of years. It’s an irrelevant compensator for human behavior. If people, in large enough numbers, are not willing to learn, and extend, their craft it will fade into obscurity.

There is no cheat sheet or magic tool for human behavior.

[keeda]

Hmm, I would say the technology to durably preserve human knowledge is actually very recent. Until just a few decades ago, other methods have been extremely lossy. Even today, with triple-redundant, geographically-distributed data replicas we see cases of catastrophic data loss. Previously, all it took was an errant spark in a library.

You're right that the main driver for loss of knowledge is human behavior, but I would posit the underlying reason is economics. Craft that reduces in economic value simply disappears because there is no incentive to preserve it.

As such, there is a cheat sheet for human behavior, it's called economics! ;-) And the already high economic value of code is only going to keep increasing sharply, so even if LLMs do most of the work there will be strong incentives for people to manualy craft code whenever LLMs, like all systems, inevitably fall short.