[agentultra]

I'm not sure I buy The Bitter Lesson, tbh.

Deep Blue wasn't a brute-force search. It did rely on heuristics and human knowledge of the domain to prune search paths. We've always known we could brute-force search the entire space but weren't satisfied with waiting until the heat death of the universe for the chance at an answer.

The advances in machine learning do use various heuristics and techniques to solve particular engineering challenges in order to solve more general problems. It hasn't all come down to Moore's Law.. which stopped bearing large fruit some time ago.

However that still comes at a cost. It requires a lot of GPUs, land, energy, and fresh water, and Freon for cooling. We'd prefer to use less of these resources if possible while still getting answers in a reasonable amount of time.

[ijustlovemath]

Deep blue had to use the techniques it did due to the limitations of the hardware of the time. Deep blue would almost certainly lose against AlphaZero, even if you tuned it to modern hardware. All you have to do 'manually' is teach it the rules/give it a loss function, then you just let it do its thing.

It's certainly true that "just throw a bunch of GPUs at it" is wasteful, but it does achieve results.

[agentultra]

Certainly does! We’ve had expert systems and various AI techniques for decades that weren’t efficient enough to run even though theoretically they would yield answers.

And even though solutions to many such problems were in the NP or NP-hard categories it didn’t mean that we couldn’t get useful results.

But it still gave us better results by applying what we know about search strategies and reinforcement to provide guidance and heuristics. Even Alpha didn’t use the most general algorithms and throw hardware at the problem. Still took quite a lot of specialized software and methods to fine-tune the overall system to produce the results we want.

[FergusArgyll]

Today's best chess models use no heuristics, I think starting with stockfish 16 they got rid of HCE (hand crafted evaluation), they're now neural nets and would absolutely eat Deep Blue