Hacker News new | ask | show | jobs
by delegate 134 days ago
It's worth remembering that this is all happening because of video games !

It is highly unlikely that the hardware which makes LLMs possible would have been developed otherwise.

Isn't that amazing ?

Just like internet grew because of p*rn, AI grew because of video games. Of course, that's just a funny angle.

The way I see it, AI isn't accidental. Its inception has been in the first chips, the Internet, Open Source, Github, ... AI is not just the neural networks - it's also the data used to train it, the OSes, APIs, the Cloud computing, the data centers, the scalable architectures.. everything we've been working on over the last decades was inevitably leading us to this. And even before the chips, it was the maths, the physics ..

Singularity it seems, is inevitable and it was inevitable for longer than we can remember.

6 comments

Remember that games are just simulations. Physics, light, sound, object boundaries - it not real, just a rough simulation of the real thing.

You can say that ML/AI/LLM's are also just very distilled simulations. Except they simulate text, speech, images, and some other niche models. It is still very rough around the edges - meaning that even though it seems intelligent, we know it doesn't really have intelligence, emotions and intentions.

Just as game simulations are 100% biased towards what the game developers, writers and artists had in mind, AI is also constrained to the dataset they were trained on.

I think it's a bit hard to say that this is definitively true: people have always been interested in running linear algebra on computers. In the absence of NVIDIA some other company would likely have found a different industry and sold linear algebra processing hardware to them!
Almost certainly not at the scale of the consumer gaming industry, however!
Google is making millions of TPUs per year. Nvidia ships more gaming GPUs, but it's not like multiple orders of magnitude off.
I'm willing to bet TPUs wouldn't be nearly as successful or sophisticated without the decades of GPU design and manufacturing that came before them.

Current manufacturing numbers are a small part of the story of the overall lineage.

It's pretty interesting that consumer GPUs started to really be a thing in the early 90s and the first Bitcoin GPU miner was around 2011. That's only 20 years. That caused a GPU and asic gold rush. The major breakthroughs around LLMs started to snowball in the academic scene right around that time. It's been a crazy and relatively quick ride in the grand scheme of things. Even this silicone shortage will pass and we'll look back on this time as quaint.
Of course you are right, but in addition they wouldn't have even made them if GPUs hadn't made ML on CPU so relatively incapable. Competition drives a lot of these decisions, not just raw performance.
You are missing his point. They very likely don't start building TPUs if there were no GPUs.
I'm not missing the point. If you recall your computer architecture class there are many vector processing architectures out there. Long before there was nvidia the world's largest and most expensive computers were vector processors. It's inaccurate to say "gaming built SIMD".
You are missing the point - it's an economic point. Very little R&D was put into said processors. The scale wasn't there. The software stack wasn't there (because the scale wasn't there).

No one is suggesting gaming chips were the first time someone thought of such an architecture or built a chip with such an architecture. They are suggesting the gaming industry produced the required scale to actually do all the work which lead to that hardware and software being really good, and useful for other purposes. In chip world, scale matters a lot.

Google DeepMind can trace part of it's evolution back to a playtester for the video game Syndicate who saw an opportunity to improve the AI of game NPCs.
Type the word porn. Self censorship is wrong.
Sci-fi ideas of robots have been around for ages and work on AI and the term singularity kicked off around 1950, so a while ago - well before chips or me being born.
what a load of utter tripe