[jiggawatts]

I wonder when the PC world will transition to unified memory and more monolithic architectures as seen in the Apple M-series chips, but also in the NVIDIA GB200 "superchip", the AMD MI300 accelerator, the X-Box and Playstation consoles, and even in some mobile phone system-on-a-chip designs. It feels like the PC is the "last holdout" of discrete upgradeable components with relatively low bandwidth between them.

Sooner or later, AI will need to run on the edge, and that'll require RAM bandwidths measured in multiple terabytes per second, as well as "tensor" compute integrated closely with CPUs.

Sure, a lot of people see LLMs as "useless toys" or "overhyped" now, but people said that about the Internet too. What it took to make everything revolve around the Internet instead of it being just a fad is broadband. When everyone had fast always-on Internet at home and in their mobile devices, then nobody could argue that the Internet wasn't useful. Build it, and the products will come!

If every gaming PC had the same spec as a GB200 or MI300, then games could do real-time voice interaction with "intelligent" NPCs with low latency. You could talk to characters, and they could talk back. Not just talk, but argue, haggle, and debate!

"No, no, no, the dragon is too powerful! ... I don't care if your sword is a unique artefact, your arm is weak!"

I feel like this is the same kind of step-change as floppy drives to hard drives, or dialup or fibre. It'll take time. People will argue that "you don't need it" or "it's for enterprise use, not for consumers", but I have faster Internet going to my apartment than my entire continent had 30 years ago.

[pquki4]

I doubt that's ever going to happen -- people build their own PCs exactly because they want the opposite of everything in a MacBook: they want modularity, upgradeability and repairability to the point that they are willing to sacrifice power efficiency.

"AI will need to run..."

Let's wait and see what actually happens to AI before being too eager to change the design of computers. I'm also pretty sure there will be a better solution than what you described.

[Semaphor]

> people build their own PCs exactly because they want the opposite of everything in a MacBook: they want modularity, upgradeability and repairability to the point that they are willing to sacrifice power efficiency.

While I’m totally one of those people, aren’t we some rather small minority, nowadays? I mean, obviously still big enough for companies to produce parts we want, but I always keep reading how more and more people are using laptops instead of desktops.

[dotnet00]

These aren't laptop parts though. Maybe they'll do integrated memory for laptops, since many solder that on anyway nowadays.

But, for the desktop parts I don't see that being worthwhile unless it's used as something like with the 'X3D' chips, a large cache layer to the expandable memory.

I feel like this is nicely indicated by how AMD's desktop APUs get a lot less interest, they were fine without it for several generations, and even now it's just an afterthought.

[smolder]

I would prefer that we figure out how to make the massive vector blobs called AI work more efficiently, versus throwing tons of hardware at what we can barely understand in perpetuity. It doesn't sit right with me that putting lots of floating point ability in edge devices for running these approximate-at-best models is considered the way forward for computing.

[ericd]

Yeah, bitnets already shift those matmuls to integer math.

[smolder]

Thanks for turning me on to this subject. It's interesting. I took a look at this paper: https://arxiv.org/abs/2310.11453

[ericd]

No problem, glad you found it interesting, I think it's pretty exciting stuff. This is a more recent paper by the same people, where they tweak slightly to ternary (-1, 0, 1), and they seem to be able to maintain network performance: https://arxiv.org/pdf/2402.17764

[aaomidi]

I can’t wait for my games to start hallucinating narration.

[fourfour3]

Exactly this.

I want plot/character driven games like RPGs to be a curated & carefully designed, plotted, and paced experience.

I want speech & narrative to be scripted! It means someone sat down and thought about the experience it produces for the player. It means a real voice actor has performed it, putting the right emotion and pacing into the line.

I don't want AI generated stilted dialogue, uncanny valley speech, etc.

And I also don't want an extra few hundred watts of power draw on gaming PCs - they're already high and in modern games the CPU is under pretty substantial load, the GPU is maxed out, and the GPU's AI/NPU style cores are being used for things like DLSS too.

Bringing in more compute resource for running speech to text, LLMs, text to speech, etc fast enough to not feel horrible is going to come at substantial power and financial cost.

[nottorp]

But... that would be ... "making video games". The large "video game companies" would rather "create content" and use the cheapest possible option.

[mavhc]

You want your NPCs to be limited in their conversations, to only follow a set few paths, if you think of a way to play the game that's not been anticipated years ago, you don't want that to be possible

[fourfour3]

Yes - I want to maintain quality, even at the expense of flexibility. I would rather there were 1, 2 or 3 high quality paths through a RPG style game or quest than hundreds, thousands, or infinite ones that are 'meh'.

(edited to expand)

[eviks]

You can still have 3 high quality ones, it's just that maintaining quality at all the side chars is too expensive, so maybe the generated experience can reach the already "meh" level for those, thus freeing resources to be invested in the bigger paths?

[fourfour3]

That’s definitely a more interesting proposition to me (and a bit different to the parent comments).

It might well result in less repetitive background chatter & stock comments from side characters without taking focus away from the primary content. Especially if those background characters could react a bit better to progression in the plot.

[WithinReason]

This but unironically

[treprinum]

Isn't CAMM2 providing higher bandwidth? MOBOs with CAMM2 for AMD are already popping up.

[cubefox]

Problem with unified memory is that software needs to support it, so there is a chicken-egg situation. Another issue is that unified memory is usually not replaceable, just like non-unified VRAM.

[bcrosby95]

How far back are you digging for calling the Internet a useless toy, the 70s? Profitable companies that provided internet like services existed at least as far back as the 80s.

I've seen this line pulled out before and it always seems like an assumption than actual reality.

[JonChesterfield]

CPU and GPU cores have been on the same package for approximately forever. I'd really like on package memory at this point, nice though upgradable memory is the performance gap it induces would be good to see gone. AMD clearly knows how to build that and Apple has thrown down the gauntlet, I just don't know when it'll happen.

Anyone know if the 'strix' apu thing is expected to be a ddr-on-package or still using with separate sticks? Search engine is not going well for me.

[eviks]

> Sure, a lot of people see LLMs as "useless toys" or "overhyped" now, but people said that about the Internet too.

Sure, you can always find naysayers about any tech, but we've also seen plenty of useless toys, so that internet fact doesn't help your argument that AI will come to the edge in any way (and no, email was not a fad even at dial-up speeds, so you don't even have the internet fact)