[chimpansteve]

So, given that I'm already aware that I almost certainly won't understand any answers to this question, can someone give me a idiot level baseline as to exactly what an AI chip is and how it would differ from a cpu/gpu?

[lsb]

The earliest Intel chips (8086) had an extra add-on to do floating point math in hardware, instead of software (8087). There’s a lot of “AI” workloads that include a lot of matrix multiplication, which the Google TPU and the Nvidia tensor cores implement in hardware instead of software.

[chimpansteve]

Ah! ok - that makes sense. Thank you. So it would be a specialised chip with a load of hardcoded functions, kind of like an ASIC?

[lsb]

Exactly, like how a “bitcoin mining” chip would implement the SHA in hardware.

And, CPUs prioritize hiding latency with all sorts of caches, and GPUs prioritize cores and bandwidth to hide latency, so there’s different tradeoffs about memory bandwidth versus latency.

[chimpansteve]

Thank you. Think we've hit the level of my "run away scared at the first sight of machine code" understanding, but I now vaguely understand what's going on.

[endisneigh]

one example of an "AI" chip might be Google's TPUs. in general it's about the optimization of calculations that are useful in the AI context.