Hacker News new | ask | show | jobs
Understanding LLM Compute Requirements (intuitiveai.substack.com)
4 points by pgspaintbrush 1113 days ago
2 comments

Inference is not that flop intense for modern IGPs... the bigger issue is the sheer memory requirements.

Training requires a ridiculous amount of both, but again the raw compute requirements would be much lower with huge memory pools.

A guide for the non-technical