[xandrius]

Just about the point about the prohibitive infrastructure at scale, why does it need to be at scale?

Over few years, we went from literally impossible to being able to run a 72B model locally on a laptop. Give it 5-10 years and we might not need to have any infrastructure at all, all served locally with switchable (and different sized) open source models.

[imhoguy]

Exactly open-source doesn't need scale in terms of user base. You can easily infer with hundreds B parameter models on pay as you go infa for a few dolars, or just build commodity rig for a few thousand. That is affordable for SMEs or even devoted hobbysts. The most important part about opensource is democratization of LLM access.