[ingenieroariel]

WebGPU would be a way to shift that cost back to each device. RedPajama 3B could become useful for some tasks and run quite fast on most hardware available. Then as users have better computers, they can get access to better models?

[snovv_crash]

You'd have to have each user download a 3GB payload first. For comparison, that's a good few hours of netflix at 1080p.

[sroussey]

Browsers and operating systems will eventually include them.

But for now, you would need a good privacy reason to go this route.