[LarsDu88]

The closest example I've seen is ChatJimmy: https://chatjimmy.ai/ a prototype from Taalas running Llama 8B

Scaling this up to 2.8 Trillion (350X increase), will certainly be challenging.

If I was younger and had the right background, I'd love to dive into attempting somethign like this

[selectodude]

Also a 3-bit quant. Useable but a long long way away from useful.