[bryanlyon]

Llama is a very cool language model, it being used for coding was all but inevitable. I especially love it being released open for everyone.

I do wonder about how much use it'll get, seeing as running a heavy language model on local hardware is kinda unlikely for most developers. Not everyone is runnning a system powerful enough to equip big AIs like this. I also doubt that companies are going to set up large AIs for their devs. It's just a weird positioning.

[int_19h]

12Gb of VRAM lets you run 13B models (4-bit quantized) with reasonable speed, and can be had for under $300 if you go for previous-generation NVidia hardware. Plenty of developers around with M1 and M2 Macs, as well.

[outside1234]

... "seeing as running a heavy language model on local hardware is kinda unlikely for most developers"

for now it is :) but with quantization advances etc. it is not hard to see the trajectory.

[ctoth]

As we all know, computers stay the same and rarely improve.