|
|
|
|
|
|
by zavertnik
936 days ago
|
|
|
For me, the gap between 3.5 and 4 is massive. If I'm stuck between using 3.5 and doing the work myself, more often than not, I'm choosing to do it myself. Not to imply 3.5 is unusable, its just my bar for minimum fidelity is closer to 4 than 3.5 with respect to tasks that I'm comfortable offloading onto an AI. What are you running goliath-120b on? Is it costly to run all day every day? How long does it take to complete an output? I've thought about building a multi GPU node for local LLMs but I always decide against it on the premise that the tech is so new I figure in the next 3-4 years we'll see specialized hardware combined with efficiency improvements that would make my node obsolete. |
|
|
> I always decide against it on the premise that the tech is so new I figure in the next 3-4 years we'll see specialized hardware combined with efficiency improvements that would make my node obsolete.
You're probably right, this happened back in the day with bitcoin mining.