|
|
|
|
|
|
by xtracto
37 days ago
|
|
|
we need this:
https://news.ycombinator.com/item?id=48516751 > distributed LLM inference. We are at a point where no single person can setup a rig to run a SOTA model, it is just too expensive.
So we must build and adopt frameworks that allow individuals to share resources to run SOTA models in a distributed manner. That way they will also be non-censorable by governments. Also The only way to prevent that one entity weaponizes it, is by giving EVERYONE access to it. |
|
|
You rent your fiber optic internet. You're doing just fine. The world isn't collapsing because you don't own the hardware racks, routers, and fiber lines.
This crazy zany P2P communal infra is Arch Linux coded - too much work, aimed at the 0.001% of users, and the juice isn't worth the squeeze.
It sounds the exact same as being mad at your ISP, so you want to build a mesh internet protocol over microwave dishes and share with everyone in your neighborhood. That was a thing in the 00's. And it predictably got nowhere and delivered no value.
Nobody's got time for this kind of stuff except for hobbyists. It's not solving a real problem. Nor does it attract business investment to grow into a healthy offering.
Just build OpenRunPod instead.
The problem is that the world doesn't have access to frontier open weights. That's the only problem to focus on and solve.
The quickest way to get there is to use large scale open weights instead of little rinky dink RTX distillations. And the easiest way to run them is by renting H100s or using a managed service offering. There's nothing wrong with either of those options.
The infrastructure is the easy part anyway.