Hacker News new | ask | show | jobs
by JohnBooty 8 days ago
At a 50-100x speedup even a GPT-4o class model could perhaps compete with much newer models simply by thinking deeper, doing harness-controlled Ralph loops, etc. Sure, then it might be "only" ~2-5x faster, but, you wouldn't need to throw all the ASICs into the trash bin.

One could also imagine hybrid models, where part of the model is burned into ASICs and part of the model exists in VRAM/HBM2 so it can be updated.

I don't have enough low-level knowledge to evaluate the technical or economic feasibility of the above ideas, however.
2 comments
boplicity 8 days ago
For many, many tasks, you only need good enough results that accomplish a clearly-defined set of criteria. Smaller models that run 50x faster, in this case, could be far superior to a much slower model that meets the same criteria.

Sometimes you need speed; sometimes you need quality. There are very different use cases for each. For my own workflows, I sometimes want something very simple done ASAP; other workflows need "subjective" reasoning and careful crafting of responses.

link
tracker1 8 days ago
Giving a literal monkey the ability to press more keys faster doesn't get a good joke from it.

A model will often come up with worse results given more cycles of compute, only because it will tailspin from second guesses, rethinking and literal flip-flopping on concepts.

--- edit: to those following the thread below... if you look at the comments from the account replying, it's pretty obviously a pro-China account and all replies are antagonistic against anything other than a total submission to the Chinese state. My responses are intentionally antagonistic as every point I've brought up is completely ignored in favor of insults, so yeah, I've been insulting back.

link
JohnBooty 6 days ago 

    Giving a literal monkey the ability to press more 
    keys faster doesn't get a good joke from it.
To some extent, it's because making a joke isn't an iterative exercise with a verifiably correct outcome.

However, a lot of jobs are. Notably, much of software engineering.

link
sdfefcxv 8 days ago
Said the OAI/Anthropic stockholder
link
tracker1 8 days ago
LOL.. neither... but good luck on your standup career with jokes by literal monkeys.
link