[enos_feedler]

Is it really local models that unlock this? Surely stateless model APIs would yield the same benefits? I get that local can be “cheaper” depending on usage, but we’ve been renting storage and compute from clouds at a premium for ages..

[asenna]

A huge thing here was the massive amount of data that was just processed - I went through about 1TB of files over 24 hours.

Using API to analyze even a subset of this would've been painful imo.

[enos_feedler]

I thought about that in this video case and it's true. I thought the parent comment was making a broader statement about local models in general. But even with video, if it was stored in private cloud storage near the LLM could this still have worked efficiently? What are the most painful elements of this whole setup / work environment if everything was cloud?

[asenna]

Oh yes, if everything is cloud, then this is a non-issue.

The few other points of consideration would be:

1) Cost - I was considering using Sonnet for this but there's always the concern of reaching limits OR the API cost if you're using the API.

The feeling of knowing you have a capable model in your hands without any limits is actually pretty awesome. Your mind starts running at what else can I throw at it to do grunt work.

2) Privacy issues - same as with moving to cloud.

3) Reliability issues - I know from experience Claude uptime has been pretty bad the past few months

4) Restrictions - Claude has been pretty heavy handed with their restrictions lately, anything which remotely triggers there flags gets an instant denial (or worse, an account ban). Often these are false-positives.

I love the value I get from Claude but there's a different kind of freedom you get with local, capable models.