[theodorewiles]

My take is that B2C AI applications are kind of structurally limited by how hard it is to build personalized context.

The idea of capable local models could be a huge unlock here if they are able to do the bottom-up context collection research / tagging / etc. at scale.

[michaelbuckbee]

I made a B2C AI app that's fully local (and free) to do AI based contextual file renaming.

So if you give it a bunch of screenshots it will try and intelligently name them based upon what is in the screenshot. Same for videos, PDFs, etc.

But to your point I haven't even tried charging money as it feels like something Apple is just going to bake in as a feature.

https://finalfinalreallyfinaluntitleddocumentv3.com/

[asenna]

This is cool. And yeah love the name!

Are you planning to open source it? Or maintain it in the future?

[michaelbuckbee]

My plan was to just see if anyone wanted to actually use it first. That if I couldn't give it away I'd not invest the time in selling or open sourcing it.

I'd sort of designed it for my own needs first and hadn't thought too far beyond that.

[ntcho]

absolutely love the domain here. great taste

[asenna]

Definitely agree with this. Here, me and Claude brainstorming together did that Research, and some trial-and-error to get to this.

But I can tell it's only a matter of time before agents become smart enough to let my non-tech friends be able to just say "Make sense of all these videos in my folder" and it just does it.

[enos_feedler]

Is it really local models that unlock this? Surely stateless model APIs would yield the same benefits? I get that local can be “cheaper” depending on usage, but we’ve been renting storage and compute from clouds at a premium for ages..

[asenna]

A huge thing here was the massive amount of data that was just processed - I went through about 1TB of files over 24 hours.

Using API to analyze even a subset of this would've been painful imo.

[enos_feedler]

I thought about that in this video case and it's true. I thought the parent comment was making a broader statement about local models in general. But even with video, if it was stored in private cloud storage near the LLM could this still have worked efficiently? What are the most painful elements of this whole setup / work environment if everything was cloud?

[asenna]

Oh yes, if everything is cloud, then this is a non-issue.

The few other points of consideration would be:

1) Cost - I was considering using Sonnet for this but there's always the concern of reaching limits OR the API cost if you're using the API.

The feeling of knowing you have a capable model in your hands without any limits is actually pretty awesome. Your mind starts running at what else can I throw at it to do grunt work.

2) Privacy issues - same as with moving to cloud.

3) Reliability issues - I know from experience Claude uptime has been pretty bad the past few months

4) Restrictions - Claude has been pretty heavy handed with their restrictions lately, anything which remotely triggers there flags gets an instant denial (or worse, an account ban). Often these are false-positives.

I love the value I get from Claude but there's a different kind of freedom you get with local, capable models.