[whatever1]

Why we don’t have an LLM based search tool for our pc / smartphones?

Specially for the smartphones all of your data is on the cloud anyway, instead of just scraping it for advertising and the FBI they could also do something useful for the user?

[rudedogg]

This is roughly what Apple Intelligence was supposed to deliver but has yet to.

[curious_cat_163]

> Why we don’t have an LLM based search tool for our pc / smartphones?

I'll offer my take as an outside observer. If someone has better insights, feel free to share as well.

In market terms, I think it is because Google, Microsoft and Apple are all still trying with varied success. It has to be them because that's where a big bulk of the users are. They are all also public companies with impatient investors wanting the stock to go up into the right. So, they are both cautious about what ship to billions of devices (brand protection) and cautious about "opening up" their OS beyond that they have already done (fear of disruption).

In technical terms, it is taking a while because if the tool is going to use LLMs, then they need to solve for 99.999% of the reliability problems (brand protection) that come with that tech. They need to solve for power consumption (either on edge or in the data centers) due to their sheer scale.

So, their choices are ship fast (which Google has been trying to do more) and iterate in public; or partner with other product companies by investing in them (which Microsoft has been doing with Open AI and Google is doing with Anthropic, etc.).

Apple is taking some middle path but they just fired the person who was heading up the initiative [1] so let's see how that goes.

My two cents.

[1] https://www.reuters.com/technology/artificial-intelligence/a...

[visarga]

I found that ChatGPT or Claude are really good at music and shopping suggestions. Just chat with them about your tastes for a while, then ask for suggestions. Compared to old recommender systems this method allows much better user guidance.

[josephg]

Yeah, Claude helped me decide what to get my girlfriend for her birthday a few weeks ago. It suggested some great gift ideas I hadn’t thought of - and my girlfriend loved them.

[Workaccount2]

I think we can expect this to be rapidly monetized.

[KoftaBob]

For shopping suggestions, I've had the best experience with Perplexity.

[GraemeMeyer]

It's coming for PCs soon: https://www.theregister.com/2025/01/20/microsoft_unveils_win...

And to a certain extent for the Microsoft cloud experience as well: https://www.theverge.com/2024/10/8/24265312/microsoft-onedri...

[wildrhythms]

Pixels already have the Screenshots app that indexes screenshots and makes them searchable. My assumption is the context window size is still too small for all of your data to go into it.

[dmbche]

It doesn't solve any problem, you can just search your files using your prefered file explorer (crtl-f)

I'd assume most people organise their files so that they know where things are as well.

[nine_k]

> you can just search your files using your prefered file explorer

This only work if you remember specific substrings. An LLM (or some other language model) can summarize and interpolate. It can be asked to find that file that mentions a transaction for buying candy, and it has a fair chance to find it, even if none of the words "transaction", "buying" or "candy" are present in the file, e.g. it says "shelled out $17 for a huge pack of gobstoppers".

> I'd assume most people organise their files

You'll be shocked, but...

[dmbche]

But isn't that candy example non-sensical? In what situation do you need some information without any of the context(or without knowing any of the context)?

i really believe that this is not an actual problem in need of solving, but instead creating a tool (personal ai assistant) and trying to find a usecase

Edit0: note to self, rambling - assuming there exist valuable information that one needa to access in their files, but one doesn't know where it is, when it was made, it's name or other information about it(as you could find said file right away with this information).

Say you need an information for some documentation like the C standard - you need precise information on some process. Is it not much simpler to just open the doc and use the index? Then again for you to be aeare of the C standard makes the query useless.

If it's from something less well organised, say you want letters you wrote to your significant other, maybe the assistant could help. But then again, what are you asking? How hard is it to keep your letters in a folder? Or even simply know what you've done (I surely can't imagine forgetting things I've created but somehow finding use in a llm that finds it for me).

Like asking it "what is my opinion on x" or "what's a good compliment I wrote" is nonsensical to me, but asking it about external ressources makes the idea of training it on your own data pointless. "How did I write X API" - just open your file, no? You know where it is, you made it.

Like saying "get me that picture of unle tony in Florida" might save you 10 seconds instead of going into your files and thinking about when you got that picture, but it's not solving a real issue or making things more efficient. (Edit1: if you don't know Tony, when you got the picture or of what it's a picture of, why are you querying? What's the usecase for this information, is it just to prove it can be done? It feels like the user needs to contorts themselves in a small niche for this product to be useful)

Either it's used for non valuable work (menial search) or you already know how to get the answer you need.

I cannot imagine a query that would be useful compared to simply being aware of what's in your computer. And if you're not aware of it, how do you search for it?

[mjlee]

I think your brain may just work differently to mine, and I don't think I'm unique.

> "get me that picture of unle tony in Florida" might save you 10 seconds instead of going into your files and thinking about when you got that picture

I don't have a memory for time, and I can't picture things in my mind. Thinking about when I took a picture does nothing for me, I could be out by years. Having some unified natural language search engine would be amazing for me. I might remember it was a sunny day and that we got ice cream, and that's what I want to search on.

The "small niche" use case for me is often my daughter wants to see a photo of a family member I'm talking about, or I want to remember some other aspect of the day and the photo triggers that for me.

[dmbche]

Makes a lot of sense. Thanks for the response, enjoy your day!

[IanCal]

> But isn't that candy example non-sensical? In what situation do you need some information without any of the context(or without knowing any of the context)?

I know the context and the content but not the specific substrings in an email I received several years ago.

Here's one of the first things that gemini in gmail actually helped with. I wanted to check when I bought a car seat for my kids, which one it was and how much it cost.

So I knew the rough time it was when I bought it, I know it's a receipt I'm looking for, it's for a child seat, and roughly when. I know the context here.

What I struggled with was finding the exact text that would be in that. There are hundreds or more emails with invoice/receipt/order in. I didn't recall exactly who I bought it from, and there are large numbers of more advertising emails with kids seats in.

I couldn't easily find it, because the actual email I wanted did not say child seat in it. It had a brand and other information, but nothing in the text had a substring I was searching for. I might have found it with "booster seat" but I didn't think of that exact phrase at the time.

Instead I asked gemini to find it. That can then trawl through a bunch of emails and find things that mean but do not say child seat.

[dmbche]

Makes total sense, I'm not entirely sure why but I had assumed we were talking about an AI assistant being device specific, which I understood as "based on my offline data and files" - I'm saying I'm not sure why because the parent comment is specifically mentionning the cloud. Anyhoot.

Enjoy your day.

[pizza]

Here’s an example of a type of feature I want: I’m looking at a menu from a popular restaurant and it has hundreds of choices. I start to feel some analysis paralysis. I say to my computer, “hey computer, I’m open to any suggestions, so long as it’s well-seasoned, spicy, salty, has some protein and fiber, easy to digest, rich in nutrients, not too dry, not too oily, pairs well with <whatever I have in my fridge>, etc..” Basically, property-oriented search queries whose answers can be verified, without having to trudge through them myself, where I don’t really care about correctness, just satisficing.

[ozim]

I think the same, people are not organized - even with things that make them money and being organized could earn them much more.

[whatever1]

But file explorer does not read the actual files and build context. Even for pure text files that sometimes search functions can also access, I need to remember exactly the string of characters I am looking for.

I was hoping an LLM would have a context of all of my content (text and visual) and for the first time use my computers data as a knowledge base.

Queries like “what was my design file for that x service” ? Today it’s impossible to answer unless you have organized your data your self.

Why do we still have to organize our data manually?

[pests]

The photos apps do this well now. Can search Apple/Google photos with questions about the content of images and videos and get useful results.

[ozim]

I think you are really wrong.

Most people I see at work and outside don’t care and they want stupid machine to deal with it.

That is why smartphones and tablets move away from providing „file system” access.

It is super annoying for me but most people want to save their tax form or their baby photo not even understanding each is different file type - because they couldn’t care less about file types let alone making folder structure to keep them organized.

[acchow]

Curiously, the things I search most often are not located in files: calendar, photo content/location, email, ChatGPT history, Spotify library, iMessage/whatsapp history, contacts, notes, Amazon order history