Hacker News new | ask | show | jobs
by idle_zealot 11 days ago
Even the aspirational use cases you're talking about basically are just "digital secretary." There's a massive problem with that even if the models end up being capable in the future. The value of a secretary is that you know them, they know you, and you trust them to do things right. There are stakes if they don't. No company can provide that as a service at scale for everyone without it being a disaster. Not because it's not technically possible, but because of the incentives. That much power over the details of so many people's lives is irresistible; there will be persistent temptation to use it. The presence of that possibility makes the secretary impossible to trust.
3 comments

The vast majority of people never even asked for a personal assistant, because that isn’t something normal people have or do need in the first place. They aren’t so occupied/privileged/posh to need someone to do the trivial tasks of daily life for them.

This whole venue of technology is an exercise in ivory tower construction completely disconnected from ordinary people.

I think you are wrong.

They never asked for one becase they never imagined being able to afford one.

The amount of administration organizing a normal household takes I suspect most would be glad to leave to someon/something they trust and that can be held accountable.

Today that someone needs to be a person (imo). But who knows, a startup may be plotting accountable digital assistans as we speak.

I would want someone who could physically work in my house, performing the chores/daily tasks I don't want to.

I have zero use for AI. What's it going to do? Read the 3 emails about bills I get?

Certainly someone out there has a use for this functionality, but when you say "household admin tasks" the last thing I think about are /digital/ admin tasks.

I think theres some real use cases in the household department. I would love an AI that I just tell my nutrition goals (I want to eat X calories, Y protein, Z fiber, hit all vitamins) and it just generates a full meal plan for me each day. Like, I go to the store and it made the complete shopping list for me. And automatically updates the rest of the day if I tell it I skipped a meal or ate a snack.
I get where you're coming from but at least for me you appear to be looking to automate away the interesting bits while being left with the tedious ones. What I want is the opposite to what you're asking for - let me dump a rough meal plan into whatever thing is doing this giving an overview of what meals I want to cook this week and then have it go place an order with the supermarket for delivery of the necessary ingredients taking into account what I've got in the house already.
Eating disorder as a service does indeed sound like a business plan.
Some people would use the GP's idea that way, yes.

But that's absolutely not what he's describing. He wants not to think about it, that's exactly the opposite of a disorder.

Anyway, an LLM assistant is also exactly the worst technology to use there, on every dimension.

People with eating disorder do not want to think about it, they cant stop thinking about it. They create all kinds of systems for themselves, but mind cant stop - and restrictions grow.
Why would having goals on fiber, protein, and vitamin intake be an eating disorder?
When the urgency and complexity of those goals becomes so high that it creates daily burden, you are in the eating disorder territory.
Pretty sure unaccountability is a desired feature of management decisions in most organisations.

That quote has an unexpressed precondition to the effect of "In order for an organisation to be objectively well run..." or "In order for an organisation to equitably benefit all stakeholders at all levels..." etc

Giving normal people something that has only been available to rich people is a staple of technological innovation. The problem in this case with Siri isn’t that people don’t want an assistant. It’s that it doesn’t actually work yet.
But normal people have entirely different problems than rich people do! The amount of administrative overhead that would warrant a personal assistant is just vastly lower - most normal people:

  - don't travel frequently, 
  - don't have so many complex inquiries that require someone to research,
  - don't have super complicated taxes to file, 
  - don't go eating out in fancy restaurants that require special skills to get reservations in, 
  - don't have so many meetings to attend, 
  - don't receive hundreds of emails per day,
  - don't work on multiple projects at the same time,
  - don't organise festivities and social gatherings all the time. 
Yeah, there probably are some things that could be simplified by delegating to someone, but they don't justify a human PA at all; and out of the remaining tasks, most are not really digital in nature: Going for groceries, doing chores, child and elderly care, interacting with other people, and so on. Digital assistants can't help you with any of these.

The one thing that would be useful - a kind of "chief of staff" that monitors your entire digital life and prioritises your every next step - is the antithesis to Siri and the like, which are merely reactive to your requests, not proactive in figuring out what needs your attention next. Let alone that that would be a total privacy nightmare, and a prime candidate for mass manipulation at scale.

Bang on.

Like you said, the non-digital things are where people need assistance most. Fold my laundry, clean my house, clean my car, make me dinner. At an affordable cost. I don't need you to book my trip to an all inclusive resort that I go on once a year at most.

We're at this place where AI/LLMs is truly incredible technology, but the futuristic vision of robot assistants doing things for you at an attainable cost isn't there yet - so a lot of companies/startups are trying to force feed purely digital consumer AI products (assistants/agents) that no one wants.

In the US, every time I file my taxes, I wonder what % of people don't meet the cognitive barrier to successfully file. I suspect a large portion of people offload tax filing to a service or accountant for numerous reasons, which is basically a personal assistant.
Personally, I'd more interested in Reminders actually being able to sync lists properly and not delayed or reshuffling items while I'm typing before they would work on a personal assistant. Reminders (like Siri) has become the favorite joke of the family by now.

They don't even seem to get the basics right, why would I want another layer on top?

That's why I like reading HN. These people are smart enough to destroy the world but too stupid to realise they're doing it
it's an interesting question if any of the AI companies would be willing to step up and absorb the risk ie: to give the AI agent a "stake".

eg: if my booking is wrong, they will cover the cost and compensate me. It would sort of just come down to buying premium travel insurance for everyone that uses it. And insurance for anything else they do. It has to be one of two things - they either believe the risks are worth it (so then there should be a financial model that can absorb the cost of insurance to do it), or in fact, the risks are too great. At some point, if they keep offering the tech on a "use at your own risk" basis, they are implicitly communicating that they themselves think the risks are too great - so YOU shouldn't trust it either.

> eg: if my booking is wrong, they will cover the cost and compensate me

That would be nice, but it's the wrong angle. The reason people like real secretaries is not because somebody is compensated when things go wrong. It's because things don't go wrong. I don't use this thing if I need to fear things go wrong, even if I'd be compensated.

Maybe it would provide the right incentives for the companies though.

Surely, if the compensation was high enough, you'd be like, "Sure, I'm happy with that outcome." And then, if the AI company thinks they have a low enough failure rate that the expected cost of paying out the compensation still lets them make a profit, then they could make that promise to all customers.

Though a compensation that high sounds like it would invite fraud, where the customer would be glad to have something go "wrong" and get a fat check. Not sure if that's a solvable problem.

they are literally burning billions of dollars and can presumably keep doing this for a while. Taking a trivial amount of additional financial liability to launder their reputation wouldn't meaningfully improve things.
These use cases will just be built as "open source" (openclawd) or even custom one off application in the future. I've been building apps to run the tedious parts of my life recently. Meal planning, personal finance, bills, tax organization... Why would I pay for services that will be enshiftified when I can build a app that does exactly what I want in an afternoon. Yes the code is shit and it wouldn't scale... But it doesn't need to
> Why would I pay for services that will be enshiftified when I can build a app that does exactly what I want in an afternoon

Because the problem now took a whole afternoon to solved and sapped your creative energy instead.

> Why would I pay for services that will be enshiftified when I can build an app that does exactly what I want in an afternoon.

When we talk about “the market”, the customer base, remember it’s a market that typically doesn’t know how to or care to even install an adblocker.

I don't see any mention of "the market" anywhere in this thread. I'm just talking about the ability for a motivated user to solve real problems with these tools. Right now these solutions are available to software developers but over time it will become approachable to more users