[jonstaab]

This is the opposite direction AI should be going. Human relationships are the most valuable thing we have, and so, naturally, technology seeks to intermediate and now replace them.

I'm not Catholic, but this podcast presents a very interesting argument against talking to AI as if they were human: https://newpolity.com/podcasts-hub/debate-chatbots

[zkmon]

Most of what AI does is already in wrong direction. Not just human-to-human interaction, it took away thinking, creative work, sensory perception (glasses) and responses. People call it as helping humans, but I call it as sucking away the "human-ness" from humans.

After the damage is done, the mega corps would simply shrug and will say "Well, we were just responding to our business competition" The business knows no human-ness, because it is not a human. Businesses and machines are creatures that see humans as their fodder. And humans created these, assuming it is progress, to have businesses and machines. We call it progress because it required our mind power and it helped us to dominate other species. Dolphins are laughing at us.

[amarant]

I don't understand this line of reasoning. How are you hindered from doing any of those things? What part of "AI can now do X" makes it so you can't also do X?

[hackrmn]

Fewer people aren't staring into their phones or talking to them -- makes your social antennas pick up automatically on not wanting to disturb them (lest you draw their ire for not having the social antennas long enough to pick up on the fact they're "busy and don't want to engage with you" like a gymrat with AirPods to signal they're there to pump in peace and quiet listening to their favourite playlist, not talk to strangers). Happened to me already many times just with people scrolling their phone instead of talking and not wanting to talk in particular either, not to me at least. And no -- I am not talking about bothering strangers in the gym etc, I am talking about sitting at the lunch table where half of the people look into their phones -- they aren't actually interested in talking, it turns out.

Our devices have now increased the distance _between_ us -- it's not about _you_ being able to "do X" -- talking to others is not _you_ doing it, it's you _and the other person_ doing it _together_. You can't be doing anything together consentually when the other person is in the habit of talking with their AI, or doomscrolling for that matter.

[batperson]

Social people will be fine, I think this tech is far more important for lonely people who for any reason don't get to socialize much (if at all), this is especially common in older people. These people might not have any other alternatives.

[hackrmn]

I think a fundamental and flawed assumption in your general line of argument boils down to that line:

> Social people will be fine

Thing is, "social" is not only a scale, it's not even a line for scale. It's this multidimensional space of different social preferences and different preferences as to _how_ to socialise, not to mention with whom and for what reason. In short, it's much more complicated than as to permit a "binary" division into the "social" and "other" people, but even if you did, I would wager the division puts it at 50% for either group, meaning you're basically describing a cataclysmic event where 50% of the population suddenly has no access to a fundamental trait of humanity that in known and unknown ways has assured our feeling of happiness and warded off a plethora of conditions of misery and worse that psychology has no names for (and won't ever have names for because that's like classifying different gradations of drinkable water, I imagine).

But yeah, humanity has always had to evolve, it's just that these changes come too fast for our adaptability to adapt to, I believe.

[anon-3988]

> I think this tech is far more important for lonely people who for any reason don't get to socialize much (if at all), this is especially common in older people.

Uhm, those lonely people need to get out and start talking. How is this going to help society? This is going to make it worse.

Oh that kid is kinda quite and sad? Throw him an iPad. Oh that adult is kinda bored and wandering aimlessly? Throw him into a casino. Oh that adult is kinda lonely and feels like they don't have anyone they can talk to about their life? Give them LLM companionship.

Yep, it is over for humanity. People simply don't understand externalities.

[batperson]

What about people with disabilities, physical or mental, who can't get out, old people with no family or family who doesn't care? In theory it would be great if everyone got some attention and socialized, in reality a lot of people in society are forgotten, no one wants to talk to them.

For some of these people even talking to a robot would likely be a huge improvement in quality of life, and that's just talking. If said robots could also help them out in real life, sort of like a personal assistant, that would be even better.

[jamilton]

>Uhm, those lonely people need to get out and start talking.

Great, what is being done to help that happen?

[andai]

There are still lots of social people. I found a lot of people actually do want to talk but are just shy.

I spent a few weeks at a hostel last year. It was always kind of depressing and tense in the shared kitchen, just this heavy silence.

I don't feel comfortable around strangers, so I solved that problem by just saying hi to everyone.

Most people didn't respond much, although most of them smiled and the tension was eased. But a few of them struck up conversation and we ended up making friends.

I ended up making like, ten new friends in two weeks. And then a bunch of them ended up becoming friends with each other as well.

[domga]

This is an individual solution to a systemic problem. On a personal level, it is possible to solve such problems, but generally no, the ship has sailed quite some time ago (I personally think cars are to blame).

Even if you do it, you are still swimming against the current.

[logicchains]

>This is an individual solution to a systemic problem. On a personal level, it is possible to solve such problems

I.e. it's only a problem if you're not willing to go and strike up conversations with people. Which is not AI or mobile phones' fault. Expecting society to come up with "systemic situations" to your personal problems is a fast path to a lifetime of disappointment.

[monkaiju]

Hasnt this already been thoroughly discussed? Your ability to do X degrades as you offload it more frequently, eventually to the point that you can no longer even vet the quality of the output.

I think the parent is saying now that that is attempting to be applied to "creativity" directly, as opposed to something like a shift of medium, that it threatens many peoples' ability to maintain creative capabilities.

Anecdotally I've already experienced this at work where post-AI we had a junior completely stagnate and a senior with over a decade of experience in the bay atrophy to the point that he had to be let go.

[xpct]

We're yet to see how this plays out, but a competing business model for creative work is emerging, where it's delegated to chatbots. Naturally, this would result in less creative work for humans.

[perching_aix]

Playing the devil's advocate a little, you say "can't also do", but that implies prohibition, not hindering. Hindering is not total like that.

It's like trying to have meaningful conversations on Twitter. You don't go to Twitter to do things like that. Can you? Sure. It's just not what the format and the conventions (and the people) lend themselves to.

I don't think there's much merit in pretending that human activities are only shaped by hard limits.

[zkmon]

I'm no longer writing code from scratch, as I used to do before. So, very soon, it will be "AI can now do X" makes it so I can't also do X? Same with many creative works. Radio music already sounds so plastic. I lost interest in crafting my text drafts because I can just dump some ugly text and get it refined by AI.

[small_model]

You no longer have to grow your own food, can go to supermarket, but doesnt stop you from still doing it.

[esafak]

I hope that's not the future of work before UBI gets settled, because almost nobody grows their own food.

[motoxpro]

I'm confused. It feels like you are saying you did those things as a means to an end.

Like saying I no longer walk because I can drive. I no longer cook because Doordash exists. I no longer play piano because midi exists.

I mean i guess, but it seems like you didn't LIKE crafting the text or coding from scratch, you just wanted the outcome. If we are talking purely about work, I understand that its about being productive and it sucks to have a job shift to something you enjoy less.

But for daily life? I dont see how it changes, maybe its a tech thing where people think about making their daily lives more productive, but most people dont.

[zkmon]

The human-ness is not about outcomes only. It is about doing things, thinking, achieving, socializing, sharing, bonding, feeling happy about what you did, using senses for what they evolved fir, recognizing and responding to body signals and being a biological creature. A lot of these is affected by AI.

[motoxpro]

None of those are affected by AI.

You can CHOOSE to let AI think for you or achieve for you, but you dont have to. You only HAVE to in a work context (for some people), which I admit sucks (or is great depending on who you are).

This feels like a touch grass moment.

[overgard]

In a personal context, you're not hindered from doing those things, so you're correct in that regard. The problem is economic and social. The AI is mediocre enough to drive the value of those things way down; possibly to zero. When something isn't valuable, people are less likely to learn it, and in the future it's less likely that anyone will actually have those skills. For example, we've slopped many illustrators out of jobs and essentially made art (an already awful paying career before AI) economically infeasible. AI illustrations kind of suck though, and even when they're technically competent they're kind of soulless. So you might say, ok, I will hire/contract an artist if I care about the quality of my illustration! Ok, but, if artists are being priced out by a machine then how long before there's no real market for finding a human to do it because all the artists gave up and got a job at starbucks and all you have left are amateurs?

This is almost certainly going to bite us in the ass long term, because eventually without human creativity you're just training AI's on other AI slop, or limiting the possible catalogue of styles to "things that existed before AI ruined every creative job". I guess the question is, what kind of future do you want to live in? One where we have a massive abundance of easy-to-create but vaguely worthless artifacts in a society that's completely devalued being good at something? It just sounds really dystopian to me.

[spongebobstoes]

"starving artist" is not a new concept

AI doesn't stop creative people from creating. it gives creative people an additional tool with which to create

[FridgeSeal]

This is repeatedly the argument rolled out by people desperate to get us to accept it at all costs.

Ai doesn’t technically stop me writing code. But it’s sucked so much joy and interest out of it, that I can’t scrape together the motivation to work on side projects anymore. It has a “chilling effect” the effects of which are notoriously indirect to observe.

[spongebobstoes]

why does me using AI affect your motivation at all?

[breezybottom]

Then why is it only the least creative people who use it?

[overgard]

> "starving artist" is not a new concept

Exactly, so why make it worse? Why try to automate an industry when the money you make off killing it wouldn't even cover the cost of your research?

> AI doesn't stop creative people from creating. it gives creative people an additional tool with which to create

I can't emphasize enough how much actual creative people hate this shit. Here's a relatively balanced take, although the headline spoils it (How AI Poisons Creativity: https://www.youtube.com/watch?v=WeCSzEtZcUw ). Long story short, the struggle is where creative ideas come from. If you automate away the process you automate away the result.

[supern0va]

Counter-point: I love that my rubber duck can talk (quack?) back, as well as record and summarize my thoughts on topics I'm working or stuck on.

I've wanted a good voice mode for precisely this reason. When I take my dog on a walk and I'm thinking about a bunch of problems/ideas, I'd love to have feedback and a record, or perhaps to even kick off research or ask questions to fill in gaps that would otherwise have me debating pulling out my phone to try to get an answer.

[zkmon]

You want to be a child forever, amazed by your toys or the new things you can do. The toys won't let you grow up and become a normal human. You don't need take a dog for a walk, if you live in a normal place that allows dogs to roam freely. And you don't need to think about a "bunch of problems/ideas". Your mind is a servant of your body. And your body never asked you for that.

[supern0va]

>You want to be a child forever, amazed by your toys or the new things you can do.

I don't think having a child-like sense of wonder at the world, or continued human ingenuity, is a sign that one wants to be a child forever.

I'm genuinely sorry if you've lost that spark.

>You don't need take a dog for a walk, if you live in a normal place that allows dogs to roam freely.

I love my dog and love walking him, even though he can freely roam on my property. I also love walking with my wife, and I don't think her agency has much reason for why I might enjoy it?

>Your mind is a servant of your body. And your body never asked you for that.

...are you okay, friend?

[ryandvm]

> "Well, we were just responding to our business competition" The business knows no humanness, because it is not a human.

Yup. Except, by God, we ought to make sure the business has unlimited free speech (i.e. campaign contributions).

[bb88]

Businesses have been anti-human for a while now.

If you think AI is bad, wait until I tell you about the horrors of social media who profit on controversy and division, US health insurance which profits on rejecting claims, and big pharma profiting on the opioid epidemic.

And it's not like this is new, either. Upton Sinclair was writing about this stuff a century ago with books like The Jungle.

The only difference between then and now is we "think" we're not evil today. We've lied to ourselves that "We're so much better than we were back then." Facebook wanted to bring people together originally, but they ended up providing the most toxic social media experience known to man. Facebook forgot to tell us they cared more about profit than people.

Please spare me the argument AI is the straw to break the camel's back here. The system has been broken a long time before that.

[Citizen_Lame]

Bit naive world view. There are forces of rich and powerful who's only purpose is to maintain their status.

So thinking corporations and such were created to push human progress is laughable.

[filoleg]

> This is the opposite direction AI should be going. Human relationships are the most valuable thing we have, and so, naturally, technology seeks to intermediate and now replace them.

Valid, but, I think, you conflate two separate things.

AI voice mode as a human socialization/conversation replacement? Cringe in my book, fully in agreement with you. Though my opinion on that aspect remains the same, regardless of whether it is done through text or voice.

AI voice mode as an alternative interface to interact with AI-as-a-tool? Great idea imo. There were a few instances where I was either too tired to type or wanted to brainstorm things in more of a freeform mode, for which a well-working voice mode would have been great.

Naturally, the current distinction between AI-as-a-personality and AI-as-a-tool exists purely on the user's end. All I know is that I want the latter a lot, and if some people want to use it for the former purpose, that's not my problem. Sadly, I think that it will be judged more on how an average person decides to use it (i.e., in the most degenerate/reductionist ways possible), as opposed to being judged on the merits of what it can actually be used for by someone who just treats it as a tool.

[streb-lo]

Basically 100% of my interaction with AI is some form of question.

But there are people that treat it like a therapist or a friend.

In my opinion, the 2nd group should be discouraged from using these agents in this way, as it cannot be healthy. But if places like Reddit etc. turn into bot towns where do the AI companies get insight into the general population? Through the eyes of their most mentally unstable users?

So I don't really see things changing, and they'll probably get worse.

[mlsu]

Yes, every minute you spend texting or talking to a chatbot is a minute that you'd have spent talking to another human beings. Literally the only important thing in life, the basis of all value, the formation of self-identity, comes from communication with other human beings.

[johnfn]

This is such a poor mischaracterization of OP that I actually started agreeing with OP more.

[enraged_camel]

It's also hilariously wrong. It essentially argues, implicitly, that those who don't communicate with other humans are missing out on the "most important thing in life" and cannot form a self-identity.

[dag100]

> Literally the only important thing in life, the basis of all value, the formation of self-identity, comes from communication with other human beings.

I think you're mistaking their sarcasm for sincerity... especially considering the emphasis on self-identity ironically juxtaposed as originating from a decidedly non-self activity, which has all the hallmarks of being intentional...

On the other hand, reading their other content leads one to believe that they may, in fact, be serious... hmm...

[samwiseg]

It is unironically correct. Well, interaction with other human beings may not be the only important thing, but it is certainly far and away the most important thing.

[mlsu]

An infant begins to mirror the emotions of his caregiver at 3-4 months. He can distinguish his own body in a physical mirror at 18 months.

[mlsu]

Human beings require interaction with other human beings to identify the self. It is impossible to live without others. Read up on solitary confinement sometime - enough time without human companionship will drive any human being (except the schizoaffective) completely insane.

[breezybottom]

Yes? That's completely true.

[broken-kebab]

Let's not overdramatize, though. I'm not in need, or even in mood to talk to fellow humans every minute, so time spent with a clanker is not necessary taken from my human relations budget

[bredren]

I also think this undersells the real value of the bot, which is to handle tasks via voice that an average human either would not or could not do.

In the video example with the grannies, the knitter is essentially wanting a PA. Regular folks don't have PAs. Even when that became a thing in the aughts they were all outsourced.

When I've used voice chat, it has often turns into rabbit holes on very niche topics. For example, I had one start about the 1996 performance of Rage Against the Machine in Portland, Oregon that was supposed to feature Wu Tang Clan. (already outside most human's knowledge) that dove into details of the club scene in Los Angeles at the time of RATM's signing to Epic Records.

Was anyone else here at that '96 show in Portland? It seems like it might be challenging to find a person on the internet able to engage on the topic.

The person may exist, but not during my fleeting interest in the subject while walking to the park.

[BeetleB]

> Yes, every minute you spend texting or talking to a chatbot is a minute that you'd have spent talking to another human beings.

Human beings tend not to be available (results vary by culture).

Also, imagine you're 82 years old and living alone (e.g. widower). It is believed that lack of interaction is a significant driver of cognitive decline (which is why being hard of hearing accelerates the onset of dementia). I wonder if having an LLM to talk to under those circumstances will decelerate cognitive decline?

[Der_Einzige]

Such radical carbon chauvinism is ontologically evil. May those who hold this view reincarnate as durian fruits or cockroaches.

[mlsu]

I'm sure you'd have no problem in solitary confinement. Why not make life easy?

[beepbooptheory]

The implication that durians and roaches are somehow ontologically subaltern is a worse chauvinism.

[wraptile]

> May those who hold this view reincarnate as durian fruits or cockroaches

don't you dare clump the best fruit in the world with cockroaches!

[rel_ic]

Just saying: those are also carbon-based lifeforms :D

[perching_aix]

> every minute you spend texting or talking to a chatbot is a minute that you'd have spent talking to another human beings

Very blatantly and obviously not though???

[mlsu]

The extent to which it's true is the extent of the evilness of the technology. Go search the phrase "ai boyfriend" on reddit sometime; imagine what it will be like when society is fully baked with this shit. You're talking to AIs all day at work. You're talking to AI's when you use social media. You're talking to AIs for therapy. You're talking to AIs on dating apps.

If your answer is "well I'll simply touch grass" I agree. But most people won't which is why this is tech is immiserating and, I would argue, evil.

[m_fayer]

There’s something phenomenally powerful, uncanny, and potentially deeply corrosive about current AI. Dismissing it as “evil” is pat, and prevents any full encounter with something that now irrevocably exists and deserves and demands the consideration of thinking people.

[mlsu]

What would you call it?

Anti human, corrosive, powerful, uncanny... but not evil?

I am fully encountering this, believe me. My conclusion is that it's evil.

[m_fayer]

Potentially corrosive. Emphasis on potentially.

Powerful and uncanny are morally neutral traits.

At root an LLM is a different kind of mind that thinks different kinds of thoughts. (I’m making no claims about consciousness here.)

It’s a novel object for us humans, and we’ll need decades to come to grips with it. In the meantime I think a bit of humility is due.

[supern0va]

I can't help but think you're conflating cause and effect. People are using a tool (AI) to apply a band-aid to a widespread social problem (loneliness and isolation).

It's possible that an "AI boyfriend" might make someone less prone to put in the continued effort to keep rolling the dice on dating apps, but the reality is that there's a more fundamental problem driving this.

Also, I want this tool for work. Just because society is fubar and people are using this tool as a crutch for their inability to find a partner, doesn't mean I should lose better tooling that makes my life easier.

Focus on fixing the actual problem.

[FridgeSeal]

“Who cares about dirty syringes being left in the street, it’s really important for my work that I have access to the heroin I need”.

Things in society don’t exist in a vacuum, and “who cares about them, so long as I get my needs” is quite literally a part of the problem being discussed.

[supern0va]

I'd argue what you're suggesting is more like trying to ban syringes, which have multiple valid uses, in order to stop dirty syringes being left in the street.

As someone with a GLP-1 user at home, responsibly depositing their syringes in an approved container that we dispose of safely at a drop site, I'd rather not have bad users inhibit my access to a tool that makes our lives better.

[perching_aix]

So do you find the selfish justification of heroin addicts in your example partially compelling, or are you betraying your own rhetorical standards?

Just because things do not exist in a vacuum doesn't mean they should be blurred and conflated until unrecognizability. There's value in explicitly anchoring the salient aspects, and getting the cause and effect right, to the extent these can be done.

Both intelligence amplification and pathological parasocialness are very real effects of this technology, and a blanket ban, a blanket halt, and similar broad and dull policies are absolutely going to be unfair and unreasonable. And even such policies would not "happen in a vacuum", there's a consequence to prohibiting something not prohibited elsewhere.

Whether we can do better is where the jury is still out. I don't think an example like your dirty needles one particularly helps with this. It's asinine and inflammatory, quite the opposite to nuance-inviting, which I believe was more towards your actual intent.

[perching_aix]

That's not what I meant, nor is "touching grass" necessarily relevant.

I use these tools at work. If I talked to other people instead of a model every time (or even just some of the time) I talk to a model, I'd be tanking everyone's productivity by constantly disrupting them, and would get worse quality responses pretty much guaranteed, if I wouldn't be just ignored outright.

This isn't to say I stopped talking to colleagues either. I reach out to them about the same amount. The time budget I took away from was the one I used to spend manually doing things, reading docs, and so on. So your entire mental model of this is just outright false there.

On a personal level, I traded off entertainment time (though for me, interacting with ChatGPT is also just a form of entertainment). Instead of staring at YouTube, I pitch silly thought experiments, explore topics, ask difficult-to-search-for questions. So once again, I did not trade off social time, meaning what you propose simply does not apply whatsoever. I do not share memes or catch up on life with it, which is what friends are for. It has no sense of actual humor, has no life of its own, and I'm not into roleplay like that. Would be just weird.

There are people who engage pathologically with LLMs, but that is neither the premiere use nor the goal. I'm sure there are also situations where people will actively approach a model over a person, but I do believe people should be able to self-regulate that decision. It gives people leverage to choose the extent and venues of their social engagement. If a topic has a community you do not wish to interact with, you can finally choose not to, while still reaping any prospective benefits (information). This can be a bad thing, but I think it's easy to see that it is also very much good.

For example, there were times where I felt more compelled to interact with ChatGPT than I felt leaving a comment here on HN. And frankly, I'm pretty sure I was way better off, and maybe the people here were too. Or a connected but different example, there were HN comments I took to heart pretty bad. Relitigating them with ChatGPT enabled me to get over them in-depth, and even consider different viewpoints properly, without having to be the resident punching bag of some asshole. It's difficult for me to see these as anything but a net improvement.

There is something to be said about interactions in one's life becoming overly structured. I think a reasonable case could be made that in less structured lives, where things just kind of happen, you get more random interactions, and it is through those random interactions that you gain valuable experiences and relationships, even if there's a lot of cruft in addition [0]. That in our effort at optimization, we're making our lives inhuman to live, for the same reason people are crapping on LLMs being statistical (unreliable, imperfect) rather than hardwired (reliable, perfect). But that implies there is an optimal life, which I'm not sure exists: specifically because of this local vs global optimality tension. In complex stateful systems, you're guaranteed to hit scenarios where locally correct and globally correct behaviors are irreconcilable [1], unless you meticulously construct them such that this cannot happen, after all.

[0] like https://en.wikipedia.org/wiki/Small-world_network

[1] like https://en.wikipedia.org/wiki/Tragedy_of_the_commons

[mlsu]

I like this reply. I think you and I use AI the same way and probably have a lot in common. But I read things more cynically than you do. Things to ponder:

Read the GPT Live announcement. Why do they want to make it more life-like, if not to encourage the pathological engagement? Why does the model say "thank you for that insightful question"?

It is not "just a tool", that is not the goal. It is designed to be a replacement for people. Even in the most charitable "LLMs are just a tool" reading. The value proposition is to have fewer people (dramatically fewer: the "one person unicorn" with fleets of AI agents!) and for those people to spend more time talking to bots. Replace your junior employees with bots. Replace your customer service people with bots. Replace a bored teenager behind the Wendy's counter with a servile golem whose smooth plastic face can easily be wiped down when spit on.

Their metric is engagement, it is how engaged are you and how long do you spend using the application. They will do absolutely anything to get that number up. Remember, after the GPT-4O debacle, when Altman talked about implementing sex bots? That's a shark, smelling blood. Something like 30% of under 18's have used AI chatbots for "companionship," a big subset of those for ERP.

> Would be just weird.

I was a weird kid who spent 10-12 hours a day on the computer, back when computers were primarily low-res screens and boring text. Now everyone is spending 10-12 hours a day on the computer. These things change in a single generation and I'm sure you know as well as I do that boy does it happen fast.

You sound like me: an older, healthy, well-adjusted person who can understand what's going on when the model says "that's a really insightful question!" Most people are not. They will not understand when it's an AI video clone of Chris Helmsworth.

> there were HN comments I took to heart pretty bad

I bet it made you feel alive.

[bossyTeacher]

> Literally the only important thing in life, the basis of all value, the formation of self-identity, comes from communication with other human beings.

Is this sarcasm?

[throwuxiytayq]

How would talking to an AI as if it were not human sound? You can probably set your system prompt to insert “beep boop” between sentences and make it refer to itself as “Cybertron9000 Personal Computing Device” if that’s what you like. Is that an improvement? Or are you against voice computer interfaces altogether?

[jonstaab]

I don't know the answer. But a robotic voice is probably not a bad idea — just having a reminder that the thing you're talking to is not actually anything like you. If you want to go full send, you could have the LLM generate a clickable interface on demand so you could interact with it as a machine. Voice/computer interfaces are obviously useful, especially for disabled folks. But the ones that existed in the past didn't pretend to laugh at your non jokes or imitate vocal fry.

[overgard]

I think you could make it impersonal sounding. Like, right now ChatGPT throws in a lot of cheeky things. They don't really bother me because I don't anthropomorphize it, but, a lot of otherwise smart people are struggling with anthropomorphizing these things right now (Richard Dawkins..) so it strikes me that the less personality they have the better. (If you want personality just throw it into the prompt, ya know?) To their credit I think this is customizable in ChatGPT right now (it's been a while since I looked), but "impersonal" should probably be the default until society adjusts to these things.

[dividedbyzero]

The Star Trek computer doesn't feel uncannily human, that would be a good starting point.

[famouswaffles]

The Star Trek computer would suck at a lot of what people use GPT for.

[SmirkingRevenge]

My preference would be to turn down the fake emotional expressions and notes in the responses. No cheeky quips, etc.

Otherwise it's kind of like being manipulated by a psychopath

[throwuxiytayq]

I would also like to tone those down, and prefer if they just focused on creating a nice, clear, natural-sounding voice. I don’t think the “emotional roleplaying” adds anything to what I perceive as a computer interaction. But that’s precisely the opposite of the problem the other commenters seem to be voicing; I have no problem recognizing what I’m interacting with, and I don’t worry about being manipulated - not by the tone of its voice, and even if it had a video avatar, all that. Do you guys seriously have this problem of being confused by LLMs that sound too much like humans, or is this a theoretical problem that you’re worried somebody else is having?

I am open to the possibility that this is a future that is coming, but as far as I’m concerned, we’re years away from that tech. Is it actually here for you?

[SmirkingRevenge]

Maybe it's just that conversing with something or someone that is acting manipulative can be unpleasant in and of itself, even if the manipulation is not successful.

Each time I hear the bot emote, it's like a subtle suggestion to just play along and anthropomorphize it - it feels weird, silly and annoying

[seizethecheese]

Isn’t voice I difference in degree rather than in kind? I definitely talk to AI as if it were human (one might say the UX of AI is to emulate a human). And a large portion of my interaction with humans is via text, for example, this post!

[1659447091]

> Isn’t voice I difference in degree rather than in kind?

There is a difference in expression / emotionality with speaking vs writing. Speaking tends to carry more emotion while writing is generally more deliberate/less-emotional*. Having a voice conversation will be more likely to get a human to engage in an emotional based expression mode, which could increase the chance of "false connections", believing the AI "gets" them or "understands" or "listens". This happens with text too, as some headlines show.

The issue is that while some people are going to "connect" with their AI in text and voice, some who do not make the connection via text may do so via voice because it tends to change a persons expression mode.

> I definitely talk to AI as if it were human

Do you talk to it as if it were a friend or family? or Do you just use natural language to give directives? The distinction, I believe, is in the kind of way we express the "talking".

I talk to AI as if it were a tool that understands human commands and then executes those commands and relays them in a human understandable format. This includes commands to provide options that I may not have covered and explain the options. If I talked to a human this way, they wouldn't be around much longer -- unless they were an employee and even then they would probably be looking for a new job

After reading some of the psychotic break headlines from AI chats, I see some people really do talk to AI as if it were human. Which I would guess includes seeking broad "thoughts and feelings" on a persons situation or asking the AI if their view/side of things is the "right or wrong" side. Basically begging the AI to be responsible for their own thoughts, or simply offloading them and taking what comes back -- which is going to be what they wanted to hear because the entire context would be full of emotion based prompts.

*I forget which books Ive read about this in. It's not an obscure concept, quick search brought this up: https://kellercenter.hankamer.baylor.edu/news/story/2023/spe...

[jonstaab]

Right, I think the challenge is that LLMs are essentially language-based, which is itself a very convenient interface. Covering the maximally humanistic default interface with something more mechanical is like tying your own shoelaces together, but it would protect us from the psychological hijacking we're so prone to when interacting with these machines.

[nullbio]

I disagree. Fluid natural conversational AI is far more productive than any other interface for working with LLMs. Although I suppose you could make the argument that it should be more... Robotic like. Like in StarTrek. Which, is honestly probably better for work, too. A "get shit done" mode, of pure, cold, efficiency.

[SmirkingRevenge]

In one of the videos the AI quips back things like "Happens to the best of us" - its basically pretending to be human and that feels kind of creepy and weird, like it's trying to cultivate a para-social emotional bond.

I just want voice assistants to reliably understand what I say and do what I mean

So hopefully you can turn that off.

There are plenty of applications for that more human conversational style though (from mock interviews, improv practice, learning languages, etc) - I just don't want that for most things.

[daveguy]

I doubt you can turn that off. That's how they upgrade it up from a tool to an engagement optimized addiction platform.

[derangedHorse]

Ironically, Data (from TNG) has a lot of heart to him

[nullbio]

Data is the goal!

[LostMyLogin]

Every new release I'm convinced we are just realizing the Knowledge Navigator.

https://www.youtube.com/watch?v=umJsITGzXd0

[hoppp]

But a lot of humans can't engage in meaningful technical conversations... So why not use a tool for that?

[FridgeSeal]

Somewhat of a self-fulfilling prophecy no?

- nobody else with the skills - talk to machine - nobody develops the skills to have a conversation - go to step 1. - bonus step: your own skills atrophy too.

[jdthedisciple]

Devil's advocate: The widespread lack of social skills was not caused by LLMs in the first place. In fact, AI ChatBots, if used right, can actually help remedy lacking socialising skills.

[FridgeSeal]

Not if there’s good money to be made exploiting people for money instead!

There’s plenty of evidence that these models worsen these issues, and we know from prior experience that companies like FB will happily exploit vulnerable users for money.

[carloseduardopx]

Again, we're paying for the fact that Sam Altman didn't understand the movie her

[jrflo]

I see it more as a way technology could be abused rather than an inherent flaw in the technology itself. If you start to replace human interaction with chatbot interaction, that's bad, but there's nothing wrong with using a human-like chatbot in moderation. So many other types of technology are fine in moderation but can be abused in a human-interaction-replacing way: television, social media, video games, etc.

[jonstaab]

Yes, but design nudges use in one direction or another. Or, as McLuhan said, "the medium is the message."

[jrflo]

I think the same applies with the examples I listed.

[jonstaab]

The thing is, individual moderation isn't sufficient to combat collective capture. This is Ivan Illich's idea of "radical monopoly", where a technology (like cars or the internet) becomes so entrenched that individuals can't realistically opt out any longer. This happens when we abdicate our collective responsibility to the internal logic and incentives of the tech.

[2001zhaozhao]

Just give it a scifi robot voice and pretend you're on a spaceship

[svantana]

I'm sorry, Dave. I'm afraid I can't do that.

[jonstaab]

This is actually not a bad option

[poly2it]

Where do you draw the line on how good human-machine interfaces should be? I'm sure this model could be a convenience for many, and while it may be social for some, I am not sure it would substitute existing human interaction for those users.

Besides, I do not think there is anything inherently immoral with not being social, or not having the ability to be. Consider for example people who do not naturally have the social network to interact with people they want to (e.g. some gifted children).

I am not convinced this model has enough empathy to satisfy most users on an emotional level. A bond is not merely an exchange of words, but prolonged and deep contemplation of the other being. We cannot introspect into these machines, and they certainly cannot yet do the same to us.

[quotemstr]

> This is the opposite direction AI should be going.

There is no moral obligation, in any domain, to refuse to make a product that adults, with full informed consent, find useful and purchase. Who are you to say you know better than the market?

[perching_aix]

Market conditions are not a moral standard either, nor do they represent any cohesive one in particular. Not sure why you're contrasting their opinion with this, it's literally no better.

[rel_ic]

I disagree. I think you absolutely have a moral obligation to consider the impacts of your product.

> Who are you to say you know better than the market?

You really don't think the scientists & engineers making these tools know some things better than the market?

[gf000]

Markets only work in well-regulated environments, this has been known since forever (Adam Smith).

There is no meaningful competition in a marathon if I can drive you over with a car at the first 100m.

And thanks fkin God that we have regulations and meaningful laws and some asshole for-profit company can't just put drugs into food.. the "informed buyer" is bullshit. Humans are faulty, and there is a billion dollar industry meant to take advantage of said faults: it's called marketing.

[FridgeSeal]

This is a cute argument to make, which patently doesn’t work in the face of decades of evidence on the pernicious effects of social media _alone_, let alone the “turbo infantilising sycophancy machine”.

Pretending that everything’s totally fine and ok because “it’s adults” really just disregards reality.

[quotemstr]

I don't accept that social media is self-evidently bad. Not everyone lives inside the bubble in which its wrongness is obvious.

[asdewqqwer]

SOME human relationships are the most valuable thing we have. Many (if not most) human relationships in many cultures are just trespassing everyone's boundary via peer pressure.

Disclaimer: I am from Scandinavian.

[standardUser]

I don't think most adults can get much social value or satisfaction from an AI conversation they know is not a real person. Those who can get that satisfaction likely have so few human-to-human interactions that an AI companion may be as good a solution to chronic loneliness as any.

I'd be more worried about the inevitable robo-nannies who could end up talking more to young children than actual people.

[akomtu]

From one perspective, AI serves as a wedge that separates those who want human culture from those who want machine culture.

[bodash]

I’m completely on your side, but also I think about all the AI saturations in the society like this:

“Driving to work or running a 10km on the weekend sharing the same surface goal: from A to B.

You may have less autonomy whether you should drive to work, but you definitely have more decision power about whether to go for that run or not”

[jstummbillig]

> This is the opposite direction AI should be going

I don't understand the counterfactual. What's the opposite direction of this direction that's desirable? Less capable voice models are obviously not it, so I am curious what direction you mean or if it's just vague indirection.

[imhoguy]

Hopefuly I see conscious young people in my surroundings who refrain from using human like AI (voice). They say "it is cringe".

Moreover I think aggresive robocalls and advertising will derail the entire trend in a few years, people will become tired of fake talk.

[mrcwinn]

Regardless how natural AI becomes I’ve yet to replace any human relationships. Where do you see the aim of technology as trying to replace people? You might be using it wrong.

[khurs]

For people with disabilities who can't type or see, ai conversations I imagine may have been life changing way to access the internet and knowledge

[Sol-]

Aren't they already very cognizant of handwringing like yours? Their article mentions various safeguards and actively steering the model away from being emotional companions and so on. It's a far cry from the OpenAI two years ago or whenever it was when they were entertaining the idea of allowing/enabling adult conversations with their models.

I personally think this is a moralistic regulatory overreach. And they definitely do that due to political pressure too, since there are various bills around the world in various legislatures that want to regulate AIs giving useful advice and being too personal to talk to.

So you can rest assured, I think, at least in that regard. The AI disempowerment will come to us anyway, just in a more sanitized corporate form.

[Unearned5161]

I want to bring some attention to a spectacular sleight of hand that occurs at minute 4 of this podcast:

"The way that I formulate the argument, just to get us started here, is that acts, the things we do, have ends. We act for an end. They have a purpose. And that it is wrong to deliberately frustrate an act from attaining its end, from attaining its purpose."

I will draw your attention to the three instances of "end", first we have the noble, tautological, "have ends", actions have ends; then, following close behind we see "an end", we act for an end, those ends have been made into one now, singular end; and finally, "its end", the act, as an entity unto itself, has an end you mustn't interrupt.