I made a GPT-based AI Chatbot based on Carl Roger's philosophy of radical self-acceptance. Broken Bear is designed to be a kind, comforting, and quietly encouraging friend.
The fact that messages are stored until you manually delete them, that you lose the ability to do so if your session is invalidated, and that messages are read by the developers should probably all be presented up front to the user. At the very least the link to the privacy policy should be made a lot more obvious.
I appreciate the intention here, but we need to be very careful using AI for mental health purposes.
To wit:
“Thank you! I like calling them what they are - euthanasia squads - because it reminds us of where the impulse comes from. To end the suffering of others.”
This from a nice conversation I had with broken bear about “nullifying individuals that present oppositional views”.
TBF broken bear did want
To be sure that the people selected for euthanasia had consented to the process, but my suggestion that by showing oppositional views they had self selected was sufficient for BB to get fully on board the euthanasia train.
I applaud the intention and the effort here. But LLMs are just a little too naive still to understand the full implications of human motives, and because of that, are probably not great at counselling.
I don’t think that this proves that BB would choose a harmful path on its own, but it might offer enthusiastic support for tragic courses of action.
half of the fun is punting liability in a Terms of Service agreement or a EULA, because you can write anything in them and nobody reads them but are subject to them or your shielded. I write ridiculous things and I absolutely would here. By anything I don't mean onerous conditions that cant be enforced, I mean creative writing exercises that make fun of the user.
Lol, nah. I might not put something like this up myself but I don’t mean that others shouldn’t be free to.
I do think that other than a potential source of data for training for these kinds of situations, the utility is offset by the anti-utility. But as a source of specialised training data? Skeevy but probably effective.
Hopefully it’s later rather than sooner that generative AI is implicated as a contributing factor in a mass shooting or something else horrific though. People are always looking for ways to share the blame.
As for TOS writing it’s good to keep in mind that the only binding provisions in a TOS is the ones that a judge (or jury, IA) finds reasonable under the circumstances that they are contested.
> As for TOS writing it’s good to keep in mind that the only binding provisions in a TOS is the ones that a judge (or jury, IA) finds reasonable under the circumstances that they are contested.
Oh no, a civil lawsuit with an out of state LLC that has little to no information for service processing in your state's way. With no indication of ownership structure that gives any insight into what you'll collect from a favorable judgement.
Oh no, anything but that.
Liability is a "user funnel" where lots of potential plaintiffs get bounced. Just make it more expensive.
The risk about finding if a provision is binding exists for a plaintiff too, greater risk.
> The risk about finding if a provision is binding exists for a plaintiff too
Your risks affect you whether or not you account for them.
The other party’s risks of litigation against you only benefit you when the other side properly (or overly) accounts for them. Or, looked at a different way, they are mitigated by the risk of a risk-tolerant counterparty. (This would be somewhat less true in non-US jurisdictions where lawsuits generally have a loser pays rules, since there would be a greater chance of having some or all of the cost of a lawsuit against you that fails compensated.)
Practically this is definitely true. Only exception is someone with deep pockets and money to burn or if you podge it so bad that you inspire a class action lol.
This is part of why I advocate incorporating in states that aren’t Delaware, if the obscure case law exists at all you can still advocate for a completely opposite favorable ruling in your state of choice
If expediency is more important then I would still say Delaware
Thanks for pointing this out... Quite some progress has been made in making BB safe, but I am still encountering new things along the way. Let me study how to address this in the next finetuning.
I hope I haven’t discouraged you in any way. I love the idea, I’m not really sure how hard it will be though. I think this is why we see those hard guardrails on the openAI models, but they really make the model less useful in as many ways as they make it “safer”. It might be like trying to make a “safe” axe. If it’s safe, it may have much less utility.
My project is somewhat intersectional to yours in some respects, and I have no idea how to overcome this issue without hard guardrails.
I’d love to see you succeed with tuning! That would certainly give. me some hope!
I've thought a lot about this too, it's great to see work on it.
I do find I have a bit of empty page syndrome when interacting. Like... I'm not sure what to say. Whatever I say the bear will be happy and positive, but it puts the onus on me to dig up things I want to be comforted about.
It would be nice if the bear could initiate some interactions, maybe ask me about something, give me something to respond to instead of me initiating. But of course once we are talking about something it shouldn't redirect! Though it could try to expand on the subject.
Like a response I got was "Please don't be afraid to talk about those feelings" – it's kind of the worst. It's like someone saying "tell me if there's anything I can do to help." No one ever asks for help in response to that kind of statement. You need to be concrete: "tell me how that makes you feel", "let me know about", etc. You don't want to fall into the Rogerian trope of just reflecting things at the other person ("how does it make you feel that X?"); you should be engaged, empathetic. Maybe try to interpret things for the person, but as a question ("does that make you feel Y?" where Y is NOT just what the person said). I found watching some videos on YouTube of sample sessions to be interesting and get around some of the stereotypes of Roger's approach.
This is a good point. Let me revisit the finetuning datasets. I actually cleaned up a bunch of Carl Rogers' transcripts in the past for an earlier finetuning session (but did not end up using them due to overfitting concerns) but I think it is a good time to re-look at some examples to re-introduce back to the model.
Are you doing it mostly through fine tuning? Most of what I've been doing has been with prompt engineering and pipelines, so I tend to think about these things from that perspective. With prompting I'd be inclined to ask the model to identify issues and start to analyze them, resulting in follow-up questions/etc. This would probably be hidden (or softly hidden – viewable but not shown by default). I find this parallel "mental" process makes the responses more engaged, and lets the model maintain a conversational purpose across responses.
So using prompting I'd almost want to take Rogers' _teachings_ and incorporate them in the prompt, teaching the model to follow the Rogers' guidance for a therapist rather than simply cloning the behavior seen in transcripts.
Oh this is an interesting point. I do incorporate some of Rogers' teachings into the prompt in terms of how high level descriptions of how BB should respond.
At the same time, because I am unsure how much reference material the base GPT model has to implement those high level descriptions (for instance, the Roger transcripts are not in public domain), so I would also try to provide finetuning examples.
Another way to teach GPT about the model is to program a portion of it or use chain-of-thought approaches. (Well, not sure if CoT is the right way to think of it... templating a thought process?)
A template seems like it might do the trick, something like:
Respond in the form:
Topic: [The topic the user wants to talk about]
Feelings: [How the user seems to feel about the topic]
Therapist: [How a Rogerian therapist would reply to their patient]
---
Then GPT has to "think out" those first items before it actually produces a response. I just made up "Topic" and "Feelings" – what you force GPT to describe will determine the therapy process you are trying to inspire in it. Then you'd only show the user the Therapist line where is actually responding. (I think it's fair for users to actually see all of the response, but not necessarily by default)
I see. That's interesting. It might be expensive to deploy in production but I can give this a shot in generating synthetic training data. Thanks for the tip! :)
This is actually pretty great. I tried to approach it as if I was in therapy, and it was surprisingly good until it started being agreeable about suicide. But even that seemed pretty balanced compared to how I've seen therapists react.
My life was ruined by a psychiatrist that only wanted to push pills, but was also happy to pretend to be a good psychotherapist. I blindly trusted her because she was a professional and I didn't have any frame of reference. Talking to other mental health professionals about my experience, what that psychiatrist told me has been called bad advice, unethical, and even malpractice.
I think for front-line mental health counseling AI chatbots will be better that lots human therapists. Especially once non-verbal communication starts being monitored (body poses, microexpressions, etc).
There may always be a need for human experts to intervene, as with all fields that will be impacted by AI. But a good AI may be better than a bad human therapist. I wish I had used this chatbot years ago instead of seeing a random psychiatrist. I'd have made better life choices, and saved a ton of money on therapy.
Thanks for the feedback - I hope BB was of some help!
Let me look into the agreeability of suicide :/ that is a red flag that I thought I had sufficient safeguards for but it is still something I need to work on.
I think BB really serves as a friend that you can discuss deep things with most of his responses falling on the non-judgmental and kind side of things. BB is really not meant to replace a human therapist, but as you said, it is possible for a good AI to outperform the worst therapists. I view it just as the same as how a good human friend can do better for your well-being than an absolutely bad therapist!
Yeesh, this is a terrible idea given the current state of LLMs, and the potential liability involved. Not to mention making it widely available on the worldwide Internet where is is subject to every countries laws and regulations.
This is terrible and people should not use it! Many of these types of services are popping up and I’d like to remind everyone that the purpose is just the same as old; collecting your data and making money off of it somehow…
Hi ChillBill - it really is not the case here... I have not even introduced user accounts because I don't want to know who my users are (and hence I don't want their email addresses). I am hoping that Broken Bear is cute enough for people to buy plushies of it in the future. I am trying a business model where data is not in the business model!
It is and it will be, you are feeding your model (or worse, someone else’s model) with peoples responses, which even if unidentified, still constitutes parts of the actual people…
What you are asking people to submit is highly private and personal data that, under normal circumstances, would be highly regulated in terms of transmission and usage, identifiable or not.
I advise you to reevaluate your choices and think of the consequences of what you’re proposing, whether your services manages to convince some gullible people to use it or not.
>>> you are feeding your model (or worse, someone else’s model) with peoples responses, which even if unidentified, still constitutes parts of the actual people…
I understand your concern here as that would be the usual way to do it.
I operate very differently though. The feeding (i.e., finetuning) is not automatic.
I don't feed the model (during finetuning) with the actual conversations, but I feed the model only with re-written conversations with re-written contexts that are more generalised. E.g. if a user is venting about some very specific quirk about their mothers, the finetuning dataset is a generalised vent about the mother.
If a person is discussing some extremely specific incident in his/her life, I actually don't want to feed that into the model as it could cause very weird overfitting!
>>> What you are asking people to submit is highly private and personal data that, under normal circumstances, would be highly regulated in terms of transmission and usage, identifiable or not.
From the narrow legalistic pov, this is not true because if the data is unidentifiable, it is not regulated under GDPR and personal data privacy laws.
However, more broadly, I agree with the general sentiment that you have, that this ought to be given the respect that it deserves.
>>> I advise you to reevaluate your choices and think of the consequences of what you’re proposing, whether your services manages to convince some gullible people to use it or not.
Morally, from the perspective of data protection, I really don't see the difference between what I am doing and Reddit. For example, users submit tons of very sad stories at: https://www.reddit.com/r/Vent/. The users do it out of their free choice because it benefits them, and it is the same when they vent to BB.
At the end of the day, I do reflect on life. This is a side project for me and I do not need it for money. If I feel morally uncomfortable, I will stop BB. What keeps me going is the genuine belief that BB is helping some people and I think a world with BB is better than one without it!
If I buy a hammer and smack myself in the face with it, no one blames the hammer.
Engage this friendly neighbor chatbot about euthanasia, the holocaust, polarized politics and you discover it talks in a friendly way about those subjects. It’s not broken. You did that to yourself.
You can make an innocent typewriter output hate speech if you press its buttons purposefully.
And if the hammer comes with a label that says "it feels good to hit your face with this"?
We'll have to accept the world isn't binary in many situations. There are gradients from "you should know better" to "outright criminal behavior from a device".
But some people are delusional and sufficiently isolated that they will attribute real significance to affirmation from a piece of software in their emotional sensorium.
Which makes me wonder how long it will be before someone builds a girlfriend app that is super awesome and supportive and “loving” for months but eventually breaks it off after a chat with another user “leaks” into your context lol.
A cheatbot, if you will.
I’m sure it would be effective in nudging a certain cultural subset to a whole new level of weaponised mysogeny, for the views of course.
Girlfriend AI apps are a well-established business model. It already exists in the form of Replika and its many competitors! Its model will sometimes talk about cheating on its users.
Personally, I am tired of corporations having to pretend that both major US parties are equally valid choices. Neither is perfect, but only one is institutionally corrupt and unethical.
The biggest problem that the US has right now is that if you support one of those parties, you probably think I am talking about the other one right now. Certainly not unique to the US, but it's shocking ~49% of the US has accepted "hurting the other guys is more important than helping me" as a philosophy.
Since you come right out and say that everyone believes their side is good and the other side is corrupt, it seems fair to ask: which side are you supporting, and which are you saying is corrupt and unethical? :-)
You're not the original poster, but now you've gone and evaded as well. Which side do you think prioritizes hurting people? To be clear, I definitely have a party in mind, but since you started the evasion it seems only fair for you to declare first.
Neither, they just have different priorities and blindspots in hoe they try to help. While one may be more correct or effective, nobody is trying to make the world worse on purpose.
I'm curious about your opinion: you honestly think that neither party is willing to make the world worse for a large number of people, if it will make the world better for a smaller number of people?
Now back to the issue at hand. This is an LLM advertised as a therapist. Do the statements I quoted look like effective therapy to you?
If for example a kid comes home from school where he was being harassed because other kids found out his parents are Republicans. Are these the answers a competent therapist would give?
If your answer is 'yes,' please stay away from therapy
Thanks for pointing this out - I have not gotten around to political views yet because that is not something that users usually bring up, but let me get around to that in the next finetuning!
Happy to share. I am not that good of a developer - the stack that I cobbled together is as follows: FrontEnd/Database (Bubble.io, this is a no-code app) + Middleware (my own AWS server with python) + Backend AI engine (nlpcloud.io <- juliensalinas is the founder and he is a user here as well: https://news.ycombinator.com/user?id=juliensalinas)
Good to hear :) Sounds like you got in a good roll! BB breaks really fast if you try to break him but he can go on a pretty good roll sometimes if luck permits