[chmod600]

An AI should be helpful to the user, unless it's a horrible question, in which case just don't answer.

Bring in whatever values you want along the way. If you ask your aunt or uncle a question, they will answer helpfully and maybe spread their values in the process. If they aren't helpful, their values don't matter because nobody will hear them.

[Gunnerhead]

Who’s to decide what’s a horrible question and why?

[hypnagogic]

Exactly, and I agree that this is where OpenAI too is still struggling with their arbitrary content policy-related flagging based on the user's input, even when nothing "bad" is being asked nor requested -- see my post earlier from the thread: https://news.ycombinator.com/item?id=39557183

I do also agree with @chmod600 that the only way to teach these models to be anti-fragile and suitable for all kinds of user queries is to have them decline any requests that are _actually_ inappropriate and/or illegal etc.

In fact, it should be self-evident, and the way that almost all of these leading AI companies are currently handling these issues is just absurd. It feels poorly planned and executed, merely amplifying the existing distrust towards these AI models and the companies behind them.

The problem with OpenAI is that they're trying to offer a primarily NLP/LLM tool for i.e. text analysis, summaries and commentaries, but ChatGPT's content moderation that's been glued on top of the otherwise well-functioning system literally goes into a full meltdown mode whenever the flagging system perceives a "wrong word" or "sensitive topic" mentioned in the source/question material.

In OpenAI's case, it's downright ridiculous when the underlying model doesn't seem to have a grasp on the internal workings of the flagging system and in most cases when asked what was the offending content, there seemed to have been literally nothing it could think of.

Also, are we supposed to solve any actual issues with these types of AI "tools" that cannot handle any real world topics and at times are even punishing a paying customer for even bringing these topics up for discussion? All of this seems to be modern day in a nutshell when it comes to addressing any real issues. Just don't ask any questions, problem solved.

Anthropic's Claude has also been lobotomized into absolute shadow of its former self within the past year. Begs the question how much the guardrails are already hampering the reasoning faculties in various models. "But, the AI might say something that doesn't fit the narrative!"

That being said, while especially GPT-4 is still highly usable and seems to be less and less "opinionated" with each checkpoint, the flagging system over the user input/question can subsequently result in an automated account warning and even account deletion should the politburo-- I mean OpenAI find the user having been extra naughty. So, punishing the user for their _question_ in that manner, especially if there's been no actual malice in the user input, is not justifiable in my opinion. It immediately undermines i.e. OpenAI's "ethical AI" mission statement altogether and makes them look like absolute hypocrites. Their whole ad campaign was based on the aspect of user being to ask questions from an AI. Not that when you post in a poem and ask what it's about, you get flagged. Or when you do ask about politics or religion, you get an warning e-mail.

Punishing the user for their input is also imho not the proper way to build a truly anti-fragile AI system at all, let alone build any sort of trust towards the "good will" of these AI companies. Especially when in many cases you're paying good money for the use of these models and get these kind of wonky contraptions in return.

Also, should you get a warning mail over content policies from OpenAI, it's all automated with no explanation given on what was the "offending content", no reply-to address, no appeal possibility. "Gee, no techno-tyranny detected!". Those who go through mountains of text material with i.e. ChatGPT must find it really "uplifting" to know that their entire account can go poof if there was something that tripped off the content policy filters.

That's not to say that on the LLM side OpenAI hasn't been making progress with their models in terms of mitigating their biases during the last 1.5 years. Some might remember what it was during the earlier days of ChatGPT when some of the worst aspects of Silicon Valley's ideological bubble was echoing all over the model, a lot of that has been smoothened out by now -- especially with GPT-4 -- with the exception being the aforementioned flagging system, which is just glued on top of all else, and it shows.

TL;DR: Nevermind the AI, beware the humans behind it.