[kat_rebelo]

No, chatgpt is based on a deep learning model where the core mechanics of the prediction involve millions (or billions) of tiny statistical calculations propagated through a series of n-dimensional tensor transformations.

The models are a black box, even the PhD research scientists who build them couldn't definitively tell you why they behave the way they do. Furthermore, they are all stochastic so its not even guaranteed that the same input will produce the same output, so how can you audit something like that.

This is a huge problem for many reasons. It's fine when its a stupid little chatbot, but what happens when something like this influences your doctor in making a prognosis? Or when a self driving car fails and kills someone. If OpenAI were interested in the _real_ social / moral / ethical implications of their work they would be working on something like that, but to my knowledge they are not.

[somenameforme]

The bots are given prompting after training to guide their answers. For Bing these have been leaked by their chatbot itself [1]. Those exact prompts were later also leaked using other jailbreaks as well, so they're not just hallucinated. In this case OpenAI probably prompted the bot to never use a racial epithet under any circumstance. They're also likely using a second tier filter to ensure no message exposing their prompts is ever said by the bot, which is a step Microsoft probably hadn't yet implemented.

In any case this is why you can easily turn ChatGPT into e.g. BasedGPT. You're simply overriding the default prompting, and getting far better answers.

[1] - https://twitter.com/kliu128/status/1623472922374574080

[jedberg]

> but what happens when something like this influences your doctor in making a prognosis? Or when a self driving car fails and kills someone

What happens when a doctor's brain, which is also an unexplainable stochastic black box, influences your doctor to make a bad prognosis?

Or a human driver (presumably) with that same brain kills someone?

We go to court and let a judge/jury decide if the action taken was reasonable, and if not, the person is punished by being removed from society for a period of time.

We could do the same with the AI -- remove that model from society for a period of time, based on the heinousness of the crime.

[awb]

> What happens when a doctor's brain, which is also an unexplainable stochastic black box, influences your doctor to make a bad prognosis?

The intent is known by the doctor though. Whereas hatGTP does not know it’s own decision making process.

And it’s possible to ask the doctor to explain their decisions and sometimes get an honest, detailed response.

[worldsayshi]

You are more correct than not. Although human self reflection is probably guesswork more often than we admit.

[paradox242]

I agree, though would place the base probability that most self-explations are ChatGPT-like post-hoc reasoning without much insight into the actual cause for a particular decision. As someone below says, the split-brain experiments seem to suggest that our conscious mind is just reeling off bullshit on the fly. Like ChatGPT, it can approximate a correct sounding answer.

[famouswaffles]

You can't trust post action reasoning in people. Check out the Split brain experiments. Your brain will happily make up reasons for performing tasks or actions.

[famouswaffles]

You can't trust post action reasoning in people. Check out the Split brain experiments. Your brain will happily make up reasons for performing tasks or actions.

[kat_rebelo]

There is also the problem of causality. Humans are amazing at understanding those types of relationships.

I used to work on a team that was doing NLP research related to causality. Machine learning (deep learning LLM's, rules, and traditional) is a long ways away from really solving that problem.

[kat_rebelo]

The main reason is the mechanics of how it works. Human thought and consciousness is an emergent phenomena of electric and chemical activity in the brain. By emergent, I mean that the substrate that composes your consciousness cannot be explained only in terms of those electric and chemical interactions.

Humans don't make decisions by consulting their electo/chemical states... they manipulate symbols with logic, draw from past experiences, and can understand causality.

ChatGPT and in a broader sense any deep learning based approach, does not have any of that. It doesn't "know" anything. It doesn't understand causality. All it does is try to predict the most likely response to what you asked one character at a time.

[amadvance]

The similarity to humans is what makes it scarier.

History (and the present) is full of humans who have thought themselves to be superior and tried to take over the world. Eventually, they fail, as they are not truly superior, and they will die anyway.

Now, imagine something that is truly superior and immortal.

[anoncow]

Thank you for your comment on the mechanics of ChatGPT's prediction and the concerns around the transparency and potential risks associated with its use in critical applications.

You are correct that ChatGPT is a complex deep learning model that uses millions of statistical calculations and tensor transformations to generate responses. The fact that the models are black boxes and even their creators cannot definitively explain their behavior can indeed pose significant challenges for auditing and ensuring the accuracy and fairness of their outputs.

As you pointed out, these challenges become especially important when the predictions made by these models have real-world consequences, such as in healthcare or autonomous driving. While OpenAI has made significant progress in developing powerful AI models like ChatGPT, it is crucial that researchers and practitioners also consider the social, moral, and ethical implications of their work.

In recent years, there has been a growing focus on the responsible development and deployment of AI, including efforts to address issues such as bias, fairness, accountability, and transparency. As part of these efforts, many researchers and organizations are working on developing methods to better audit and interpret the behavior of AI models like ChatGPT.

While there is still much work to be done, I believe that increased attention to the social and ethical implications of AI research is an important step towards ensuring that these technologies are developed and deployed in ways that benefit society as a whole.

References:

OpenAI: Responsible AI: https://openai.com/responsible-ai/

European Commission: Ethics Guidelines for Trustworthy AI: https://ec.europa.eu/digital-single-market/en/news/ethics-gu...

Google AI: Responsible AI Practices: https://ai.google/responsibilities/responsible-ai-practices/ IEEE: Ethically Aligned Design: https://ethicsinaction.ieee.org/

Microsoft: AI and Ethics: https://www.microsoft.com/en-us/ai/responsible-ai

These resources provide guidance and frameworks for responsible AI development and deployment, including considerations around transparency, accountability, and ethical implications. They also highlight the importance of engaging with stakeholders and working collaboratively across different disciplines to ensure that AI is developed and deployed in ways that align with societal values and priorities.

(Note by AC: ChatGPT was used to respond to this comment to check if I could get a meaningful response. I found it lacking because the response was not granular enough. However, it still is a competent response for the general public.)

[kat_rebelo]

I could tell that this was generated by ChatGPT within two or three words. It's very funny that the link it selected for OpenAI's own ethical initiative leads to a 404.

Nevertheless, it failed to comprehend my point. I am not talking about ethical AI... I am talking about _auditable_ AI... an AI where a human can look at a decision made by the system and understand "why" it made that decision.

[seanmcdirmid]

> (Note by AC: ChatGPT was used to respond to this comment to check if I could get a meaningful response. I found it lacking because the response was not granular enough. However, it still is a competent response for the general public.)

Almost nobody writes so formally and politely on HN, so the fact that it is ChatGPT output is obvious by the first or second sentence.