[ale42]

It's hilarious to see how these models can hallucinate stuff that doesn't exist...

Q: What is the meaning of ABBA's song "I have a dream"?

A: "I Have a Dream" is a song by the Swedish pop group ABBA, released in 1970. The song was written by Benny Andersson and Björn Ulvaeus, and it was the group's first hit single. The song is a ballad that talks about a person's dream of finding love and happiness. The lyrics describe the dream as a "beautiful light in the dark" and a "feeling inside that no one can steal." The song has become a classic of the ABBA catalog and is still widely played and enjoyed today.

(Didn't check the details but seems okish). But let's not ask about a song that doesn't exist:

Q: And what did they mean with the song "General Trouble"?

A: "General Trouble" is a song by the Swedish pop group ABBA, released in 1969. The song was written by Benny Andersson and Björn Ulvaeus, and it was the group's first single. The song is a pop ballad that talks about a person who is experiencing a lot of problems in their life, including relationship troubles, financial troubles, and feelings of isolation. The lyrics describe the person as being in "general trouble" and feeling like they are "losing ground." The song was not a big hit for ABBA, but it is still an interesting early work from the group and provides insight into their early sound and songwriting style.

[ale42]

I also asked about a recipe with mushrooms, tea, and dishwasher soap. It happily gave me one... mentioning at some point "Add the dishwasher soap to the tea, if desired. The soap will help to release the flavor of the mushrooms and give the tea a slightly soapy taste."

[antman]

Presumably dishwasher soap is somewhat edible or it would not be on dishes? So it might actually be correct just weird, but it's what you ordered

[mx20]

HN had a not so good looking study about it.

"Gut epithelial barrier damage caused by dishwasher detergents and rinse aids "

https://news.ycombinator.com/item?id=33822149

[ale42]

Right now it tells me "Dishwasher soap is not intended for consumption and can be harmful if ingested." Not sure if I used a different prompt before, or if I was lucky.

[bongobingo1]

Well... it's not wrong...

[zhisbug]

The recipe indeed may not be a bad idea lol

[tim333]

GTP-4 seems to have improved:

Q1: A: ABBA's "I Have a Dream" is a song from their 1979 album, "Voulez-Vous." The song's lyrics express themes of hope, self-belief, and optimism...

Q2: A:ABBA does not have a song called "General Trouble" in their discography. It's possible that you may be referring to a different artist or misremembering the title of the song. If you could provide more information or the correct title, I would be happy to help you understand the meaning behind the song.

[bazzargh]

I Have a Dream wasn't 1970-Abba hadn't formed yet! - it was 79, I remember my sisters buying it. And it wasnt their first single, not even the first off that album. And it doesn't have those lyrics, which I don't recognise, I don't think they're even from Abba songs.

[ale42]

My bad, I should have noticed this!

[Terretta]

These are generative models where the LLM prompting works by examples it tries to match while generating new tokens into the provided pattern.

Chat implementations provide your recent back and forth history as example to the next prompt. This is how providing an example reply in JSON can get it to reply with new data using JSON.

Ongoing replies reference what it interprets as labels (the blanks to fill in) provided in the examples, replacing those tokens with new tokens.

Look at your first question and second question again. Pretend the first reply is in JSON but with placeholders for song title, artists, topic, lyrical analysis, and critique.

Pretend the second reply was supposed to use the first reply as its example, and replace only new data.

You see it put new generation into the provided example structure.

Now you can see what it did there. It did exactly what you prompted it to do.

[CapsAdmin]

I feel like hallucinations in the LLM sense is something we do a lot too. It's also where I feel people can come up with good ideas and different perspectives.

Many times in my life I've explained a solution to a problem to someone else, they say something like "oh that's genius! we can just do X Y Z!", but actually it's not what I meant and instead their solution is much better and often simpler than mine. Same thing has happened in reverse too of course.

One time I even hallucinated the meaning behind a song with my partner. We thought it was about the tragedy of the Hiroshima nuclear bomb, but actually it was about the first time snow falls in a city. I really had to double check many times when my friend told me the original meaning behind the song. Once I realized everything clicked as if I had watched a movie with a big reveal to the mystery at the end.

Maybe it's not useful in a LLM, but I think it can be a beautiful thing about humans.

[DeusExMachina]

I was doing some market research with ChatGPT some time ago, and I started asking about some of my competitors. The output matched what I saw people say in conversations online, so I got curious about what it knew about myself. I asked and the answer was completely false.

[alexvoda]

But did it match what people were saying about yourself?

[DeusExMachina]

No, it just invented some expertise I don't have and things I never did, like books I didn't publish.

[nathanasmith]

Sounds like you have some work to do!