[sagaro]

I have used chatgpt, sage, bing exclusively for gettibg code snippets and ideas on how to solve something complicated. They have significantly improved my productivity and enabled me to make many small tools to help me and my friends.

When the ai is wrong, I just move on.

I fear some small portion of people who want to prompt engineer and prove the AI is dumb or wrong or evil will end up winning and make all these companies scale down to avoid the backlash and in the end people like me will have to live without the good things.

[gladiatr72]

There was a blog post about the problem with almost-good self-driving systems. It takes care of 95% of anything the car might encounter.

The problem is that the leftover 5% are the hard cases. When a driver is completely unengaged 95% of the time, the liklihood of (human) failure is much greater than if they were piloting the vehicle full-time.

You glide by with _when the LM(Ai) is wrong, I just move on.._ as if you were magically born with some intuitive ability to detect wrongness.

I imagine you have _experience_, prior to LM, that gives that ability. Where will that experience come from?

[sagaro]

Like I said, I use it for coding. I don't possess any magical intuition. When it is wrong the program doesn't work and so I know it is wrong. If it gets an algorithm worng and the output is not what I expect, i don't go about trying to school it and make it blunder more and take screenshots to show off how it is dumb and I schooled it.

[noduerme]

Why is preserving the reputation of a broken system more important than pointing out its flaws?

Let's say you bought 30 Surface Duos for your company, and 10 of them had broken hinges out of the box. Should no one document that or complain, lest Microsoft pull back from making dual-screen tablets? Who would be helped by that?

On the other hand, some people would take one of the working machines and stress-test the hinge, to find out why it was breaking, and perhaps take pictures and warn others and the company about its weaknesses. Assuming that everyone already knows these things break easily, it's useful to know how they break and what happens next. Just like on GitHub issues, if users assume everyone else is running into the same problem and don't give detailed bug reports, it's impossible to know how widespread a problem is.

[gladiatr72]

It's not about the tool. It's about what you become by using it.

[noduerme]

I'm more concerned that only a small percentage of people will actually check the citations (or the code) before putting the AI's results into circulation. Sort of similar to how people retweet or forward misinformation, thus lending it a personal stamp of human approval. When it comes from someone you know, most people don't check every reference. The scale of the fact-checking problem on social media and in academic papers is already obvious, along with its societal ramifications, but the addition of machines that gleefully spew factual-sounding garbage with false citations just puts that into overdrive.

I'm not afraid of a few people calling out that the AI is wrong. It's much scarier to envision a world where no one even tries to debunk AI-generated false facts. Part of what was so maddening about this conversation with Bing was the idea that it was rewriting history. Without recourse to Archive.org, could I have even proven that it was wrong or that the page hadn't existed? Since it's the kind of a thing a human would be very unlikely to just make up, it sounds more plausible; but then false assertions will be built upon other false assertions, until historical fact is buried under a mountain of hallucinated documents.