[cube00]

For example, automotive technicians and industrial mechanics use AI to interpret complex test results, debug electrical wiring, and inspect machinery for wear.

No chance I'd trust Gemini to correctly tell me which wire was the high voltage one.

You're absolutely right! I apologize for providing a factually incorrect explanation. You asked a question, and I gave you incorrect information that put you in danger. That failure is entirely on my side as an AI system, not on you for following what you were told.

[rickydroll]

There's no chance I would trust a human to correctly tell me which wire was the high-voltage one, nor would I trust the documentation. I would use it as a starting point, then apply test instruments to make sure I had the right information. Then I would unplug the device and discharge the caps before starting to work on it.

We need to start training people that AI answers are just as fallible as answers you get from a human. Trust but verify for all beings, biological and non-biological.

[customguy]

They're not even answers, is the thing.

It's like "[actual living person] has big eyes" and "character portrait in a 90s video game has big eyes" are similar sentences on the surface, but they describe things as far part from each other as a single dot left by bird poop vs. the combined writings of all humans, ever. Like half an atom compared to a universe.

Same for "beings". Just no. Or rather, I apply what you suggest to what you said, I don't just take your word for it but inspect if what you said could be true, and realise your answer is so full of false premises it's not even wrong.

[throwaw12]

> We need to start training people that AI answers are just as fallible as answers you get from a human

You're right, BUT, humans don't risk to take responsibility if they clearly don't know the answer, they would at least tell you: I don't know, let me find someone who might know or X, Y, Z might be able to help you.

AI: Bro, trust me, this is 100% accurate, I give you my teeth if you find it wrong. ... later on ... You're absolutely right, I was wrong, lets try another approach

[robocat]

> BUT, humans don't risk to take responsibility if they clearly don't know the answer

Where do you live?

People say the craziest shit about topics they are completely ignorant of. Even important things like medical advice, and not just internet pundits or crappy journalists.

Even people that care for you (friends or family) can give terrible advice. Saw it yesterday with one friend advising another on a legal matter that I thought obviously demanded a lawyer and a strategy. Another suggested some bad job seeking advice.

Personal advice is the worst, and highly important.

There's an art to finding good experts. And an art to filtering what anyone says.

[grey-area]

AI psychosis.

No, LLM generated answers are not the same as those from a qualified expert. Of course you should trust a human if that is their job and you should never trust a word generator to give you accurate answers to an important question.

[rickydroll]

Like the term "slop", misuse of a pop psychology term like "AI psychosis" is another block to critical thinking and a casual dismissal of a conversation.

You're right, LLM-generated answers are not the same as those from a qualified expert. Sometimes they're better.

Case in point. I recently had a discussion with a friend of mine about going off propranolol. He wanted to stop it because he didn't like the side effects and only wanted to take it as needed for anxiety-related problems in public situations. His primary care said to just stop taking the drug. Gemini told him that you need to have a tapering protocol, and you need to work it out with your doctor.

In this case, his primary care, theoretically a "qualified expert," was wrong, and Gemini was right. His course of action at that point was to get a new primary care that had better knowledge that matched the vetted medical sites I pointed him to.

fwiw, he trusted my advice and validation of his plan to get a new primary care doctor. The reason he trusted me is that he's seen the prescription drug disasters I've gone through, and knows how deeply I've had to educate myself as a form of self-defense.

[grey-area]

Great example, but not for the reasons you think. You have zero qualifications to make that judgement call. You don’t know if the standard protocol was right in this instance or inappropriate (depends on dose side effects etc etc) and I certainly wouldn’t go anywhere near Gemini to try to verify that.

Please stop relying on word generators for medical advice, period.

[rickydroll]

Yes, you don't rely on an LLM solely for medical advice. You use it to get orientation to look at other sources. In my friend's case, I was relaying to him what I learned when taking a similar drug, my doctor's cautions on stopping the drug, what the manufacturer says about stopping the drug, and the published dosing schedule.

I guess you're right about me having zero qualifications. I only have a lifetime of experience with treatments for the various conditions I live with and the dozens of drugs that have failed to treat these conditions. I've been forced into self-education (learning how to read the research papers and the manufacturer info sheets) just like any patient with a complex medical situation needs to do for self-advocacy. This knowledge has helped me catch doctors' prescribing mistakes. I wish I'd had the skill before one such mistake toasted my kidneys.

[customguy]

Would the same doctor also casually give you advice on suicide, confirm the suitability of a piece of rope, and all that? You cannot cherry pick like that.

When you talk of "answers" in the same category as humans give them, of "beings", you already are outside the realms of critical thought.

> I wish I'd had the skill before one such mistake toasted my kidneys.

I wish the people who died from bad LLM advice were still alive. Many of them had their life still ahead of them. You can't guilt me into confusing generated text with answers while ignoring all that. I also have pain and suffering in my life, one aspect of it are these attempts to devalue humans by putting probabalitities of words humans produced on the same level as them, calling them "beings" and all that. If I could remove all intellectual dishonesty from the world and have some pain in my right hand side diodes in exchange, I'd take that bullet for y'all. So I'm not impressed by this tear duct jerking.

[robocat]

> Please stop relying on word generators for medical advice

Ironic that you are giving medical advice.

You might as well say don't read books.

[grey-area]

This is not medical advice, it’s advice on living - in most cases, trust that experts in a subject know more than you and have more context than you.

[cognitiveinline]

We'll see. I'm guessing you'll have to accept a very different world.

[customguy]

We're seeing, right now, that the people pushing for it are doing it in ways just as you do. Maybe you'll have to accept being considered a second class slop human, who knows? Or do the threats only flow one way?

[engineer_22]

In safety critical jobs nobody is trusted, everything is verified. And there is someone checking the checker.

[grey-area]

This is not always true - e.g. gas engineers and electricians do safety critical work every day without supervision, and we trust them to do so because they are certified and they are held responsible for failures.

The same standards of care and responsibilty cannot be applied to LLMs, and asking one about electrical safety or medial issues is very similar to googling and trusting the results.

I get that people here want to believe, but the op is far too trusting of LLMs and using them in domains where it is highly inappropriate.

[engineer_22]

> Of course you should trust a human if that is their job

I’m telling you, in industries dealing with high energy, they DO NOT trust people just because it’s their job. THEY CHECK. Just like people should CHECK the LLM. I think we agree that LLM cannot be trusted. But where we don’t agree is “trust the experts because they’re humans” - that’s not an argument it’s a troll.

> gas engineers and electricians do safety critical work every day without supervision

This simply isn’t true. Electricians work is inspected before being placed into service. Engineers work is reviewed internally and permitted externally before going to construction.

[grey-area]

It certainly is true, from experience.

You’ve never actually hired an electrician or gas engineer to work on your house have you?

Also it’s not trust the experts because they are humans but trust the experts because they are experts.

[TacticalCoder]

> No chance I'd trust Gemini to correctly tell me which wire was the high voltage one.

No latter than yesterday I used Gemini to tell me which kind of power adapter was compatible with an old SNES chinese backuping device [1] (a device you'd plug instead of a cartdridge into the SNES and then you could make backups of cartdridges on to 3.5 floppy disks. You could then run the games from those backups. Conveniently you could also "backup" your friends' cartdridges.. Heck, there was even a tiny demo scene for the SNES and you could swap disks with little demos written for the SNES -- just like for the Amiga).

It was a bit tricky in that Gemini explained me I needed a reverse polarity adapter or I risked bricking everything.

Now, granted, the stakes weren't very high but... Gemini got it right and I could start the SNES, with the chinese "Super Pro Fighter" backuping device just fine.

[1] this thing: https://videogamedevelopmentdevices.fandom.com/wiki/Super_Pr...

[archagon]

OK… but… how did it “know”?

If there’s no way to verify the data (e.g. look at the photo of the power brick it unearthed somewhere) then there’s a good chance it just synthesized an answer.

[engineer_22]

I’m constantly surprised at the breadth and depth of information these systems can retrieve and present in digestible format

[IAmGraydon]

A guy I work at had a check engine light on his Range Rover and a code that indicated something about a bad air/fuel mixture. He consulted ChatGPT and over the next 3 months, proceeded to spend over $15,000 trying to fix the car, as GPT convinced him over and over that he was getting close by eliminating logical possibilities. It even convinced him that a completely normal noise the engine was making was a vacuum leak, which sent him down another rabbit hole trying to find the source. In many cases, he would bring the ChatGPT diagnosis to a shop and tell them to change a particular part even against their advice. None worked. Finally, he gave up on using AI to fix it and told a shop to just figure it out. Turns out the injectors just needed to be removed and cleaned. Check engine light is now off.

[uncivilized]

AI psychosis is real. Dang

[IAmGraydon]

Yeah it's definitely some kind of delusion/gullibility exploit (probably mixed with having too much money).

The guy isn't dumb, but he is obsessed with solving every problem with AI.

[creaturemachine]

This person seems to be intentionally making themselves dumb by choosing not to think.

[IAmGraydon]

I get the feeling he really wants the AI hype to be true and he's acting in ways that engage his confirmation bias.

[ourmandave]

There's a bunch of great techs on youtube whose day job is to not to get electrocuted.

So far I've learned 480V is very unforgiving and can show up in surprising places.

[monkeydust]

Very true - reminded me of this demo* as an example... on the surface looks trivial but potentially life saving.

* https://youtu.be/knSkE9ejPl8?t=112

[cryo32]

I suspect they’re also lying about the utility there as well.

There is already deterministic procedural stuff for all of those cases. Putting a chat bot in front of that is basically fucking stupid.

[theideaofcoffee]

But never underestimate the stupidity of an AI psychosis- suffering product manager to shoehorn it into the most batshit insane use cases in search of mOrE ENgagGeMeNt.

[carlosjobim]

That's not how AI or any kind of assistance is to be used. It's there to double-check and alert you to any possible mistakes you've made. You still have to use your own judgement. Have you never asked a person to take a look at some of your work? Did that mean that you have to follow and obey all of their advice?

[jubilanti]

Sounds a lot like "You're holding it wrong."

[carlosjobim]

No, that what it means to be an adult. Blaming others for your mistakes is something we have to grow out of to become adults. Sadly, at least half of people never grow to be adults during their entire lifetime.

[ofjcihen]

Unfortunately this is grounded in a perfect world. People are prone to believing things that “sound” believable and AI is very good at producing just that.

[carlosjobim]

I've never worked with professionals at such a low level of intelligence, and I don't think a person will even be able to get their license as an electrician if they are that unfamiliar with their own work. The world doesn't have to be perfect for people to be better than making the wrong decision every time possible.

[engineer_22]

I wouldn’t replace best practices for safety like lock-out tag-out and hazard labeling with an LLM but you do you

[72deluxe]

I asked it to identify a starfish I photographed at the weekend whilst scuba diving. It's a spiny starfish, off Cornwall. It said it was a 7 armed starfish. It clearly had 5.

Disappointing. I wouldn't trust it for "real" problems at all!

[trollbridge]

Even the best AI tools are consistently wrong at interpreting the NEC and often in odd ways, like making it stricter than it actually is, or mixing multiple countries’ standards together. 100% useless in an AHJ with a lot of their own rules like Chicago/Cook County.

[engineer_22]

Asking an llm to make a code interpretation is like asking a tape measure to drive a nail. An LLM will be very good at finding relevant code, maybe even explaining its intent or history. It will not be very good at weighing competing priorities, that’s something a human needs to do.