[Toutouxc]

I use it daily, sort of like a professional site/forum where someone immediately answers any question you may have. It may not be 100 % accurate, just like real people, but it's instant and personalized. (And in my experience GPT-4 is just spot on for anything a beginner might ask.) It's like training wheels for anything.

How do I set up some backup infrastructure for my home server? How does Btrfs handle read errors? How do I cook X if I don't have ingredient Y? What's the correct syntax for this command? How to analyse and interpret these measurements from this vermicomposting experiment (helping my girlfriend with statistics)? How do noise cancelling headphones REALLY work? Is it okay to have my samba.conf set like this (it wasn't)?

[spicybright]

This is what I'm most curious about with AI. How often do you catch it being incorrect? The questions you're asking a fairly inconsequential if wrong, but how often do you fact check what it says?

[standardly]

You have to tell it. "Please be correct" it usually works

[moonchrome]

What was the accuracy and the difference between googling the question ?

[Toutouxc]

I can't say anything specific about the accuracy, other than that it's good enough, but the difference between asking GPT-4 and googling the same question is night and day. And that's comparing it to Kagi, with my own search filters with boosted wikipedia, stackoverflow and generally scientific sources. Do you know the feeling that you want to ask a specific question about a specific, technical aspect of something, and you wish you could just tell the search engine somehow that you're just not interested in hand-wavy popular science average Joe half-incorrect answers? And if you actually find the one good answer on page 14 of some god-forgotten thread from 2006, the question differs in some tiny but important detail from what you're after?

Well that feeling doesn't exist with GPT-4. So far we've always been able to come up with something, together. If you don't like the first answer, you ask more questions. You can dig deeper. You can tell it what your guess is, what your intuition tells you about the problem, where exactly your uncertainty lies, e.g.:

  You: I don't understand phenomenon X.
  GPT: Oh that's easy, X is just [parrots wikipedia].
  You: That's fine, but I don't understand how X differs from Y, they sound like the same phenomena.
  GPT: I see why you'd think that, but Y and X actually differ in this detail called Foo that makes all the difference.
  You: I still don't get it, compare Foo to something similar that I know from normal life.
  GPT: Okay, so Foo is like an elephant who's too large to drive a car.
  You: Ooh, I get it now.

[moonchrome]

But here's what happens when I try that :

  Me : I don't understand this publicly documented AWS service with plenty of OSS examples, can you suggest how I would solve some edge case X
  GPT : Oh that's easy, invents bullshit that sounds exactly like what I need 
  Me : Googles the shit the GPT came up with and can't find any references
  Me : Maybe there is something similar - spends more time searching
  Me : GPT you're wrong - your solution doesn't exist 
  GPT : Oh sorry, here's the correct solution, comes up with more bulshit
  Me : Googles the shit that GPT came up with and can't find any references
  Me : Starts googling and solving the problem on my own
  Me : Finds out X can't be done with AWS service

I've had this flow many times now, it never resulted in valuable output from GPT vs just Googling since I need to verify everything anyway.

Or

  Me : Review and come up with improvements to this code : CP code
  GPT : Suggest bullshit improvements, gives factually wrong reasoning, makes code worse
  Me : Points out everything wrong 
  GPT : Suggest other bullshit improvements
  Me : Points out everything wrong
  GPT : Returns basically identical code to input

Or

  Me : Propose a design to this technical problem 
  GPT : Implements the solution in the most roundabout way and suboptimal way

IMO if I'm not sure what the output should be GPT is less than worthless it's actually convincingly misleading.

[Toutouxc]

Is that with 3.5 or with 4? 4 is usually not afraid to tell it doesn’t know.

[moonchrome]

I've upgraded to 4 ASAP, all those examples are with 4. I tried 3.5 buy it was completely useless, 4 actually started giving sane output but still lying and hard to verify.

My only use case so far is stuff like "translate this model to openapi schema" because it was faster than setting up the tool, and similar tasks that are easy to verify and boring to type.