[hypeatei]

I agree about an abundance of apps, but what type of value are LLMs adding?

It can sometimes be useful to input a more "human" search and have something get spit out but 60% of the time it completely lies to you. I'm talking about questions related to web specifications which are public documents. Section numbers, standards names, etc.. will be completely made up.

[rtsil]

Off the top of my head, and just for the last couple of months, and only outside of work (where its value is even more immense), it has saved one of my indoor plants, told me how to handle a major boiler problem that would have left us without a working boiler during a weekend in the winder, with the next "emergency" repairman only available on Monday, advised me to use Kopia as backup solution for my personal files instead of Syncthing, helped me choose the right type of glass for a painting frame, answered a couple of questions about bikes and helped me when I was stuck in an harmonic analysis of a piece of music. All of that are extremely valuable to me (if only for the time not wasted googling answers), and in none of them its potential hallucinating would have been an issue. And I can't count the number of times where "specialists" in bike repairs or plumbing told me something incorrect or outright false, so I've learned to deal with hallucinations already!

[vel0city]

> And I can't count the number of times where "specialists" in bike repairs or plumbing told me something incorrect or outright false, so I've learned to deal with hallucinations already!

So much this. So many times I've argued with hired experts saying "can't be done" just to see yes, it can be done.

[swatcoder]

Yes, but which of those things would you not have resolved just as well 10 years ago? All those possibilities were added by the maturing web itself, as a genuinely novel change from having to source books or experts/friends in the days before.

I'm glad ChatGPT didn't lead you astray, but I'm not seeing what it's added here besides shuffling up the user interface in a way that you presently and subjectively prefer?

[TeMPOraL]

> I'm not seeing what it's added here besides shuffling up the user interface in a way that you presently and subjectively prefer?

This. But in the same sense the past 50 years merely changed interface from dusty textbooks in libraries to Google Search, and the past 100 years gave us dusty textbooks over writing to Royal Society, and that just replaced the option of asking a local whisperer or hoping you'll find answers on the Sunday mass.

Do not underestimate the power of being able to get an answer to your problem described, visualized, and perhaps complete with interactive demo to explore it further, in time it would previously take you to formulate the right search query that finally gives you relevant information.

EDIT:

And that's on top of all the arbitrary data transformations prior tools couldn't do. E.g. I'm increasingly often using GPT and Claude models to turn photos of (possibly hand-written) notes or posters into iCAL files I can immediately import into our family shared calendar.

Another frequent use case, data normalization. Paste a whole dump of inconsistently structured data multiple people collected (say, addresses of various local businesses that helped a local NGO and now are supposed to get a thank-you card for Christmas). Like, you get 200 rows of addresses in a single column, with spelling mistakes, repetitions, junk at the end, arbitrary capitalization, wrong order of address segments, and such; you need to separate it out into 5+ columns (name line 1, name line 2, street address, zip code, city, etc.) and have it all normalized.

The fastest and most robust way to do it as a one-off job, today, is to paste the whole thing to GPT-4o or Claude 3.5 Sonnet, tell it how the output should look (give one-two examples, mention some mistakes you saw), then send the message and wait 30 seconds for the job to be done for you.

(Yes, it may make mistakes - it didn't for me in recent memory, but it can. But for that, I quickly add an extra verification column for each one in LLM output, and do a simple case-insensitive substring match with original, and eyeball any data row that shows an error. And guess what, the formulas don't take much time either, since LLMs are good at writing them for you, too!)

[rtsil]

My plant would have been dead. As for the rest, sure, I would have resolved them eventually, after many frustrated hours of googling and trial and error.

Time is my most precious thing, I already don't have enough time to do all the things that I want to do, I don't want to waste that trying to find and test solutions when ChatGPT gives me instant answers. I'd rather spend time playing with my cats or riding a bike instead. It's not a matter of UI, it's a matter of preventing waste of time, energy and money, and less frustration. For that alone, €20/month is a very good value. And that's just for my personal life.

[swatcoder]

"many hours of frustrated googling and trial and error" isn't a familiar experience to me, but I'll trust that it is for you. I'm glad you see that as behind you now with this. I suppose you must not be alone.

[djeastm]

>besides shuffling up the user interface

I wouldn't discount this effect. As someone with sensory issues, one thing I like about ChatGPT as opposed to the "raw" internet is that I can see the answer to my questions in a nice and calm textual format without some website who created the article specifically to catch my search terms, but is trying to get me to deceptively click on ads or pull me into buying something through their affiliate links. That's absolutely increased my own enjoyment and productivity.

[fragmede]

objectively, it takes less time to ask a question and get a direct answer than it does to search for some words, leaf through a couple of results, find one that has the information you want, and then read that page. If I want to know the height of the Eiffel tower, being told it's 1083 meters tall is faster than searching for its website, finding the stats section, then locating that information on the page. Google realizes that, so they pull that info out of the page and just put it on the results page for you.

[dayjah]

This is a thin edge of the wedge issue, right? ChatGPT is pretty darn good for most things. I’ve used it extensively for the past 18 months and only in a few cases would I say it “completely lied to me”.

My general rubric is: “would I trust someone on Reddit to correctly guide me on this”. If the answer is “yes” then ChatGPT is likely going to do well. If the volume on a particular subject is low / susceptible to false information then it’ll lie.

Recently it lied hard about how to configure MikroTik routers. I lost many hours. But for a large construction project recently it completely balled out.

Are you doing cutting edge / complicated stuff? Have you examples of where it lies?

[hypeatei]

> Have you examples of where it lies?

No specific prompts, but most were related to the XHR/Fetch specs and behaviors within. It would say "X.Y.Z sections defines this" but that section didn't exist at all and the answer provided was not accurate.

> My general rubric is: “would I trust someone on Reddit to correctly guide me on this”. If the answer is “yes” then ChatGPT is likely going to do well

I see. Well, I don't know if I find that very valuable but if others do, then so be it.

[adamc]

I've asked it for things like book recommendations and gotten:

  - completely made up books
  - real books that were only marginally related
  - real books with really bad reviews

I'd estimate that only 30-40% of the time did I find the results at all useful.

[airstrike]

stop using it like a database that you can query

[bigfudge]

Agreed this is a bad idea in the case you are replying to, but I love ChatGPT as a way to recover the name of a book or film I’ve forgotten. I recently prompted for “a book about nuclear wasteland dominated by a church” and it gave me A canticle for Leibowitz (which is great). I’m not sure how easy that would be any other way.

[airstrike]

absolutely, I've been using it for that at lot as well and it's remarkably well suited for it. there's really no better tool for the job.

it's just that every thread about LLMs in AI invariably has someone complaining about best results from a query best described as `SELECT * FROM...`

[cruffle_duffle]

I wonder how many people are promoting it correctly. You can’t just query it like you might for google or something. It works best with lots of context and back and forth. And yeah, for many things you are going to get directional answers not exact ones (esp with “rote memory” like exact quotes from a book or something.)

[therein]

I don't want to turn this into another Claude lies less than ChatGPT subthread but since you mentioned configuration of MikroTik routers I felt like I should.

ChatGPT lies a lot about RouterOS, I don't know why. Claude helped me a lot on the other hand with all things MikroTik.

[dayjah]

Thanks! I’ll give it a shot when I get to the vlan stuff I’ve on deck

[dghlsakjg]

I find it useful, and it brings value to me (literally: I exchange valuable money for API access), even if it doesn't for you. Many other people report the exact same thing. Just because you don't find value in a technology, doesn't mean that others don't.

In the past week I have used it for helping write a script in a framework I'm not super familiar with (OpenSCAD), I was able to finish a project in 5 minutes that otherwise would have taken me hours. I have used it to help make movie recommendations (none of them were hallucinated). I have used it to translate a conversation with a non-english speaker, etc. There are other tools that can help me do all of these things, but none quite as fast or painlessly.

It might not be useful for your use case of asking questions related to specific web specs, but that doesn't mean that the technology has no value. Horses for courses...

[rajamaka]

The value to me is by having an on-demand junior developer working alongside me for the price of $20 a month

[hypeatei]

My experience with code completion tools (i.e. single line/method snippets) has been positive. But, anything more complicated seems to fall apart rather quickly.

[stonedge]

I have upgraded to the $200 Pro tier, and, with o1-pro, all of my tasks delegated to the "junior" have been so much better. It takes longer to complete, of course, but the overall duration is less because I'm not having to go back and correct it as much as I was with 4o. It's been able to figure out problems that 4o continually failed on.

[ipaddr]

Mine is a senior developer with memory lapses.

[airstrike]

And that sometimes you need to bully a bit to get coerced answers out of... feels bad

[drusepth]

LLMs have been a personal tutor to me for the last year, able to explain anything and everything I've been curious about professionally and personally. I changed jobs to new technologies in large part because I effectively had an assistant able to help cover any gaps in knowledge I had, train me up quickly, and offer ongoing help on the job.

They can make stuff up, but saying "60% of the time they lie to you" hasn't been true for years.

[krger]

>They can make stuff up, but saying "60% of the time they lie to you" hasn't been true for years.

If you're using them to fill knowledge gaps, what scaffolding have you set up to ensure that those gaps aren't being filled with incorrect-but-plausible-sounding information?

[lxgr]

That's because we're currently largely not using them correctly, i.e. hooked up to RAG instead of hoping that they've memorized enough of the training data verbatim, which is arguably a waste of neurons in a foundational model.

Imaging being graded on your ability to quote exact line numbers of particular parts of your codebase as a senior software engineer without being able to look at it!

LLMs are not, in isolation, a search product.

[jklinger410]

> but 60% of the time it completely lies to you

This is such an exhausting conversation

[hansonkd]

i think when people say things like this it indicates that they tried LLMs in 2022 and solidified their opinion there.

I had the same impression about the hallucinations 2 years ago. The reality is in at the end of 2024, you can get incredible value from LLMs.

I've used copilot to code almost exclusively now for the past few months. Anyone still comparing it to text completion I feel is operating on completely out of date information either intentionally or unintentionally.

[beefnugs]

Wait do you expect people to retry every failed thing they have tried due to marketing lies every how often exactly?

[jklinger410]

Do you expect the first iteration of every product to be perfect?