[josephwegner]

I become more and more convinced with each of these tweets/blogs/threads that using LLMs well is a skill set akin to using Search well.

It’s been a common mantra - at least in my bubble of technologists - that a good majority of the software engineering skill set is knowing how to search well. Knowing when search is the right tool, how to format a query, how to peruse the results and find the useful ones, what results indicate a bad query you should adjust… these all sort of become second nature the longer you’ve been using Search, but I also have noticed them as an obvious difference between people that are tech-adept vs not.

LLMs seems to have a very similar usability pattern. They’re not always the right tool, and are crippled by bad prompting. Even with good prompting, you need to know how to notice good results vs bad, how to cherry-pick and refine the useful bits, and have a sense for when to start over with a fresh prompt. And none of this is really _hard_ - just like Search, none of us need to go take a course on prompting - IMO folks jusr need to engage with LLMs as a non-perfect tool they are learning how to wield.

The fact that we have to learn a tool doesn’t make it a bad one. The fact that a tool doesn’t always get it 100% on the first try doesn’t make it useless. I strip a lot of screws with my screwdriver, but I don’t blame the screwdriver.

[mexicocitinluez]

Agree. It's a tool like anything else.

On a side note, this lady is a fraud: https://www.youtube.com/watch?v=nJjPH3TQif0&themeRefresh=1

[sitkack]

I don't know if she is a fraud, but she has definitely greatly amplified Rage Bait Farming and talking about things that are far outside of her domain of expertise as if she were an expert.

In no way am I credentialing her, lots of people can make astute observations about things they weren't trained in, but she both mastered sounding authoritative and at the same time, presenting things to go the most engagement possible.

[joquarky]

I've frequently heard that once you get sucked into the YouTube algorithm, you have to keep making content to maintain rankings.

This trap reminds me of the Perry Bible Fellowship comic "Catch Phrase" which has been removed for being too dark but can still be found with a search.

[sitkack]

https://www.youtube.com/watch?v=4L0PAXOH718 Sponge Bob realizes he has played himself out with his own catchphrase!

Wow, thank you. I rarely good a good cultural recommendation here, but PBF I didn't know about.

I raise you, Joan Cornellà

[refulgentis]

Thanks for sharing this. I was heavily involved in graduate physics when I was in school, and was very worried about what direction shed take after the first big viral vid "telling her story." I wasn't sure it was well understood, or even understood at all, how blinkered her...viewpoint?...was.

[Spivak]

LLMs function as a new kind of search engine, one that is amazingly useful because it can surface things that traditional search could never dream of. Don't know the name of a concept, just describe it vaguely and the LLM will pull out the term. Are you not sure what kind of information even goes into a cover letter or what's customary to talk about? Ask an LLM to write you one, it will be bland and generic sure but that's not the point because you now know the "shape" of what they're supposed to look like and that's great for getting unblocked. Have you stumbled across a passage of text that's almost English but you're not really sure what to look up to decipher it? Paste it into the LLM and it will tell you that it's "Early Modern English" which you can look up to confirm and get a dictionary for.

[cainxinth]

Broader than that, it’s critical thinking skills. Using search and LLMs requires analyzing the results and being able to separate what is accurate and useful from what isn’t.

[giraffe_lady]

From my experience this is less an application of critical skills and more a domain knowledge check. If you know enough about the subject to have accumulated heuristics for correctness and intuition for "lgtm" in the specific context, then it's not very difficult or intellectually demanding to apply them.

If you don't have that experience in this domain, you will spend approximately as much effort validating output as you would have creating it yourself, but the process is less demanding of your critical skills.

[sitkack]

No, it is critical thinking skills, because the LLMs can teach you the domain, but you have to then understand what they are saying enough to tell if they are bsing you.

> you don't have that experience in this domain, you will spend approximately as much effort validating output as you would have creating it yourself,

Not true.

LLMs are amazing tutors. You have to use outside information, they test you, you test them, but they aren't pathologically wrong in the way that they are trying to do a Gaussian magic smoke psyop against you.

[cainxinth]

Knowledge certainly helps, but I’m talking about something more fundamental: your bullshit detector.

Even when you lack subject matter expertise about something, there are certain universal red flags that skeptics key in on. One of the biggest ones is: “There’s no such thing as a free lunch” and its corollary: “If it sounds too good to be true, it probably is.”

[jpc0]

I'm not so sure about that. I was really Anti llm in the previous generation of LLMs (GPT3.5/4) but never stopped trying them out. I just found the results to be subpar.

Since reasoning models came about I've been significantly more bullish on them purely because they are less bad. They are still not amazing but they are at a poiny where I feel like including them in my workflow isn't an impediment.

They can now reliably complete a subset of tasks without me needing to rewrite large chunks of it myself.

They are still pretty terrible at edge cases ( uncommon patterns / libraries etc ) but when on the beaten path they can actually pretty decently improve productivity. I still don't think 10x ( well today was the first time I felt a 10x improvement but I was moving frontend code from a custom framework to react, more tedium than anything else in that and the AI did a spectacular job ).