[potsandpans]

Counterexample: Ive been able to complete more side projects in the last month leveraging llms than i have ever in my life. One of which I believe to have potential as a viable product, and another which involved complicated rust `no_std` and linker setup for compiling rust code onto bare metal RISCV from scratch.

I think the key to being successful here is to realize that you're still at the wheel as an engineer. The llm is there to rapidly synthesize the universe of information.

You still need to 1) have solid fundamentals in order to have an intuition against that synthesis, and 2) be experienced enough to translate that synthesis into actionable outcomes.

If youre lacking in either, youre at the same whims of copypasta that have always existed.

[pfannkuchen]

I’ve found LLMs to basically be a more fluent but also more lossy way of interfacing with stack overflow and tutorials.

If a topic is well represented in those places, then you will get your answer quicker and it can be to some extent shaped to your use case.

If the topic is not well represented there, then you will get circular nonsense.

You can say “obviously, that’s the training data”, and that’s true, and I do find it obvious personally, but the reaction to LLMs as some kind of second coming does not align with this reality.

[biohcacker84]

That matches my experience too. I wonder how fast they'll improve and if LLMs will hit a wall, as some AI experts think.

[janderson215]

Is it possible that you’re both using LLMs the same way you’d use SO and that’s the reason you see such similarities? The reason I ask is because it doesn’t not match my experience. It feels more like I’m able to Matrix-upload docs into my brain like Trinity learning to fly a helicopter.

[pfannkuchen]

I am using it like stack overflow in the sense that I’m solving a problem and I’m using it to answer questions when I’m in an unfamiliar or non-obvious place in the problem space.

If I have a question about a first order language or framework feature or pattern, it works great. If I have a question about a second order problem, like an interaction between language or framework features, or a logical inconsistency in feature behavior, then it usually has no idea what’s going on, unless it turns out to be a really common problem such as something that would come up when working through a tutorial.

For code completion, I’ve just turned it off. It saves time on boilerplate typing for sure, but the actual content pieces are so consistently wrong that on balance I find it distracting.

Maybe I have a weird programming style that doesn’t mesh well with the broader code training corpus, not sure. Or maybe a lot of people spend more time in the part of problem-space that intersects with tutorial-space? I am not very junior these days.

That being said I definitely do use LLMs to engage with tutorial type content. For that it is useful. And outside of software it is quite a bit better for interfacing with Wikipedia type content. Except for the part where it lies to your face. But it will get better! Extrapolating never hurt anyone.

[x0x0]

Same. I use it to bootstrap my writing a react native app from pretty low familiarity with React.

It's pretty good at writing screens in broad strokes. You will have to fill in some details.

The exact details of correctly threading data through; or prop drilling vs alternatives; the rules around wrapping screens to use them in React Navigation? It's terrible at them.

[h0l0cube]

> I’m able to Matrix-upload docs into my brain like Trinity learning to fly a helicopter.

So you're using it like Wikipedia? I find when learning something new (and non coding related) YouTube is infinitely better than an LLM. But then I prefer visual demonstration to tutorial or verbal explanation.

[almostdeadguy]

I'm sorry, what?

[ASalazarMX]

> It feels more like I’m able to Matrix-upload docs into my brain like Trinity learning to fly a helicopter

Did you puzzle about this sentence specifically? Imagine you don't know jack about flying helicopters, then Tank uploads the Helicopter Pilot Program (TM) directly to your brain; it would feel like magic.

Conversely, if you know a lot about helicopters, just not enough to fly a B-212, and the program includes instructions like "Press (Y) and Left Stick to hover", you'd know it's confabulating real world piloting with videogames.

That's the same with LLMs, you need to know a lot of the field to ask the right questions, and recognize/correct slop or confabulation, otherwise they seem much more powerful and smart than they really are.

[EarthMephit]

I find that LLMs are almost comically bad at projects that have a hardware component like RaspberryPi or Pico, or Ardunio.

I think that its because often the libraries you use are niche or have a a few similar versions, the LLM really commonly hallucinated solutions and would continually suggest that library X did have that capability. I think because often in hardware projects you often hit a point where you can't do something or you need to modify a library, but the LLM tries to be "helpful" and it makes up a solution.

[jerf]

Based on my own modestly successful forays into that world, I have to imagine one problem the LLMs have in that space is terrible training data. A good three quarters of any search result you search for in that space will be straight-up out-of-date and not work on your system. Then you've got all the tiny variations between dozens of chipsets, and all the confidently wrong people on the internet telling you to do nonsensical things, entire ecosystems that basically poofed out of existence three years ago but are still full of all kinds of juicy search terms and Google juice... if I can hardly paw through this stuff with years of experience and the physical hardware right in front of me to verify claims with, I don't know how an LLM is supposed to paw through all that and produce a valid answer to any question in that space, without its own hardware to fiddle with directly.

[lostlogin]

You’re making my eye twitch.

The number of projects I’ve done where my notes are the difference between hours of relearning The Way or instant success. Google doesn’t work as some niche issue is blocking the path.

ESP32, Arduino, Home Assistant And various media server things.

[chamomeal]

They’re also pretty bad at typescript generics. They’re quite good at explaining concepts (like mapped types), but when push comes to shove they generate all sorts of stuff that looks convincing, but doesn’t pass the type checker.

And then you’ll paste in the error, and they’ll just say “ok I see the problem” and output the exact same broken code lol.

I’m guessing the problem is lack of training data. Most TS codebases are mostly just JS with a few types and zod schemas. All of the neat generic stuff happens in libraries or a few utilities

[swatcoder]

Actually, it's because many of the people writing tutorials and sharing answers about that stuff don't know what the hell they're doing or grasp the fundamentals of how those systems work and so most of the source material the LLM's are trained on is absolute garbage.

Public Arduino, RPi, Pico communities are basically peak cargo cult, with the blind leading the blind through things they don't understand. The noise is vastly louder than the signal.

There's a basically giant chasm between expereinced or professional embedded developers that mostly have no need to ever touch those things or visit their forums, and the confused hobbyists on those forums randomly slapping together code until something sorta works while trying to share their discoveries.

Presumably, those communities and their internal knowledge will mature eventually, but it's taking a long long time and it's still an absolute mess.

If you're genuinely interested in embedded development and IoT stuff, and are willing to put in the time to learn, put those platforms away and challenge yourself to at least learn how to directly work with production-track SoC'a from Nordic or ESP or whatever. And buy some books or take some courses instead of relying on forums or LLM's. You'll find yourself rewarded for the effort.

[bsder]

> Presumably, those communities and their internal knowledge will mature eventually, but it's taking a long long time and it's still an absolute mess.

It won't because the RPi are all undocumented, closed-source toys.

It would be an interesting experiment to see which chips an LLM is better at helping out with: RPi's with its hallucinatory ecosystem or something like the BeagleY-AI which has thousands of pages of actual TI documentation for its chips.

It would be really nice if the LLMs could cover for this and circumvent where RPi's keep getting used because they were dumped under cost to bootstrap a network effect.

[rcxdude]

>Presumably, those communities and their internal knowledge will mature eventually, but it's taking a long long time and it's still an absolute mess.

I'm not sure they will. There's a kind of evaporative cooling effect where once you get to a certain level of understanding you switch around your tools enough that there's not much point interacting with the community anymore.

[tacticalDonut]

Any book or course recommendations for someone who has a senior SWE background but has never touched hardware/embedded systems?

[jrmg]

I was just today trying to fix some errors in an old Linux kernel version 3.x.x .dts file for some old hardware, so that I could get a modern kernel to use it. ChatGPT seemed very helpful at first - and I was super impressed. I thought it was giving me great insight into why the old files were now producing errors … except the changes it proposed never actually fixed anything.

Eventually I read some actual documentation and realised it was just spouting very plausible sounding nonsense - and confident at it!

The same thing happened a year or so ago when I tried to get a much older ChatGPT to help me with with USB protocol problems in some microcontroller code. It just hallucinated APIs and protocol features that didn’t actually exist. I really expected more by now - but I now suspect it’ll just never be good at niche tasks (and these two things are not particularly niche compared to some).

[dagw]

Eventually I read some actual documentation...

For the best of both worlds make the LLM first 'read' the documentation, and then ask for help. Make a huge difference in the quality and relevance of the answers you get.

[ASalazarMX]

And hope the docs aren't too large. LLMs tend to confabulate more with longer contexts.

[onemoresoop]

I find LLMs to be decent unblockers. I only turn to them form time to time though, unless Im in a playful mode and try poking out various ideas. As a coder I also ask for snippets when Im lazy. I tried growing a slightly larger solution a few times and it failed in dumb ways. It was clear it doesn’t really comprehend we do, it’s not aware it’s moving in circles and so on. All these things will probably see a lot incremental of improvents and as a tool will definitely stay but fundamentally LLMs can’t really think, at least the way we do and expecting that is also foolish.

[talldayo]

> which involved complicated rust `no_std` and linker setup for compiling rust code onto bare metal RISCV from scratch.

That's complicated, but I wouldn't say the resulting software is complex. You gave an LLM a repetitive, translation-based job, and you got good results back. I can also believe that an LLM could write up a dopey SAAS in half the time it would take a human to do the same.

But having the right parameters only takes you so far. Once you click generate, you are trusting that the model has some familiarity with your problem and can guide you without needing assistance. Most people I've seen rely entirely on linting and runtime errors to debug AI code, not "solid fundamentals" that can fact-check a problem they needed ChatGPT to solve first place. And the "experience" required to iterate and deploy AI-generated code basically boils down to your copy-and-paste skills. I like my UNIX knowledge, but it's not a big enough gate to keep out ChatGPT Andy and his cohort of enthusiastic morons.

We're going to see thousands of AI-assisted success stories come out of this. But we already had those "pennies on the dollar" success stories from hiring underpaid workers out of India and Pakistan. AI will not solve the unsolved problems of our industry and in many ways it will exacerbate the preexisting issues.

[simonw]

A tool that can "write up a dopey SAAS in half the time it would take a human to do" is a pretty incredible thing to add to your toolbox!

[talldayo]

If the summary goal of your existence is to be the most delirious waste of resources that humanity has yet known, sure. It's the hammer and nail of spoiled burnouts everywhere that need a credible ruse to help them out of the bottle.

Some of us are capable of wanting for things better than a coin-operated REST API. The kind of imagination used to put people on the moon, that now helps today's business leaders imagine more profitable ways to sell anime pornography on iPhone. (Don't worry, AI will disrupt that industry too.)

[tonyhart7]

I'm sorry, but generated REST API boilerplate has probably fed more people than putting people on the moon.

[justinclift]

Pity about the direct injection SQL vulnerabilities though?

[HPsquared]

What it will do is to free up a lot of brainpower to think about those hard problems and empower people to try our their ideas.

[talldayo]

I used to think the exact same thing would happen when we paid Pakistani and Indian labor to do America's busywork.

That was about 15 years ago, I no longer have the same enthusiasm you do.

[tonyhart7]

Now, you are paying a Taiwanese or American company to produce GPUs for you. This allows you to use open-source models like DeepSeek R1, significantly reducing your reliance on Indian tech labor

[sgc]

I believe they are saying we did not learn to think more deeply then, so why should we expect to learn how now.

[baxtr]

Is it reasonable to assume that more senior devs benefit more from LLMs?

[adamtaylor_13]

I believe so. In my experience, you need to have that gut intuition (or experience) to say, “No way. That’s totally wrong.”

Since AI will capitulate and give you whatever you want.

You also have to learn how to ask without suggesting because it will take whatever you give it and agree.

[j_bum]

Yep. I think a default state of skepticism is an absolute necessity when working with these tools.

I love LLMs. I agree with OP them expanding my hobby capacity as well. But I am constantly saying (in effect) “you sure…?” and tend to have a pretty good bs meter.

I’m still working to get my partner to that stage. They’re a little too happy to accept an answer without pushback or skepticism.

I think being ‘eager to accept an answer’ is the default mode of most people anyway. These tools are likely enabling faster disinformation consumption for the unaware.

[Spivak]

Yes, you essentially have an impossibly well read junior engineer you can task with quick research questions like, "I'm trying to do x using lib y, can you figure that out for me." This is incredibly productive because in the answer is typically all the pieces you need but not always assembled right.

Getting the LLM to pull out well-known names of concepts is for me the skill you can't get anywhere else. You can describe a way to complete a task and ask for what it's called and you'll be heading down arxiv links right away. Like yes the algorithm to find the closest in edit distance and length needle string in a haystack is called Needleman–Wunsch, of course Claude, everyone knows that.

[ahi]

Once it gives me the names for the concepts I'm struggling with, I often end up finding the stackoverflow or documentation it's copy pasting.

[simonw]

I think so.

Junior devs can get plenty of value out of them too, if they have discipline in how they use them - as a learning tool, not as a replacement for thinking about projects.

Senior devs can get SO much more power from these things, because they can lean on many years of experience to help them evaluate if the tool is producing useful results - and to help them prompt it in the most effective way possible.

A junior engineer might not have the conceptual knowledge or vocabulary to say things like "write tests for this using pytest, include a fixture that starts the development server once before running all of the tests against it".

[RickS]

IMO experience provides better immunity for common hangups. Generated code tends to be directionally pretty good, but with lots of minor/esoteric failures. The experience to spot those fast and tidy them makes all the difference. Copilot helps me move 10x faster with tedious arduino stuff, but I can easily see where if I didn't have decent intuition around debugging and troubleshooting, there'd be almost zero traction since it'd be hard to clear that last 10% hurdle needed to even run the thing.

[jaredcwhite]

I wouldn't assume that at all. Most of the senior devs I talk to on a regular basis think commercial* LLMs are ridiculous and the AI hype is nonsensical.

* I put commercial there as a qualifier because there's some thought that in the future, very specifically-trained smaller models (open source) on particular technologies and corpuses (opt-in) might yield useful results without many of the ethical minefields we are currently dealing with.

[dogma1138]

It depends it think it’s less about how senior they are and how good they are at writing requirements, and knowing what directives should be explicitly stated and what can be safely inferred.

Basically if they are good at utilizing junior developers and interns or apprentices they probably will do well with an LLM assistant.

[jumpman500]

Ya. I think people that have better technical vocabulary and an understanding of what should be possible with code do better. That’s usually a senior engineer, but not always

[satvikpendem]

It's the LLM paradox, seniors get more senior with them while juniors get more junior, creating a bimodal distribution in the future simply because juniors will start depending on them too much to learn how to code properly while seniors (who some may also exhibit the previous trait) will by and large be able to rapidly synthesize information from LLMs with their own understanding.

[ern]

I had a couple of the most capable senior developers reach out to me to tell me how Github Copilot accelerated their productivity, which surprised me initially. So I think there's something to it.

[chamomeal]

I agree with his point about asking AI to “fix” problems though. It’s really nice that you don’t have to fully understand something to use it, but that becomes a problem if you lean on it too much

[pinoy420]

Exactly this. OP, credit where credit is due, appears to be someone who “hacks things together” copy pasting solutions blindly from the internet - with little intuition gained along the way.

[gsf_emergency_2]

Ime engineers who find LLM useful have misunderstood their reasons for existence outside of being salaried..

What is your main project ? Do you LLM that? (

I wager you're not a rust expert and should maybe reconsider using rust in your main project.

FWIW asking LLM whether you should use rust ~ asking it about the meaning of life. Important questions that need answers, but not right away! (A week or 2 tops)

If you need to synthesize the universe of information with LLM.. that is not the universe you want to live or play in

[dogma1138]

Indeed LLMs are useful as an intern, they are at the “cocky grad” stage of their careers. If you don’t understand the problem and can’t steer the solution and worse has only limited understanding of the code they produce you are unlikely to be productive.

On the other hand if you understand what needs to be done, and how to direct the work the productivity boost can be massive.

Claude 3.5 sonnet and O1 are awesome at code generation even with relatively complex tasks and they have a long enough context and attention windows that the code they produce even on relatively large projects can be consistent.

I also found a useful method of using LLMs to “summarize” code in an instructive manner which can be used for future prompts. For example summarizing a large base class that may be reused in multiple other classes can be more effective than having to overload a large part of your context window with a bunch do code.

[irthomasthomas]

Another data point. Plugin ideation to publication in 2 minutes, from a single prompt, albeit a multi-step shell prompt. 3 packages published in 24 hrs https://x.com/xundecidability/status/1884077427871342955

[mythrwy]

I've had both experiences strangely enough.

[blast]

> The llm is there to rapidly synthesize the universe of information

That's a nice way of putting it.

[zackproser]

This

[maybesomaybenot]

Claude is like having my own college professor. I've learned more in the past month with Claude then I learned in the past year. I can ask questions repeatedly and get clarification as fine as a need it. Granted, Claude has limits, but its a game-changer.

> I think the key to being successful here is to realize that you're still at the wheel as an engineer. The llm is there to rapidly synthesize the universe of information.

Bingo. OP is like someone who is complaining about the tools, when they should be working on their talent. I have a LOT of hobbies (circuits, woodworking, surfing, playing live music, cycling, photography) and there will always be people who buy the best gear and complain that the gear sucks. (NOTE: I"m not implying claude is "the best gear", but it's a big big help.)

I think the only problem with LLMs is synthesis of new knowledge is severely limited. They are great at explaining things others have explained, but suck hard at inventing new things. At least that's my experience with Claude: it's terrible as a "greenfield" dev.

[BalinKing]

> Claude is like having my own college professor.

I don't use Claude, so maybe there's a huge gap in reliability between it and ChatGPT 4o. But with that disclaimer out of the way, I'm always fairly confused when people report experiences like these—IME, LLMs fall over miserably at even very simple pure math questions. Grammatical breakdowns of sentences (for a major language like Japanese) are also very hit-or-miss. I could see an LLM taking the place of, like, an undergrad TA, but even then only for very well-trod material in its training data.

(Or maybe I've just had better experiences with professors, making my standard for this comparison abnormally high :-P )

EDIT: Also, I figure this sort of thing must be highly dependent on which field you're trying to learn. But that decreases the utility of LLMs a lot for me, because it means I have to have enough existing experience in whatever I'm trying to learn about so that I can first probe whether I'm in safe territory or not.

[joseda-hg]

Major in the context of Japanese is rough, I can see a significant drop in quality when interacting with the same model in say Spanish vs English

For as rich a culture the Japanese have, there's only about 1XX million speakers and the size of the text corpus really matters here, the couple billion of English speakers are also highly motivated to choose English over anything else because Lingua Franca has homefield advantage

To use LLM's efectively you have to work with knowledge of their weaknesses, Math is a good example, you'll get better results from Wolphram Alpha even for the simple things, which is expected

Broad reasoning and explanations tend to be better than overly specific topics, the more common a language, the better the response If a topic has a billion tutorials online, an LLM has a really high chance of figuring out first try

Be smart with the context you provide, the more you actively constrain an LLM, the more likely it is to work with you I have friends that just use it to feed class notes to generate questions and probe it for blindspots until they're satisfied, the improvements on their grade s make it seem like a good approach, but they know that just feeding responses to the LLM isn't trustworthy, so they do and then they also check by themselves, the extra time valuable by itself, if just to improve familiarity with the subject

[satvikpendem]

> LLMs fall over miserably at even very simple pure math questions

They are language models, not calculators or logic languages like Prolog or proof languages like Coq. If you go in with that understanding, it makes a lot more sense as to their capabilities. I would understand the parent poster to mean that they are able to ask and rapidly synthesize information from what the LLM tells them, as a first start rather than necessarily being 100% correct on everything.

[BalinKing]

Of course that's fair advice in itself, but the parent specifically equated them to a "college professor."

[justinclift]

Maybe that should be "college art professor" then? :)

[maybesomaybenot]

I think a lot of these people object to AI probably see the gross amounts of energy it is using, or the trillions of dollars going to fewer than half a dozen men (most american, mostly white).

But, once you've had AI help you solve some gnarly problems, it is hard not to be amazed.

And this is coming from a gal who thinks the idea of self-driving cars is the biggest waste of resources ever.

[BalinKing]

(EDIT: Upon rereading this, it feels unintentionally blunt. I'm not trying to argue, and I apologize if my tone is somewhat unfriendly—that's purely a reflection of the fact that I'm a bad writer!)

Sorry, maybe I should've been clearer in my response—I specifically disagree with the "college professor" comparison. That is to say, in the areas I've tried using them for, LLM's can't even help me solve simple problems, let alone gnarly ones. Which is why hearing about experiences like yours leaves me genuinely confused.

I do get your point about people disagreeing with modern AI for "political" reasons, but I think it's inaccurate to lump everyone into that bucket. I, for one, am not trying to make any broader political statements or anything—I just genuinely can't see how LLMs are as practically useful as other people claim, outside of specific use cases.

[maybesomaybenot]

Like I made very clear, it is great at some things and terrible at others.

YMMV. /shrugs/

[KerrAvon]

How do you know it's accurate?

[selcuka]

They are reasonably accurate, and no tutor is perfect. How do you know your college professor is accurate?

[malfist]

My college professor has certifications and has passed tests that weren't in their training data.

My college professor was also willing to say "I don't know, ask me next class"

[paulryanrogers]

> My college professor was also willing to say "I don't know, ask me next class"

This is a key differentiator that I see in humans over LLMs: knowing ones limits.

[selcuka]

> My college professor has certifications and has passed tests that weren't in their training data.

Granted, they are not (can't be) as rigorous as the tests your professor took, but new models are run through test suites before being released, too.

That being said, I saw my college professors making up things, too (mind you, they were all graduated from very good schools). One example I remember was our argument with a professor who argued that there is a theoretical limit for the coefficient of friction, and it is 1. That can potentially be categorised as a hallucination as it was completely made up and didn't make sense. Maybe it was in his training data (i.e. his own professors).

I agree with the "I don't know" part, though. This is something that LLMs are notoriously bad.

[Eisenstein]

What do you consider 'not in its training data'?

I just asked Claude a question I am pretty sure was not in its training data.

* https://i.imgur.com/XjvImeT.jpeg

[mahogany]

It is immediately wrong in Step 1. A newborn is not a 2:1 ratio of height:width. Certainly not 25cm width (what does that even mean? Shoulder to shoulder?).

This is a perfect example of where not knowing the “domain” leads you astray. As far as I know “newborn width” is not something typically measured, so Claude is pulling something out of thin air.

Indeed you are showing that something not in the training data leads to failure.

[selcuka]

That's almost in the training data:

https://www.quora.com/How-many-Humans-can-we-fit-on-the-Moon

[satvikpendem]

Did you actually test the math done? Usually LLMs are terrible at math as, as I mentioned in another comment, they are language models, not calculators. Hopefully that changes when LLMs leverage other apps like calculators to get their results, I am not sure if Claude does that already or it's still in development.

[maybesomaybenot]

You can also test your professor's answers. I don't just walk around going "Oh, Claude was right", I'm literally using what I just learned and am generating correct results. I'm not learning facts like dates, or subject things, I'm learning laws, equations, theories, proofs, etc. (Like how to apply Euler's totient or his extended theories on factorization... there's only one "right answer").

Also, you method for attesting your professors accuracy is inherently flawed. That little piece of paper on their wall doesn't correlate with how accurate they are; it doesn't mean zero, but it isn't foolproof. Hate to break it to you, but even heroes are fallible.

[maybesomaybenot]

Simple: one thing I'm learning about is RFCs for TCP/IP. I can literally go test it. It's like saying, "How do you know it is right when it says 2+2=4"? Some knowledge when taught is self-correcting. Other things I'm studying, like, say, tensor calculus, I can immediately use and know I learned it correctly.

[kortilla]

TCP/IP is a great example though of something you can get seemingly correct and then be subject to all kinds of failure modes in edge cases you didn’t handle correctly (fragmentation, silly windows, options changing header sizes, etc).

[maybesomaybenot]

Thanks for telling me how wrong I am! I bet you're fun at parties.

[namaria]

This is not a party, this is a technology forum and we prize being right

[XenophileJKO]

I would add though, that they can be very good at combining known concepts. Which can create a non-trivial set of "new knowledge".

[maybesomaybenot]

Creating new knowledge from current knowledge is called "synthesis" (ancient term, nothing modern). I'm hoping you're right, it would be amazing.