[fuzzfactor]

I had a feeling it had to be something like massive waste due to a misguided feature of the algorithms that shouldn't have been there in the first place.

Once the "math is done" quite likely it would have paid off better than most investments for the top people to have spent a few short years working with grossly underpowered hardware until they could come up with amazing results there before scaling up. Rather than grossly overpowered hardware before there was even deep understanding of the underlying processes.

When you think about it, what we have seen from the latest ultra-high-powered "thinking" machines is truly so impressive. But if you are trying to fool somebody into believing that it's a real person it's still not "quite" there.

Maybe a good benchmark would be to take a regular PC, and without reliance on AI just pull out all the stops and put all the effort into fakery itself. No holds barred, any trick you can think of. See what the electronics is capable of this way. There are some smart engineers, this would only take a few years but looks like it would have been a lot more affordable.

Then with the same hardware if an AI alternative is not as convincing, something has got to be wrong.

It's good to find out this type of thing before you go overboard.

Regardless of speed or power, I never could have gotten an 8-bit computer to match the output of a 32-bit floating-point algorithm by using floating-point myself. Integers all the way and place the decimal where it's supposed to be when you're done.

Once it's really figured out, how do you think it would feel being the one paying the electric bills up until now?

[jimmaswell]

Faster progress was absolutely worth it. Spending years agonizing over theory to save a bit of electric would have been a massive disservice to the world.

[rossjudson]

You're sort of presuming that LLMs are going to be a massive service to the world there, aren't you? I think the jury is still out on that one.

[jimmaswell]

They already have been. Even just in programming, even just Copilot has been a life changing productivity booster.

[recursive]

I've been using copilot for several months. If I could figure out a way to measure its impact on my productivity, I'd probably see a single digit percentage boost in "productivity". This is not life-changing for me. And for some tasks, it's actually worse than nothing. As in, I spend time feeding it a task, and it just completely fails to do anything useful.

[jimmaswell]

I've been using it for over a year I think. I don't often feed it tasks with comments so much as go about things the same as usual and let it autocomplete. The time and cognitive load saved adds up massively. I've had to go without it for a bit while my workplace gets its license in order for the corporate version and the personal version has an issue with the proxy, and it's been agonizing going without it again. I almost forgot how much it sucks having to jump to google every other minute, and it was easy to start to take for granted how much context copilot was letting me not have to hold onto in my head. It really lets me work on the problem as opposed to being mired in immaterial details. It feels like I'm at least 2x slower overall without it.

[atq2119]

> I almost forgot how much it sucks having to jump to google every other minute

Even allowing for some hyperbole, your programming experience is extremely different from mine. Looking anything up outside the IDE, let alone via Google, is by far the exception for me rather than the rule.

I've long suspected that this kind of difference explains a lot of the difference in how Copilot is perceived.

[rockskon]

I don't know about you but LLMs spit out garbage nonsense frequent enough that I can't trust their output in any context I cannot personally verify the validity of.

[framapotari]

If you're already a competent developer, I think that's a reasonable expectation of impact on productivity. I think the 'life-changing' part comes in helping someone get to the point of building things with code where before they couldn't (or believed they couldn't). It does a lot better job of turning the enthusiasts and code-curious into amateurs vs. empowering professionals.

[jimmaswell]

> turning the enthusiasts and code-curious into amateurs vs. empowering professionals.

I'm firmly in #2. My other comment goes over how.

I'm intrigued to see how devs in #1 grow. One might be wary those devs would grow into bad habits and not thinking for themselves, but it might be a case of the ancient Greek rant against written books hindering memorization. Could be that they'll actually grow to be even better devs unburdened by time wasted on trivial details.

[datavirtue]

If you are in maintenance mode your visits to Copilot will be rare. If you are building greenfield, use goes through the roof. All those test cases, nevermind all the POC and framework scaffolding and other boilerplate that is now completely unacceptable as a use of developer time.

[recursive]

I'm building "greenfield". I still use it at least daily, but the benefit just struggles to outweigh the cost of invoking it. Maybe I don't understand how to use it.

[giraffe_lady]

"Even just in programming" the jury is still out. None of my coworkers using these are noticeably more productive than the ones who don't. Outside of programming no one gives a shit except scammers and hype chasers.

[JonChesterfield]

The people writing articles for journals that aggregate and approximate other sources are in mortal terror of LLMs. Likewise graphic designers and anyone working in (human language) translation.

I don't fear that LLMs are going to take my job as a developer. I'm pretty sure they mark a further decrease in the quality and coherence of software, along with a rapid increase in the quantity of code out there, and that seems likely to provide me with reliable employment forever. I'm basically employed in fixing bugs that didn't need to exist in the first place and that seems to cover a lot of software dev.

[giraffe_lady]

They're not scared of LLMs because of anything about LLMs. It's just that everyone with power is publicly horny to delete the remaining middle class jobs and are happy to use LLMs as a justification whether it can functionally replace those workers or not. So it's not that everyone has evaluated chatgpt and cannily realized it can do their job, they're just reading the room.

[wruza]

Are you sure it’s a life changing productivity booster? Sometimes I look at my projects and wonder how would I explain it to an LLM what this code should have done if it didn’t exist yet. Must be a shitton of boilerplate programming for copilot to be a life-changing experience.

[AYBABTME]

You haven't used them enough. Everytime an LLM reduces my search from 1min to 5s, the LLM pays.

Just summary features: save me 20min of reading a transcript, turn it into 20s. That's a huge enabler.

[wruza]

Overviews aren’t code though. In code, for me, they don’t pass 80/20 tests well enough, sometimes even on simple cases. (You get 50-80% of an existing function/block with some important context prepended and a comment, let it write the rest and check if it succeeds). It doesn’t mean that LLMs are useless. Or that I am antillamist or a denier - I’m actually an enthusiast. But this specific claim I hear often and don’t find true. Maybe true for repetitive code in boring environments where typing and remembering formats/params over and over is the main issue. Not in actual code.

If I paste the actual non-trivial code, it starts deviating fast. And it isn’t too complex, it’s just less like “parallel sort two arrays” and more like “wait for an image on a screenshot by execing scrot (with no sound) repeatedly and passing the result to this detect-cv2.py script and use all matching options described in this ts type, get stdout json as in this ts type, and if there’s a match, wait for the specified anim timeout and test again to get the settled match coords after an animation finishes; throw after a total timeout”. Not a rocket science, pretty dumb shit, but right there they fall flat and start imagining things, heavily.

I guess it shines if you ask it to make an html form, but I couldn’t call that life-changing unless I had to make these damn forms all day.

[mecsred]

If 20 mins of informations can legitimately be condensed into 20 seconds, it sounds like the original wasn't worth reading in the first place. Could have skipped the llm entirely.

[andrei_says_]

My experience with overviews is that they are often subtly or not so subtly inaccurate. LLMs not understanding meaning or intent carries risk of misrepresentation.

[haakonhr]

And here you're assuming that making software engineers more productive would be a service to the world. I think the jury is out on that one as well. At least for the majority of software engineering since 2010.

[vrighter]

actually, studies seem to show it makes code worse. Just like llms can confidently spout junk, devs using llms confidently check in more bugs.

[Forgeties79]

“A bit”?

[bartread]

Yes, a large amount for - in the grand scheme of things - a short period of time (i.e., a quantity of energy usage in an intense spike that will be dwarfed by energy usage over time) can accurately be described as “a bit”.

Of course, the impact is that AI will continue to become cheaper to use, and induced demand will continue the feedback loop driving the market as a result.

[michaelmrose]

This comment lives in a fictional world where there is a singular group that could have collectively acted counterfactually. In the real world any actor that individually went this route would have gone bankrupt while the others collected money by showing actual results even if ineffeciently earned.

[newyankee]

Also it is likely that the rise of LLMs gave many researchers in allied fields the impetus to tackle with the problems that are relevant to making it more efficient and people stumbled upon a solution hiding there.

The momentum with LLMs and allied technology may last till it keeps on improving even by a few percentage points and keeps shattering human created new benchmarks every few months

[Scene_Cast2]

This is a bit like recommending to skip vacuum tubes, think hard and invent transistors.

[fuzzfactor]

This is kind of thought-provoking.

That is a good correlation when you think about how much more energy-efficient transistors are than vacuum tubes.

Vacuum tube computers were a thing for a while, but it was more out of desperation than systematic intellectual progress.

OTOH you could look at the present accomplishments like it was throwing more vacuum tubes at a problem that can not be adequately addressed that way.

What turned out to be a solid-state solution was a completely different approach from the ground up.

To the extent a more power-saving technique using the same hardware is only a matter of different software approaches, that would be something that realistically could have been accomplished before so much energy was expended.

Even though I've always thought application-specific circuits would be what really helps ML and AI a lot, and that would end up not being the exact same hardware at all.

If power is truly being wasted enough to start rearing its ugly head, somebody should be able to figure out how to fix it before it gets out-of-hand.

Ironically enough with my experience using vacuum tubes, I've felt that there were some serious losses in technology when the research momentum involved was so rapidly abandoned in favor of "solid-state everything" at any cost.

Maybe it is a good idea to abandon the energy-intensive approaches, as soon as anything completely different that's the least bit promising can barely be seen by a gifted visionary to have a glimmer of potential.

[VagabundoP]

That's just not how progress works.

Its iteritive, there are plenty of cul-de-sacs and failures. You can't really optimise until you have something that works and its a messy process that is inefficient.

You're looking at this with hindsight.

[pcl]

Isn’t this paper pretty much about spending a few short years to improve the performance? Or are you arguing that the same people who made breakthroughs over the last few years should have also done the optimization work?

[fuzzfactor]

>the same people who made breakthroughs over the last few years should have also done the optimization work

I never thought it would be ideal if it was otherwise, so I guess so.

When I first considered neural nets from state-of-the art vendors to assist with some non-linguistic situations over 30 years ago, it wasn't quite ready for prime time and I could accept that.

I just don't have generic situations all the time which would benefit me, so it's clearly my problems that have the deficiencies ;\

What's being done now with all the resources being thrown at it is highly impressive, and gaining all the time, no doubt about it. It's nice to know there are people that can afford it.

I truly look forward to more progress, and this may be the previously unreached milestone I have been detecting that might be a big one.

Still not good enough for what I need yet so far though. And I can accept that as easily as ever.

That's why I put up my estimation that not all of those 30+ years has been spent without agonizing over something ;)