[livid-neuro]

The first cars broke down all the time. They had a limited range. There wasn't a vast supply of parts for them. There wasn't a vast industry of experts who could work on them. There wasn't a vast network of fuel stations to provide energy for them. The horse was a proven method.

What an LLM cannot do today is almost irrelevant in the tide of change upon the industry. The fact is, with improvements, it doesn't mean an LLM cannot do it tomorrow.

[Night_Thastus]

The difference is that the weaknesses of cars were problems of engineering, and some of infrastructure. Both aren't very hard to solve, though they take time. The fundamental way cars operated worked and just needed revision, sanding off rough edges.

LLMs are not like this. The fundamental way they operate, the core of their design is faulty. They don't understand rules or knowledge. They can't, despite marketing, really reason. They can't learn with each interaction. They don't understand what they write.

All they do is spit out the most likely text to follow some other text based on probability. For casual discussion about well-written topics, that's more than good enough. But for unique problems in a non-English language, it struggles. It always will. It doesn't matter how big you make the model.

They're great for writing boilerplate that has been written a million times with different variations - which can save programmers a LOT of time. The moment you hand them anything more complex it's asking for disaster.

[programd]

> [LLMs] spit out the most likely text to follow some other text based on probability.

Modern coding AI models are not just probability crunching transformers. They haven't been just that for some time. In current coding models the transformer bit is just one part of what is really an expert system. The complete package includes things like highly curated training data, specialized tokenizers, pre and post training regimens, guardrails, optimized system prompts etc, all tuned to coding. Put it all together and you get one shot performance on generating the type of code that was unthinkable even a year ago.

The point is that the entire expert system is getting better at a rapid pace and the probability bit is just one part of it. The complexity frontier for code generation keeps moving and there's still a lot of low hanging fruit to be had in pushing it forward.

> They're great for writing boilerplate that has been written a million times with different variations

That's >90% of all code in the wild. Probably more. We have three quarters of a century of code in our history so there is very little that's original anymore. Maybe original to the human coder fresh out of school, but the models have all this history to draw upon. So if the models produce the boilerplate reliably then human toil in writing if/then statements is at an end. Kind of like - barring the occasional mad genious [0] - the vast majority of coders don't write assembly to create a website anymore.

[0] https://asm32.info/index.cgi?page=content/0_MiniMagAsm/index...

[motorest]

> Modern coding AI models are not just probability crunching transformers. (...) The complete package includes things like highly curated training data, specialized tokenizers, pre and post training regimens, guardrails, optimized system prompts etc, all tuned to coding.

It seems you were not aware you ended up describing probabilistic coding transformers. Each and every single one of those details are nothing more than strategies to apply constraints to the probability distributions used by the probability crunching transformers. I mean, read what you wrote: what do you think that "curated training data" means?

> Put it all together and you get one shot performance on generating the type of code that was unthinkable even a year ago.

This bit here says absolutely nothing.

[leptons]

>The complete package includes things like highly curated training data, specialized tokenizers, pre and post training regimens, guardrails, optimized system prompts etc, all tuned to coding.

And even with all that, they still produce garbage way too often. If we continue the "car" analogy, the car would crash randomly sometimes when you leave the driveway, and sometimes it would just drive into the house. So you add all kinds of fancy bumpers to the car and guard rails to the roads, and the car still runs off the road way too often.

[mgaunard]

Except we should aim to reduce the boilerplate through good design, instead of creating more of it on an industrial scale.

[patrickmay]

I regret that I have but one upvote to give to this comment.

Every time someone says "LLMs are good at boilerplate" my immediate response is "Why haven't you abstracted away the boilerplate?"

[exe34]

what we should and what we are forced to do are very different things. if I can get a machine to do the stuff I hate dealing with, I'll take it every time.

[mgaunard]

who's going to be held accountable when the boilerplate fails? the AI?

[danielbln]

The buck stops with the engineer, always. AI or no AI.

[exe34]

no, I'm testing it the same way I test my own code!

[skydhash]

It's like the xkcd on automation

https://xkcd.com/1205/

After a while, it just make sense to redesign the boilerplate and build some abstraction instead. Duplicated logic and data is hard to change and fix. The frustration is a clear signal to take a step back and take an holistic view of the system.

[gibbitz]

And this is a great example of something I rarely see LLMs doing. I think we're approaching a point where we will use LLMs to manage code the way we use React to manage the DOM. You need an update to a feature? The LLM will just recode it wholesale. All of the problems we have in software development will dissolve in mountains of disposable code. I could see enterprise systems being replaced hourly for security reasons. Less chance of abusing a vulnerability if it only exists for an hour to find and exploit. Since the popularity of LLMs proves that as a society we've stopped caring about quality, I have a hard time seeing any other future.

[Night_Thastus]

>In current coding models the transformer bit is just one part of what is really an expert system. The complete package includes things like highly curated training data, specialized tokenizers, pre and post training regimens, guardrails, optimized system prompts etc, all tuned to coding. Put it all together and you get one shot performance on generating the type of code that was unthinkable even a year ago.

This is lipstick on a pig. All those methods are impressive, but ultimately workarounds for an idea that is fundamentally unsuitable for programming.

>That's >90% of all code in the wild. Probably more.

Maybe, but not 90% of time spent on programming. Boilerplate is easy. It's the 20%/80% rule in action.

I don't deny these tools can be useful and save time - but they can't be left to their own devices. They need to be tightly controlled and given narrow scopes, with heavy oversight by an SME who knows what the code is supposed to be doing. "Design W module with X interface designed to do Y in Z way", keeping it as small as possible and reviewing it to hell and back. And keeping it accountable by making tests yourself. Never let it test itself, it simply cannot be trusted to do so.

LLMs are incredibly good at writing something that looks reasonable, but is complete nonsense. That's horrible from a code maintenance perspective.

[motorest]

> For casual discussion about well-written topics, that's more than good enough. But for unique problems in a non-English language, it struggles. It always will. It doesn't matter how big you make the model.

Not to disagree, but "non-english" isn't exactly relevant. For unique problems, LLMs can still manage to output hallucinations that end up being right or useful. For example, LLMs can predict what an API looks like and how it works even if they do not have the API in context if the API was designed following standard design principles and best practices. LLMs can also build up context while you interact with them, which means that iteratively prompting them that X works while Y doesn't will help them build the necessary and sufficient context to output accurate responses.

[windward]

>hallucinations

This is the first word that came to mind when reading the comment above yours. Like:

>They can't, despite marketing, really reason

They aren't, despite marketing, really hallucinations.

Now I understand why these companies don't want to market using terms like "extrapolated bullshit", but I don't understand how there is any technological solution to it without starting from a fresh base.

[motorest]

> They aren't, despite marketing, really hallucinations.

They are hallucinations. You might not be aware of what that concept means in terms of LLMs but just because you are oblivious to the definition of a concept that does not mean it doesn't exist.

You can learn about the concept by spending a couple of minutes reading this article on Wikipedia.

https://en.wikipedia.org/wiki/Hallucination_(artificial_inte...

> Now I understand why these companies don't want to market using terms like "extrapolated bullshit", (...)

That's literally in the definition. Please do yourself a favour and get acquainted with the topic before posting comments.

[zahlman]

> You might not be aware of what that concept means in terms of LLMs

GP is perfectly aware of this, and disagrees that the metaphor used to apply the term is apt.

Just because you use a word to describe a phenomenon doesn't actually make the phenomenon similar to others that were previously described with that word, in all the ways that everyone will find salient.

When AIs generate code that makes a call to a non-existent function, it's not because they are temporarily mistakenly perceiving (i.e., "hallucinating") that function to be mentioned in the documentation. It's because the name they've chosen for the function fits their model for what a function that performs the necessary task might be called.

And even that is accepting that they model the task itself (as opposed to words and phrases that describe the task) and that they somehow have the capability to reason about that task, which has somehow arisen from a pure language model (whereas humans can, from infancy, actually observe reality, and contemplate the effect of their actions upon the real world around them). Knowing that e.g. the word "oven" often follows the word "hot" is not, in fact, tantamount to understanding heat.

In short, they don't perceive, at all. So how can they be mistaken in their perception?

[windward]

That page was made in December 2022, requires specifying '(artificial intelligence)' and says:

>(also called bullshitting,[1][2] confabulation,[3] or delusion)[4]

Here's the first linked source:

https://www.psypost.org/scholars-ai-isnt-hallucinating-its-b...

[motorest]

> That page was made in December 2022, (...)

Irrelevant. Wikipedia does not create concepts. Again, if you take a few minutes to learn about the topic you will eventually understand the concept was coined a couple of decades ago, and has a specific meaning.

Either you opt to learn, or you don't. Your choice.

> Here's the first linked source:

Irrelevant. Your argument is as pointless and silly as claiming rubber duck debugging doesn't exist because no rubber duck is involved.

[withinboredom]

> Not to disagree, but "non-english" isn't exactly relevant.

how so? programs might use english words but are decidedly not english.

[motorest]

> how so? programs might use english words but are decidedly not english.

I pointed out the fact that the concept of a language doesn't exist in token predictors. They are trained with a corpus, and LLMs generate outputs that reflect how the input is mapped in accordance to how the were trains with said corpus. Natural language makes the problem harder, but not being English is only relevant in terms of what corpus was used to train them.

[mfbx9da4]

How can you tell a human actually understands? Prove to me that human thought is not predicting the most probable next token. If it quacks like duck. In psychology research the only way to research if a human is happy is to ask them.

[alpaca128]

Does speaking in your native language, speaking in a second language, thinking about your life and doing maths feel exactly the same to you?

> Prove to me that human thought is not predicting the most probable next token.

Explain the concept of color to a completely blind person. If their brain does nothing but process tokens this should be easy.

> How can you tell a human actually understands?

What a strange question coming from a human. I would say if you are a human with a consciousness you are able to answer this for yourself, and if you aren't no answer will help.

[randallsquared]

> What a strange question coming from a human.

Oh, I dunno. The whole "mappers vs packers" and "wordcels vs shape rotators" dichotomies point at an underlying truth, which is that humans don't always actually understand what they're talking about, even when they're saying all the "right" words. This is one reason why tech interviewing is so difficult: it's partly a task of figuring out if someone understands, or has just learned the right phrases and superficial exercises.

[0xak]

That is ill-posed. Take any algorithm at all, e.g. a TSP solver. Make a "most probable next token predictor" that takes the given traveling salesman problem, runs the solver, and emits the first token of the solution, then reruns the solver and emits the next token, and so on.

By this thought experiment you can make any computational process into "predict the most probable next token" - at an extreme runtime cost. But if you do so, you arguably empty the concept "token predictor" of most of its meaning. So you would need to more accurately specify what you mean by a token predictor so that the answer isn't trivially true (for every kind of thought that's computation-like).

[exe34]

I take it you haven't tried an LLM in a few years?

[Night_Thastus]

Just a couple of weeks ago on mid-range models. The problem is not implementation or refinement - the core idea is fundamentally flawed.

[nativeit]

The problem is we’re now arguing with religious zealots. I am not being sarcastic.

[gibbitz]

I feel this way about Typescript too. There are a lot of people in engineering these days who don't think critically or exercise full observation when using popular technologies. I don't feel like it was like this 15 years ago, but it probably was...

[oinfoalgo]

I actually don't know if you are referring to anti-LLM/"ai slop" software engineers or irrationally bullish LLM "the singularity is near" enthusiast.

Religious fervor in one's own opinion on the state of the world seems to be the zeitgeist.

[exe34]

that's correct. those who believe only carbon can achieve intelligence.

[windward]

This stops being an interesting philosophical problem when you recognise the vast complexity of animal brains that LLMs fail to replicate or substitute.

[shkkmo]

If you stereotype the people who disagree with you, you'll have a very hard time understanding their actual arguments.

[melagonster]

Yes, Carbon do not give them human rights.

[exe34]

why not the top few? mid-range is such a cop out if you're going to cast doubt.

[Night_Thastus]

Realistically, most people are not going to be on top end models. They're expensive and will get far, far FAR more expensive once these companies feel they're sufficiently entrenched enough to crank up pricing.

It's basically the silicon valley playbook to offer a service for dirt cheap (completely unprofitable) and then once they secure the market they make skyrocket the price.

A mid range model is what most people will be able to use.

[exe34]

If you want to know what's possible, you look at the frontier models. If you want it cheap, you wait a year and it'll get distilled into the cheaper ones.

[bitwize]

> LLMs are not like this. The fundamental way they operate, the core of their design is faulty. They don't understand rules or knowledge. They can't, despite marketing, really reason. They can't learn with each interaction. They don't understand what they write.

Said like a true software person. I'm to understand that computer people are looking at LLMs from the wrong end of the telescope; and that from a neuroscience perspective, there's a growing consensus among neuroscientists that the brain is fundamentally a token predictor, and that it works on exactly the same principles as LLMs. The only difference between a brain and an LLM maybe the size of its memory, and what kind and quality of data it's trained on.

[Night_Thastus]

>from a neuroscience perspective, there's a growing consensus among neuroscientists that the brain is fundamentally a token predictor, and that it works on exactly the same principles as LLMs

Hahahahahaha.

Oh god, you're serious.

Sure, let's just completely ignore all the other types of processing that the brain does. Sensory input processing, emotional regulation, social behavior, spatial reasoning, long and short term planning, the complex communication and feedback between every part of the body - even down to the gut microbiome.

The brain (human or otherwise) is incredibly complex and we've barely scraped the surface of how it works. It's not just nuerons (which are themselves complex), it's interactions between thousands of types of cells performing multiple functions each. It will likely be hundreds of years before we get a full grasp on how it truly works - if we ever do at all.

[N_Lens]

Reminds me of that old chestnut - “if the human brain were simpler to understand, we wouldn’t be smart enough to do so”

[fzeroracer]

> The only difference between a brain and an LLM maybe the size of its memory, and what kind and quality of data it's trained on.

This is trivially proven false, because LLMs have far larger memory than your average human brain and are trained on far more data. Yet they do not come even close to approximating human cognition.

[alternatex]

>are trained on far more data

I feel like we're underestimating how much data we as humans are exposed to. There's a reason AI struggles to generate an image of a full glass of wine. It has no concept of what wine is. It probably knows way more theory about it than any human, but it's missing the physical.

In order to train AIs the way we train ourselves, we'll need to give it more senses, and I'm no data scientist but that's presumably an inordinate amount of data. Training AI to feel, smell, see in 3D, etc is probably going to cost exponentially more than what the AI companies make now or ever will. But that is the only way to make AI understand rather than know.

We often like to state how much more capacity for knowledge AI has than the average human, but in reality we are just underestimating ourselves as humans.

[gibbitz]

I think this conversation is dancing around the relationship of memory and knowledge. Simply storing information is different than knowing it. One of you is thinking book learning while the other is thinking street smarts.

[zahlman]

> and that from a neuroscience perspective, there's a growing consensus among neuroscientists that the brain is fundamentally a token predictor, and that it works on exactly the same principles as LLMs

Can you cite at least one recognized, credible neuroscientist who makes this claim?

[imtringued]

Look you don't have to lie at every opportunity you get. You are fully aware and know what you've written is bullshit.

Tokens are a highly specific transformer exclusive concept. The human brain doesn't run a byte pair encoding (BPE) tokenizer [0] in their head. anything as tokens. It uses asynchronous time varying spiking analog signals. Humans are the inventors of human languages and are not bound to any static token encoding scheme, so this view of what humans do as "token prediction" requires either a gross misrepresentation of what a token is or what humans do.

If I had to argue that humans are similar to anything in machine learning research specifically, I would have to argue that they extremely loosely follow the following principles:

* reinforcement learning with the non-brain parts defining the reward function (primarily hormones and pain receptors)

* an extremely complicated non-linear kalman filter that not only estimates the current state of the human body, but also "estimates" the parameters of a sensor fusing model

* there is a necessary projection of the sensor fused result that then serves as available data/input to the reinforcement learning part of the brain

Now here are two big reasons why the model I describe is a better fit:

The first reason is that I am extremely loose and vague. By playing word games I have weaseled myself out of any specific technology and am on the level of concepts.

The second reason is that the kalman filter concept here is general enough that it also includes predictor models, but the predictor model here is not the output that drives human action, because that would logically require the dataset to already contain human actions, which is what you did, you assume that all learning is imitation learning.

In my model, any internal predictor model that is part of the kalman filter is used to collect data, not drive human action. Actions like eating or drinking are instead driven by the state of the human body, e.g. hunger is controlled through leptin and insulin and others. All forms of work, no matter how much of a detour it represents, ultimately has the goal of feeding yourself or your family (=reproduction).

[0] A BPE tokenizer is a piece of human written software that was given a dataset to generate an efficient encoding scheme and the idea itself is completely independent of machine learning and neural networks. The fundamental idea behind BPE is that you generate a static compression dictionary and never change it.

[zahlman]

> Look you don't have to lie at every opportunity you get. You are fully aware and know what you've written is bullshit.

As much as I may agree with your subsequent claims, this is not how users are expected to engage with each other on HN.

[N_Lens]

You seem to be an LLM

[jerf]

AI != LLM.

We can reasonably speak about certain fundamental limitations of LLMs without those being claims about what AI may ever do.

I would agree they fundamentally lack models of the current task and that it is not very likely that continually growing the context will solve that problem, since it hasn't already. That doesn't mean there won't someday be an AI that has a model much as we humans do. But I'm fairly confident it won't be an LLM. It may have an LLM as a component but the AI component won't be primarily an LLM. It'll be something else.

[xenadu02]

Every AI-related invention is hyped as "intelligence" but turns out to be "Necessary but Not Sufficient" for true intelligence.

Neural networks are necessary but not sufficient. LLMs are necessary but not sufficient.

I have no doubt that there are multiple (perhaps thousands? more?) of LLM-like subsystems in our brains. They appear to be a necessary part of creating useful intelligence. My pet theory is that LLMs are used for associative memory purposes. They help generate new ideas and make predictions. They extract information buried in other memory. Clearly there is another system on top that tests, refines, and organizes the output. And probably does many more things we haven't even thought to name yet.

[Ferret7446]

Most adult humans don't have "true intelligence" so I don't quite get the point

[Jensson]

What do you mean? Most adult humans can learn to drive a car, book a plain ticket, get a passport, fly abroad, navigate in a foreign country etc. There is a variation in human intelligence, but almost all humans are very intelligent compared to everything else we know about.

[JackFr]

> Every AI-related invention is hyped as "intelligence" but turns out to be "Necessary but Not Sufficient" for true intelligence.

Alternatively, the goalposts keep being moved.

[ezst]

Not really, only "merchants" are trying to package and sell LLMs as "artificial intelligence". To this day AI still very much is the name of a research field focused on computational methods: it's not a discovery, it's not a singular product or tool at or disposal (or it is in no greater capacity than Markov chains, support vector machines or other techniques that came before). If you ever expect the goalposts to settle, you are essentially wishing for research to stop.

[ithkuil]

Both things can be true:

1. People are trying to sell a product that is not ready and thus are overhyping it

2. The tech is in its early days and may evolve into something useful via refinement and not necessarily by some radical paradigm shift

In order for (2) to happen it helps if the field is well motivated and funded (1)

[xenadu02]

I don't think we fully understand all the aspects of intelligence. What the potential feature set is. How to categorize or break it down into parts. We have some data and some categories but we are so far away from a full description that it only makes sense we must move the goalposts constantly.

[lbrandy]

> has a model much as we humans do

The premise that an AI needs to do Y "as we do" to be good at X because humans use Y to be good at X needs closer examination. This presumption seems to be omnipresent in these conversations and I find it so strange. Alpha Zero doesn't model chess "the way we do".

[klabb3]

Both that, and that we should not expect LLMs to achieve ability with humans as baseline comparison. It’s as if cars were rapidly getting better due to some new innovation, and expecting them to fly within a year. It’s a new, and different thing, where the universality of ”plausibly sounding” coherent text appeared to be general, when it’s advanced pattern matching. Nothing wrong with that, pattern matching is extremely useful, but drawing the equal sign to human cognition is extremely premature, and a bet that is very likely be wrong.

[shkkmo]

Alpha Zero is not trying to be AGI.

> The premise that an AI needs to do Y "as we do" to be good at X because humans use Y to be good at X needs closer examination.

I don't see it being used as a premise. It see it as speculation that is trying to understand why this type of AI underperforms at certain types of tasks. Y may not be necessary to do X well, but if a system is doing X poorly and the difference between that system and another system seems to be Y, it's worth exploring if adding Y would improve the performance.

[byteknight]

I have to disagree. Anyone that says LLMs do not qualify as AI are the same people who will continue to move the goal posts for AGI. "Well it doesn't do this!". No one here is trying to replicate a human brain or condition in its entirety. They just want to replicate the thinking ability of one. LLMs represent the closest parallel we have experienced thus far to that goal. Saying that LLMs are not AI feel disingenuous at best and entirely purposely dishonest at the worst (perhaps perceived as staving off the impending demise of a profession).

The sooner people stop worrying about a label for what you feel fits LLMs best, the sooner they can find the things they (LLMs) absolutely excel at and improve their (the user's) workflows.

Stop fighting the future. Its not replacing right now. Later? Maybe. But right now the developers and users fully embracing it are experiencing productivity boosts unseen previously.

Language is what people use it as.

[sarchertech]

> the developers and users fully embracing it are experiencing productivity boosts unseen previously

This is the kind of thing that I disagree with. Over the last 75 years we’ve seen enormous productivity gains.

You think that LLMs are a bigger productivity boost than moving from physically rewiring computers to using punch cards, from running programs as batch processes with printed output to getting immediate output, from programming in assembly to higher level languages, or even just moving from enterprise Java to Rails?

[skydhash]

Even learning your current $EDITOR and $SHELL can be a great productivity booster. I see people claiming AI is helping them and you see them hunting for files in the file manager tree instead of using `grep` or `find` (Unix).

[Espressosaurus]

Or the invention of the container, or hell, the invention of the filing cabinet (back when computer was a job)

[overgard]

The studies I've seen for AI actually improving productivity are a lot more modest than what the hype would have you believe. For example: https://www.youtube.com/watch?v=tbDDYKRFjhk

Skepticism isn't the same thing as fighting the future.

I will call something AGI when it can reliably solve novel problems it hasn't been pre-trained on. That's my goal post and I haven't moved it.

[jerf]

!= is "not equal". The symbol for "not a subset of" is ⊄, which you will note, I did not use.

[byteknight]

I think you replied in the wrong place, bud. All the best.

EDIT - I see now. sorry.

For all intents and purposes of the public. AI == LLM. End of story. Doesn't matter what developers say.

[marcus_holmes]

> For all intents and purposes of the public. AI == LLM. End of story. Doesn't matter what developers say.

This is interesting, because it's so clearly wrong. The developers are also the people who develop the LLMs, so obviously what they say is actually the factual matter of the situation. It absolutely does matter what they say.

But the public perception is that AI == LLM, agreed. Until it changes and the next development comes along, when suddenly public perception will change and LLMs will be old news, obviously not AI, and the new shiny will be AI. So not End of Story.

People are morons. Individuals are smart, intelligent, funny, interesting, etc. But in groups we're moronic.

[leptons]

So when an LLM all-too-often produces garbage, can we then call it "Artificial Stupidity"?

[byteknight]

Not sure how that fits. Do you produce good results every time, first try? Didn't think so.

[leptons]

>Do you produce good results every time, first try?

Almost always, yes, because I know what I'm doing and I have a brain that can think. I actually think before I do anything, which leads to good results. Don't assume everyone is a junior.

>Didn't think so.

You don't know me at all.

[neoromantique]

Sr. "human" here.

If you always use your first output then you are not a senior engineer, either your problem space is THAT simple that you can fit all your context in your head at the same time first try, or quite frankly you just bodge things together in non-optimal way.

It always takes some tries at a problem to grasp edge cases and to easier visualize the problem space.

[danielbln]

Here you have it folks, seniors don't make mistakes.

[oinfoalgo]

In cybernetics, this label has existed for a long time.

Unfortunately, discourse has followed an epistemic trajectory influenced by Hollywood and science fiction, making clear communication on the subject nearly impossible without substantial misunderstanding.

[parineum]

> Anyone that says LLMs do not qualify as AI are the same people who will continue to move the goal posts for AGI.

I have the complete opposite feeling. The layman understanding of the term "AI" is AGI, a term that only needs to exist because researchers and businessmen hype their latest creations as AI.

The goalposts for AI don't move but the definition isn't precise but we know it when we see it.

AI, to the layman, is Skynet/Terminator, Asimov's robots, Data, etc.

The goalposts moving that you're seeing is when something the tech bubble calls AI escapes the tech bubble and everyone else looks at it and says, no, that's not AI.

The problem is that everything that comes out of the research efforts toward AI, the tech industry calls AI despite it not achieving that goal by the common understanding of the term. LLMs were/are a hopeful AI candidate but, as of today, they aren't but that doesn't stop OpenAI from trying to raise money using the term.

[shkkmo]

AI has had many, many lay meanings over the years. Simplistic decision trees and heuristics for video games is called AI. It is a loose term and trying to apply it with semantic rigour is useless, as is trying to tell people that it should only be used to match one of its many meanings.

If you want some semantic rigour use more specific terms like AGI, human equivalent AGI, super human AGI, exponentially self improving AGI, etc. Even those labels lack rigour, but at least they are less ambiguous.

LLMs are pretty clearly AI and AGI under commonly understood, lay definitions. LLMs are not human level AGI and perhaps will never be by themselves.

[parineum]

> LLMs are pretty clearly AI and AGI under commonly understood, lay definitions.

That's certainly not clear. For starters, I don't think there is a lay definition of AGI which is largely my point.

The only reason people are willing to call LLMs AI is because that's how they are being sold and the shine isn't yet off the rose.

How many people call Siri AI? It used to be but people have had time to feel around the edges where it fails to meet their expectations of AI.

You can tell what people think of AI by the kind of click bait surrounding LLMs. I read an article not too long ago with the headline about an LLM lying to try and not be turned off. Turns out it was intentionally prompted to do that but the point is that that kind of self preservation is what people expect of AI. Implicitly, they expect that AI has a "self".

ChatGPT doesn't have a self.

[shkkmo]

AI and AGI are broad umbrella terms. Stuff like Alpha Zero is AI but not AGI while LLMs are both.

Engaging in semantic battles to try to change the meanings of those terms is just going to create more confusion, not less. Instead why not use more specific and descriptive labels to be clear about what you are saying.

Self-Aware AGI, Human Level AGI, Super-Human ANI, are all much more useful than trying to force general label to be used a specific way.

[byteknight]

"Just ask AI" is a phrase you will hear around enterprises now. You less often hear "Google it". You hear "ChatGPT it".

[imiric]

> The sooner people stop worrying about a label for what you feel fits LLMs best, the sooner they can find the things they (LLMs) absolutely excel at and improve their (the user's) workflows.

This is not a fault of the users. These labels are pushed primarily by "AI" companies in order to hype their products to be far more capable than they are, which in turn increases their financial valuation. Starting with "AI" itself, "superintelligence", "reasoning", "chain of thought", "mixture of experts", and a bunch of other labels that anthropomorphize and aggrandize their products. This is a grifting tactic old as time itself.

From Sam Altman[1]:

> We are past the event horizon; the takeoff has started. Humanity is close to building digital superintelligence

Apologists will say "they're just words that best describe these products", repeat Dijkstra's "submarines don't swim" quote, but all of this is missing the point. These words are used deliberately because of their association to human concepts, when in reality the way the products work is not even close to what those words mean. In fact, the fuzzier the word's definition ("intelligence", "reasoning", "thought"), the more valuable it is, since it makes the product sound mysterious and magical, and makes it easier to shake off critics. This is an absolutely insidious marketing tactic.

The sooner companies start promoting their products honestly, the sooner their products will actually benefit humanity. Until then, we'll keep drowning in disinformation, and reaping the consequences of an unregulated marketplace of grifters.

[1]: https://blog.samaltman.com/the-gentle-singularity

[skydhash]

When the first cars broke down, people were not saying: One day, we’ll go to the moon with one of these.

LLMs may get better, but it will not be what people are clamoring them to be.

[serf]

>When the first cars broke down, people were not saying: One day, we’ll go to the moon with one of these.

maybe they should have; a lot of the engineering techniques and methodologies that produced the assembly line and the mass produced vehicle also lead the way into space exploration.

[tobr]

The article has a very nuanced point about why it’s not just a matter of today’s vs tomorrow’s LLMs. What’s lacking is a fundamental capacity to build mental models and learn new things specific to the problem at hand. Maybe this can be fixed in theory with some kind of on-the-fly finetuning, but it’s not just about more context.

[ako]

You can give it some documents, or classroom textbooks, and it can turn those into rdf graphs, explaining what the main concepts are, and how they are related. This can then be used by an llm to solve other problems.

It can also learn new things using trial and error with mcp tools. Once it has figured out some problem, you can ask it to summarize the insights for later use.

What would define as an AI mental model?

[tobr]

I’m not an expert on this, so I’m not familiar with what RDF graphs are, but I feel like everything you’re describing happens textually, and used as context? That is, it’s not at all ”learning” the way it’s learning during training, but by writing things down to refer to them later? As you say - ”ask it to summarize the insights for later use” - this is fundamentally different from the types of ”insights” it can have during training. So, it can take notes about your code and refer back to them, but it only has meaningful ”knowledge” about code it came across in training.

To me as a layman, this feels like a clear explanation of how these tools break down, why they start going in circles when you reach a certain complexity, why they make a mess of unusual requirements, and why they have such an incredible nuanced grasp of complex ideas that are widely publicized, while being unable to draw basic conclusions about specific constraints in your project.

[ako]

To me it feels very much like a brain: my brain often lacks knowledge, but i can use external documents to augment it. My brain also has limitations in what it can remember, I hardly remember anything I learned in high school or university on science, chemistry, math, so I need to write things down to bring back knowledge later.

Text and words are the concepts we use to transfer knowledge in schools, across generations, etc. we describe concepts in words, so other people can learn these concepts.

Without words and text we would be like animals unable to express and think about concepts

[tobr]

The point isn’t that writing and reading aren’t useful. The point is that they’re different from forming new neurological connections as you familiarize yourself with a problem. LLMs, as far as I know, can’t do that when you use them.

[ako]

Does that really matter if the result is the same, they have a brain, they have additional instructions, and with these they can achieve specified outcomes. Would be interesting to see how far we can shrink the brains to get desired outcomes with the right instructions.

[shalmanese]

The analogy is very apt because the first cars:

* are many times the size of the occupants, greatly constricting throughput.

* are many times heavier than humans, requiring vastly more energy to move.

* travel at speeds and weights that are danger to humans, thus requiring strictly segregated spaces.

* are only used less than 5% of the day, requiring places to store them when unused.

* require extremely wide turning radiuses when traveling at speed (there’s a viral photo showing the entire historical city of Florence fit inside a single US cloverleaf interchange)

Not only have none of these flaws been fixed, many of them have gotten worse with advancing technology because they’re baked into the nature of cars.

Anyone at the invention of automobiles with sufficient foresight could have seen the intersecting incentives that cars would wreak, same as how many of the future impacts of LLMs are foreseeable today, independent of technical progress.

[oblio]

> Anyone at the invention of automobiles with sufficient foresight could have seen the intersecting incentives that cars would wreak, same as how many of the future impacts of LLMs are foreseeable today, independent of technical progress.

Yeah, but where's the money to be made in not selling people stuff?

https://imgur.com/few-shareholders-had-good-value-least-jpsP...

[dml2135]

This is like saying that because of all the advancements that automobiles have made, teleportation is right around the corner.

[brandon272]

The question is, when is “tomorrow”?

Dismissing a concern with “LLMs/AI can’t do it today but they will probably be able to do it tomorrow” isn’t all that useful or helpful when “tomorrow” in this context could just as easily be “two months from now” or “50 years from now”.

[windward]

How do you differentiate between tech that's 'first cars' and tech that's 'first passenger supersonic aircraft'?

[card_zero]

When monowheels were first invented, they were very difficult to steer due to the gyroscopic effects inherent to a large wheel model (LWM).

[aaroninsf]

My preferred formulation is Ximm's Law,

"Every critique of AI assumes to some degree that contemporary implementations will not, or cannot, be improved upon.

Lemma: any statement about AI which uses the word "never" to preclude some feature from future realization is false.

Lemma: contemporary implementations have almost always already been improved upon, but are unevenly distributed."

[moregrist]

Replace “AI” with “fusion” and you immediately see the problem: there’s no concept of timescale or cost.

And with fusion, we already have a working prototype (the Sun). And if we could just scale our tech up enough, maybe we’d have usable fusion.

[dpatterbee]

Heck, replace "AI" with almost any noun and you can close your eyes to any and all criticism!

[gjm11]

Only to criticism of the form "X can never ...", and some such criticism richly deserves to be ignored.

(Sometimes that sort of criticism is spot on. If someone says they've got a brilliant new design for a perpetual motion machine, go ahead and tell them it'll never work. But in the general case it's overconfident.)

[latexr]

> Every critique of AI assumes to some degree that contemporary implementations will not, or cannot, be improved upon.

That is too reductive and simply not true. Contemporary critiques of AI include that they waste precious resources (such as water and energy) and accelerate bad environmental and societal outcomes (such as climate change, the spread of misinformation, loss of expertise), among others. Critiques go far beyond “hur dur, LLM can’t code good”, and those problems are both serious and urgent. Keep sweeping critiques under the rug because “they’ll be solved in the next five years” (eternally away) and it may be too late. Critiques have to take into account the now and the very real repercussions already happening.

[antod]

Agreed. I find LLMs incredibly useful for my work and I'm amazed at what they can do.

But I'm really worried that the benefits are very localized, and that the externalized costs are vast, and the damage and potential damage isn't being addressed. I think that they could be one of the greatest ever drivers of inequality as a privileged few profit at the expense of the many.

Any debates seem neglect this as they veer off into AGI Skynet fantasy land damage rather than grounded real world damage. This seems to be deliberate distraction.

[ajuc]

It also doesn't mean they can. LLMs may be the steam-powered planes of our times.

A crucial ingredient might be missing.

[jedimastert]

> The first cars broke down all the time. They had a limited range. There wasn't a vast supply of parts for them. There wasn't a vast industry of experts who could work on them.

I mean, there was and then there wasn't. All of those things are shrinking fast because we handed over control to people who care more about profits than customers because we got too comfy and too cheap, and now right to repair is screwed.

Honestly, I see llm-driven development as a threat to open source and right to repair, among the litany of other things

[apwell23]

ugh.. no analogies pls

[ants_everywhere]

The anti-LLM chorus hates when you bring up the history of technological change