[fnordpiglet]

What I find amazing about the original exchange was the profound lack of curiosity Knuth demonstrated. Because the model wasn’t flawless in performance he pinned it as a curiosity that was good at grammar and vacuous otherwise and wasn’t interested to hear how it improves. This reminds me of an awful lot of the computing field in this drama as it plays out. People that literally know how implausible any of these feats have been using traditional approaches immediately discount the entire thing the moment it hallucinates - and it feels like the more deterministic the bent of the person the more absolutely dismissive they are of what’s transpiring in front of us.

These models are doing feats that are stupendous and impossible before their advent. Not just a little bit, but the capability differences are so vast that it’s perhaps not even recognizable by people as being as vast as it is. I am impressed that Wolfram seems to have immediately grasped its significance and is running with it.

The fact this gist demonstrates essentially every single flaw was addressed. But that Knuth apparently doesn’t know / care months after GPT4’s introduction is demonstrative of a different type of personality.

I know which I aspire to be.

[manquer]

What do you expect ? He is one the person in the world who has most the earned the right to take that attitude .

Both Knuth and GPTs are aggregators and presenters of knowledge, Knuth is however the antithesis of a LLM .

He has painstakingly spent years to make sure not a single mistake, not even a typo is there in material he publishes , he has devoted years developing a better typesetting so he can present his material accurately.

His obsession with accuracy is unparalleled and his dedication and mastery over communication to explain complex topics precisely and with an approachability that no one else comes close to .

He has strived for perfection all his life and not been far of the mark .ChatGPT for its all powers will never share that idealogy,

so I am more surprised that he was complimentary at all, and actually appreciated many of its skills

[fnordpiglet]

That’s actually not exactly my point - my point is his lack of curiosity … 3.5’s answered poorly but sounded convincing. But his dismissiveness of the potential and future advances bothered me.

[manquer]

He is 85! I would hope to be that disciplined about what what I can spend time on at that age

He was curious enough to spend some time on it and was worried it would sink more of his time with all the sub problems it is presented and asks specifically Stephan wolfram to disengage on this

He talks about his preference of working with authentic and trustworthy .

Maybe a younger Knuth may have spent more time , but I perhaps think not that likely really .

This is simply not a area of interest for him, he does truly understand the impact and potential - When he talks about novelists not capturing precursors to singularity and how millions of people have access to 0.01 % intelligence for free .

I don’t think he is dismissive of its potential and future , he is not working on everything that can change the world in computing just his areas of interest.

Perhaps you (I am certainly) disappointed that someone of Knuth’s stature is not going to spend time on an emerging field and that’s what really bothers us..

[lupire]

I can't comprehend this comment. Kunths commentary was glowing praise for the AI's thinking ability (and none of the "it's not AI" BS that is so popular), plus a statement that he believes accuracy is more important than raw power, so he wants "you" to work on that. Knuth commented on GPT 4 at the start, and complimented its power and correctness at the end.

[jiggawatts]

I much prefer the attitude of the chap that made the video "GPT 4 is smarter than you think" https://youtu.be/wVzuvf9D9BU

Instead of nit-picking flaws in what is a very early iteration of a revolutionary technology, he instead immediately started exploring ways of making it better and more useful.

Even with minimal effort that was essentially just copy-pasting some text around, he was able to show that the current way we use LLMs like GPT 4 is not the be-all and end-all of this type of technology.

I'm entirely convinced that we're just scratching the surface. It's like the first transistor, which was a crude, ugly, useless thing: https://images.computerhistory.org/siliconengine/1947-1-1.jp...

Just in the last two weeks(!), I've read about the following still-experimental methods for enhancing LLMs:

1. Plugging in "calculators" like Wolfram Alpha.

2. Adding vision input so they can understand equations, graphs, etc...

3. Filtering the output probability vector for certain allowed terms only ("YES", "NO", "MAYBE"), making them more useful in programmatically-invoked scenarios.

4. Similarly, filtering the output token list for syntax-validity, such as "valid JSON", "valid XML", etc... That is, instead of a purely random selection between to "top-n" output tokens, only valid tokens can be chosen, based on contextual syntax.

5. Storing embeddings in a vector database, giving LLMs medium-term memory, and the ability to index and reference sources precisely.

6. Efficient fine-tuning through Low-Rank Adaptation (LoRA), which allows desktop GPUs to tune a model overnight! This overcomes the "stale long-term memory" issue of ChatGPT, which only knows things up to September 2021. It could now read the news daily and "keep up".

7. External script harnesses that run multiple LLMs in parallel, with different prompts and/or different system messages. Some optimised for "idea generation", some optimised for "task completion", and then finally models tuned for "review and verification". Almost like a human team, multiple ideas can be generated, merged, reviewed, planned out, and then actioned. Check out "smol developer", which utilises Anthropic's 100K context window for this: https://www.youtube.com/watch?v=UCo7YeTy-aE

This is just the beginning. Chat GPT 4 hasn't even been available for 3 months yet, and practically all of the above experimentation has been done with weaker models because GPT 4 still doesn't have generally-available API access! Similarly, the 32K context window version of the GPT 4 model isn't available to anyone except a lucky few.

What will 2024 bring!? Heck... what will H2 2023 bring?

[fnordpiglet]

100% agree - the magic comes when you constrain, inform, and integrate them in a feedback cycle with various multimodal inputs and classical optimization, solvers, agents, inference engines, etc. The criticism seems to be that this solution to a problem space doesn’t solve all problem spaces we’ve already done a good job solving and ignoring the fact it solves the spaces we have done a crap job solving. The fact it’s so powerful by itself is amazing. As we integrate it tightly with all the other techniques of the last 80 years of computing the emergent abilities will be mind-blowing. What baffles me is how few people seem to see it clearly.

[cubefox]

And if you look a few years into the future: What will happen in five years from now? Isn't it plausible that we will have another revolution like LLMs? What will they be able to do? Or rather, what won't they be able to do?

What happens if we get strongly superhuman intelligence in just a few years? Is that really so implausible?

[carapace]

It sounds like you profoundly misunderstand Knuth, and LLMs.

I recommend a dose of Mickens: https://www.youtube.com/watch?v=ajGX7odA87k

[fnordpiglet]

I don’t know Knuth. I understand LLMs for precisely what they are, how they’re built, the math behind them, the limits of what they’re doing, and I don’t over estimate the illusion. However while I see people over estimating them I think they’re extrapolating the current state to a state where it’s limits are restricted and augmented with other techniques and models that address their short comings. Lack of agency? We have agent techniques. Lack of consistency with reality? We have information retrieval and semantic inference systems. LLMs bring an unreasonably powerful ability to semantically interpret in a space of ambiguity and approximate enough reasoning and inference to tie together all the pieces we’ve built into an ensemble model that’s so close to AGI that it likely doesn’t matter. People look at LLMs and shake their head failing to realize it’s a single model and single technique that we haven’t even attempted to augment and fail to realize that it’s even possible to augment and constrain LLM with other techniques to address their non trivial failings.

[carapace]

> I don’t know Knuth.

Well you should before taking unwarranted potshots at the man. He's done more for humanity than you or I ever will, eh?

Anyway, you do sound like you know about LLMs, so apologies for that bit.

> People look at LLMs and shake their head failing to realize it’s a single model and single technique that we haven’t even attempted to augment and fail to realize that it’s even possible to augment and constrain LLM with other techniques to address their non trivial failings.

I doubt Knuth is doing that, rather I think the whole thing is orthogonal to his life's work. FWIW, I would love to know his thoughts after reading the GPT4 version of the answers to his questions, eh?

- - - - - -

> I think they’re extrapolating the current state to a state where it’s limits are restricted and [not] augmented with other techniques and models that address their short comings.

I think you might have dropped a negation in that sentence?

> Lack of agency? We have agent techniques. Lack of consistency with reality? We have information retrieval and semantic inference systems. LLMs bring an unreasonably powerful ability to semantically interpret in a space of ambiguity and approximate enough reasoning and inference to tie together all the pieces we’ve built into an ensemble model that’s so close to AGI that it likely doesn’t matter.

I agree! I've been saying for a few minutes now that we'll connect these LLMs to empirical feedback devices and they'll become scientists. Schmidhuber says his goal is "to create an automatic scientist and then retire.", eh?

(FWIW I think there are serious metaphysical ramifications of the pseudo- vs. real- AGI issue, but this isn't the forum for that.)