[xpe]

I welcome any substantive commentary and disagreement, as always.

I’m happy to stray outside the herd. HN needs more clearly articulated disagreement regarding AI regulation. I made my comment in response to what seemed like a simplistic, ideology-driven claim. Few wise hackers would make an analogous claim about a system they actually worked on. Thinking carefully about tech but phoning it in for other topics is a double standard.

Bare downvotes don't indicate (much less explain) one's rationale. I can’t tell if I (1) struck a nerve (an emotional response); (2) conjured a contentious philosophy (i.e. we have a difference in values or preferences or priorities), (3) made a logical error, (4) broke some norm or expectation, or (5) something else. Such a conflation of downvotes pushes me away from HN for meaningful discussion.

I’ve lived/worked on both coasts, Austin, and more, and worked at many places (startups, academic projects, research labs, gov't, not-for-profits, big tech) and I don’t consider myself defined by any one place or culture. But for the context of AI regulation, I have more fundamental priorities than anything close to "technical innovation at all costs".

P.S. (1) If a downvote here is merely an expression of “I’m a techno-libertarian” or "how dare you read someone's HN profile page and state the obvious?" or any such shallow disagreement, then IMO that’s counterproductive. If you want to express your viewpoint, make it persuasive rather than vaguely dismissive with an anonymous, unexplained downvote. (2) Some people do the thing where they guess at why someone else downvoted. That’s often speculation.

[srj]

FWIW I didn't downvote you. I don't work on AI personally, and while I have no way of proving it to you I certainly am not trying to shill for my employer.

My skepticism of AI safety is just because of skepticism of AI generally. These are amazing things, but I don't believe the technology is even a road to AGI. There's a reason it can give a chess move when prompted and explain all the rules and notation, but can't actually play chess: it's not in the training data. I simply think the hype and anxiety is unnecessary, is my issue. Now this is most definitely just my opinion and has nothing to do with that company I work for who I'd bet would disagree with me on all of this anyway. If I did believe this was a road to AGI I actually would be in favor of AI safety regulation.

[xpe]

> My skepticism of AI safety is just because of skepticism of AI generally. These are amazing things, but I don't believe the technology is even a road to AGI.

Thanks for your response. I'm curious how to state your claim in a way that you would feel is accurate. Would you say "LLMs are not a road to AGI"?

I put ~zero weight on what an arbitrary person believes until they clarify their ideas, show me their model, and give me a prediction. So:

- Clarify: What exactly do you mean by "a road to"? Does this mean you are saying any future technology that uses LLMs (for training? for inference? something else) won't assist the development of AGI?

- Model: On what model(s) of how the world works do you make your claims?

- Prediction: If you are right, when will we know and what will be observe?

[srj]

Yes I'm talking about LLMs in particular. I'm in the stochastic parrot camp. Though I could be convinced humans are no more than stochastic parrots, in which case it does have a path for development of AGI.

If I'm right the breakthroughs will plateau even while applications of the technology continue to advance for the next several years.

[xpe]

Here is my take. When people use the stochastic parrots phrase, very often they use it as an explanation of what is happening. But in many cases, I don't think they appreciate: (1) good explanations must be testable models; (2) different explanations exist at different levels of abstraction; (3) having one useful level of explanation does not mean that other levels of explanation are not accurate nor useful.

Sure, optimization based on predicting the next word is indeed the base optimizer for LLMs. This doesn't prevent the resulting behavior from demonstrating behavior that corresponds with some measurable levels of intelligence, as in problem-solving in particular domains! Nor does it prevent fine-tuning from modifying the LLMs behavior considerably.

One might say e.g. "LLMs only learn to predict the next word." The word only is misleading. Yes, models learn to predict the next word, and they build a lot of internal structures to help them do that. These structures enable capabilities much greater than merely parroting text. This is a narrow claim, but it is enough to do serious damage to the causal wielder of the "stochastic parrots" phrase. (To be clear, I'm not making any claims about consciousness or human-anchored notions of intelligence.)

[xpe]

> I'm in the stochastic parrot camp.

If you wouldn't mind doing me a favor?... For a few minutes, can we avoid this phrase? I don't know what people really mean by it.* Can you in plain English translate your view into sentence of theses form:

1. "Based on my understanding of LLMs, X1 and X2 are impossible."

2. "This leads to predictions such as P1 and P2."

3. "If I observed X3 or X4, it would challenge my current beliefs of LLMs."

* I've read the original stochastic parrots paper. In my view, the paper does not match how many people talk about it. Quite the opposite. It is likely many people name drop it but haven't read it carefully. I may have some misinterpretations, sure, but at least I'm actively seeking to question them.

[xpe]

> There's a reason it can give a chess move when prompted and explain all the rules and notation, but can't actually play chess: it's not in the training data.

I don't understand how you can claim an LLM can't play chess. Just as one example, see: https://dynomight.net/chess/