| HN Mirror

Sure, as I said, a better search can bring pretty dramatic change in how you work.

> Every line of code I commit is generated by LLMs...

Imagine if someone told you "Hey, this stackoverflow site is great. Everything I commit in my work is copy pasted from it!". What would you think about their work? Is that something worth bragging about?

mgfist 21 days ago

Those are 2 different things so yeah I naturally would have a different reaction.

snapcaster 21 days ago

i'm imagining this, and don't see how it has any relation to the discussion

No, no, you don't get it. See, it's just like a text search, if a text search could return text that never existed before, and that solves original math problems, and answer your emails for you, ... and ...

kakacik 21 days ago

But most of the stuff it returns existed before, and llm's parent company during their training part stole all that info, legal or not who cares right. The rest is combined in sort of least-resistance-path which can produce impressive results but its not what you wrote. Many people don't actually care much about morality in their lives only when its convenient for them, and this is a prime example of such tunnel vision.

Start with clean llm, no external previous ideas of humans inserted into it, and let it generate some wisdom on its own and then lets talk. (btw thats how I would expect we could get closer to AGI with these statistical models, but thats just my opinion)

But most of the stuff it returns existed before

No, it did not.

Start with clean llm, no external previous ideas of humans inserted into it, and let it generate some wisdom on its own and then lets talk.

An LLM is about as likely to do that as you are. The ability to generate "wisdom" ab initio cannot possibly be a criterion for intelligent reasoning. The ability to arrive at novel mathematics proofs, on the other hand, is good enough for me.

Intelligence means making the most of the resources and information available, not the ability to speedrun the Big Bang. LLMs are certainly smarter than humans who dismiss them as "text searchers."

>No, it did not.

It did. In the form of a pattern that humans are too incapable of recongizing. LLMs identify and repeat the pattern. That is all.

For example, if A -> B and B -> C, logic dictates A -> C. But LLMs will be able to state that A -> C, without actually using logic, if there is sufficient statements in its training data that says A -> B and B -> C and A -> C. So now if you say P -> Q and Q -> R, it will say that P -> R, when there is no explict P -> R in the training data and NOT using logic. For you, it looks like a new discovery inferred using logic when it is not. But that is how that happens..

It is just pattern recognition masquarading as logic, x, y, or z.

And it doesn't bother you in the slightest that "masquerading" is enough to take gold at IMO?

You're not feeling even the faintest stirrings of cognitive dissonance when a "text search" solves math problems you couldn't solve yourself?

Ever heard of "prompt injection" attacks?

This "super intelligent" and "capable" thing cannot even understand that your ssh keys are private and should not be sent to randos. It can solve complex math, but does not understand basic security/privacy.

What does that say to you?

This "super intelligent" and "capable" thing cannot even understand that your ssh keys are private and should not be sent to randos.

When somebody posts their private keys to Github, it's usually a human. Enough said.

(And if you had ever used Claude Code, you'd know that it nags you endlessly about key hygiene.)

maybe_pablo 21 days ago

Ever heard of social engineering? Also, models nowadays are way sharper than they were even a year ago. They’re not going to make stupid mistakes like that unless you basically ask them to. GPT-5.x for example would bend over backwards to avoid even reading your passwords into context.

> Ever heard of social engineering?

Oh wait, I thought these things were super smart. I didn't expect "social engineering" to work on them.

> models nowadays are way sharper than they were even a year ago.

You are missing the point. If the thing can solve complex math problems and at the same time be so dumb as to fall for "social engineering", then that means that it is not "smartness" or "reasoning" that is helping it to solve those problems. Just some form of advanced, but yet dumb, search algorithm.

maybe_pablo 21 days ago

By "heard of social engineering?" I meant that humans are vulnerable to malicious input too. Prompt injection is basically a simplified form of social engineering for language models. It looks different because models operate over much smaller and more explicit contexts than humans do and are explicitly trained to follow instructions, but the general idea is similar: malicious input tries to manipulate how the system interprets trust and instructions. This is why we need protocols, permissions, and opsec for both agents and humans. That said, I’m not criticizing how you choose to use, or not use, these models, though.

>I meant that humans are vulnerable to malicious input too.

No they are not. Social engineering won't work on a human security expert who knows and understands the implications of the information they are giving away. Your analogy is pointless.

retsibsi 21 days ago

> If the thing can solve complex math problems and at the same time be so dumb as to fall for "social engineering", then that means that it is not "smartness" or "reasoning" that is helping it to solve those problems. Just some form of advanced, but yet dumb, search algorithm.

I'm not just trying to be snarky, but I have no idea how to read this without taking the implication that humans are advanced, yet dumb, search algorithms.

A human being who states X (implying they know it to be true) will behave in a way that is consistent with X being true.

An LLM will happily say X and behaves in contradiction to X. Because it does not reason. Its behavior is not derived from things that it claim (or appears) to know.