[ben_w]

Eventually, that becomes interesting. Before we get to that stage, I think we pass through (almost) everyone following almost the exact same advice as each other.

Such people are extremely predictable.

You may have already noticed, before LLMs, cliché? Talking points that make a group identifiable? Words and phrases that act as applause lights or cognitive stop signs? (That last sentence itself being a pair of clichés that you can use to identify where I hang out online).

Anyway, point is, LLMs will give us a memetic monoculture before they turn us all into a world of correctly personalised Whispering Earring wearers. That makes them predictable, that makes them exploitable. It'll be like playing chess against someone you know is using a specific version of Stockfish: even though it would beat you if you tried to fight the system unaided, you can win by asking your own copy of the AI to go one step further ahead, and it will be accurate precisely because it's playing against itself and reacting to its own moves.

(Of course, the fact I've said this in writing means this is in the training data; in the general case this means the LLMs will know that and account for that, but I suspect comments like this won't shift the needle all that much compared to the aggregate output of 3 billion people reacting to short-form emotional manipulation A/B slop)

[csh0]

This is already underway.

My dog often gets misidentified as a restricted breed. This used to make apartment hunting difficult because, occasionally, the property manager would visually ID the dog breed as banned, I’d have to go to the vet and get paperwork, potentially gene testing, arguing she wasn't, it was a whole thing.

But, recently, the apartment I moved into had an online portal where I had to upload a photo and it would identify the breed to determine if it was approved.

I correctly assumed the portal was using an LLM for this purpose. I wrote a script which submitted different photos of my dog to the major LLM providers until it found a photo which all the LLMs would identify as the correct breed.

I simply submitted that photo and, as expected, passed with flying colors.

[ForOldHack]

I was anticipating that it would have reversed course and flagged you. Blessings on you, and your four footed friend.

[randcraw]

Or you could just submit a photo of someone else's poodle.

[sebastiennight]

> It'll be like playing chess against someone you know is using a specific version of Stockfish: even though it would beat you if you tried to fight the system unaided, you can win by asking your own copy of the AI to go one step further ahead, and it will be accurate precisely because it's playing against itself and reacting to its own moves.

I don't believe this is how chess works, and I don't believe this is how Stockfish works, and I don't believe this is how AI works.

Stockfish isn't winning because it's playing a better sequence of programmed steps, and having access to "the next version of Stockfish" doesn't mean it can "guess the next move" and play against that.

[ben_w]

I didn't say "the next version of Stockfish".

You have Stockfish version n, see board state s. I have Stockfish version n, see board state s. I want to know what you're about to do, so I put Stockfish into state s, ask it what the best move is, and I know you'll make that move because I know you'll ask Stockfish version n the same question of the same state. I now know board state s+1.

The steps are not pre-programmed, but the program itself is (modulo hardware imprecision) deterministic. If there's a RNG in there then sure, this doesn't work as easily as I wrote it; and there may be randomness in the thing that this is a metaphor for, regardless of if there's one in Stockfish or not, but that's not hard to work with when you want to win against an aggregate: we invented the field of statistics to deal with random numbers because they come up so often.

[muvlon]

There is deliberate randomness in stockfish. The easiest way to see this is from the fact that, when playing the white pieces, it won't play the same opening move every time. Often it's e5, but it also goes for e4 or Nf3 or something else entirely.

This is by design, and very much necessary for a competitive chess engine. Otherwise, people could do basically what you say: Run an offline (as in, ahead of time, with ample compute resources) search against stockfish that finds a line where it loses, then make an engine that plays that every time.

As a consequence, even if you know that your opponent is running stockfish, you can't really use that against them. Your best bet is also just running stockfish.

[drybjed]

Jacek Dukaj in his book "Perfect Imperfection" (Polish hard sci-fi about alternative physics, mind uploading and enclosed universes) wrote about entities vastly superior to human-level intelligence having an ability to simulate humans to such a degree of precision that they were able to determine what a given human would say or do based on his own internal experience and knowledge in real time. The superintelligent beings didn't scan his brain patterns, they just had multiple simulations of him and chose the one that would best match his actions based only on external cues like position of his hands, body language, speech patterns, etc.

A solution to avoid being read to such a degree that your free will effectively is destroyed was to introduce randomness as a program in your own mind processed in an equivalent of a "cloud VM" - random changes in your body posture, hand movements, weird phrases introduced while you spoke could confuse the internal models of you enough to avoid hyper-predictability.

[lioeters]

That's really good, I'll have to read some of his work (Jacek Dukaj). It makes me think about the role of indeterminacy and unpredictability in physics. And reminds me of..who was it, Gurdjieff maybe, who taught that humans are like sleeping machines, and that constant conscious effort is necessary to "stay awake". Many of his exercises for students had the purpose of recognizing one's own habits, routine patterns of thinking and behaving, and breaking out of them.

The sci-fi story also reminds me of, I think it was related to Karl Popper's book about "objective knowledge". There was a thought experiment about Mozart, if a scientist could quantify everything external about the composer, like the brain, the body, the room he's in, his life experiences - is it possible to deterministically compose new works exactly as the real Mozart would. And if not, what is the difference between the perfectly simulated Mozart and the real one?

This "randomness" is related to the ancient Greek concept of clinamen.

> Clinamen, derived from clinare, to incline: the unpredictable swerve of atoms in the atomistic doctrine of Epicurus. This swerving, according to Lucretius, provides the "free will which living things throughout the world have".

[chongli]

Stockfish already does that. You run it as long as you want. When it's calculating evaluations for move k+1, it already knows what move k was, because it has that evaluation. It can explore that tree first, before exploring other trees to try to improve on move k.

Stockfish never settles down to an exact move, unless it sees a forced mate sequence. You could leave it running indefinitely, but the exponential blowup will soon grind you to a halt.

What new versions of Stockfish buy you are optimizations and better heuristics. An older version of Stockfish can still beat a new one, if given enough of a compute advantage.

[im3w1l]

Your whole strategy basically involves having more compute than the other guy so you can look one step further. And yeah in an AI world having more compute or some exclusive data seem like obvious ways to get a leg up.

[ben_w]

Yes.

Have you met most people?

Most people won't bother, find this kind of thought process to be somewhere between nerdy and paranoid.