[catfacts]

Cognitive load in LLMs: When LLMs are faced with syntactic complexity (Lisp/J parentheses/RL-NOP), distractors (cat facts), or unfamiliar paradigms (right-to-left evaluation), the model’s performance degrades because its "attention bandwidth" is split or overwhelmed. This mirrors human cognitive overload.

My question: is there a way to reduce cognitive load in LLMs?, one solution seems to be process the input and output format so that the LLM can use a more common format. I don't know if there is a more general solution.

Edit: Cat attack https://the-decoder.com/cat-attack-on-reasoning-model-shows-...

[umanwizard]

Isn't the whole idea of Lisp that there is _no_ syntactic complexity? Lisp programs are roughly a serialized AST.

[pornel]

LLMs use tokens, with 1d positions and rich complex fuzzy meanings, as their native "syntax", so for them LISP is alien and hard to process.

That's like reading binary for humans. 1s and 0s may be the simplest possible representation of information, but not the one your wet neural network recognizes.

[kazinator]

Already over two years ago, using GPT4, I experimented with code generation using a relatively unknown dialect of Lisp for which there are few online materials or discussions. Yet, the results were good. The LLM slightly hallucinated between that dialect and Scheme and Common Lisp, but corrected itself when instructed clearly. When given a verbal description of a macro that is available in the dialect, it was able to refactor the code to take advantage of it.

[jdiff]

Agreed, Gleam as a language has very few, generalized syntactic constructs compared to most procedural languages. There's enough of a signal in the data to be able to answer queries about the language; but when writing, LLMs universally trip over themselves. The signal from other nearby languages is too strong and it ends up trying to do early returns, if statements, even loops on occasion.

[catfacts]

I usually use deepseek (gratis) for code, and when using defun and let it usually lacks one (or more) closing parenthesis. So the way to mark the end is not well understood by this LLM, or perhaps that the height of the AST is usually bigger than in python.

[kazinator]

The idea in Lisp is that there is low complexity in encoding abstract syntax into surface syntax.

There can be considerable complexity in Lisp abstract syntax.

[umanwizard]

Fair enough.

[gabiteodoru]

I think a translation layer to a lower-density language might be a good solution; e.g. Iverson's divisible-by-11 check, 0=11|-/d, can be verbosely done in Python with

import numpy as np

def flippedSubtract(a, b): return b - a

flipSubUfunc = np.frompyfunc(flippedSubtract, 2, 1)

def isDivBy11(number): digits = list(map(int, str(number))) discriminant = flipSubUfunc.reduce(digits) return (discriminant % 11) == 0

Though Claude already understands (has already seen?) 0=11|-/d so it's hard to tell for this example

As for the cat attack, my gut feeling is that it has to do with the LLM having been trained/instructed to be kind