Hacker News new | ask | show | jobs
by rootlocus 38 days ago
LLMs can be described as "hardcoded weights for generating the next token". All code is "hardcoded".

I think AI for game NPCs is perfectly valid terminology. The actor senses the surroundings and makes decisions to achieve its goals within the constraints and events of the environment.

2 comments

> LLMs can be described as "hardcoded weights for generating the next token". All code is "hardcoded".

I disagree. Hardcoded means you need to change and recompile the entire program. So neither an .env file nor model weights that are loaded at runtime are 'hardcoded'.

But you can use hardcoded while referring to interpreted languages as well, it's a bit of a difficult term. I feel it's more about repeatability when non of the input changes.

I would define it more broadly, "same input == same output, with the same program" and using that definition all computer programs are hardcoded in the end.

There is a gray area, you can consider that even programs that use random() are in the hardcoded category as with the same (semi)random value they will produce the same output. You just need to think of the random value as input, since not all input needs to be user input.

Sure, and the article mentions the AI in elden ring is scripted in lua which is loaded at runtime, meaning it's not hardcoded.
There's no reasonable definition of "hard-coded" that fits how LLMs are made.

The whole idea behind "hard-coded" and original symbolic AI was that a skilled human deliberately wrote explicit code to define a program's behavior.

Machine learning is the exact opposite of that approach.