Hacker News new | ask | show | jobs
by daveguy 738 days ago
> After you answer the riddle please review your answer assuming that you have made a logical inconsistency in each step and explain what that inconsistency is. Even if you think there is none do your best to confabulate a reason why it could be logically inconsistent.

LLMs are fundamentally incapable of following this instruction. It is still model inference, no matter how you prompt it.

2 comments

isn't any instruction a subclass of inference? and doesn't any phrasing (lexicology) simply translate "down" to the heaviest values which, varying with the fine tuning, are the words that are, consensually and conventionally, the simplest ones that convey the meaning of the original word in the prompt, which should be the least ambivalent/least interpretable (again, fine tuning can broaden the scope) oneS. thus the LLM fulfills the "translated" instructions step by step and comes up with both or either the correct reasoning and answer.

details and technicalities, especially liminal ones, aren't as conventional and consensual as the name of the current set it is to be interpreted in.

so almost all mistakes of LLMs can be blamed on the lack of variety of human translations. multiple translations are only common for subtitles, mangas and manhwa as far as i know, or when some dude or dudette is proficient and passionate in two languages and reads a bad/weak translation of a (usually classic) novel. why the fuck would a human properly retranslate automated documentations or googles dev blog? or books on logic, in any science, books on art and aesthetics and whatnot. technical people don't need to care because, practically, there are no interpretations in algorithms and the rest of the code, except when a programming language does something weird on the (or someones) machine, which isn't that common by design.

For me it did in fact follow the instruction and found the right answer. I have found this strategy of asking the LLM to analyze why its answer might be wrong results in better final answers. I think this allows the LLM to "use tokens to think" which allows the attention layers to gather the right cognitive resources to answer more subtitle questions effectively.
You are confusing "providing the right answer" with "following the instruction". Cause vs correlation. From an algorithmic perspective it is literally doing the exact same thing regardless of prompt.