[wizzwizz4]

If you can extract anything resembling a declarative rule from the weights of a transformer, I will put you in for a Turing award.

Expert systems are a specific kind of thing (see https://en.wikipedia.org/wiki/Expert_system#Software_archite...): any definition you've read is a description. If the definition includes GPT models, the definition is imprecise.

[HarHarVeryFunny]

Well, OK, perhaps not a declarative rule, more a procedural one (induction heads copying data around, and all that) given the mechanics of transformer layers, but does it really make a conceptual difference?

Would you quibble if an expert system was procedurally coded in C++ rather than in Prolog? "You see this pattern, do this".

[wizzwizz4]

Yes, it makes a conceptual difference. Expert systems make decisions according to an explicit, explicable world model consisting of a database of facts, which can be cleanly separated from the I/O subsystems. This does not describe a transformer-based generative language model. The mathematical approaches for bounding the behaviour of a language model are completely different to those involved in bounding the behaviour of an expert system. (And I do mean completely different: computer programs and formal logic are unified in fields like descriptive complexity theory, but I'm not aware of any way to sensibly unify mathematical models of expert systems and LLMs under the same umbrella – unless you cheat and say something like cybernetics.)

You could compile an expert system into C++, and I'd still call it an expert system (even if the declarative version was never written down), but most C++ programs are not expert systems. Heck, a lot of Prolog programs aren't! To the extent a C++ program representing GPT inference is an expert system, it's the trivial expert system with one fact.