[thomastjeffery]

Objectivity. Independent thought.

The technologies we have are models, not actors. They are each the static result of a process that determines boundaries and relationships between pieces of data. These models do not, however, organize or label the boundaries or relationships themselves.

For example, you can model a dataset of human-written text into an LLM. Using that LLM, you can transform a human-written prompt into a "continuation" that incorporates the modeled dataset. The resulting continuation will contain a new organization of both text from your prompt and/or text from the dataset: nothing more.

The boundaries and relationships modeled by an LLM are not categorized. The model does not contain any objective observation about its data. It only provides a structure that is intended to "align to" (simulate) the already-present semantics of natural language. It was not the model's intention to align with language semantics: that comes from the authors of the model, and from the presence of patterns (we recognize as language semantics) in the dataset itself. Without the presence of natural language patterns, there would be no boundary to align to in the first place.

To contrast, a human can read text into ideas, then think objectively about those ideas, produce new ideas, and finally express those ideas into text by structuring them into language semantics. Nothing like that happens in any software I am aware of.