[nonrandomstring]

> Notation matters in how we explore ideas.

Indeed, historically. But are we not moving into a society where thought is unwelcome? We build tools to hide underlying notation and structure, not because it affords abstraction but because its "efficient". Is there not a tragedy afoot, by which technology, at its peak, nullifies all its foundations? Those who can do mental formalism, mathematics, code etc, I doubt we will have any place in a future society that values only superficial convenience, the appearance of correctness, and shuns as "slow old throwbacks" those who reason symbolically, "the hard way" (without AI).

(cue a dozen comments on how "AI actually helps" and amplifies symbolic human thought processes)

[PaulRobinson]

Let's think about how an abstraction can be useful, and then redundant.

Logarithms allow us to simplify a hard problem (multiplying large numbers), into a simpler problem (addition), but the abstraction results in an approximation. It's a good enough approximation for lots of situations, but it's a map, not the territory. You could also solve division, which means you could take decent stabs at powers and roots and voila, once you made that good enough and a bit faster, an engineering and scientific revolution can take place. Marvelous.

For centuries people produced log tables - some so frustratingly inaccurate that Charles Babbage thought of a machine to automate their calculation - and we had slide rules and we made progress.

And then a descendant of Babbage's machine arrived - the calculator, or computer - and we didn't need the abstraction any more. We could quickly type 35325 x 948572 and far faster than any log table lookup, be confident that the answer was exactly 33,508,305,900. And a new revolution is born.

This is the path we're on. You don't need to know how multiplication by hand works in order to be able to do multiplication - you use the tool available to you. For a while we had a tool that helped (roughly), and then we got a better tool thanks to that tool. And we might be about to get a better tool again where instead of doing the maths, the tool can use more impressive models of physics and engineering to help us build things.

The metaphor I often use is that these tools don't replace people, they just give them better tools. There will always be a place for being able to work from fundamentals, but most people don't need those fundamentals - you don't need to understand the foundations of how calculus was invented to use it, the same way you don't need to build a toaster from scratch to have breakfast, or how to build your car from base materials to get to the mountains at the weekend.

[ryandv]

> This is the path we're on. You don't need to know how multiplication by hand works in order to be able to do multiplication - you use the tool available to you.

What tool exactly are you referring to? If you mean LLMs, I actually view them as a regression with respect to basically every one of the "characteristics of notation" desired by the article. There is a reason mathematics is no longer done with long-form prose and instead uses its own, more economical notation that is sufficiently precise as to even be evaluated and analyzed by computers.

Natural languages have a lot of ambiguity, and their grammars allow nonsense to be expressed in them ("colorless green ideas sleep furiously"). Moreover two people can read the same word and connect two different senses or ideas to them ("si duo idem faciunt, non est idem").

Practice with expressing thoughts in formal language is essential for actually patterning your thoughts against the structures of logic. You would not say that someone who is completely ignorant of Nihongo understands Japanese culture, and custom, and manner of expression; similarly, you cannot say that someone ignorant of the language of syllogism and modus tollens actually knows how to reason logically.

You can, of course, get a translator - and that is what maybe some people think the LLM can do for you, both with Nihongo, and with programming languages or formal mathematics.

Otherwise, if you already know how to express what you want with sufficient precision, you're going to just express your ideas in the symbolic, formal language itself; you're not going to just randomly throw in some nondeterminism at the end by leaving the output up to the caprice of some statistical model, or allow something to get "lost in translation."

[TuringTest]

> If you mean LLMs, I actually view them as a regression with respect to basically every one of the "characteristics of notation" desired by the article.

LLMs are not used for notation; you are right that they're not precise enough for accurate knowledge.

What LLMs do as a tool is solving the Frame Problem, allowing the reasoning system to have access to the "common sense" knowledge that is needed for a specific situation, retrieving it from a humongous amount of background corpus of diverse knowledge, in an efficient way.

Classic AI based on logical inference was never able to achieve this retrieval, thus the unfulfilled promises in the 2000s to have autonomous agents based on ontologies. Those promises seem approachable now thank to the huge statistical databases of all topics stored in compressed LLM models.

A viable problem-solving system should combine the precision of symbolic reasoning with the breadth of generative models, to create checks and heuristics that guide the autonomous agents to interact with the real world in ways that make sense given the background relevant cultural knowledge.

https://en.wikipedia.org/wiki/Frame_problem

[PaulRobinson]

You need to see the comment I was replying to, in order to understand the context I was making.

LLMs are part of what I was thinking of, but not the totality.

We're pretty close to Generative AI - and by that I don't just mean LLMs, but the entire space - being able to use formal notations and abstractions more usefully and correctly, and therefore improve reasoning.

The comment I was replying to complained about this shifting value away from fundamentals and this being a tragedy. My point is that this is just human progress. It's what we do. You buy a microwave, you don't build one yourself. You use a calculator app on your phone, you don't work out the fundamentals of multiplication and division from first principles when you're working out how to split the bill at dinner.

I agree with your general take on all of this, but I'd add that AI will get to the point where it can express "thoughts" in formal language, and then provide appropriate tools to get the job done, and that's fine.

I might not understand Japanese culture without knowledge of Nihongo, but if I'm trying to get across Tokyo in rush hour traffic and don't know how to, do I need to understand Japanese culture, or do I need a tool to help me get my objective done?

If I care deeply about understanding Japanese culture, I will want to dive deep. And I should. But for many people, that's not their thing, and we can't all dive deep on everything, so having tools that do that for us better than existing tools is useful. That's my point: abstractions and tools allow people to get stuff done that ultimately leads to better tools and better abstractions, and so on. Complaining that people don't have a first principle grasp of everything isn't useful.

[WJW]

> But are we not moving into a society where thought is unwelcome?

Not really, no. If anything clear thinking and insight will give an even bigger advantage in a society with pervasive LLM usage. Good prompts don't write themselves.