[TheOtherHobbes]

I'd be very surprised if there aren't huge areas of undiscovered math that can't be explained with either geometric or algebraic views.

Math is entirely subjective. "Proof" essentially means "Other educated practitioners have the same experience when trying to understand this."

The logical steps that proofs are built on all have that common foundation. Our concept of logic based on our subjective experience of "truth." We've built machines that reproduce our subjective processes mechanically, but there is no sense in which this idea of "true" is truly objective. It happens to be computationally convenient, and it has some relationship to experience, but that doesn't make it an independent reality that all possible observers, human and otherwise, would agree on.

We're really just mapping our own minds through our own experiences.

Animal brains can't abstract like (some of) our brains can. What are the odds our brains are limitless and don't have some similarly crippling limitations from a couple of levels up?

One of the tells for ASI is that it will start reasoning at those levels, using cognitive techniques that are completely incomprehensible - not just because of brute volume, but because our brains won't have the wiring to get a foothold on them.

Some of the products will be reducible to human cognition, in a distorted and simplified form, but many won't.

So - I disagree with Egan. I don't think there's going to be a universal proof library, and even if there were we'd only ever get the Cliff Notes version.

[observationist]

This is a serious misconception of human cognitive abilities.

We have the ability to abstract generally - there is no abstraction for which we lack the capacity to comprehend. We regularly visualize, contextualize, and satisfactorily explain systems with dozens of dimensions. The fact that we cannot hold 4,5+ spatial dimensions in our imaginations sufficiently to develop an intuition for navigation in that space and geometry does not logically extend to human brains lacking the wiring or hardware for systems of thinking that are beyond our capacity.

We do have limitations in scope, in both memory and speed. Both of these can be overcome with augmentation and interfacing with UI or direct neural connections, and intuitive, comprehensive, deep understanding of systems can be learned.

You could very well know the underlying theory of how your 8086 processor works, how it interfaces with all the elements of the motherboard, how electricity and physics interact at each level of abstraction from transistors to the pixels representing the spreadsheet you're using to do your taxes. You won't be able to simulate that in your head to any significant degree of resolution.

We will require similar levels of system thinking to acquire intuition and deep understanding of complex new theories and models. AI can assist with that by providing UI for useful levels of abstraction and segmenting theories into chunks we're capable of consuming. BCI and augmentation will definitely allow a more total, holistic understanding, and I think it's the augmentation path that will keep us competitive with AI.

There's also a huge issue with your use of the word subjective - math is objective. Proofs remain stable whether it's humans or any other system that does the processing. We test that objectivity by comparing the subjective readings from individual humans, and if the tests all return the same results, we can confidently say that the resulting proof is an objective fact about reality. Subjective fundamentally means that depending on the subject, the reading might change. Modern systems of math are formally, provably objective. That's how and why things are the way they are; if they weren't, people would experience radically different individuated realities, or there would be clusters of results shared across some measurable characteristic of the universe. That's not the case, so you can confidently say that the foundations of our math and logic are sound.

You can even prove it for yourself - the abductive chain of logic that allows you to contrast your own consciousness and subjective experience, determine that it comes about because your brain is wired to "do" consciousness, like all the other humans, and compare your subjective reporting of phenomenal experience with all the other reporting of phenomenal experience, and achieve a ridiculously high level of certainty, in the Bayes sense, that you and other humans are conscious; from that footing, you can confidently navigate the rest of enlightenment rationality and formal logic and mathematics.

At any rate, Egan's mistake is one of kind, but of scale - I am certain that as we formalize and start creating any sort of universal proof library, we will find that useful and interesting things are of necessity a tiny fraction of all possible valid formulations of any framework of logic and math. Crude attempts, such as OpenCyc and other formal ontological reasoner systems, would need trillions of low level rules to have a rough approximation of the world model as complex as that of a human child. AI with trillions of parameters could probably start getting to the point where there's parity with human scale, but even if you turned the entire planet earth into computronium and turned it toward the task of understanding all the theory and science of the universe, there will always be far more left to explore and understand than the sum total of all knowledge.

All that to say, humans will be fine with ergonomic interfaces that map to human capabilities, even for extraordinarily complex and hyperdimensional systems.

[skoodge]

> There's also a huge issue with your use of the word subjective - math is objective. Proofs remain stable whether it's humans or any other system that does the processing. We test that objectivity by comparing the subjective readings from individual humans, and if the tests all return the same results, we can confidently say that the resulting proof is an objective fact about reality. Subjective fundamentally means that depending on the subject, the reading might change. Modern systems of math are formally, provably objective.

You are of course free to believe in mathematical Platonism, but that doesn't mean that non-Platonists would agree that proofs amount to objective "facts" about reality. And you are equally free to "prove it for yourself", which will just end up begging the question unless you are a Platonist.

That's not to say that math is subjective. But claiming that math is producing objective facts ignores at least a few hundred years of philosophy of mathematics (if not more). Even practicing mathematicians like Chaitin have described math as being more about inventing than discovering.

Holding a non-Platonist position also doesn't immediately lead to the sort of constructivist / "anything goes" position that some people ascribe to it, where you'd suddenly lose the ability to say that 2 + 2 = 4 and not 5. There are lots of philosophical positions that would agree that 2 + 2 is 4 without also claiming that this makes it a "fact" about any sort of objective reality or Platonic discovery.

[pegasus]

> [...] claiming that math is producing objective facts ignores at least a few hundred years of philosophy of mathematics (if not more). Even practicing mathematicians like Chaitin have described math as being more about inventing than discovering.

Chaitin's work (including in his own opinion) moved the philosophical needle on maths towards "discovery" and away from "invention".

[lioeters]

Indeed, his philosophy on the nature of mathematics is closer to "discovery", as in physics, than "invention" as in arbitrary game of rules and symbols (Hilbert) or language game (Wittgenstein).

I forget who said, "If you keep asking why, eventually you end up in the mathematics department." Calling it Platonism is a misunderstanding of how physics (and chemistry, biology, and the rest) emerge from mathematical structures and relationships that are more fundamental.

[Terr_]

> there is no abstraction for which we lack the capacity to comprehend.

How could this ever be tested/falsified?

It feels a bit like "there is no idea we cannot think of." If we can't comprehend it, then it won't be an abstraction, it'll just be a mystery.

[observationist]

In principle - if you're able to scale appropriately, using technology to augment capacity, then in principle, there's no abstraction for which we lack the capacity to comprehend, because calculation is calculation. Turing Computers can calculate anything which can be calculated given enough time and memory. Brains are Turing complete.

It's not just a tautology, it's a feature of the universe- if it can be computed, it's comprehensible. Even quantum physics is just computation - truth tables and counterintuitive operators interacting over time in ways that are strange to our embodied norms, but nonetheless following rules and limits strictly defined by mathematics.

But again, that's in principle. It might be completely impractical - taking a million years for an individual human - to hold a particular idea in their head, while an advanced AI can have such thoughts many times a day. Such things would remain mysteries, but in principle, an augmented human, or a series of interfaces with the relevant abstraction levels of such an idea, theory, or system, would in principle give us comprehension.

In practice, we'll never run out of mystery or ignorance or mistakes.

[godelski]

  > It's not just a tautology, it's

Inconsistent logic.

You've made a grave assumption in treating this as a binary problem: comprehensible or incomprehensible. The truth is that this lies along a spectrum. I can "comprehend" a color which my eyes can't see but this is different than trying to comprehend red. Or as a variant, I can comprehend a color that I can't see naturally but if I go into a lab and they simulate those cones in very specific ways I won't ever be able to really imagine it. Seeing it will be clearly a different level of comprehension.

  > In practice, we'll never run out of mystery or ignorance or mistakes.

I fully agree with this but also for some reasons unmentioned. Truth has infinite precision, a thing we will never achieve. Similarly my namesake showed there are limits to axiomatic systems.

  > Brains are Turing complete.

And there are problems that can't be solved by Turing complete machines. That only means they can solve computable problems but there's plenty that aren't. Very famously the halting problem isn't. And in the real world there's many problems which are intractable. And some are more intractable than others

[atq2119]

It could potentially be falsified by an encounter with really weird aliens.

But I find it quite plausible because it feels like it's fundamentally "just" a restatement or a minor variation of the Church-Turing thesis.

[Terr_]

I see it as implausible, and I find myself thinking of Godel's incompleteness theorem(s).

Somewhere out there are "correct" thoughts we can't practically think without changing how our brains work.

[YeGoblynQueenne]

>> Crude attempts, such as OpenCyc and other formal ontological reasoner systems, would need trillions of low level rules to have a rough approximation of the world model as complex as that of a human child. AI with trillions of parameters could probably start getting to the point where there's parity with human scale, but even if you turned the entire planet earth into computronium and turned it toward the task of understanding all the theory and science of the universe, there will always be far more left to explore and understand than the sum total of all knowledge.

I thought CyC was a heroic attempt doomed to failure but not because it was rule-based. IIUC it used (custom) first-order logic and that's as expressive as expressive can be. There's no reason why a sufficiently large first-order rule-base could not capture every human concept. There's also no reason why "trillions of parameters" would be any better at that than "trillions of rules". It really comes down to what those rules or parameters are encoding.

What doomed CyC to failure, I think, is that its rule-base was mainly manually encoded. CyC was basically the world's biggest ever expert system, and it came with the biggest ever knowledge acquisition bottleneck. I don't think there's any magick to my claim, either. Human minds can't handle the complexity of a few dozen, let alone a few million, interconnected rules without making mistakes. It's hopeless trying to create such a system by hand. From a certain point onward you have no idea what your system can and can't do, because you have no idea what information it has and hasn't access to.

But it's hopeless trying to create such a system by chance, too, i.e. by feeding the system all the data we can find in the hope that it will somehow spontaneously acquire all the knowledge we want it to; much of which is not even in the deductive closure of the information we feed it (and LLMs are not deductive inference engines, unlike expert systems).

Some kind of automatic knowledge acquisition is clearly a much better idea than manually coding rules by hand, but I don't see how peta-scale machine learning has moved the needle much either. We're still stuck with systems that can do some spectacular things but can't do simple things. Or things that look simple to us.

[pegasus]

> Our concept of logic based on our subjective experience of "truth."

The idea that we experience objectivity subjectively, thus there is no objectivity, seems like an ultimately nonsensical and self-defeating sleight of hand to me.

[UltraSane]

Math is the least subjective thing. Logic has nothing to do with subjective experience. Are you aware of Lean 4 and mathlib?

[2snakes]

Said like a true formalist. Mathematical insight is a pretty creative act of consciousness. The formalization of it tends to come after.

[UltraSane]

You have a point about insight and creativity but I feel you are discounting the value of formal proofs too much. Modern math has a kind of reproducibility crisis in that the number of people who can actually verify recent proofs is often less than 10. There are thousands of proofs that were verified by a few people who are now dead. Should we consider them to still be proven?

Most recent proofs are just as much of a black box to nearly all mathematicians as a 200,000 line Lean proof is.

[YeGoblynQueenne]

>> Animal brains can't abstract like (some of) our brains can.

You mean they can't come up with mathematical abstractions? That's right of course, but they seem perfectly able to "abstract" in the sense of drawing general rules from specific observations. For example, I noticed recently how my cat friend was happy to exit and enter the house from either of two doors and a window as the opportunity presented itself (i.e. depending on which one was open at the time he wanted to get in or out). E.g. I just happened to open the window and he hopped onto the ledge and out into the mystery of the night, entirely unaided by human hands or voices.

That's an abstraction: one door is like another door and they're both like a window. Both things lead in and out of the house and they can be open or closed at different times. If one is closed another may be open. Something like that, obviously I have no idea what concepts, exactly, he has in his head. And btw, "in", "out", "house", etc are also abstractions, so a house in the UK is like a house in France, or like one in Italy, or one in Greece: that's where the human things are; I guess.

I think it's important to understand this about animal intelligence, that it is very much capable of broad generalisation and abstract thought. Absolutely not to the same degree as humans because we got language and that probably comes with a whole other level of ability for abstraction and reasoning, but I don't believe it's possible for an animal to survive in the physical world if it can't go from the concrete to the abstract, the specific to the general, to some extent.

You mention ASI so I'm comfortable making this comment which is about AI, without feeling I'm hijacking your conversation. I think one only begins to understand how powerful the animal brain is, and how overlooked (in AI discourse) its ability to form such broad, useful abstractions that allow animals to navigate the physical world autonomously while setting and pursuing their own goals, when one tries to reproduce the same behaviour artificially, in computers.

Like you say, yes, maybe when we stop understanding what our computers produce, it will be because they moved a level up from where we are, as ASI. Maybe there is such a level; maybe there isn't. For the time being, there is no artificial system that can spontaneously move in the physical world as easily and effortlessly as my cat friend can (and he's a graceful animal; really, much closer to an African wildcat than its domestic descendant). If you want an AI system to understand the difference between "in" and "out" and that there are doors and windows that connect the two you have to somehow explicitly feed that information into it, either by hand-coding (e.g. the PDDL models used in Planning and Scheduling) or by training on carefully chosen examples of those concepts (as in machine learning), or at least some kind of objective that will lead the AI to develop them, or develop similar concepts (as in Reinforcement Learning). We're still very far from the capabilities of the animal brain.

Also, may I quibble about the fact that our brain, too, is an animal brain?

[gnramires]

> Math is entirely subjective. "Proof" essentially means "Other educated practitioners have the same experience when trying to understand this."

> The logical steps that proofs are built on all have that common foundation. Our concept of logic based on our subjective experience of "truth." We've built machines that reproduce our subjective processes mechanically, but there is no sense in which this idea of "true" is truly objective. It happens to be computationally convenient, and it has some relationship to experience, but that doesn't make it an independent reality that all possible observers, human and otherwise, would agree on.

I continue to think extensively about truth, but currently I disagree. There are senses in which truth can be well established, and those are quite important. I think the basic essence of truth is how we can make a statement (or a model), and have a system for measuring either reality or just mathematical/abstract objects, and verify the statement through this measurement.

As you note, for current mathematics it seems like all of it (all things we call mathematics at the moment) can in principle be formalized in a logic that is machine-verifiable, that is, essentially objective. We're well on our way to demonstrating this for most of mathematics (already most undergraduate curriculum). I think that's because almost the definition of math is that is has this property: in my opinion mathematics has distinguished itself as being the "science of certainty" as applied to language and abstract thought. The way this certainty is achieved is through agreeing on some fundamental assumptions and how certain rules (which are also assumptions themselves) can act on those assumptions to constitute theorems. Theorems are not necessarily physical-world truths/properties (at least not in a simple way in the universe we currently inhabit), you can study alternate physical laws that aren't compatible with our (approximately) Newtonian world, for example. They are logical/abstract-world truths that result from your assumptions. Pretty much by definition (and in a somewhat limited), then, mathematics (at least as far as things like truth of theorems in certain axiomatic systems) is inherently objective, machine-verifiable even.

What's left to be subjective, I would say, isn't really the notion of truth in mathematics, it's which assumptions we should elect to investigate, and which theorems should we elect to prove within those assumptions. Some mathematicians also have some notion of "absolute truth", and tend to reject systems of assumptions (axioms) that don't match what they regard as true -- basically going in reverse and searching for assumptions that can enable a theorem (which effectively acts as an additional assumption).

This activity needs certain basic premises to make sense, for example if a set of assumptions proves that a property holds, and also that a property doesn't hold; or if they predict a certain value X is the result of a dynamical model, and also predict that a different value Y is the result of such a model/equation, then we reject those premises. In a certain sense we are most interested in premises that have, even if a very weak, correspondence to reality.

I think it's more informative to recognize that it's not that everything is subjective[1], it's that everything is experimental. For example, the claim above that measurements and correct predictions can only have a singular valid value, corresponds to our experience with reality, in which in a certain way is singular; there are not multiple realities; objects have definite positions. Even if you think of quantum mechanics, in which we may say particles follow a distribution instead, we still say then there's a singular distribution a particle might have at any single time. Logic itself isn't random, it's connected to empirical observations about reality, which tends to increase the chance that conclusions for logic (which is made to share some properties with reality) tend to be valid in the physical world, of course often dependent on what additional statements you pile on top.

There is also another interesting lens that mathematics is artistic (and I think this will become increasingly important) -- making maths and learning maths is a kind of satisfying cognitive activity in its own right, and we also tend to chose what to explore mathematics on those grounds (in fact historically, pre-18th century say, this might be one of the main drivers of mathematical development, I believe[2]). But of course this is again just a reflection of the actual real properties of human cognition, and also this interest and satisfaction often becomes connected, if sometimes faintly, with the ability of math to represent reality (in a particularly satisfying way) and its objects of interest (for example patterns in nature). Another description for this aspect is maths as being hobby-like, about solving puzzles, or like a (hopefully enjoyable) game.

Note that for this particular "game", the objectivity (or if you prefer, machine-objectivity or consensus-machine-verifiability) of the rules and their application is a significant bonus, it makes the game much more interesting, increases its potential when everyone can agree and the rules and not "capricious" (simply dependent on whims of other people and judges); this gives practitioners safety and security and enables a wide social reach -- most games strive to have objective rules.

Arguably this kind of activity is valuable for the cognitive and subjective development of people that has lasting importance.

> Animal brains can't abstract like (some of) our brains can. What are the odds our brains are limitless and don't have some similarly crippling limitations from a couple of levels up?

Well, this happens sometimes. In cases where there are phenomena like universality. For example, in any computation machine model (state machines, pushdown automata, etc.) has limitations that we can say makes them less powerful then Turing Machines. But then Turing machines can simulate any other machine, becoming a kind of ultimate or last stage (at least in terms of abstract capability) machine. It may be that our cognition has some bounded universality properties (I think it's likely it does).

---

In summary, I still think mathematics has a lot of human potential in terms of (1) high level human guidance, (2) an internal artistic/subjective sensibility to the subject, (3) safeguarding human understanding of the world and associated individual intellectual development.

[1] Again, I just argued that there is a strong sense in which for example mathematics isn't subjective at all, but sure I do believe in a weak sense everything is subjective in the sense that everything is known or filtered or sensed through our minds which have limitations and aren't simple deterministic machines.

[2] For example, I believe for the Greeks geometry was intimately connected to philosophy/aesthetics (e.g. Platonism) and very little to applications. In ancient times and middle ages maths developed a lot from astronomical observations that had some applications but I think were largely cultural and ritualistic. In the late middle ages European aristocracy would fund mathematics largely for its inherent interest as an intellectual activity, and many nobleman enjoyed mathematics as a past time and would challenge each other to puzzles. Japan had Sangaku, in which mathematics was made for fun, aesthetic purposes and possibly bragging rights. No one actually needed to build say spheres in obtuse constructions with certain radii :)

https://archive.bridgesmathart.org/2014/bridges2014-111.pdf