YES! That's the one :-)
Thanks for identifying it. And it was in that anthology "Nanotech" having editors Jack Dann and Gardner Dozois. I found this interesting article about the story, with links to other similar stories:
"Marsden's data stores contain a fragmented catalogue of mathematical variants. All founded on the postulates of arithmetic, but differing in their resolution of undecidable hypothesis."
"Undecidability. You're talking about the incompleteness theorems,"
"Right. No logical system rich enough to contain the axioms of simple arithmetic can ever be made complete. It is always possible to construct statements that can be neither disproved nor proved by deduction from the axioms; instead the logical system must be enriched by incorporating the truth or falsehood of such statements as additional axioms..."
The Continuum Hypothesis was an example.
There were several orders of infinity. There were 'more' real numbers, scattered like dust in the interval between zero and one, that there were integers. Was there an order of infinity between the reals and the integers. This was undecidable, within logically simpler systems like set theory; additional assumptions had to be made.
"So one can generate many versions of mathematics, by adding these true-false axioms."
"And then searching on, seeking out statements which are undecidable in the new system. Yes. Because of incompleteness, there is an infinite number of such mathematical variants, spreading like the branches of a tree...."
This time he would reach the Sky. This time, before the Culling cut him away...
The tree of axiomatic systems beneath him was broad, deep, strong. He looked around him, at sibling-twins who had branched at choice-points, most of them thin, insipid structures. They spread into the distance, infiltrating the Pool with their webs of logic. He almost pitied their attenuated forms as he reached upwards, his own rich growth path assured...
Almost pitied. But when the Sky was so close there was no time for pity, no time for awareness of anything but growth, extension.
Little consciousness persisted between Cullings. But he could remember a little of his last birthing; and surely he had never risen so high, never felt the logical richness of the tree beneath him surge upwards through him like this, empowering him.
Now there was something ahead of him: a new postulate, hanging above him like some immense fruit. He approached it warily, savoring its compact, elegant form.
The fibers of his being pulsed as the few, strong axioms at the core of his structure sought to envelop this new statement. But they could not. They could not. The new statement was undecidable, not deducible from the set within him.
His excitement grew. The new hypothesis was simple of expression, yet ....
https://kasmana.people.cofc.edu/MATHFICT/mfview.php?callnumb...
And this quote:
"Marsden's data stores contain a fragmented catalogue of mathematical variants. All founded on the postulates of arithmetic, but differing in their resolution of undecidable hypothesis." "Undecidability. You're talking about the incompleteness theorems," "Right. No logical system rich enough to contain the axioms of simple arithmetic can ever be made complete. It is always possible to construct statements that can be neither disproved nor proved by deduction from the axioms; instead the logical system must be enriched by incorporating the truth or falsehood of such statements as additional axioms..." The Continuum Hypothesis was an example. There were several orders of infinity. There were 'more' real numbers, scattered like dust in the interval between zero and one, that there were integers. Was there an order of infinity between the reals and the integers. This was undecidable, within logically simpler systems like set theory; additional assumptions had to be made. "So one can generate many versions of mathematics, by adding these true-false axioms." "And then searching on, seeking out statements which are undecidable in the new system. Yes. Because of incompleteness, there is an infinite number of such mathematical variants, spreading like the branches of a tree...."