[murftown]

Cool, I'll bite. I don't know some of your mathy terms like "putative extensive transduction" or "anisotropy" so go easy on me.

But I hear you say things like "Neural nets are nonlinear iterated function systems by construction.", and it resonates with me. I've often tried to draw a parallel between the recursive tree structure of a program with the matrix structure of a neural network, which as I understand it is mathematically still a graph of functional nodes feeding into one another - it's just that treating it like a matrix allows the programs to be much more performant by using low-level structures like numpy arrays.

In any case, I often think about the tree structure of programs, and we whether could get to a place of programs that can analyze, understand and evolve the abilities of other programs. This could be seen as the old-school "symbolic AI" mindset. And then, the similarities and differences of that with the matrix-based neural network revolution that's happened in the last years. I wonder about possible holomorphisms and interfaces between those two paradigms.

I don't doubt that I barely skimmed the surface of what you were actually saying. The very general and meta way you were talking about information processing is intriguing to me, and I'd love to understand more. Thanks for sharing!

[curuinor]

I am afraid that I am running the other way: looking hard at the Zipfian criticality and thinking that the seeming hierarchy of symbolic AI is even more an illusion.

Many seeming hierarchies are at heart the formalization of a power-law phenomenon.