[nyrikki]

The one place that I think the previous discussion lost something important, at least to me with functions.

The popular lens is the porcupine concept when infinite dimensions for functions is often more effective when thought of as around 8:00 in this video.

https://youtu.be/q8gng_2gn70

While that video obviously is not fancy, it will help with building an intuition about fixed points.

Explaining how the dimensions are points needed to describe a functions in a plane and not as much about orthogonal dimensions.

Specifically with fixed points and non-expansive mappings.

Hopefully this helps someone build intuitions.

[chongli]

I see this a lot with math concepts as they begin to get more abstract: strange visualizations to try to build intuition. I think this is ultimately a dead-end approach which misleads rather than enlightens.

To me, the proper way of continuing to develop intuition is to abandon visualization entirely and start thinking about the math in a linguistic mode. Thus, continuous functions (perhaps on the closed interval [0,1] for example) are vectors precisely because this space of functions meet the criteria for a vector space:

* (+) vector addition where adding two continuous functions on a domain yields another continuous function on that domain

* (.) scalar multiplication where multiplying a continuous function by a real number yields another continuous function with the same domain

* (0) the existence of the zero vector which is simply the function that maps its entire domain of [0,1] to 0 (and we can easily verify that this function is continuous)

We can further verify the other properties of this vector space which are:

* associativity of vector addition

* commutativity of vector addition

* identity element for vector addition (just the zero vector)

* additive inverse elements (just multiply f by -1 to get -f)

* compatibility of scalar multiplication with field multiplication (i.e a(bf) = (ab)f, where a and b are real numbers and f is a function)

* identity element for scalar multiplication (just the number 1)

* distributivity of scalar multiplication over vector addition (so a(f + g) = af + ag)

* distributivity of scalar multiplication over scalar addition (so (a + b)f = af + bf)

So in other words, instead of trying to visualize an infinite-dimensional space, we’re just doing high school algebra with which we should already be familiar. We’re just manipulating symbols on paper and seeing how far the rules take us. This approach can take us much further when we continue on to the ideas of normed vector spaces (abstracting the idea of length), sequences of vectors (a sequence of functions), and Banach spaces (giving us convergence and the existence of limits of sequences of functions).

[tsimionescu]

> I see this a lot with math concepts as they begin to get more abstract: strange visualizations to try to build intuition. I think this is ultimately a dead-end approach which misleads rather than enlightens.

Isn't this how people arrived at most of these concepts historically, how the intuition arose that these are meaningful concepts at all?

For example, the notion of a continuous function arose from a desire to explicitly classify functions whose graph "looks smooth and unbroken". People started with the visual representation, and then started to build a formalism that explains it. Once they found a formalism that was satisfying for regular cases, they could now apply it to cases where the visual intuition fails, such as functions on infinite-dimensional spaces. But the concept of a continuous function remains tied to the visual idea, fundamentally that's where it comes from.

Similalrly with vectors, you have to first develop an intuition of the visual representation of what vector operations mean in a simple to understand vector space like Newtonian two-dimensional or three-dimensional space. Only after you build this clean and visual intuition can you really start understanding the formalization of vectors, and then start extending the same concepts to spaces that are much harder or impossible to visualize. But that doesn't mean that vector addition is an arbitrary operation labeled + - vector addition is a meaningful concept for spatial vectors, one that you can formally extend to other operations if they follow certain rules while retaining many properties of the two-dimensional case.

[chongli]

the notion of a continuous function arose from a desire to explicitly classify functions whose graph "looks smooth and unbroken"

This notion falls down when you get to topology where you have continuous functions on topological spaces (which need not have any concept of distance nor even "smoothness"), since a topology can be defined on a finite (or infinite) set (of objects which may not even be numbers).

But that doesn't mean that vector addition is an arbitrary operation labeled + - vector addition is a meaningful concept for spatial vectors, one that you can formally extend to other operations if they follow certain rules while retaining many properties of the two-dimensional case

Vector addition need not even look like addition. For example, the positive real numbers can be defined as a vector space over the real numbers, with:

* (+) vector addition: u + v = uv (adding two vectors by multiplying two positive real numbers)

* (.) scalar multiplication: av = v^a

* (0) zero vector: 1 (the identity for multiplication)

Now we can verify some of the vector space axioms:

* let a be a scalar and u, v be vectors, then: a(u + v) = a(uv) = (uv)^a = (u^a)(v^a) = au + av, thus distributivity of scalar multiplication over vector addition holds

* let a, b be scalars and v be a vector, then: (a + b)v = v^(a + b) = (v^a)(v^b) = av + bv, thus distributivity of scalar multiplication over scalar addition holds

The rest can also be similarly verified but you get the picture.

Another weird vector space is the set of spanning subgraphs of a finite, simple, undirected graph over the finite field F[2] (which yields only 0 and 1 as scalars). In this one the idea of vector addition between subgraphs G + H is about determining whether two vertices are adjacent in one or the other of G or H, or adjacent in neither or both. This isn't really like addition at all, so none of the intuitions you might develop for vector addition in a Euclidean two-dimensional space would apply at all.

[tsimionescu]

None of these examples contradict what I've said. The historical fact is that continuous functions and vector addition were discovered as interesting mathematical properties based on certain visual intuitions. Only later were did they come to be extended to more general notions applicable to wildly different conditions.

What I claim in addition is that it's still useful for your intuition to understand this history and the leaps that were made by extremely talented mathematicians of the past who came up with these formalizations of intuitive properties.

I'd also claim that they couldn't have ever arrived at the current formal systems if they hadn't started with certain intuitions for simple systems.

[Tainnor]

You're arguing for including historical motivation in order to make a definition seem more immediately useful and intuitive. I agree with that.

What the person you replied to is suggesting, however, is that this can only get you so far. At some you have to sit down and study the formal definition and understand how it relates to the examples you've seen so far. Otherwise it's very easy to have your intuition lead you astray.

[tsimionescu]

I'd that's what they're arguing, then I fully agree. My fear was that they're arguing that the formal definition is the only thing that you should spend time on, that the original motivation is irrelevant - this is what I was trying to argue against.

[ajkjk]

Funny, I agree that visualizations aren't that useful after a point, but when you said "start thinking about the math in a linguistic mode" I thought you were going to describe what I do, but then you described an entirely different thing! I can't learn math the way you described at all: when things are described by definitions, my eyes glaze over, and nothing is retained. I think the way you are describing filters out a large percentage of people who would enjoy knowing the concepts, leaving only the people whose minds work in that certain way, a fairly small subset of the interested population.

My third way is that I learn math by learning to "talk" in the concepts, which is I think much more common in physics than pure mathematics (and I gravitated to physics because I loved math but can't stand learning it the way math classes wanted me to). For example, thinking of functions as vectors went kinda like this:

* first I learned about vectors in physics and multivariable calculus, where they were arrows in space

* at some point in a differential equations class (while calculating inner products of orthogonal hermite polynomials, iirc) I realized that integrals were like giant dot products of infinite-dimensional vectors, and I was annoyed that nobody had just told me that because I would have gotten it instantly.

* then I had to repair my understanding of the word "vector" (and grumble about the people who had overloaded it). I began to think of vectors as the N=3 case and functions as the N=infinity case of the same concept. Around this time I also learned quantum mechanics where thinking about a list of binary values as a vector ( |000> + |001> + |010> + etc, for example) was common, which made this easier. It also helped that in mechanics we created larger vectors out of tuples of smaller ones: spatial vector always has N=3 dimensions, a pair of spatial vectors is a single 2N = 6-dimensional vector (albeit with different properties under transformations), and that is much easier to think about than a single vector in R^6. It was also easy to compare it to programming, where there was little difference between an array with 3 elements, an array with 100 elements, and a function that computed a value on every positive integer on request.

* once this is the case, the Fourier transform, Laplace transform, etc are trivial consequences of the model. Give me a basis of orthogonal functions and of course I'll write a function in that basis, no problem, no proofs necessary. I'm vaguely aware there are analytic limitations on when it works but they seem like failures of the formalism, not failures of the technique (as evidenced by how most of them fall away when you switch to doing everything on distributions).

* eventually I learned some differential geometry and Lie theory and learned that addition is actually a pretty weird concept; in most geometries you can't "add" vectors that are far apart; only things that are locally linear can be added. So I had to repair my intuition again: a vector is a local linearization of something that might be macroscopically, and the linearity is what makes it possible to add and scalar-multiply it. And also that there is functionally no difference between composing vectors with addition or multiplication, they're just notations.

At no point in this were the axioms of vector spaces (or normed vector spaces, Banach spaces, etc) useful at all for understanding. I still find them completely unhelpful and would love to read books on higher mathematics that omit all of the axiomatizations in favor of intuition. Unfortunately the more advanced the mathematics, the more formalized the texts on it get, which makes me very sad. It seems very clear that there are two (or more) distinct ways of thinking that are at odds here; the mathematical tradition heavily favors one (especially since Bourbaki, in my impression) and physics is where everyone who can't stand it ends up.

[chongli]

I can't learn math the way you described at all: when things are described by definitions, my eyes glaze over, and nothing is retained. I think the way you are describing filters out a large percentage of people who would enjoy knowing the concepts, leaving only the people whose minds work in that certain way, a fairly small subset of the interested population.

If you told me this in the first year of my math degree I would have included myself in that group. I think you’re right that a lot of people are filtered out by higher math’s focus on definitions and theorems, although I think there’s an argument to be made that many people filter themselves out before really giving themselves the chance to learn it. It took me another year or two to begin to get comfortable working that way. Then at some point it started to click.

I think it’s similar to learning to program. When I’m trying to write a proof, I think of the definitions and theorems as my standard library. I look at the conclusion of the theorem to prove as the result I need to obtain and then think about how to build it using my library.

So for me it’s a linguistic approach but not a natural language one. It’s like a programming language and the proofs are programs. Believe it or not, this isn’t a hand-wavey concept either, it’s a rigorous one [1].

[1] https://en.wikipedia.org/wiki/Curry%E2%80%93Howard_correspon...

[Tainnor]

> When I’m trying to write a proof, I think of the definitions and theorems as my standard library. I look at the conclusion of the theorem to prove as the result I need to obtain and then think about how to build it using my library.

fwiw, this is exactly the thing that you when you're trying to formally prove some theorem in a language like Lean.

[chongli]

I do want to learn theorem proving in Lean just for a hobby at some point. I haven't found a great resource for it though.

[Tainnor]

Have you seen: https://leanprover-community.github.io/mathematics_in_lean/

[Tainnor]

> So I had to repair my intuition again: a vector is a local linearization of something that might be macroscopically, and the linearity is what makes it possible to add and scalar-multiply it. And also that there is functionally no difference between composing vectors with addition or multiplication, they're just notations.

Except none of this is true of vectors in general, although it might be true of very specific vector spaces in physics that you may have looked at. Matrices or continuous functions form vector spaces where you can add any vectors, no matter how far apart. Maybe what you're referring to is that differentiability allows us to locally approximate nonlinear problems with linear methods but that doesn't mean that other things aren't globally linear.

I also don't understand what you mean by "no difference between composing vectors with addition or multiplication", there's obviously a difference between adding and multiplying functions, for example (and vector spaces in which you can also multiply are another interesting structure called an algebra).

That's the problem if you just go from intuition to intuition without caring about the formalism. You may end up with the wrong understanding.

Intuition is good when guided by rigour. Terence Tao has written about this: https://terrytao.wordpress.com/career-advice/theres-more-to-...

The vector space axioms in the end are nothing more than saying: here's a set of objects that you can add and scale and here's a set of rules that makes sure these operations behave like they're supposed to.

[ajkjk]

You have unfortunately written off everything I said by assuming the most naive understanding of it.

The general theme is that I am interested in the metaphysical concept of vectors, not the thing that human mathematicians have labeled vectors. The universe doesn't care if you write ax+by or x^a y^b, hence addition vs multiplication is just a choice of coordinate system. And matrices and functions are vector spaces sure, but out in the world, when they show up in modeling things, they are local linearizations of curved things. Every linear algebra is (inevitably) a local point in a nonlinear one, as far as I can tell. Not in a formal sense, but in the sense that when you go out into the world and find them, it turns out to be the case.

The general theme is: I don't want to spend my life mastering the rigor of these simplistic models so that I can do it intuitively (in Tao's sense); I want to use them to learn intuition of the things that they are simplistic models of, and then master that.

[Tainnor]

> The general theme is that I am interested in the metaphysical concept of vectors, not the thing that human mathematicians have labeled vectors.

That's fine but then you shouldn't be surprised that you can't read higher level mathematics textbooks, because those are not about metaphysics.

The rest of what you wrote is... just not true. Matrices are used in plenty of areas where they are not mere approximations (e.g. cryptography), and spaces of functions remain vector spaces even when the functions themselves are not linear (because there's a difference between requiring that f(x+y)=f(x)+f(y) and that (f+g)(x) = f(x)+g(x)).

[MalbertKerman]

> and I was annoyed that nobody had just told me that because I would have gotten it instantly.

Right?! In my path through the physics curriculum, this whole area was presented in one of two ways. It went straight from "You don't need to worry about the details of this yet, so we'll just present a few conclusions that you will take on faith for now" to "You've already deeply and thoroughly learned the details of this, so we trust that you can trivially extend it to new problems." More time in the math department would have been awfully useful, but somehow that was never suggested by the prerequisites or advisors.

[ajkjk]

oh, my point was the opposite of that. The math department was totally useless for learning how anything made sense. I only understood linear algebra when I took quantum mechanics for instance. The math department couldn't be bothered to explain anything in any sort of useful way; you were supposed to prove pointless theorems about things you didn't understand.

[MalbertKerman]

I did get a lot of that in the lower level math courses, where it kinda felt like the math faculty were grudgingly letting in the unwashed masses to learn some primitive skills to apply [spit] to their various fields, and didn't really give a shit if anybody understood anything as long as the morons could repeat some rituals for moving x around on the page. I didn't really understand integrals until the intermediate classical mechanics prof took an hour or two to explain what the hell we had been doing for three semesters of calculus.

But when I did go past the required courses and into math for math majors, things got a lot better. I just didn't find that out until I was about to graduate.

[olddustytrail]

> infinite dimensions for functions is often more effective when thought of as around 8:00

I guess it works if you look at it sideways.