[ttoinou]

Isn't this the opposite way ? Vectors are functions whose input space are discrete dimensions. Let's not pretend going from natural numbers to real is "simple", reals numbers are a fascinating non-obvious math discovery. And also the passage from a few numbers to all natural numbers (aleph0) is non obvious. So basically we have two alephs passages to transforms N-D vectors as functions over reals.

[xeonmc]

Vectors are not necessarily discrete-domained. Anything that satisfies the vector space properties is a vector.

[ttoinou]

I agree but I'm operating under the assumption of the article

  Conceptualizing functions as infinite-dimensional vectors lets us apply the tools of linear algebra to a vast landscape of new problems

[MITSardine]

That's not really the assumption of the article, that's the assumption the article makes of the reader's knowledge of vectors as they begin reading. The article is trying to make the point that functions are also vectors, strictly speaking, and give some intuition as to what that means.

Basically it's an introduction to "functional analysis", the field of math that looks at functions like vectors. The term is in there sneakily, but this is really what the article is all about.

I had classes on this subject and we were encouraged to build upon linear algebra intuition (much like the author does), with some caveats (such as a bounded closed set is not necessarily compact in infinite dimension). Things break down, as the author hints at when they mention they didn't prove the Laplacian to be self-adjoint but only symmetric (this has to do with the operator's domain, a concern that doesn't exist for linear operators on finite dimensional spaces)... but it's still a very convenient way to think and talk about things, conceptually.

[layer8]

Linear algebra isn’t limited to discrete-dimensional vector spaces. Or what do you mean?

[ttoinou]

See my other comment sibling.

And he's starting from the assumption vectors are finite (cf. the article)

[Sharlin]

He does not assume anything! Any assumption is in your head only. Of course he starts from the specific type of vector spaces that's the most familiar to readers. But then he shows that there's nothing that requires a vector space to have a finite, or even countably infinite, dimension. What matters are the axioms.

[ttoinou]

The article :

  You may recall that this representation is only one example of an abstract vector space. There are many other types of vectors, such as lists of complex numbers, graph cycles, and even magic squares.

  However, all of these vector spaces have one thing in common: a finite number of dimensions. That is, each kind of vector can be represented as a collection of N N numbers, though the definition of “number” varies.

So the goal is to impress the reader by letting him believe we will easily apply our easy linear algebra to real -> real functions. But we can't.

[gizmo686]

Vectors are an abstract notion. If you have two sets and two operations that satisfy the definition of a vector space, then you have a vector space; and we refer to elements of the vector set as "vectors" within that vector space.

The observation here is that set of real value functions, combined with the set of real numbers, and the natural notion of function addition and multiplication by a real number satisfies the definition of a vector space. As a result all the results of linear algebra can be applied to real valued functions.

It is true that any vector space is isomorphic to a vector space whose vectors are functions. Linear algebra does make a lot of usage of that result, but it is different from what the article is discussing.

[ttoinou]

I agree but we're using functions for different things here. Yes some specific families of functions can be treated as vector spaces. In this article it seems like the author is pretending to take all real->real functions and treating them as if they are a vector space, whatever the content of the functions, quote :

  we’ve built a vector space of functions

and later he admits it is impossible

  Ideally, we could express an arbitrary function f as a linear combination of these basis functions. However, there are uncountably many of them—and we can’t simply write down a sum over the reals. Still, considering their linear combination is illustrative:

They are uncountable because they are aleph1

[gizmo686]

It is not required for vector spaces to have a basis. As it turns out, the claim that every vector space has a basis is equivalent to the axiom of choice, which seems well beyond the scope of the article.

However, the particular vector space in question (functions from R to R) does have a basis, which the author describes. That basis is not as useful as a basis typically is for finite dimensional (or even countably unfitine dimensional) vector spaces, but it still exists.

[bmacho]

> As it turns out, the claim that every vector space has a basis is equivalent to the axiom of choice, which seems well beyond the scope of the article.

> However, the particular vector space in question (functions from R to R) does have a basis, which the author describes.

No, there is no known constructible basis for R -> R functions.

[ttoinou]

But it's the article talking about vectors as a sequence of reals and having a basis, then extending that to infinite sequences of reals. The author is playing on multiple definitions of vector to produce a "woaw that's cool" effect, and that's bad maths

[Sharlin]

There is only one definition of "vector space" (up to isomorphism anyway), and that's what the author uses. You'll note that he doesn't talk about bases at all, the assumption of a basis is entirely in your mind. The entire point of the article is that the ℝ→ℝ function space is a vector space. A vector space is not required to have a basis, but assuming the axiom of choice, every vector space does have (at least) one, including that of ℝ→ℝ functions.

[ttoinou]

The choice of basis is very important for the applications. It doesnt matter that function could be vector spaces in theory without constructing a base, the article is about "hey look at functions they're as easy to operate in as this thing you already know with 3D vectors"

[MarkusQ]

You are misreading the article. At a guess, you have come into it "knowing what a vector is" from physics or computer science. Specifically, a vector is not "a sequence of reals", it is any object that obeys a certain set of properties. He is not "extending" your definition, he's trying to teach you the actual definition, which is more general than the examples you are used to.

(Imagine/remember what it feels like when children first learn that the integers aren't the only number, there are also fractions, then irrationals, then complex numbers...this is a very similar situation).

With that in mind, you may want to reread the text and pay attention to the definitions he is using, and not assume that your definitions are the whole story.

[ttoinou]

I learned vector spaces in maths and I'm aware of your definition, that's how I learned it. I'm also aware that the same word can be used with different meanings depending on the context.

  (Imagine/remember what it feels like when children first learn that the integers aren't the only number, there are also fractions, then irrationals, then complex numbers...this is a very similar situation).

Imagine now an article saying "complex numbers are just natural numbers, you can apply euclidean division to them the exact same way"

[998244353]

The set of all real->real functions is still a vector space.

This vector space also has a basis (even if it is not as useful): there is a (uncountably infinite) subset of real->real functions such that every function can be expressed as a linear combination of a finite number of these basis functions, in exactly one way.

There isn't a clean way to write down this basis, though, as you need to use Zorn's lemma or equivalent to construct it.

[seanhunter]

If you restrict yourself to Lebesque-integrable functions, can’t you take the complex Fourier transform of the function and call the terms of the Fourier series a basis, with the coefficients being the components of the vector of the function? This is a bit above my current mathematical paygrade, so forgive me if I’m not expressing the idea accurately but I’m learning a lot both from the article and the ensuing discussion - hopefully you understand what I’m getting at.

I think what I may be asking is “Does the complex Fourier transform make a Hilbert space?” but I might be wrong both about that and about that being the right question.

[MITSardine]

Sorry, I couldn't find the page in English, but what you're talking about is a Hilbert basis: https://fr.wikipedia.org/wiki/Base_de_Hilbert . There is a paragraph on this in the orthonormal basis English page: https://en.wikipedia.org/wiki/Orthonormal_basis

Another example is the eigenvectors of linear operators like the Laplacian. Recall how, in finite dimension, the eigenvectors of a full rank operator (matrix) form an orthonormal basis of the vector space. There is a similar notion in infinite dimension. I can't find an English page that covers this very well, but there's a couple of paragraphs in the Spectral Theorem page (https://en.wikipedia.org/wiki/Spectral_theorem#Unbounded_sel... ). The article linked here also touches on this.

Regarding your last sentence, one thing to note is that having a basis is not what makes you a Hilbert space, but rather having an inner product! In fact, to get the Fourier coefficients, you need to use that inner product.

[seanhunter]

That's awesome info thank you so much. Reading it, a Hilbert basis is exactly what I am talking about. It's always exciting when my intuition guides me on the right path. I'll check out the Spectral theorem page also.

[998244353]

No, and this is where this formal notion of basis I mentioned unfortunately diverges from what is perhaps more useful in practice.

You can represent any function f: [-pi, pi] -> R as an infinite sum

    f(x) = sum_(k = 0 to infinity) (a_k sin(kx) + b_k cos(kx))

for some coefficients a_k and b_k as long as f is sufficiently nice (I don't remember the exact condition, sorry).

This is very useful, but the functions sin(x), sin(2x), ... , cos(x), cos(2x), ... don't constitute a basis in the formal sense I mentioned above as you need an infinite sum to represent most functions. It is still often called a basis though.

[seanhunter]

Thanks for that. That’s the trigonometric Fourier series. A complex Fourier series (which is what I mentioned) is equivalent and works similarly except that the terms are all uniform, so

   f(x) = sum n=-infty to +infty C_n e^{i n x}

You can derive one from the other by using the identities

  sin x = (e^(ix) - e^(-ix))/2i

  cos x = (e^(ix) + e^(-ix))/2 

I specifically mentioned the complex series because I didn’t like the fact that the alternating terms use a different trig function and it seemed weird to me to have every second dimension in a space be different in that way but they are equivalent.

The convergence criteria for Fourier series vary depending on how strongly you need convergence but I think basically if a function is differentiable on the interval you care about then the Fourier series provably converges on that interval and otherwise if it has jump discontinuities and that sort of thing, then depending on whether it is square-integrable or a bunch of other properties) you can prove weaker forms of convergence (absolute, pointwise etc).

To address your comment I don’t see why an infinite sum prevents something being a basis. In fact I would specifically say that can’t be true because then there would never be a basis for any infinite-dimensional space- any time you want to take an inner product in such a space you need an infinite sum, and you need such an inner product to construct the basis. A sibling comment pointed me in the direction of a Hilbert basis, which seems to be what I was thinking of.

[eapriv]

You can’t, because a Fourier series is not a linear combination.

[ttoinou]

Linear combination of what ?

[ttoinou]

I'd love to read more about that, he's not talking about that at all in this article though

[tel]

If you're familiar with Zorn's Lemma, the construction is just to order bases by inclusion and to consider chains created by noting that there must be an independent dimension and adding it inductively. You can upper bound each of these chains by unioning the members of the chain (which preserves linear independence). By Zorn's Lemma that means there is a maximal linearly independent system and if an element existed outside of that system's span it would contradict that maximality.

[lmm]

He doesn't need to talk about it (though you might like to look up the notorious Hilbert's Basis Theorem); it happens to be the case that any vector space has a basis, but even if you don't know that, a vector space is still a vector space and its elements are still vectors.

[ttoinou]

The choice of basis is very important for the applications, which is what the article is about

[bmacho]

> Vectors are functions whose input space are discrete dimensions.

This is _also_ true. But the fact that finite dimensional real vectors can be viewed as a special case of set-theoretic functions or as a special case of real functions on a discrete finite space is probably less useful than the opposite: the set of real functions have a vector-space structure, and you can use all the neat theorems about (finite or infinite dimensional) vector spaces.

Well, at least the article is about the latter direction.