[pengaru]

It's ridiculous how much random college-level linear algebra textbook material I stared at before things clicked in the course of just jumping in and exploring 3D graphics and writing my own 3D vector, matrix multiplication and 3D transform headers and using them in making some games in plain C.

At some point it's like "Wait, is linear algebra really just about heaps of multiplication and addition? Like every dimension gets multiplied by values for every dimension, and values 0 and 1 are way more interesting than I previously appreciated. That funny identity matrix with the diagonal 1s in a sea of 0s, that's just an orthonormal basis where each corresponding dimension's axis is getting 100% of the multiplication like a noop. This is ridiculously simple yet unlocks an entire new world of understanding, why the hell couldn't my textbooks explain it in these terms on page 1? FML"

I'm still a noob when it comes to linear algebra and 3D stuff, but it feels like all the textbooks in the world couldn't have taught me what some hands-on 3D graphics programming impressed upon me rather quickly. Maybe my understanding is all wrong, feel free to correct me, as my understanding on this subject is entirely self-taught.

[dxbydt]

> Maybe my understanding is all wrong, feel free to correct me, as my understanding on this subject is entirely self-taught.

I wouldn't say it is all wrong. Just that the stuff you are talking about is a very tiny fraction of LA. I took a graduate class in LA, based on Strang's book. I have the book right here in front of me. So the stuff you allude to, i.e. rotation matrix, reflection matrix & projection matrix, is on p130 of Chapter 2. We got to that in the 1st month of the semester, & it got about 1 hour of classtime total. That's it. An LA class is like 4 months, or 50 hours. If the point of LA to derive those matrices so one can do 3D computer graphics with scaling, rotation & projection ? No, that stuff is too basic. We got 1 homework problem on that, that's it.

The stuff that most of the class struggled with ( & still struggle with, because Strang goes over it rather quickly in his book), is function spaces ( chapter 3, p182), Gram Schmidt for functions ( p184), FFTs, (p195), fibonacci & lucas numbers (p255), the whole stability of differential equations chapter ( he gives these hard and fast rules like a Differential Equation is stable if trace is negative & determinant is positive, but its not too clear why. ), quadratic forms & minimum principles - that whole 6th chapter glosses over too much material imo.

Overall, Strang's book is a solid A+ on how to get stuff done, but maybe a B- on why stuff works the way it works. Like, why should I find Rayleigh quotient if I want to minimize one quadratic divided by another ? Strang just says, do it & you'll get the minimum. How to find a quadratic over [-1,1] that is the least distance away from a cubic in that same space ? Again, Strang gives a method but the why part of it is quite mysterious.

[pengaru]

Thanks for writing this. I intend to pick up a physical copy of Strang's book as it's been repeatedly mentioned in a positive light on HN.

So does LA get substantially more involved than just lots of multiplications and additions or is it always at the end of the day still just bags of floats getting multiplied and summed? Is it just a fantastic rabbit hole describing what values you put where in those bags of numbers?

[Phlogistique]

Is it the same issue as the infamous "monads tutorials" problem, where the understanding takes a lot of time to infuse but looks obvious in retrospect when it finally clicks?

[jerf]

The "monad" problem is worse; many of the "tutorials" were actively wrong about some critical element, often more than one. I don't think I've seen someone claim to have linear algebra "click" but be fundamentally wrong about it somehow.

One advantage of linear algebra is that it is, well, linear. Linear is nice. It means you can decompose things into their independent elements, and put them all together again, without loss. The monad interface, as simple as it is, is not linear; specific implementations of it can have levels of complexity more like a Turing machine.

[bencw]

Maybe I'm biased, but I really don't think so. Monads are quite a bit more abstract than the concepts in linear algebra. Linear algebra is both geometric and algorithmic and therefore very intuitive. Most of the difficulty people have learning linear algebra can be attributed to poor teaching methods.

[madhadron]

That depends on the part of linear algebra. In an abstract function space when you start calculating dimensions of kernels and the like and get ready to make the jump to infinite dimensions, Banach spaces, and Hilbert spaces, it's about as abstract as monads.

[Koshkin]

Well, to be fair, functional analysis is not part of linear algebra proper. (If you want to get more abstract, you go to rings and modules and from there to category theory.)

[Tainnor]

Typically, linear algebra is understood to be the study of finite-dimensional vector spaces, so functional analysis is not necessarily part of it.

However, things like the vector space of polynomials of degree at most n, the vector space of all homomorphisms between two vector spaces, the dual space of a vector space, etc. are all concepts that belong to linear algebra proper yet are more "abstract" than just "computations with matrices".

[madhadron]

That's fair. I may have a bias coming from physics because quantum mechanics demands Hilbert Space Now! from the students.

[ianai]

LA is one of those topics that, to an extent, is built on a handful of core capabilities and concepts. Once you master those much of what follows are logical extensions or combinations of the previous. It goes on from there, but the value returned from the core material is wide reaching.

[basic_bgnr]

"whenever somebody gets a deeper understanding of monads, they immediately lose the ability to explain it to other" I don't remember where I've read this but it still holds even today.

[DagAgren]

I am convinced that monads induce a very specific kind of brain damage that makes a person incapable of ever explaining monads.

[dleslie]

Start with a container.

    M a

Then add a way to put things in the container.

    a -> M a

Then add a way to use the thing in the container.

    M a -> (a -> M b) -> M b

[jerf]

Well, you see, that's one of the problems... monad implementations don't have to be "containers", or at least not the way most people mean. This was one of the critical errors in many of the aforementioned "tutorials". IO, the quintessential monad, is not a container, for instance.

(A nearly-exact parallel can be seen in the Iterator interface. You can describe it as "a thing that walks through a container presenting the items in order"... and yeah, that's the majority use case and where the idea came from... but it's also wrong. What it really is is just "a thing that presents items in some order". It doesn't have to be from "a container". You can have an iterator that produces integers in order, or strings in lexigraphic order, or yields bytes from a socket as they come in, or other things that have no "container" anywhere to be found. If you have "from a container" in your mental model then those things are confusing; if you understand it simply as "presenting items in order" then having an iterator that just yields integers makes perfect sense. A lot of the Monad confusion comes from adding extra clauses to what it is. Though by no means all of it.)

[dleslie]

I wouldn't over-think it and over-describe it.

The "aha" realization that the "container" can be an ephemeral concept and not resident at run time can come later.

FWIW, I think of IO as a container: it contains the risk of side-effects within. All the examples you gave are containers in their own way.

[jerf]

The problem is telling people it's a container is "over describing" it. We don't need to hypothesize about that. We have the space suits and burritos to prove it is not a good didactic approach. It is not removing from the definition to simplify, it is adding to the definition, exactly as I carefully showed in my description of "Iterator". An Iterator is "a thing that presents a series of items". It does not simplify the discussion of Iterator to say "It's a thing that presents a series of items out of a container, but also, it doesn't have to be a container". It's not the first definition that's "overdescribing", it's the second.

[DagAgren]

Yes, well, good example.

[vajrabum]

This explanation did it for me. https://www.reddit.com/r/math/comments/ap25mr/a_monad_is_a_m...

[DagAgren]

Pretty sure my point still stands.

[madhadron]

I've had good luck with explaining it as a characteristic of a programming language. In a language consisting of sequences of statements with bindings and function calls, we expect that

f(x)

is the same as

a = x; f(a)

and the same as

g = f; g(x);

That's the monad laws. Whatever craziness you want to put in the semantics, those are properties you probably would like to preserve in your language.

[DagAgren]

I think I understand monads less now.

[madhadron]

Do you really understand them less, or has it dislodged ideas that you thought were true? Moving towards zero is not always decreasing.

[blablabla123]

I'm no expert, although I think I remember that a Monad is basically just like allowing a sequence of statements to be executed. Like executing a code file ;)

Functional languages are really weird, for instance it's possible to switch line order of statements and the compiler will still figure out how to stitch that together. I think even JS in parts has or at least had that behaviour. (Actually that's useful when having mathematical formulas that are interdependent and you're too lazy to order them topologically by dependence)

On the other hand, just executing a sequence of commands in order to do I/O is only a normal thing to do since recently as far as I understand. The sweet spot for FP is IMHO something like React where state is strictly separated from the functions. (Imagine writing Hello World using Normal Maths)

(Please correct me if I'm wrong, which is probably quite likely ;))

[cbm-vic-20]

"A monad is just a monoid in the category of endofunctors, what's the problem?"

[Koshkin]

I don't think this definition is correct. (A monad is an endofunctor.)

[crdrost]

It's correct but jargony. So in the category of sets there is a notion of product between two sets called the Cartesian product, and one can do a couple things to endow this product with an identity element, for example one might use {{}} as that object in the category of sets.

The claim is that in other categories, there might be other natural combinations between two objects, for example a tensor product of Abelian groups combined with the integers Z as unit, or a composition of two endofunctors into a new endofunctor F ∘ F combined with the identity functor.

So the idea is that a monoid is somehow a destroyer of this combination operation; a monoid in sets un-combines the Cartesian product M × M back into the set M, and indeed this is a function (a set-arrow) from the combined objects to the underlying object.

By having an endofunctor combined with a natural transformation from F ∘ F back to F (natural transformations are the arrows in the category of endofunctors) a monad is therefore doing exactly what a monoid does, if you replace the "pre-monoid" combination step of the Cartesian product with instead a new "pre-monoid" combination step of endofunctor composition.

[howling]

The definition is correct. A monad is an endofunctor with return and join functions. Just like a monoid in the category of sets is a set with identity and multiplication.

[bencw]

Honestly, I think working with computers (possibly some programming) should be more frequently integrated into math courses. A computer is a natural to really interact with the material, like labs in the natural sciences. We're lucky to live in a time where this is possible, but sadly math education is taking it's sweet time taking advantage of this possibility.

[layoutIfNeeded]

Sounds like a case of You Can't Tell People Anything: http://habitatchronicles.com/2004/04/you-cant-tell-people-an...

[nothis]

I guess a mathematician might look down upon sticking with 2d and 3d stuff because it leaves out all the interesting things that happen at 92382 or negative infinity. But yea, matrices are basically just a convenient way to write rows and rows of "ax + by + cz...". In linear algebra, you just do it so often, people made up their own syntax. And nothing can visualize it like transforming graphics, IMO.

You don't even have to go 3D, just starting with the points of a rectangle in 2D and asking, "how do you put the edge points of this rectangle 10px to the left, rotate them 45° and stretch them 200% vertically?" and you've applied a matrix. Even if you're not using the fancy brackets, you're using a matrix, and understanding it.

[dmvaldman]

I think these are good examples, but to me "linear algebra thinking" lies in it's generality. For example, the derivative is a linear operator, so how do you write it down as a matrix? Google's PageRank is a solution of a matrix equation, what does that matrix represent? Etc.

[vlasev]

> For example, the derivative is a linear operator, so how do you write it down as a matrix?

Consider polynomials in X of degree up, but not including N. The powers 1,X,...,X^(n-1) form a basis. Then the coefficients of the polynomial can be put in a column vector. If D is the derivative operator, DX^n = nX^(n-1), so the derivative matrix can be expressed as a sparse matrix with D_(n,n+1) = n. Visually, it's a matrix with the integers 1,2,...,n-1 on the super-diagonal.

You can also see that this is a nilpotent matrix for finite N, since repeated multiplication sends the entries further up into the upper right corner.

You can extend this to the infinite case for formal power series in X, too, where you don't worry about convergence.

> Google's PageRank is a solution of a matrix equation, what does that matrix represent?

Isn't it just the adjacency matrix of a big graph?

Anyway, I agree with you. Matrices and linear algebra is a really good inspiration for higher level concepts like vector spaces and Hilbert spaces and so on. That's where the real power lies. But even in such general domains, matrices are often used to do concrete computations on them, because we have a lot of tools for matrices.

[swiley]

I think the best thing anyone told me about linear algebra was that “matrices are just the coordinate form of a linear map.” So applying the map is equivalent to multiplying it’s matrix etc.

[Koshkin]

Reality, as always, is not that simple. Matrix analysis is a huge area in itself; and matrices can also be used to represent tensors (which generally are not seen as linear maps) and some other things.

[gowld]

> Wait, is linear algebra really just about heaps of multiplication and addition?

That's just one of dozens of things LA is "about"

> why the hell couldn't my textbooks explain it in these terms on page 1

Because you wouldn't have understood terms like

> orthonormal

and because it would have been unhelpful to everyone else who want in LA for the exact same reason you were.

Being obviously in retrospect doesn't mean it was obvious in forespect. You had to learn the material first.

[mangoman]

Do you have any recommendations on videos/tutorials with graphics stuff that would help me grasp some of the linear algebra and basics about 2D/3D graphics? Your comment about 'staring at random college-level linear algebra' stuff resonated deeply with me, and I've always felt like I'm just not understanding how it all connects.

[pengaru]

Not really, I didn't follow any specific guide. But if you like learning from youtube videos Casey Muratori has some decent streams about this stuff on his Handmade Hero channel. The 3Blue1Brown channel also has some relevant videos.

If you've never written a standalone software-rendered ray tracer, I found that to be a very useful exercise early on. There are plenty of tutorials for those on the interwebs.

[samvher]

Not the person you responded to but I found this course to be very good: https://www.edx.org/course/computer-graphics-2