[quanto]

The article/webpage is a nice walk-through for the uninitiated. Half the challenge of doing matrix calculus is remembering the dimension of the object you are dealing with (scalar, vector, matrix, higher-dim tensor).

Ultimately, the point of using matrix calculus (or matrices in general) is not just concision of notation but also understanding that matrices are operators acting on members of some spaces, i.e. vectors. It is this higher level abstraction that makes matrices powerful.

For people who are familiar with the concepts but need a concise refresher, the Wikipedia page serves well:

https://en.wikipedia.org/wiki/Matrix_calculus

[PartiallyTyped]

Adding, these operators are also "polymorphic"; for matrix multiplication the only operations you need are (non commutative) multiplication and addition; thus you can use elements of any non-commutative ring, i.e. a set of elements with those two operations :D

Matrices themselves form non-commutative rings too; and based on this, you can think of a 4N x 4N matrix as a 4x4 matrix whose elements are NxN matrices [1] :D

[1] https://youtu.be/FX4C-JpTFgY?list=PL49CF3715CB9EF31D&t=1107

You already know whose lecture it is :D

I love math.. I should have become a mathematician ...

[tikhonj]

You can even generalize linear algebra algorithms to closed semirings and have some really cool algorithms pop out, like finding the shortest path in graphs. There's a great paper called "Fun with Semirings" that goes into more details; unfortunately looks like the PDF isn't easily available online any more, but I found some slides[1] that seem to cover the same ideas well enough.

[1]: https://pdfs.semanticscholar.org/2e43/477e26a54b2d1a046c2140...

[PartiallyTyped]

Okay I went over the slides and good lord this would have made my life easier not too long ago.

[PartiallyTyped]

This deserves its own HN post imho.

[mrfox321]

Re [1]: it's fairly concrete to simply say that matrix multiplication can be performed block-wise.

[PartiallyTyped]

I don’t disagree; but that is just an example of MM. The gist is not that you can do block multiplication; but that you can define matrices over any non commutative ring, which includes other matrices - ie blocks.

[mrfox321]

Yeah matrices are more abstract. I guess I am just pointing out that your concrete example of non-commutative rings (matrices of matrices) still needs a proof to demonstrate bijection between 4N x 4N (scalar) and 4 x 4 (N x N(scalar)).

Block MM demonstrates the equivalence.