[blackbear_]

The fundamental truth is that matrices represent linear transformations, and all of linear algebra is developed in terms of linear transformations rather than just grid of numbers. It all becomes much clearer when you let go of the tabular representation and study the original intentions that motivated the operations you do on matrices.

My appreciation for the subject grew considerably after working through the book "Linear Algebra done right" by Axler https://linear.axler.net

[srean]

> The fundamental truth is that matrices represent linear transformations, and all of linear algebra is developed in terms of linear transformations...

Not at all. It is most definitely not a fundamental truth. I think you are conflating matrix multiplication with matrices, as if that's the only operation that one does on matrices.

Matrices are a way to organize data and what operations you define over the matrices define what it represents.

There are other useful operations on matrices, for example Hadamard products, Schur products, Kronecker product, face split products of matrices that finds use in Physics and Engineering (antenna design for one) where the operation does not represent a linear transformation. They can form an algebra and/or algebraic structures different from the very familiar linear algebraic one.

[zkmon]

Spatial transformations? Take a look at the complex matrices in Fourier transforms with nth roots of unity as its elements. The values are cyclic, and do not represent points in an n-D space of Euclidean coordinates.

[blackbear_]

Yes; I wrote linear transformation on purpose not to remain constrained on spatial or geometric interpretations.

The (discrete) Fourier transform is also a linear transformation, which is why the initial effort of thinking abstractly in terms of vector spaces and transformations between them pays lots of dividends when it's time to understand more advanced topics such as the DFT, which is "just" a change of basis.