[bmacho]

The first two interpretations are about matrix-vector products, and the third is about matrix-matrix product. This is a bit surprising, as I was expecting different interpretations of the same thing.

Anyway these are basic stuff, but it's nice to see them formulated!

[nivter]

Originally I had planned to add a line or two explaining how a matrix-matrix product is just a _list_ of matrix-vector products but then dropped the idea to keep it solely focused on the interpretations. I will probably add it to make it all about matrix-matrix products.

[adrian_b]

The 3 interpretations correspond to the 3 possible choices for the innermost loop when you reorder the 3 nested loops of a matrix-matrix product (tensor product of 2 vectors, AXPY operation of 2 vectors and scalar product of 2 vectors).

For a matrix-vector product, where there are only 2 nested loops, only 2 of these choices are applicable (AXPY operation of 2 vectors and scalar product of 2 vectors).

For a vector-vector product, where there is a single loop, only 1 of these choices is applicable (scalar product of 2 vectors).

For numeric computations, where possible, AXPY is preferable to scalar product, and where possible, tensor product is preferable to AXPY. Therefore matrix-vector products should be done with AXPY and matrix-matrix products should be done with tensor products of vectors.

I strongly dislike the misuse of the term "outer product" for the tensor product of 2 vectors, like in the parent article, because this is ambiguous. The original definition of the outer product of 2 vectors (due to Grassmann) is related to the so-called vector product of 2 vectors and "outer" refers to the fact that one vector is multiplied by the component of the other vector that is orthogonal to it, so it points outwards. The tensor product, whose value is a matrix, is completely different. For 2 vector operands, there are 3 distinct products, the first is inner a.k.a. scalar, the second is outer product or (only in 3 dimensions) its variant "vector product" and the third is the tensor product.

The name "tensor product" is also historically and semantically incorrect, but at least it is not ambiguous.

The "tensor product" should have been named "Zehfuss product", after the mathematician who has defined it in 1858 (about 60 years before the word "tensor" had any relationship with it). "Tensor" was originally a name for symmetric matrices, which correspond to affine transformations where a body is stretched in the directions of certain axes. The name was introduced by Hamilton, together with scalar (similarity affine transformation), vector (translation affine transformation) and versor (rotation affine transformation). For unknown reasons Einstein has chosen to use "tensor" with the meaning of "multi-dimensional array", breaking with the tradition, then everybody has imitated him (due to the huge popularity of the Theory of Relativity among non-mathematicians and non-physicists, after the end of WWI, which prompted some book editors to insert the word "tensor" everywhere in several mathematics books and advertise them as being useful to understand the theory of Einstein).