[alextgordon]

The traditional pure mathematical of looking at it is that vectors are members of a vector space. Matrices are linear maps, and matrix multiplication is composition of linear maps.

So what algebraic concept do tensors correspond to?

[jwmerrill]

Multilinear maps. You can view contraction with a vector (a dot product) as mapping a vector into the scalars, contraction with a matrix as mapping two vectors into the scalars, and contraction with a tensor as mapping several vectors into the scalars. You don't always have to contract all the indices at once, so with a rank m tensor, you can map n vectors into a collection of m - n vectors.