[dbranes]

Sorry this is just wrong. Maybe you’re trying to get at the “co/contravariant” properties of tensors, in which case your statement can be more clearly stated as, e.g., the space rank (2, 0), and rank (1,1) vectors, admit different interpretations as internal hom spaces of vector spaces. But in any interpretation of your statement the distinction is never important because all spaces distinguished by this distinction are isomorphic via cononical isomorphisms.

[ska]

You might want to think that through a bit more, leaving aside the issue of needing the underlying vector space, are you sure you are comfortable with the statement that all matrices are tensors?

[dbranes]

Yes absolutely. Given a matrix as an array of numbers there's a number of natural ways to interpret it as a tensor. What are you uncomfortable about?