Hacker News new | ask | show | jobs
by kaashif 1561 days ago
It's not clear to me what you're annoyed about exactly. The way I see it, there are a few options:

You're getting annoyed that people are confusing the map with the territory [1]. Multidimensional arrays with certain properties can be used to represent tensors, but aren't tensors. In the same way a diagram of torus isn't a topological space, or a multiplication table isn't a group, or a matrix is not a linear map. Isomorphic but not literally the thing.

Or you're annoyed that people forget an array representing a tensor needs to satisfy some transformation law and can't just be any big array with some numbers in it.

Or maybe you're a fan of basis-free linear algebra!

Which one is it?

1: https://en.wikipedia.org/wiki/Map%E2%80%93territory_relation

2 comments

I mean, if I wanted to refer to the reals exclusively as a vectorspace I wouldn't be wrong, but if you aren't actually using what makes it a vectorspace why would you choose to call it that? Hell, call 7 a tensor. There's more than a map-territory distinction (I'd argue formal mathematics is perhaps the only realm where the two are one and the same, but I see what you're saying), it's a convention of language more generally. You typically use the most necessary term, rather than a random also accurate label. If you don't care about invariance under coordinate transformations (and most machine learning does not), why would you call it a tensor?
I personally am a fan of basis free linear algebra.

More importantly though, “tensors” as commonly used in machine learning seem to rely on a single special basis, so they really are just multidimensional arrays. A machine learning algorithm isn’t really invariant under a change of basis. For example, the ReLU activation function is not independent of a change of basis.