Linear transformations can be more easily visualized as scaling coordinate systems - this is actually a pretty easy way to grok what's actually going on with the math. Big bang theory is pointlessly obscure (and uses four dimensions in its zero-zero-zero-zero because apparently four dimensions is enough to be "ooh mysterious" without making anyone feel like a dolt - just use two dimensions guys - everyone has worked with graph paper) but here's an interesting video[1] that sort of touches on the fact that linear transformations (by definition) keep the origin consistent.
That assumes that the world is naturally described in four dimensions (the three primary spacial ones and time) - but that's just a habit of humans. It's perfectly valid to describe space-time in one dimension or, if you're a bosonic string theorist, 26. The habit of identifying three dimensions (and leaving out obvious things like spin) is just one that's indoctrinated into us by our common mode of education. 3+1D is certainly very sensible - but it isn't the only right answer by a long shot. And, again, 2D is much more socially common - going to 4D really just feels like being pointlessly obscure.
1. Just as an aside - 3Blue 1Brown is an amazing resource https://www.youtube.com/watch?v=kYB8IZa5AuE