There is a way to look at mathematics as just a bunch of rewrite rules for things on paper. It might not be particularly inspiring, but it's a valid way to look at things.
Indeed, there's a way to get a semantics for free, based on the syntax alone. For example, in the first order logic this is the Herbrand interpretation
https://en.wikipedia.org/wiki/Herbrand_interpretation
The point of mathematical semantics is that for any given theory, we can have other interpretations that don't just interpret symbols as themselves.
So we could conceivably imagine an interpretation where ∞ doesn't just mean literally ∞ and nothing more.