[dojomouse]

++ this. The branches of maths that are relevant to ML are all pretty extensive, and you can do a huge amount of applied ML and understand the underlying theory while understanding only a fairly small subset of (for e.g.) probability theory.

Starting by learning the maths will mean you learn a lot of stuff which isn't directly relevant. Not the worst thing that could happen, but you'll be a hell of a lot more directed (even if you want to learn the theory - and I would recommend learning at least some) if you pick a decent ML course and learn the maths you bump into as you go.

http://cs231n.github.io/ is one of the best general hands-on introductions I've found. The TF tutorials are pretty good too if you just want to try some things out, but I predict that once you've worked your way through the TF tuts you'll still not really understand what's going on and will feel a bit like you just learned the magic words that made the black box dance some particular dances.

As for notation - https://en.wikipedia.org/wiki/List_of_mathematical_symbols is really good for when you stumble on something unfamiliar.

Good luck.