However pretty early in the lectures they jump right into taking partial derivatives of matrices for calculating gradient descent. I think by 'no particular math background' they mean 'assuming nothing more than single and basic multivariable calculus plus linear algebra'. If your linear algebra and/or multi-variable calculus are weak/fuzzy (as they are for me) I think those sections will take quite a while to really understand.
The reference course on Stanford is CS229A instead of CS229, which is much lighter on Math. I have watched the videos on Linear and Logistic regression and the hardest derivations are elided, while the emphasis is put on intuitions.
No particular math background is assumed, and we will go over the math concepts you need in the first couple of weeks.