Looking at the table of contents (for someone who is not familiar with the term 'Probabilistic Machine Learning'), is this just covering typical ML methods through the lens of probability?
Answer is not so black and white as everything in ml has to use probability. You can ignore this unless you are among 20 top researchers who are working on frontier of ml. Bayesian probabilistic techniques does not work or are very slow for any practical purpose.
Yeh for sure, but it's an overkill. It's like reading quantum mechanics to understand Newtonian mechanics. If you want to get a feel of bayesian ml here is an easier book, "Regression and Other Stories" https://avehtari.github.io/ROS-Examples/