Are you trying to learn Dijkstra/Hoare/Floyd style programming techniques or are you trying to understand the idea of usage of "Markov Processes" (totally unnecessary) in the comment?
Ah, this is a more general question on which it is hard to give specific suggestions because "problem-solving" can mean anything/everything without knowing your exact context.
However you might want to start with The Thinker's Toolkit by Morgan Jones. This gives you a catalog of models which teaches you to structure your requirements born from problem analysis in various ways to aid problem-solving.