[evanb]

You will have a hard time constructing Markov Chain that correctly models the real-time evolution of physical quantum-mechanical observables. The problem is that the transition matrix that governs quantum-mechanical evolution is exp[i t H], which is not a well-formed probability distribution.