What the hell is a markov model ?

Markov Models represent things that are happening in the real world

Say your at t a casino . 

• a markov model (poker table, where states depend on previous states),

• a multinomial distribution (where things are stateless, so every time you select something it will have the same probability), 

• a hidden markov model (is a poker game where you spin the roulette wheel every turn and change the deck based on the wheel result, in other words, the MARKOV MODEL is HIDEN BEHIND A ROULETTE WHEEL).

RJ has written some interesting posts lately about the transition from a markov model , which was not working in an ideal way,  to a much simpler multinomial distribution, in BigPetStore http://rnowling.github.io/math/2015/07/11/customer-segmentation-bps-multinomial.html.


It seems to me that a markov model is better for interesting patterns - it has the possibility of creating "wells" and loops which are salient and unique even in very long running simulations.   A multinomial distribution of course will be less likely to have such effects, because the behvaiour of the model doesnt vary as much in local time slices.