15 3 First Order Markov Chain | Machine Learning

*FIRST-ORDER MARKOV CHAIN (GENERAL)* Again, we encode this more general probability distribution in a matrix: Mij = p(st = jjst−1 = i) We will adopt the notation that rows are distributions. I M is a transition matrix, or Markov matrix. I M is S × S and each row sums to one. I Mij is the probability of transitioning to state j given we are in state i. Given a starting state, s0, we generate a sequence (s1; : : : ; st) by sampling st j st−1 ∼ Discrete(Mst−1;:): We can model the starting state with its own separate distribution.M