khadar134 / markovprocess

EXP NO-5

Markov Process

Aim :

Software required :

Python

Theory:

Markov chains, named after Andrey Markov, a stochastic model that depicts a sequence of possible events where predictions or probabilities for the next state are based solely on its previous event state, not the states before. In simple words, the probability that n+1th steps will be x depends only on the nth steps not the complete sequence of steps that came before n. This property is known as Markov Property or Memorylessness.

Assumptions for Markov Chain :

1. The statistical system contains a finite number of states.
2. The states are mutually exclusive and collectively exhaustive.
3. The transition probability from one state to another state is constant over time.

Procedure :

Program

# Developed by
# Name: SHAIK KHADAR BASHA
# Register Number: 212220230045

import numpy as np
P0=[0.3,0.2,0.5]
P=[[0,2/3,1/3],[1/2,0,1/2],[1/2,1/2,0]]
n=8
for i in range(1,n+1):
    P0=np.multiply(P0,P)
    print("\nThe %d - step probability distribution is\n"%i)
    print(P0)

Output

Result

Thus, the n-th step probability distribution matrix of the three state Markov chain was calculated.