Markov chain with external input

by Miss Q   Last Updated May 22, 2020 23:20 PM

Could anyone explain to me this Markov chain model? $$S_{k+1}= P(S_k+S_k^0).$$

Please allow me to give a link from the paper I was read this equation $(6)$ here

I was actually reading this paper but they refer to the above model which I didn't understand. Thanks for a bit of discussion and over-all idea about the model. Thanks a lot.

Answers 1

A simple Markov chain model multiplies an $N\times N$ square matrix of transition probabilities ($P$ in your question) by a state vector of height $N$ ($S_k$) in your question. (I'm sure you know this, but sometimes $P$ is on the right or left, which affects the structure of that matrix--i.e. whether it's the rows or columns that must each sum to 1 because they represent probabilities. This difference is illustrated in the two papers.)

So in a simple Markov chain model, the only thing that produces a new state $S_{k+1}$ is the transition probability matrix $P$ operating on the old state $S_k$. We multiply the the square matrix $P$ and the column vector $S_k$ to produce the new state vector $S_{k+1}$.

In the model you are asking about, there is also an external source of change at time $k$, a vector of the same size as $S_k$, named $S^0_k$. This represents changes to the values represented in $S_k$ from some other source. It looks like this represents newly infected people to be added in to the vector of states of previously infected people, but I have not read the article closely.

Once the vectors $S_k$ and $S^0_k$ are added together, the next step proceeds just as in the simple Markov model: $P$ is multiplied by the resulting column vector.

(You really should provide more background in the question, explaining exactly where you got stuck, rather than asking someone to get all of the relevant background information from external sources.)

May 22, 2020 23:17 PM

Related Questions

Markov distribution

Updated April 21, 2017 16:20 PM

Expected value for Markov CHAIN

Updated December 12, 2017 00:20 AM