What is a Markov Chain?
A Markov Chain is a special stochastic process. The aim is to give probabilities for future events when applying this chain. This Markov chain is defined in such a way that even with knowledge of only a limited past history, a forecast of a future development is just as good as with knowledge of the entire past history of a process.
Markov chains can be distinguished according to different orders. Thus, a first-order Markov chain is defined in such a way that the future state of a process is conditioned only by the current state and is not influenced by past states. The mathematical formulation may require only the notion of a discrete distribution and a conditional probability in the case of a finite set of states and the concepts of a filtration and a conditional expectation in the continuous-time case.
Markov chains can be excellently used to model random state changes in a system, if there is a reason to assume that the state changes only influence each other over limited periods of time or that they are even memoryless.
There are discrete finite Markov chains and discrete infinite Markov chains.
What are Markov chains used for?
Markov chains are simple and descriptive models to represent real-world processes mathematically accurately. For example, given known and assumed constant probabilities, it is possible that the probable state of a system can be predicted for any future. Markov chains are the basis for stochastic processes, which on the one hand can be based on memoryless randomness, and on the other hand can be possible with state transitions to given probabilities in each case. The Markov chains can model general stochastic Petri nets and rankings based on subjective recommendations.
What are the areas of application?
Markov chains are used in spam filters, for example, which are far more effective than Bayesian filters. Queuing processes and probability distributions of objects in moving systems can also be calculated very easily. In addition, it is possible to create an objective ranking for the entire system based on many subjective recommendations.
Google's PageRank is also based on Markov chains. Classic applications are queues and exchange rates. The behaviour of a dam can also be modelled with the help of a Markov chain. It is also possible to model a speed control system for motor vehicles. The analytical evaluation of mobility algorithms, such as random walk, is also possible.
Population dynamics can also be modelled to predict population growth of humans or animals. Brownian molecular motion can also be modelled. Statistical programming and the simulation of equilibrium distributions with the software "Statistik Software R" are particularly relevant.
