Markov chain models in practice: a review of low cost software options

Abstract

Markov processes (or Markov chains) are used for modeling a phenomenon in which changes over time of a random variable comprise a sequence of values in the future, each of which depends only on the immediately preceding state, not on other past states. A Markov process (PM) is completely characterized by specifying the finite set S of possible states and the stationary probabilities (i.e. time-invariant) of transition between these states. The software most used in medical applications is produced by TreeAge, since it offers many advantages to the user. But, the cost of the Treeage software is relatively high. Therefore in this article two software alternatives are presented: Sto Tree and the zero cost add-in package "markovchain" implemented in R. An example of a cost-effectiveness analysis of two possible treatments for advanced cervical cancer, previously conducted with the Treeage software, is re-analyzed with these two low cost software packages.