A Time-Inhomogeneous Markov Chain with Rate Control
摘要
In this paper, we implement control over the Markov chain, which causes it to become a time-inhomogeneous Markov chain. One of its most prominent features is that the transition rate matrix varies over time, which makes the time-inhomogeneous Markov chain with rate control can be more congruent with practical situations. Nevertheless, it is precisely this characteristic that renders the analysis of it more difficult. Thus we present a framework to analyze its asymptotic behavior based on the corresponding probability equations. In particular, by deriving the convergence of its solution, we can get the steady-state distribution of the underlying time-inhomogeneous Markov chain. In addition, we also provide an example to demonstrate this framework for substantiating its feasibility. Finally, we conducted numerical experiments and the results indicate that it can reach equilibrium as time approaches infinity, which is consistent with the convergence of the solution to its probability equations.