A Summary of Markov Chains
摘要
This chapter reviews the concepts of discrete-time Markov process and matrix-based transition probabilities, which are central tools in this book. We also cover related techniques for the computation of hitting probabilities and mean hitting and absorption times, which will be applied in subsequent chapters. This chapter is mostly self-contained, to the exception of some proofs for which the reader is referred for conciseness to the relevant statements in the literature.