Markov chains are indispensable models for cascaded systems in signal processing, communications, and information theory. Information theoretic descriptions of systems using Markov chains play a dominant role in communications and compression applications. In this chapter, we provide a brief introduction to results from communications and compression but in a more general context. Plus, we relate the classical measure of performance in estimation and prediction, mean squared error, to information theoretic descriptions of Markov chain models of cascaded systems.