Entropy: Discrete Case
摘要
The main character of this chapter is the entropy function ( 1.2.9 ), \(\displaystyle H=-\sum _{k=1}^Kp_k{\log _2}p_k, \) introduced via the source partition theorem. Most of the chapter is devoted to the case of random variables defined on a finite probability space (but see Remark 8.1.10 for an interpretation of the result in an infinite probability space).