Information and Entropy

This chapter presents the notions connected to Shannon’s entropy and information, namely the joint, conditional, relative (Kullback–Leibler) entropies, and the mutual information, with their implementations in R. Applications to surprise quantification are shown, with their implementation in R. Examples show the use of the programs presented.