Density Functions and Entropy

Probability measures on the Borel σ-field \(\mathcal{B}^{n}\) are often represented by density functions. For this representation, an integration theory is needed, which we now recapitulate (see [Bau92]). With \(\bar{\mathbb{R}}:=\mathbb{R}\cup\{-\infty,+\infty\}\) an extension of the set of all real numbers is defined. The algebraic structure of \(\mathbb{R}\) is extended to \(\bar{\mathbb{R}}\) as follows: For all \(a\in\mathbb{R}\) the following applies: \(\begin{aligned}\displaystyle a+(\pm\infty)=(\pm\infty)+a=(\pm\infty)+(\pm\infty)=(\pm\infty),\quad+\infty-(-\infty)=+\infty,\\ \displaystyle a\cdot(\pm\infty)=(\pm\infty)\cdot a=\begin{cases}(\pm\infty),&\text{ for }a> 0,\\ 0,&\text{ for }a=0,\\ (\mp\infty),&\text{ for }a<0,\end{cases}\\ \displaystyle(\pm\infty)\cdot(\pm\infty)=+\infty,\quad(\pm\infty)\cdot(\mp\infty)=-\infty,\quad\frac{a}{\pm\infty}=0.\end{aligned}\) Thus, \(\bar{\mathbb{R}}\) is not a field . The signs at \(\pm\infty\) must not be combined in the above formulas, because the expressions “ \(+\infty+(-\infty)\) ” and “ \(-\infty+(+\infty)\) ” are not defined. Caution is required with the limit theorems: \(\lim\limits_{x\to+\infty}\left(x\cdot\frac{1}{x}\right)\neq(+\infty)\cdot 0=0.\)