Entropy Estimation

In this chapter, we face the issue of making inference on the heterogeneity of a system, i.e. of estimating entropy accounting for, possibly complex, dependence structures. The idea is that the available observations are a (temporary) picture of an underlying and unknown process that governs the actual diversity of the system. Therefore, data are considered as a sample extracted from such process, and used for understanding what the latent heterogeneity of the system is. Under this approach, entropy is not a descriptive measure that synthesizes the observations into a single number, rather it aims at capturing the possible sources of heterogeneity, which can be related to environmental factors and/or spatial or temporal effects. In the literature, hardly any work is available dealing with the consideration of covariates or unobservable effects into entropy indices, and the existing entropy estimators are based on the very strong hypothesis of independence between realizations. The present chapter starts by offering an extensive review, in a unified framework, of the main proposals for entropy estimation coming from frequentist, non-parametric and Bayesian approaches. Then, a Bayesian model-based approach for multinomial data is presented, able to properly estimate local (e.g. location- or time-specific) probabilities for the categories of the study variable, which can include any data dependence on fixed or random effects. We also summarize the main findings of an empirical study based on assessing the different performance of such proposal wrt the existing methods, and lastly show how to estimate entropy over the case studies used in the present book.