<p>Shannon entropy for discrete distributions is a fundamental and widely used concept, but its continuous analogue, known as differential entropy, lacks essential properties such as positivity and compatibility with the discrete case. In this paper, we analyze this incompatibility in detail and illustrate it through examples. To overcome these limitations, we proposemodified versions of Shannon and Rényi entropy that retain key properties, including positivity, while remaining close to the classical forms. We also define compatible discrete functionals and study the behavior of the proposed entropies for the normal and exponential distributions.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Differential Shannon and Rényi entropies revisited

  • Yuliya Mishura,
  • Kostiantyn Ralchenko

摘要

Shannon entropy for discrete distributions is a fundamental and widely used concept, but its continuous analogue, known as differential entropy, lacks essential properties such as positivity and compatibility with the discrete case. In this paper, we analyze this incompatibility in detail and illustrate it through examples. To overcome these limitations, we proposemodified versions of Shannon and Rényi entropy that retain key properties, including positivity, while remaining close to the classical forms. We also define compatible discrete functionals and study the behavior of the proposed entropies for the normal and exponential distributions.