The chapter covers information theoretic definitions and properties that will be required in further chapters. It is hence not a comprehensive IT text. It introduces Shannon’s information measures, including Entropy and Differential Entropy, Conditional Entropies (Equivocation and Irrelevance), Mutual Information and Channel Capacity, the Chain Rule, Typical Sequences and the Asymptotic Equipartition Property. Channel Capacities of typical channels are derived: the BSC, the BEC, AWGN with discrete or continuous input, and the bandlimited Gaussian channel. The chapter also introduces the Bhattacharyya bound for maximum likelihood decoding and some Rate-Distortion Theory.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Introduction to Information Theory

  • Werner Henkel

摘要

The chapter covers information theoretic definitions and properties that will be required in further chapters. It is hence not a comprehensive IT text. It introduces Shannon’s information measures, including Entropy and Differential Entropy, Conditional Entropies (Equivocation and Irrelevance), Mutual Information and Channel Capacity, the Chain Rule, Typical Sequences and the Asymptotic Equipartition Property. Channel Capacities of typical channels are derived: the BSC, the BEC, AWGN with discrete or continuous input, and the bandlimited Gaussian channel. The chapter also introduces the Bhattacharyya bound for maximum likelihood decoding and some Rate-Distortion Theory.