Probability in Human Communication: Shifting Perspectives

The problem of irreconcilable differences in the understanding of information, whether as syntactic or semantic, physical or epistemic, which I discussed in the previous chapter, is also related to fundamental differences in the interpretation of the central concept of information theory: probability. As we remember, Shannon’s information theory, as well as other theories based on it (such as algorithmic information theory), utilises the concept of entropy from statistical mechanics, which is mathematically equivalent to Gibbs entropy. The concepts of probability and randomness connect the informational and thermodynamic concepts of entropy, defining information as the negative logarithm of probability.