We propose a new concept of codivergence, which quantifies the similarity between two probability measures \(P_1, P_2\) relative to a reference probability measure \(P_0\) . In the neighborhood of the reference measure \(P_0\) , a codivergence behaves like an inner product between the measures \(P_1-P_0\) and \(P_2-P_0\) . Codivergences of covariance-type and correlation-type are introduced and studied with a focus on two specific correlation-type codivergences, the \(\chi ^2\) -codivergence and the Hellinger codivergence. We derive explicit expressions for several common parametric families of probability distributions. For a codivergence, we introduce moreover the divergence matrix as an analogue of the Gram matrix. It is shown that the \(\chi ^2\) -divergence matrix satisfies a data-processing inequality.
