The Pythagorean law of relative entropy was proposed to attribute the conditional mutual information (MI) in an \(I\times J\times K\) contingency table to the sum of two orthogonal components: one is the goodness-of-fit deviance between the observed table and fitted one assuming ( \(I-1)\times (J-1)\) odds ratios (ORs) to be homogeneous across \(K\) levels, and the other is the deviance between the table of homogeneous ORs and the one fitted by assuming ORs = 1 (Cheng, Liou, & Aston, 2010). Expected cell counts in both fitted tables are estimated by the maximum likelihood estimation method. In this study, we elaborated the key idea behind the Pythagorean law and illustrated its potential role in categorical data analysis and modern big-data applications. Computations of empirical examples were executed using SPSS and R packages such that interested readers may easily replicate similar data analysis. An empirical comparison between methods for assessing differential item functioning (DIF) was conducted and the MI decomposition approach was shown to be a potential competitor to other existing methods for detecting uniform and nonuniform DIF. In summary, empirical results suggested that conventional approaches to modeling associations between categorical variables could lead to less precise inference if the Pythagorean law and MI decomposition rules are overlooked. Because the Pythagorean law has innovated new challenges and insights into modeling associations in contingency table analysis, further applications, such as feature selection in high-dimensional data, are anticipated.
