In the domain of Deep Neural Network (DNN) optimization, the precision of quantization emerges as a critical factor in harmonizing computational demands with performance efficacy. Traditional mixed-precision quantization methods primarily rely on various approximations and relaxations, and the search for the optimal bit-width configuration is also time-consuming. This study introduces a novel approach, Mutual Information-based Mixed Precision Quantization (MIMPQ), which adeptly fine-tunes bit-width allocation across different layers of DNNs. Utilizing mutual information as a metric, MIMPQ thoroughly assesses the informational interplay between layers, facilitating a more informed and dynamic quantization process. This methodology stands in stark contrast to traditional sensitivity-driven approaches that might neglect the nuanced inter-layer relationships, potentially culminating in suboptimal quantization schemes. MIMPQ has been systematically evaluated on benchmark datasets such as ImageNet, employing prevalent architectures like ResNet18/50 and MobileNet. By leveraging Pareto optimization strategies, MIMPQ effectively navigates the trade-off between model compactness and predictive accuracy. Empirical results highlight MIMPQ’s prowess in achieving a superior balance, often outperforming other mixed-precision quantization methods in both model size and accuracy.