Disentanglement via Adaptive Information Bottleneck in Latent Dimensions
摘要
Disentangled representation learning aims to identify the ground-truth factors underlying the data and represent them independently. Since the ground-truth factors cannot be directly accessed, achieving disentanglement in a purely unsupervised manner is infeasible. We argue that inductive bias is crucial for enabling disentanglement. In this work, we demonstrate the role of inductive bias in the successful learning of disentangled representations with \(\beta\) -VAE. Additionally, we utilize inductive biases to implement distinct information bottlenecks for each latent variable dimension, adjusting them adaptively to replace the overall constraint on the latent variables. This method allows the model to learn disentangled representations that exceed those produced by a single global constraint. Our method’s robustness and effectiveness were validated on several benchmark datasets, including dSprites and shapes3d.