Disentangled representation learning aims to identify the ground-truth factors underlying the data and represent them independently. Since the ground-truth factors cannot be directly accessed, achieving disentanglement in a purely unsupervised manner is infeasible. We argue that inductive bias is crucial for enabling disentanglement. In this work, we demonstrate the role of inductive bias in the successful learning of disentangled representations with \(\beta\) -VAE. Additionally, we utilize inductive biases to implement distinct information bottlenecks for each latent variable dimension, adjusting them adaptively to replace the overall constraint on the latent variables. This method allows the model to learn disentangled representations that exceed those produced by a single global constraint. Our method’s robustness and effectiveness were validated on several benchmark datasets, including dSprites and shapes3d.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Disentanglement via Adaptive Information Bottleneck in Latent Dimensions

  • Xiangtian Zheng,
  • Yuehui Chen,
  • Yi Cao,
  • Yaou Zhao,
  • Dong Wang

摘要

Disentangled representation learning aims to identify the ground-truth factors underlying the data and represent them independently. Since the ground-truth factors cannot be directly accessed, achieving disentanglement in a purely unsupervised manner is infeasible. We argue that inductive bias is crucial for enabling disentanglement. In this work, we demonstrate the role of inductive bias in the successful learning of disentangled representations with \(\beta\) -VAE. Additionally, we utilize inductive biases to implement distinct information bottlenecks for each latent variable dimension, adjusting them adaptively to replace the overall constraint on the latent variables. This method allows the model to learn disentangled representations that exceed those produced by a single global constraint. Our method’s robustness and effectiveness were validated on several benchmark datasets, including dSprites and shapes3d.