Rethinking the Information Bottleneck: Structured Decomposition under Label-Induced Partitions

Excerpt

arXiv:2610.01175v1 Announce Type: new Abstract: Standard information bottleneck (IB) regularization constrains representations via a single scalar I(Z;X), implicitlytreating all information as homogeneous. However, a single global compression control couples label-relevant structurewith residual within-condition variation, rather than regulating their allocation independently, allowing nuisanceinformation to persist in learned representations. For example, in medical imaging applications, residu