Deep representation learning for domain generalization with information bottleneck principle.

• A theoretical framework for domain generalization (DG) from the perspective of information bottleneck (IB) principle is established. • Based on the framework, a feasible solution by class-wise instance discrimination (CID) combined with maximizing feature entropy (MFE) is proposed to learn the desired representation for DG. • The proposed method achieves excellent performance on multiple datasets without knowing domain labels. • The proposed regularization rule (MFE) improves other invariance-based DG methods consistently. Although deep neural networks have achieved superior performance on many classical tasks, they deteriorate in real applications due to the unpredictable distribution shift. Domain generalization (DG) focuses on improving the generalization ability of the predictive model in unseen domains by training on multiple available source domains. All these domains share the same categories but commonly obey different distributions. In this paper, we establish a new theoretical framework for domain generalization from the perspective of the information bottleneck (IB) principle, which links representation learning in DG with domain-invariant representation learning and maximizing feature entropy (MFE). Based on the theoretical framework, we provide a feasible solution by class-wise instance discrimination combined with inter-dimension decorrelation and intra-dimension uniformity to learn the desired representation for domain generalization, which achieves excellent performance on multiple datasets without knowing domain labels. Extensive experiments show that the proposed regularization rule (MFE) can improve invariance-based DG methods consistently. Moreover, as an extreme case of domain generalization, we also show that MFE is promising to improve adversarial robustness. [ABSTRACT FROM AUTHOR]