Abstract:
Learning domain-invariant representations is a popular approach to unsupervised domain adaptation, i.e., generalizing from a source domain with labels to an unlabeled target domain. In this work, we aim to better understand and estimate the effect of domain-invariant representations on generalization to the target. In particular, we study the effect of the complexity of the latent, domain-invariant representation, and find that it has a significant influence on the target risk. Based on these findings, we propose a general approach for addressing this complexity tradeoff in neural networks. We also propose a method for estimating how well a model based on domain-invariant representations will perform on the target domain, without having seen any target labels. Applications of our results include model selection, deciding early stopping, and predicting the adaptability of a model between domains.
