Abstract: We show that the bijectivity of normalising flows means they are misspecified for modelling target densities whose support has a different topology from the prior. In this case, we prove that the flow must become arbitrarily close to noninvertible in order even to approximate the target closely. This result has implications for all flow-based models, and particularly residual flows (ResFlows), which explicitly control the Lipschitz constant of the bijection used. To address this, we propose continuously indexed flows (CIFs), which replace the single bijection used by normalising flows with a continuously indexed family of bijections, and which intuitively allow rerouting mass that would be misplaced by a single bijection. We prove that CIFs can exactly match the support of the target even when its topology differs from the prior, and obtain empirically better performance for a variety of models on a variety of benchmarks.