Abstract:
To use multiple attentions in semantic segmentation task, attentions are generally arranged in parallel and then combined by concatenation or summation. In this work, we use Neural Architecture Search (NAS) to explore attentions and their combination patterns (e.g., parallel, hybrid parallel+sequential) in semantic segmentation. We propose AttNAS, which searches both backbone and attention for a lightweight semantic segmentation model. In particular, we define a new attention search space with a two-layer structure and propose a unified differentiable formulation to support both attention type search and attention combination search. Our experimental results show that the model searched with AttNAS achieves state-of-the-art compared to existing lightweight methods on Cityscapes, CamVid and Pascal VOC 2012. Moreover, the attention patterns obtained by AttNAS have robust generalization capability and can be used in combination with existing backbones, such as MobileNetV2 and ResNet18, to improve segmentation performance.
