Enhancing feature fusion with spatial aggregation and channel fusion for semantic segmentation

Semantic segmentation is crucial to the autonomous driving, as an accurate recognition and location of the surrounding scenes can be provided for the street scenes understanding task. Many existing segmentation networks usually fuse high‐level and low‐level features to boost segmentation performance. However, the simple fusion may impose a limited performance improvement because of the gap between high‐level and low‐level features. To alleviate this limitation, we respectively propose spatial aggregation and channel fusion to bridge the gap. Our implementation, inspired by the attention mechanism, consists of two steps: (1) Spatial aggregation relies on the proposed pyramid spatial context aggregation module to capture spatial similarities to enhance the spatial representation of high‐level features, which is more effective for the latter fusion. (2) Channel fusion relies on the proposed attention‐based channel fusion module to weight channel maps on different levels to enhance the fusion. In addition, the complete network with U‐shape structure is constructed. A series of ablation experiments are conducted to demonstrate the effectiveness of our designs, and the network achieves mIoU score of 81.4% on Cityscapes test dataset and 84.6% on PASCALVOC 2012 test dataset.