  Audio-Visual Segmentation (AVS) aims to extract the sounding object from a
video frame, which is represented by a pixel-wise segmentation mask. The
pioneering work conducts this task through dense feature-level audio-visual
interaction, which ignores the dimension gap between different modalities. More
specifically, the audio clip could only provide a Global semantic label in each
sequence, but the video frame covers multiple semantic objects across different
Local regions, which leads to mislocalization of the representationally similar
but semantically different object. In this paper, we propose a Cross-modal
Cognitive Consensus guided Network (C3N) to align the audio-visual semantics
from the global dimension and progressively inject them into the local regions
via an attention mechanism. Firstly, a Cross-modal Cognitive Consensus
Inference Module (C3IM) is developed to extract a unified-modal label by
integrating audio/visual classification confidence and similarities of
modality-specific label embeddings. Then, we feed the unified-modal label back
to the visual backbone as the explicit semantic-level guidance via a Cognitive
Consensus guided Attention Module (CCAM), which highlights the local features
corresponding to the interested object. Extensive experiments on the Single
Sound Source Segmentation (S4) setting and Multiple Sound Source Segmentation
(MS3) setting of the AVSBench dataset demonstrate the effectiveness of the
proposed method, which achieves state-of-the-art performance. Code is available
at https://github.com/ZhaofengSHI/AVS-C3N.
