  Visual Abductive Reasoning is an emerging vision-language (VL) topic where
the model needs to retrieve/generate a likely textual hypothesis from a visual
input (image or its part) using backward reasoning based on commonsense. Unlike
in conventional VL retrieval or captioning tasks, where entities of texts
appear in the image, in abductive inferences, the relevant facts about
inferences are not readily apparent in the input images. Besides, these
inferences are causally linked to specific regional visual cues and would
change as cues change. Existing works highlight cues utilizing a specific
prompt (e.g., colorful prompt). Then, a full fine-tuning of a VL foundation
model is launched to tweak its function from perception to deduction. However,
the colorful prompt uniformly patchify ``regional hints'' and ``global
context'' at the same granularity level and may lose fine-grained visual
details crucial for VAR. Meanwhile, full fine-tuning of VLF on limited data
would easily be overfitted.
  To tackle this, we propose a simple yet effective Region-Prompted Adapter
(RPA), a hybrid parameter-efficient fine-tuning method that leverages the
strengths of detailed cues and efficient training for the VAR task.
RPA~consists of two novel modules: Regional Prompt Generator (RPG) and
Adapter$^\textbf{+}$. The prior encodes ``regional visual hints'' and ``global
contexts'' into visual prompts separately at fine and coarse-grained levels.
The latter extends the vanilla adapters with a new Map Adapter, which modifies
the attention map using a trainable low-dim query/key projection. Additionally,
we propose a new Dual-Contrastive Loss to regress the visual feature toward
features of factual description and plausible hypothesis. Experiments on the
Sherlock demonstrate that RPA outperforms previous SOTAs, achieving the 1st
rank on leaderboards (Comparison to Human Accuracy: RPA~31.74 vs CPT-CLIP
29.58).
