task_categories:image-text-to-text task_categories:visual-question-answering task_categories:any-to-any language:en license:cc-by-nc-4.0 size_categories:10K<n<100K arxiv:2405.17220 arxiv:2509.18154 arxiv:2312.00849 region:us multimodal feedback preference-alignment mllm