  Just as other disciplines, the humanities explore how computational research
approaches and tools can meaningfully contribute to scholarly knowledge
production. We approach the design of computational tools through the
analytical lens of 'human-AI collaboration.' However, there is no generalizable
concept of what constitutes 'meaningful' human-AI collaboration. In terms of
genuinely human competencies, we consider criticality and reflection as guiding
principles of scholarly knowledge production. Although (designing for)
reflection is a recurring topic in CSCW and HCI discourses, it has not been
centered in work on human-AI collaboration. We posit that integrating both
concepts is a viable approach to supporting 'meaningful' human-AI collaboration
in the humanities. Our research, thus, is guided by the question of how
critical reflection can be enabled in human-AI collaboration. We address this
question with a use case that centers on computer vision (CV) tools for art
historical image retrieval. Specifically, we conducted a qualitative interview
study with art historians and extended the interviews with a think-aloud
software exploration. We observed and recorded our participants' interaction
with a ready-to-use CV tool in a possible research scenario. We found that
critical reflection, indeed, constitutes a core prerequisite for 'meaningful'
human-AI collaboration in humanities research contexts. However, we observed
that critical reflection was not fully realized during interaction with the CV
tool. We interpret this divergence as supporting our hypothesis that
computational tools need to be intentionally designed in such a way that they
actively scaffold and support critical reflection during interaction. Based on
our findings, we suggest four empirically grounded design implications for
'critical-reflective human-AI collaboration'.
