  Scholars and practitioners across domains are increasingly concerned with
algorithmic transparency and opacity, interrogating the values and assumptions
embedded in automated, black-boxed systems, particularly in user-generated
content platforms. I report from an ethnography of infrastructure in Wikipedia
to discuss an often understudied aspect of this topic: the local, contextual,
learned expertise involved in participating in a highly automated
social-technical environment. Today, the organizational culture of Wikipedia is
deeply intertwined with various data-driven algorithmic systems, which
Wikipedians rely on to help manage and govern the "anyone can edit"
encyclopedia at a massive scale. These bots, scripts, tools, plugins, and
dashboards make Wikipedia more efficient for those who know how to work with
them, but like all organizational culture, newcomers must learn them if they
want to fully participate. I illustrate how cultural and organizational
expertise is enacted around algorithmic agents by discussing two
autoethnographic vignettes, which relate my personal experience as a veteran in
Wikipedia. I present thick descriptions of how governance and gatekeeping
practices are articulated through and in alignment with these automated
infrastructures. Over the past 15 years, Wikipedian veterans and administrators
have made specific decisions to support administrative and editorial workflows
with automation in particular ways and not others. I use these cases of
Wikipedia's bot-supported bureaucracy to discuss several issues in the fields
of critical algorithms studies, critical data studies, and fairness,
accountability, and transparency in machine learning -- most principally
arguing that scholarship and practice must go beyond trying to "open up the
black box" of such systems and also examine sociocultural processes like
newcomer socialization.
