  Today's online platforms rely heavily on recommendation systems to serve
content to their users; social media is a prime example. In turn,
recommendation systems largely depend on artificial intelligence algorithms to
decide who gets to see what. While the content social media platforms deliver
is as varied as the users who engage with them, it has been shown that
platforms can contribute to serious harm to individuals, groups and societies.
Studies have suggested that these negative impacts range from worsening an
individual's mental health to driving society-wide polarisation capable of
putting democracies at risk. To better safeguard people from these harms, the
European Union's Digital Services Act (DSA) requires platforms, especially
those with large numbers of users, to make their algorithmic systems more
transparent and follow due diligence obligations. These requirements constitute
an important legislative step towards mitigating the systemic risks posed by
online platforms. However, the DSA lacks concrete guidelines to operationalise
a viable audit process that would allow auditors to hold these platforms
accountable. This void could foster the spread of 'audit-washing', that is,
platforms exploiting audits to legitimise their practices and neglect
responsibility.
  To fill this gap, we propose a risk-scenario-based audit process. We explain
in detail what audits and assessments of recommender systems according to the
DSA should look like. Our approach also considers the evolving nature of
platforms and emphasises the observability of their recommender systems'
components. The resulting audit facilitates internal (among audits of the same
system at different moments in time) and external comparability (among audits
of different platforms) while also affording the evaluation of mitigation
measures implemented by the platforms themselves.
