  The automotive industry has witnessed an increasing level of development in
the past decades; from manufacturing manually operated vehicles to
manufacturing vehicles with a high level of automation. With the recent
developments in Artificial Intelligence (AI), automotive companies now employ
blackbox AI models to enable vehicles to perceive their environments and make
driving decisions with little or no input from a human. With the hope to deploy
autonomous vehicles (AV) on a commercial scale, the acceptance of AV by society
becomes paramount and may largely depend on their degree of transparency,
trustworthiness, and compliance with regulations. The assessment of the
compliance of AVs to these acceptance requirements can be facilitated through
the provision of explanations for AVs' behaviour. Explainability is therefore
seen as an important requirement for AVs. AVs should be able to explain what
they have 'seen', done, and might do in environments in which they operate.
  In this paper, we provide a comprehensive survey of the existing body of work
around explainable autonomous driving. First, we open with a motivation for
explanations by highlighting and emphasising the importance of transparency,
accountability, and trust in AVs; and examining existing regulations and
standards related to AVs. Second, we identify and categorise the different
stakeholders involved in the development, use, and regulation of AVs and elicit
their explanation requirements for AV. Third, we provide a rigorous review of
previous work on explanations for the different AV operations (i.e.,
perception, localisation, planning, control, and system management). Finally,
we identify pertinent challenges and provide recommendations, such as a
conceptual framework for AV explainability. This survey aims to provide the
fundamental knowledge required of researchers who are interested in
explainability in AVs.
