  In the last four years, the number of distinct autonomous vehicles platforms
deployed in the streets of California increased 6-fold, while the reported
accidents increased 12-fold. This can become a trend with no signs of subsiding
as it is fueled by a constant stream of innovations in hardware sensors and
machine learning software. Meanwhile, if we expect the public and regulators to
trust the autonomous vehicle platforms, we need to find better ways to solve
the problem of adding technological complexity without increasing the risk of
accidents. We studied this problem from the perspective of reliability
engineering in which a given risk of an accident has severity and probability
of occurring. Timely information on accidents is important for engineers to
anticipate and reuse previous failures to approximate the risk of accidents in
a new city. However, this is challenging in the context of autonomous vehicles
because of the sparse nature of data on the operational scenarios (driving
trajectories in a new city). Our approach was to mitigate data sparsity by
reducing the state space through monitoring of multiple-vehicles operations. We
then minimized the risk of accidents by determining proper allocation of tests
for each equivalence class. Our contributions comprise (1) a set of strategies
to monitor the operational data of multiple autonomous vehicles, (2) a Bayesian
model that estimates changes in the risk of accidents, and (3) a feedback
control-loop that minimizes these risks by reallocating test effort. Our
results are promising in the sense that we were able to measure and control
risk for a diversity of changes in the operational scenarios. We evaluated our
models with data from two real cities with distinct traffic patterns and made
the data available for the community.
