  Context: Demonstrating high reliability and safety for safety-critical
systems (SCSs) remains a hard problem. Diverse evidence needs to be combined in
a rigorous way: in particular, results of operational testing with other
evidence from design and verification. Growing use of machine learning in SCSs,
by precluding most established methods for gaining assurance, makes operational
testing even more important for supporting safety and reliability claims.
Objective: We use Autonomous Vehicles (AVs) as a current example to revisit the
problem of demonstrating high reliability. AVs are making their debut on public
roads: methods for assessing whether an AV is safe enough are urgently needed.
We demonstrate how to answer 5 questions that would arise in assessing an AV
type, starting with those proposed by a highly-cited study. Method: We apply
new theorems extending Conservative Bayesian Inference (CBI), which exploit the
rigour of Bayesian methods while reducing the risk of involuntary misuse
associated with now-common applications of Bayesian inference; we define
additional conditions needed for applying these methods to AVs. Results: Prior
knowledge can bring substantial advantages if the AV design allows strong
expectations of safety before road testing. We also show how naive attempts at
conservative assessment may lead to over-optimism instead; why extrapolating
the trend of disengagements is not suitable for safety claims; use of knowledge
that an AV has moved to a less stressful environment. Conclusion: While some
reliability targets will remain too high to be practically verifiable, CBI
removes a major source of doubt: it allows use of prior knowledge without
inducing dangerously optimistic biases. For certain ranges of required
reliability and prior beliefs, CBI thus supports feasible, sound arguments.
Useful conservative claims can be derived from limited prior knowledge.
