  With increasing complexity of Automated Driving Systems (ADS), ensuring their
safety and reliability has become a critical challenge. The Verification and
Validation (V&V) of these systems are particularly demanding when AI components
are employed to implement perception and/or control functions. In ESA-funded
project VIVAS, we developed a generic framework for system-level
simulation-based V&V of autonomous systems. The approach is based on a
simulation model of the system, an abstract model that describes symbolically
the system behavior, and formal methods to generate scenarios and verify the
simulation executions. Various coverage criteria can be defined to guide the
automated generation of the scenarios.
  In this paper, we describe the instantiation of the VIVAS framework for an
ADS case study. This is based on the integration of CARLA, a widely-used
driving simulator, and its ScenarioRunner tool, which enables the creation of
diverse and complex driving scenarios. This is also used in the CARLA
Autonomous Driving Challenge to validate different ADS agents for perception
and control based on AI, shared by the CARLA community. We describe the
development of an abstract ADS model and the formulation of a coverage
criterion that focuses on the behaviors of vehicles relative to the vehicle
with ADS under verification. Leveraging the VIVAS framework, we generate and
execute various driving scenarios, thus testing the capabilities of the AI
components. The results show the effectiveness of VIVAS in automatically
generating scenarios for system-level simulation-based V&V of an automated
driving system using CARLA and ScenarioRunner. Therefore, they highlight the
potential of the approach as a powerful tool in the future of ADS V&V
methodologies.
