  The Cancer Registry of Norway (CRN) collects, curates, and manages data
related to cancer patients in Norway, supported by an interactive,
human-in-the-loop, socio-technical decision support software system. Automated
software testing of this software system is inevitable; however, currently, it
is limited in CRN's practice. To this end, we present an industrial case study
to evaluate an AI-based system-level testing tool, i.e., EvoMaster, in terms of
its effectiveness in testing CRN's software system. In particular, we focus on
GURI, CRN's medical rule engine, which is a key component at the CRN. We test
GURI with EvoMaster's black-box and white-box tools and study their test
effectiveness regarding code coverage, errors found, and domain-specific rule
coverage. The results show that all EvoMaster tools achieve a similar code
coverage; i.e., around 19% line, 13% branch, and 20% method; and find a similar
number of errors; i.e., 1 in GURI's code. Concerning domain-specific coverage,
EvoMaster's black-box tool is the most effective in generating tests that lead
to applied rules; i.e., 100% of the aggregation rules and between 12.86% and
25.81% of the validation rules; and to diverse rule execution results; i.e.,
86.84% to 89.95% of the aggregation rules and 0.93% to 1.72% of the validation
rules pass, and 1.70% to 3.12% of the aggregation rules and 1.58% to 3.74% of
the validation rules fail. We further observe that the results are consistent
across 10 versions of the rules. Based on these results, we recommend using
EvoMaster's black-box tool to test GURI since it provides good results and
advances the current state of practice at the CRN. Nonetheless, EvoMaster needs
to be extended to employ domain-specific optimization objectives to improve
test effectiveness further. Finally, we conclude with lessons learned and
potential research directions, which we believe are generally applicable.
