  There is often a scarcity of training data for machine learning (ML)
classification and regression models in industrial production, especially for
time-consuming or sparsely run manufacturing processes. A majority of the
limited ground-truth data is used for training, while a handful of samples are
left for testing. Here, the number of test samples is inadequate to properly
evaluate the robustness of the ML models under test for classification and
regression. Furthermore, the output of these ML models may be inaccurate or
even fail if the input data differ from the expected. This is the case for ML
models used in the Electroslag Remelting (ESR) process in the refined steel
industry to predict the pressure in a vacuum chamber. A vacuum pumping event
that occurs once a workday generates a few hundred samples in a year of pumping
for training and testing. In the absence of adequate training and test samples,
this paper first presents a method to generate a fresh set of augmented samples
based on vacuum pumping principles. Based on the generated augmented samples,
three test scenarios and one test oracle are presented to assess the robustness
of an ML model used for production on an industrial scale. Experiments are
conducted with real industrial production data obtained from Uddeholms AB steel
company. The evaluations indicate that Ensemble and Neural Network are the most
robust when trained on augmented data using the proposed testing strategy. The
evaluation also demonstrates the proposed method's effectiveness in checking
and improving ML algorithms' robustness in such situations. The work improves
software testing's state-of-the-art robustness testing in similar settings.
Finally, the paper presents an MLOps implementation of the proposed approach
for real-time ML model prediction and action on the edge node and automated
continuous delivery of ML software from the cloud.
