  Context. Advancements in Machine Learning (ML) are revolutionizing every
application domain, driving unprecedented transformations and fostering
innovation. However, despite these advances, several organizations are
experiencing friction in the adoption of ML-based technologies, mainly due to
the shortage of ML professionals. In this context, Automated Machine Learning
(AutoML) techniques have been presented as a promising solution to democratize
ML adoption. Objective. We aim to provide an overview of the evidence on the
benefits and limitations of using AutoML tools. Method. We conducted a
multivocal literature review, which allowed us to identify 54 sources from the
academic literature and 108 sources from the grey literature reporting on
AutoML benefits and limitations. We extracted reported benefits and limitations
from the papers and applied thematic analysis. Results. We identified 18
benefits and 25 limitations. Concerning the benefits, we highlight that AutoML
tools can help streamline the core steps of ML workflows, namely data
preparation, feature engineering, model construction, and hyperparameter
tuning, with concrete benefits on model performance, efficiency, and
scalability. In addition, AutoML empowers both novice and experienced data
scientists, promoting ML accessibility. On the other hand, we highlight several
limitations that may represent obstacles to the widespread adoption of AutoML.
For instance, AutoML tools may introduce barriers to transparency and
interoperability, exhibit limited flexibility for complex scenarios, and offer
inconsistent coverage of the ML workflow. Conclusions. The effectiveness of
AutoML in facilitating the adoption of machine learning by users may vary
depending on the tool and the context in which it is used. As of today, AutoML
tools are used to increase human expertise rather than replace it, and, as
such, they require skilled users.
