  Data pipelines are an integral part of various modern data-driven systems.
However, despite their importance, they are often unreliable and deliver
poor-quality data. A critical step toward improving this situation is a solid
understanding of the aspects contributing to the quality of data pipelines.
Therefore, this article first introduces a taxonomy of 41 factors that
influence the ability of data pipelines to provide quality data. The taxonomy
is based on a multivocal literature review and validated by eight interviews
with experts from the data engineering domain. Data, infrastructure, life cycle
management, development & deployment, and processing were found to be the main
influencing themes. Second, we investigate the root causes of data-related
issues, their location in data pipelines, and the main topics of data pipeline
processing issues for developers by mining GitHub projects and Stack Overflow
posts. We found data-related issues to be primarily caused by incorrect data
types (33%), mainly occurring in the data cleaning stage of pipelines (35%).
Data integration and ingestion tasks were found to be the most asked topics of
developers, accounting for nearly half (47%) of all questions. Compatibility
issues were found to be a separate problem area in addition to issues
corresponding to the usual data pipeline processing areas (i.e., data loading,
ingestion, integration, cleaning, and transformation). These findings suggest
that future research efforts should focus on analyzing compatibility and data
type issues in more depth and assisting developers in data integration and
ingestion tasks. The proposed taxonomy is valuable to practitioners in the
context of quality assurance activities and fosters future research into data
pipeline quality.
