  Recently, there has been tremendous progress in developing each individual
module of the standard perception-planning robot autonomy pipeline, including
detection, tracking, prediction of other agents' trajectories, and ego-agent
trajectory planning. Nevertheless, there has been less attention given to the
principled integration of these components, particularly in terms of the
characterization and mitigation of cascading errors. This paper addresses the
problem of cascading errors by focusing on the coupling between the tracking
and prediction modules. First, by using state-of-the-art tracking and
prediction tools, we conduct a comprehensive experimental evaluation of how
severely errors stemming from tracking can impact prediction performance. On
the KITTI and nuScenes datasets, we find that predictions consuming tracked
trajectories as inputs (the typical case in practice) can experience a
significant (even order of magnitude) drop in performance in comparison to the
idealized setting where ground truth past trajectories are used as inputs. To
address this issue, we propose a multi-hypothesis tracking and prediction
framework. Rather than relying on a single set of tracking results for
prediction, our framework simultaneously reasons about multiple sets of
tracking results, thereby increasing the likelihood of including accurate
tracking results as inputs to prediction. We show that this framework improves
overall prediction performance over the standard single-hypothesis
tracking-prediction pipeline by up to 34.2% on the nuScenes dataset, with even
more significant improvements (up to ~70%) when restricting the evaluation to
challenging scenarios involving identity switches and fragments -- all with an
acceptable computation overhead.
