  Image-based navigation is widely considered the next frontier of minimally
invasive surgery. It is believed that image-based navigation will increase the
access to reproducible, safe, and high-precision surgery as it may then be
performed at acceptable costs and effort. This is because image-based
techniques avoid the need of specialized equipment and seamlessly integrate
with contemporary workflows. Further, it is expected that image-based
navigation will play a major role in enabling mixed reality environments and
autonomous, robotic workflows. A critical component of image guidance is 2D/3D
registration, a technique to estimate the spatial relationships between 3D
structures, e.g., volumetric imagery or tool models, and 2D images thereof,
such as fluoroscopy or endoscopy. While image-based 2D/3D registration is a
mature technique, its transition from the bench to the bedside has been
restrained by well-known challenges, including brittleness of the optimization
objective, hyperparameter selection, and initialization, difficulties around
inconsistencies or multiple objects, and limited single-view performance. One
reason these challenges persist today is that analytical solutions are likely
inadequate considering the complexity, variability, and high-dimensionality of
generic 2D/3D registration problems. The recent advent of machine
learning-based approaches to imaging problems that, rather than specifying the
desired functional mapping, approximate it using highly expressive parametric
models holds promise for solving some of the notorious challenges in 2D/3D
registration. In this manuscript, we review the impact of machine learning on
2D/3D registration to systematically summarize the recent advances made by
introduction of this novel technology. Grounded in these insights, we then
offer our perspective on the most pressing needs, significant open problems,
and possible next steps.
