  Many tasks performed in image-guided, mini-invasive, medical procedures can
be cast as pose estimation problems, where an X-ray projection is utilized to
reach a target in 3D space. Expanding on recent advances in the differentiable
rendering of optically reflective materials, we introduce new methods for pose
estimation of radiolucent objects using X-ray projections, and we demonstrate
the critical role of optimal view synthesis in performing this task. We first
develop an algorithm (DiffDRR) that efficiently computes Digitally
Reconstructed Radiographs (DRRs) and leverages automatic differentiation within
TensorFlow. Pose estimation is performed by iterative gradient descent using a
loss function that quantifies the similarity of the DRR synthesized from a
randomly initialized pose and the true fluoroscopic image at the target pose.
We propose two novel methods for high-fidelity view synthesis, Neural Tuned
Tomography (NeTT) and masked Neural Radiance Fields (mNeRF). Both methods rely
on classic Cone-Beam Computerized Tomography (CBCT); NeTT directly optimizes
the CBCT densities, while the non-zero values of mNeRF are constrained by a 3D
mask of the anatomic region segmented from CBCT. We demonstrate that both NeTT
and mNeRF distinctly improve pose estimation within our framework. By defining
a successful pose estimate to be a 3D angle error of less than 3 deg, we find
that NeTT and mNeRF can achieve similar results, both with overall success
rates more than 93%. However, the computational cost of NeTT is significantly
lower than mNeRF in both training and pose estimation. Furthermore, we show
that a NeTT trained for a single subject can generalize to synthesize
high-fidelity DRRs and ensure robust pose estimations for all other subjects.
Therefore, we suggest that NeTT is an attractive option for robust pose
estimation using fluoroscopic projections.
