\section{Introduction}
\input{sections/s1_introduction}

\section{Related Work}
\label{sec:related}
\input{sections/s2_related}

\input{sections/f2_model}

\section{Method}
\label{sec:approach}
\input{sections/s3_approach}

\input{sections/t1_scannet_reg}

\section{Experiments}
\label{sec:experiments}
\input{sections/s4_experiments}

\section{Conclusion}
\label{sec:conclusion}
\input{sections/s5_conclusion}

\vspace{6pt} \noindent
\textbf{Acknowledgments}
We would like to thank the anonymous reviewers for their valuable comments and suggestions.
We also thank Nilesh Kulkarni, Karan Desai, Richard Higgins, and Max Smith for many helpful discussions and feedback on early drafts of this work.

\clearpage
\newpage

{\small
\bibliographystyle{format/ieee}

\subsection{Pairwise Registration}
\label{sec:exp_pcreg}

We first evaluate our approach on point cloud registration.
Given two RGB-D images, we estimate the 6-DOF pose that would best align the first input image to the second.
The transformation is represented by a rotation matrix $\mathbf{R}$ and translation vector $\mathbf{t}$.

\lsparagraph{Evaluation Metrics.}
We evaluate pairwise registration by evaluating the pose prediction as well as the chamfer distance between the estimated and ground-truth alignments.
We compute the angular and translation errors as follows:
$$
E_{\text{rotation}} = \arccos(\frac{Tr(\mathbf{R}_{pr}\mathbf{R}_{gt}^\top) - 1}{2}),
$$$$
E_{\text{translation}} = ||\mathbf{t}_{pr} - \mathbf{t}_{gt}||_2 .
$$
We report the translation error in centimeters and the rotation errors in degrees.

While pose gives us a good measure of performance, some scenes are inherently ambiguous and multiple alignments can explain the scene appearance; \eg, walls, floors, symmetric objects.
To address these cases, we compute the chamfer distance between the scene and our reconstruction.
Given two point clouds where $\mathcal{P}$ represents the correct alignment of the scene and $\mathcal{Q}$ represents our reconstruction of the scene,
we can define the closest pairs between the point clouds as set $\Lambda_{\mathcal{P}, \mathcal{Q}} = \{(p, \argmin_{q \in \mathcal{Q}} ||p - q||) : p \in \mathcal{P})$.
We then compute the chamfer error as follows:
$$
E_{\text{cham}} =
|\mathcal{P}|^{-1} \sum_{\mathclap{({p, q) \in \Lambda_{\mathcal{P}, \mathcal{Q}}}}} ||\mathbf{x}_p - \mathbf{x}_q||
+
|\mathcal{Q}|^{-1}  \sum_{\mathclap{{(q, p) \in \Lambda_{\mathcal{Q}, \mathcal{P}}}}} ||\mathbf{x}_q - \mathbf{x}_p||.
$$

For each of these error metrics, we report the mean and median errors over the dataset as well as the accuracy for different thresholds.

We conduct our experiments on ScanNet and report the results in Table~\ref{tab:pose_scannet}.
We find that our model learns accurate point cloud registration; outperforming prior feature descriptors and performing on-par with supervised geometric registration approaches.
We next analyze our results through the questions posed at the start of this section.

\lsparagraph{Does unsupervised learning improve over off-the-shelf descriptors? }
Yes. We evaluate our approach against the traditional pipeline for registration: feature extraction using an off-the-shelf keypoint descriptor and alignment via RANSAC.
We show large performance gains over both traditional and learned descriptors.
It is important to note that FCGF and SuperPoint currently represent the state-of-the-art for feature descriptors.
Furthermore, both methods have been used directly, without further fine-tuning, to achieve the highest performance on image registration benchmarks~\cite{sarlin2020superglue} and geometric registration benchmarks~\cite{choy2020deep,gojcic2020learning}.
We also find that our approach learns features that can generalize to similar datasets. As shown in Table~\ref{tab:pose_scannet}, our model trained on 3D Match outperforms the off-the-shelf descriptors while being competitive with supervised geometric registration approaches.

\lsparagraph{Does RGB-D training alleviate the need for pose supervision? }
Yes.
We compare our approach to two recently proposed supervised point cloud registration approaches: DGR~\cite{choy2020deep} and 3D Multi-view Registration~\cite{gojcic2020learning}.
Since their model was trained on 3D Match, we also train our model on 3D match and report the numbers.
We find that our model is competitive with supervised approaches when trained on their dataset, and can outperform them when trained on ScanNet.
However, a direct comparison is more nuanced since those two classes of methods differ in two key ways: training supervision and input modality.

We argue that the recent rise in RGB-D cameras on both hand-held devices and robotic systems supports our setup.
First, the rise in devices suggests a corresponding increase in RGB-D raw data that will not necessarily be annotated with pose information.
This increase provides a great opportunity for unsupervised learning to leverage this data stream.
Second, while there are cases where depth sensing might be the better or only option (\eg, dark environment or highly reflective surfaces.), there are many cases where one has access to both RGB and depth information.
The ability to leverage both can increase the effectiveness and robustness of a registration system.
Finally, while we only learn visual features in this work, we note that our approach is easily extensible to learning both geometric and visual features since it is agnostic to how the features are calculated.

\subsection{Ablations}
\label{sec:exp_ablations}

We perform several ablation studies to better understand the model’s performance and its various components.
In particular, we are interested in better understanding the impact of the optimization and rendering parameters on the overall model performance.
While some ablations can only be applied during training (\eg, rendering choice), ablations that affect the correspondence estimation and fitting  can be selectively applied during training, inference, or both.
Hence, we consider all the variants.

\vspace{0.2cm}
\lsparagraph{Joint Rendering.}
Our first ablation investigates the impact of our rendering choices by rendering the output images from the joint point cloud.
In \S~\ref{sec:method_render}, we discuss rendering alternate views to force the model to align the pointclouds to produce accurate renders.
As shown in Table~\ref{tab:ablations}, we find that naively rendering the joint point cloud results in a significant performance drop.
This supports our claim that a joint render would negatively impact the features learned since the model can achieve good photometric consistency even if the pointclouds are not accurately aligned.

\vspace{0.2cm}
\lsparagraph{Ratio Test.}
In our approach, we use Lowe's ratio test to estimate the weight for each correspondence.
We ablate this component by instead using the feature distance between the corresponding points to rank the correspondences.
Since this ablation can be applied to training or inference independently, we apply it to training, inference, or both.
Our results indicate that the ratio test is critical to our model's performance, as ablating it results in the largest performance drop.
This supports our initial claims about the utility of the ratio test as a strong heuristic for filtering correspondences.
It is worth noting that Lowe's ratio test~\cite{lowe2004distinctive} shows incredible efficacy in determining correspondence weights; a function often undertaken by far more complex models in recent work~\cite{choy2020deep,gojcic2020learning,ranftl2018deepfundamental,sarlin2020superglue}.
Our approach is able to perform well using such a simple filtering heuristic since it is also learning the features, not just matching them.

\vspace{0.2cm}
\lsparagraph{Randomized Subsets.}
In our model, we estimate $t$ transformations based on $t$ randomly sampled subsets. This is inspired by RANSAC~\cite{RANSAC} as it allows us to better handle outliers.
We ablate this module by estimating a single transformation based on all the correspondences.
Similar to the ratio test, this ablation can be applied to training or inference independently.
As shown in Table~\ref{tab:ablations}, ablating this component at test time results in a significant drop in performance.
Interestingly, we find that applying it during training and relieving it during testing improves performance.
We posit that this ablation acts similarly to DropOut~\cite{srivastava2014dropout} which forces the model to predict using a subset of the features and is only applied during training. As a result, the model is forced to learn better features during training, while gaining the benefits of randomized optimization during inference.

\vspace{0.2cm}
\lsparagraph{Number of subsets. }
We find that the number of subsets chosen has a significant impact on both run-time and performance.
During training, we sample 10 subsets of 80 correspondences each. During testing, we sample 100 subsets of 80 correspondences each.
For this set of experiments, we used the same pretrained weights and only vary the number of subsets used. Each subset still contains 80 correspondences.
As shown in Table~\ref{tab:runtime}, using a larger number of subsets improves the performance while also increasing the run-time.
Additionally, we find that the performance gains saturate at 100 subsets.

\subsection{Point Cloud Generation}
\label{sec:method_pcgen}

Given an input RGB-D image, $I \in \mathbb{R}^{4 \times H \times W}$, we would like to generate a point cloud $\mathcal{P} \in \mathbb{R}^{(6 + F) \times N}$.
Each point $p \in \mathcal{P}$ is represented by a 3D coordinate $\mathbf{x}_p \in \mathbb{R}^{3}$, a color $\mathbf{c}_p \in \mathbb{R}^{3}$, and a feature vector $\mathbf{f}_p \in \mathbb{R}^{F}$.
We first use a feature encoder to extract a feature map using each image's RGB channels.
The extracted feature map has the same spatial resolution as the input image.
As a result, one can easily convert the extracted features and input RGB into a point cloud using the input depth and known camera intrinsic matrix.
However, given that current depth sensors do not predict depth for every pixel, we omit pixels with missing depth from our generated point cloud.
In order to avoid heterogeneous batches, we mark points with missing depths so that subsequent operations ignore them.

\subsection{Correspondence Estimation}
\label{sec:method_corr}

Given two feature point clouds\footnote{As noted in Sec~\ref{sec:method_pcgen}, point clouds will have different numbers of valid points based on the input depth. While our method deals with this by tracking those points and omitting them from subsequent operations, we assume all the points are valid in our model description to enhance clarity.},
$\mathcal{P}$, $\mathcal{Q} \in \mathbb{R}^{(6 + F) \times N}$,
we would like to find the correspondences between the point clouds.
Specifically, for each point in $p \in \mathcal{P}$, we would like to find the point $q_p$ such that
\begin{equation}
q_{p} = \operatorname*{arg\,min}_{q\in{\mathcal{Q}}} D(\mathbf{f}_p, \mathbf{f}_q),
\end{equation}
where $D(p, q)$ is a distance-metric defined on the feature space.
In our experiments, we use cosine distance to determine the closest features.

We extract such correspondences for all the points in both $\mathcal{P}$ and $\mathcal{Q}$ since correspondence is not guaranteed to be bijective.
As a result, we have two sets of correspondences, $\mathcal{C}_{\mathcal{P} \to \mathcal{Q}}$ and $\mathcal{C}_{\mathcal{Q} \to \mathcal{P}}$, where each set consists of $N$ pairs.

\paragraph{Ratio Test.}
Determining the quality of each correspondence is a challenge faced by any correspondence-based geometric fitting approach.
Extracting correspondences based on only the nearest neighbor will result in many false positives due to falsely matching repetitive pairs or non-mutually visible portions of the images.

The standard approach is to estimate a weight for each correspondence that captures the quality of this correspondence.
Recent approaches estimate a correspondence weight for each match using self-attention graph networks~\cite{sarlin2020superglue}, PointNets~\cite{gojcic2020learning,yi2018learning}, and CNNs~\cite{choy2020deep}.
In our experiments, we found that a much simpler approach based on Lowe's ratio test~\cite{lowe2004distinctive} works well without requiring any additional parameters in the network.
The basic intuition behind the ratio test is that unique correspondences are more likely to be true matches.
As a result, the quality of correspondence $(p, q_p)$ is not simply determined by $D(p, q_p)$, but rather between the ratio $r$ which is defined as
\begin{equation}
r = \frac{D(p, q_{p, 1})}{D(p, q_{p, 2})},
\end{equation}
where $q_{p, i}$ is the $i$-th nearest neighbor to point $p$ in $\mathcal{Q}$.
Since $0 \leq r_p \leq 1$ and a lower ratio indicates a better match, we weigh each correspondence by $w = 1 - r$.

In the traditional formulation, one would define a distance ratio threshold for inlier vs outliers.
Instead, we rank the correspondences using their ratio weight and pick the top $k$ correspondences.
We pick an equal number of correspondences from $\mathcal{C}_{\mathcal{P} \to \mathcal{Q}}$ and $\mathcal{C}_{\mathcal{Q} \to \mathcal{P}}$.
Additionally, we keep the weights for each correspondence to use in the geometric fitting step.
Hence, we end up with a correspondence set $\mathcal{M} = \{(p, q, w)_i: 0 \leq i < k \}$ where $k{=}400$.

\subsection{Geometric Fitting}
\label{sec:method_fitting}

Given a set of correspondences $\mathcal{M}$, we would like to find the transformation, $\mathcal{T^{*}} \in \text{SE(3)}$ that would minimize the error between the correspondences
\begin{equation}
    \mathcal{T^{*}} = \argmin_{\mathcal{T} \in~\text{SE}(3)} E(\mathcal{M}, \mathcal{T})
    \label{eq:w_proc}
\end{equation}
where the error $E(\mathcal{M}, \mathcal{T})$ is defined as:
\begin{equation}
    E(\mathcal{M}, \mathcal{T}) = |\mathcal{M}|^{-1} \sum_{(p, q, w) \in \mathcal{M}} w~(\mathbf{x}_p - \mathcal{T}(\mathbf{x}_q))^2
    \label{eq:corr_err}
\end{equation}
This can be framed as a weighted Procrustes problem and solved using a weighted variant of Kabsch's algorithm~\cite{kabsch1976solution}.

While the original Procrustes problem minimizes the distance between a set of unweighted correspondences~\cite{gower1975generalized}, Choy \etal~\cite{choy2020deep} have shown that one can integrate weights into this optimization.
This is done by calculating the covariance matrix between the centered and weighted point clouds, followed by calculating the SVD on the covariance matrix.
For more details, see~\cite{choy2020deep, kabsch1976solution}.

Integrating weights into the optimization is important for two reasons.
First, it allows us to build  robust estimators that can weigh correspondences based on our confidence in their uniqueness.
More importantly, it makes the optimization differentiable with respect to the weights, allowing us to backpropagate the losses back to the encoder for feature learning.

\paragraph{Randomized Optimization. }
While this approach is capable of integrating the weights into the optimization, it can still be sensitive to outliers with non-zero weights.
We take inspiration from RANSAC and use random sampling to mitigate the problem of outliers.
More specifically, we sample $t$ subsets of $\mathcal{M}$, and use Equation~\ref{eq:w_proc} to find $t$ candidate transformations.
We then choose the candidate that minimizes the weighted error on the full correspondence set.
Since the $t$ optimizations on the correspondence subsets are all independent, we are able to run them in parallel to make the optimization more efficient.
We deviate from classic RANSAC pipelines in that we choose the transformation that minimizes a weighted error, instead of maximizing inlier count, to avoid having to define an arbitrary inlier threshold.

It is worth noting that the model can be trained and tested with a different number of random subsets.
In our experiments, we train the model with 10 randomly sampled subsets of 80 correspondences each.
At test time, we use 100 subsets with 20 correspondences each.
We evaluate the impact of those choices on performance and run time in \S~\ref{sec:exp_ablations}.

\subsection{Point Cloud Rendering}
\label{sec:method_render}

The final step of our approach is to render the RGB-D images from the aligned point clouds. This provides us with our primary learning signals: photometric and depth consistency.
The core idea is that if the camera locations are estimated correctly, the point cloud renders will be consistent with the input images.
We use differentiable rendering to project the colored point clouds onto an image using the estimated camera pose and known intrinsics. Our pipeline is very similar to Wiles \etal~\cite{wiles2019synsin}.

A naive approach of simply rendering both point clouds suffers from a degenerate solution: the rendering will be accurate even if the alignment is incorrect.
An extreme case of this would be to always estimate cameras looking in opposite directions.
In that case, each image is projected in a different location of space and the output will be consistent without alignment.
We address this issue by forcing the network to render each view using only the other image's point cloud, as shown in Fig.~\ref{fig:mask_render}.
This forces the network to learn consistent alignment as a correct reconstruction requires the mutually visible parts of the scene to be correctly aligned.
This introduces another challenge: \textit{how to handle the non-mutually visible surfaces of the scene? }

While view synthesis approaches hallucinate the missing regions to output photo-realistic imagery~\cite{wiles2019synsin}, earlier work in differentiable SfM observed that the gradients coming from the hallucinated region negatively impact the learning~\cite{zhou2017unsupervised}.
Our solution to this problem is to only evaluate the loss for valid pixels.
Valid pixels, as shown in Fig~\ref{fig:mask_render}, are ones for which rendering was possible; \ie, there were points along the viewing ray for those pixels.
This is important in this work since invalid pixels can occur due to two reasons: non-mutually visible surfaces and pixels with missing depth.
While the first reason is due to our approach, the second reason for invalid pixels is governed by current depth sensors which do not produce a depth value for each pixel.

In our experiments, we found that pose networks are very susceptible to the issues above; the network starts estimating very large poses within the first hundred iterations and never recovers.
We also experimented with rendering the features and decoding them, similar to~\cite{wiles2019synsin}, but found that this resulted in worse alignment performance.

\input{sections/f4_model_visual}

\subsection{Losses}

We use three consistency losses to train our model: photometric, depth, and correspondence.
The photometric and depth losses are the L1 losses applied between the rendered and input RGB-D frames.
Those losses are masked to only apply to valid pixels, as discussed in \S~\ref{sec:method_render}.
Additionally, we use the correspondence error calculated in Eq.~\ref{eq:corr_err} as our correspondence loss.
We weight the photometric and depth losses with a weighting of 1 while the correspondence loss receives a weighting of 0.1.


\section{Introduction}
\label{sec:intro}
Image fusion is frequently involved in modern \mbox{image-guided} medical interventions, typically augmenting \mbox{intra-operatively} acquired \mbox{2-D}\xspace \mbox{X-ray}\xspace images with \mbox{pre-operative} \mbox{3-D}\xspace CT or MRI images. Accurate alignment between the fused images is essential for clinical applications and can be achieved using \mbox{2-D/3-D}\xspace rigid registration, which aims at finding the pose of a \mbox{3-D}\xspace volume in order to align its projections to \mbox{2-D}\xspace \mbox{X-ray}\xspace images. Most commonly, \mbox{intensity-based} methods are employed~\cite{markelj2010review}, where a similarity measure between the \mbox{2-D}\xspace image and the projection of the \mbox{3-D}\xspace image is defined and optimized as e.\,g.\xspace~described by Kubias~et~al.\xspace~\cite{IMG08}. Despite decades of investigations, \mbox{2-D/3-D}\xspace registration remains challenging. The difference in dimensionality of the input images results in an \mbox{ill-posed} problem. In addition, content mismatch between the \mbox{pre-operative} and \mbox{intra-operative} images, poor image quality and a limited field of view challenge the robustness and accuracy of registration algorithms. Miao~et~al.\xspace~\cite{DFM17} propose a \mbox{learning-based} registration method that is build upon the intensity-based approach. While they achieve a high robustness, registration accuracy remains challenging.

The intuition of \mbox{2-D/3-D}\xspace rigid registration is to globally minimize the visual misalignment between \mbox{2-D}\xspace images and the projections of the \mbox{3-D}\xspace image.
Based on this intuition, Schmid and Ch{\^e}nes~\cite{segm2014Schmid} decompose the target structure to local shape patches and model image forces using Hooke's law of a spring from image block matching.
Wang~et~al.\xspace~\cite{DRR17} propose a \mbox{point-to-plane} \mbox{correspondence (PPC)} model for \mbox{2-D/3-D}\xspace registration, which linearly constrains the global differential motion update using local correspondences. Registration is performed by iteratively establishing correspondences and performing the motion estimation.
During the intervention, devices and implants, as well as locally similar anatomies, can introduce outliers for local correspondence search (see Fig. \ref{fig:sample:td} and \ref{fig:sample:NGC}). Weighting of local correspondences, in order to emphasize the correct correspondences, directly influences the accuracy and robustness of the registration.
An iterative reweighted scheme is suggested by Wang~et~al.\xspace~\cite{DRR17} to enhance the robustness against outliers. However, this scheme only works when outliers are a minority of the measurements.

Recently, Qi~et~al.\xspace~\cite{PND17} proposed the PointNet, a type of neural network directly processing point clouds. PointNet is capable of internally extracting global features of the cloud and relating them to local features of individual points. Thus, it is well suited for correspondence weighting in \mbox{2-D/3-D}\xspace registration.
Yi~et~al.\xspace~\cite{LFG18} propose to learn the selection of correct correspondences for wide-baseline stereo images. As a basis, candidates are established, e.\,g.\xspace~using SIFT features. Ground truth labels are generated by exploiting the epipolar constraint. This way, an outlier label is generated. Additionally, a regression loss is introduced, which is based on the error in the estimation of a known essential matrix between two images. Both losses are combined during training. While including the regression
loss improves the results, the classification loss is shown to be important to find highly accurate correspondences.
The performance of iterative correspondence-based registration algorithms
(e.\,g.\xspace~\cite{segm2014Schmid}, \cite{DRR17})
can be improved by learning a weighting strategy for the correspondences.
However, automatic labeling of the correspondences is not practical for iterative methods as even correct correspondences may have large errors in the first few iterations.
This means that labeling cannot be performed by applying a simple rule such as a threshold based on the ground truth position of a point.

In this paper, we propose a method to learn an optimal weighting strategy for the local correspondences for rigid \mbox{2-D/3-D}\xspace registration directly with the criterion to minimize the registration error, without the need of per-correspondence ground truth annotations.
We treat the correspondences as a point cloud with extended \mbox{per-point} features and use a modified PointNet architecture to learn global interdependencies of local correspondences according to the PPC registration metric.
We choose to use the PPC model as it was shown to enable a high registration accuracy as well as robustness~\cite{DRR17}. Furthermore, it is differentiable and therefore lends itself to the use in our training objective function.
To train the network, we propose a novel training objective function, which is composed of the motion estimation according to the PPC model and the registration error computation steps. It allows us to learn a correspondence weighting strategy by minimizing the registration error.
We demonstrate the effectiveness of the learned weighting strategy by evaluating our method on \mbox{single-vertebra} registration, where we show a highly improved robustness compared to the original PPC registration.

\section{Registration and Learned Correspondence Weighting}

In the following section, we begin with an overview of the registration method using the PPC model.
Then, further details on motion estimation (see Sec.~\ref{sec:motionEstimation}) and registration error computation (see Sec.~\ref{sec:errorComputation}) are given, as these two steps play a crucial role in our objective function.
The architecture of our network is discussed in Sec.~\ref{sec:architecture}, followed by the introduction of our objective function in Sec.~\ref{sec:objective}.
At last, important details regarding the training procedure are given in Sec.~\ref{sec:training}.

\subsection{Registration Using Point-to-Plane Correspondences}
Wang~et~al.\xspace~\cite{DRR17} measure the local misalignment between the projection of a \mbox{3-D}\xspace volume $V$ and the \mbox{2-D}\xspace fluoroscopic (live \mbox{X-ray}\xspace) image $I^\mathrm{FL}$ and compute a motion which compensates for this misalignment.
Surface points are extracted from $V$ using the \mbox{3-D}\xspace Canny detector~\cite{CAE86}.
A set of contour generator points~\cite{hartley03contGen} $\set{\mathbf{w}_i}$, i.\,e.\xspace~surface points $\mathbf{w}_i\in\mathbb{R}^3$ which correspond to contours in the projection of $V$, are projected onto the image as $\set{\mathbf{p}_i}$, i.\,e.\xspace~a set of points $\mathbf{p}_i\in\mathbb{R}^3$ on the image plane.
Additionally, gradient projection images of $V$ are generated and used to perform local patch matching to find correspondences for $\mathbf{p}_i$ in $I^\mathrm{FL}$.
Assuming that the motion along contours is not detectable, the patch matching is only performed in the orthogonal direction to the contour.
Therefore, the displacement of $\mathbf{w}_i$ along the contour is not known, as well as the displacement along the viewing direction. These unknown directions span the plane $\mathrm{\Pi}_i$ with the normal $\mathbf{n}_i\in\mathbb{R}^3$. After the registration, a point $\mathbf{w}_i$ should be located on the plane $\mathrm{\Pi}_i$.
To minimize the point-to-plane distances $\distance{\mathbf{w}_i}{\mathrm{\Pi}_i}$, a linear equation is defined for each correspondence under the small angle assumption.
The resulting system of equations is solved for the differential motion $\delta\mathbf{v}\in\mathbb{R}^6$, which contains both rotational components in the axis-angle representation $\delta{\boldsymbol{\omega}}\in\mathbb{R}^3$ and translational components $\delta{\boldsymbol{\nu}}\in\mathbb{R}^3$, i.\,e.\xspace~$\delta\mathbf{v}=(\delta{\boldsymbol{\omega}}^\intercal, \delta{\boldsymbol{\nu}}^\intercal)^\intercal$.
The correspondence search and motion estimation steps are applied iteratively over multiple resolution levels.
To increase the robustness of the motion estimation, the maximum correntropy criterion for regression (MCCR)~\cite{LMC15} is used to solve the system of linear equations~\cite{DRR17}.
The motion estimation is extended to coordinate systems related to the camera coordinates by a rigid transformation by Schaffert~et~al.\xspace~\cite{MVD17}.

The PPC model sets up a linear relationship between the local point-to-plane correspondences and the differential transformation, i.\,e.\xspace a linear misalignment metric based on the found correspondences.
In this paper, we introduce a learning method for correspondence weighting, where the PPC metric is used during training to optimize the weighting strategy for the used correspondences with respect to the registration error.

\subsection{Weighted Motion Estimation}
\label{sec:motionEstimation}
Motion estimation according to the PPC model is performed by solving a linear system of equations  defined by $\matr{A}\in\mathbb{R}^{N\times6}$ and $\vect{b}\in\mathbb{R}^N$, where each equation corresponds to one point-to-plane correspondence and $N$ is the number of used correspondences.
We perform the motion estimation in the camera coordinate system with the origin shifted to the centroid of $\set{\mathbf{w}_i}$. This allows us to use the regularized least-squares estimation
\begin{equation}
\delta\mathbf{v} = \underset{\delta\mathbf{v}'}{\arg\min}\left(\dfrac{1}{N}\norm{\matr{A}_s\delta\mathbf{v}'-\vect{b}_s}_2^2 + \lambda \norm{\delta\mathbf{v}'}_2^2\right)
\label{eq:LS}
\end{equation}
in order to improve the robustness of the estimation.
Here, $\matr{A}_s=\matr{S}\cdot\matr{A}$, $\vect{b}_s=\matr{S}\cdot\vect{b}$ and $\lambda$ is the regularizer weight. The diagonal matrix $\matr{S}=\text{diag}(\vect{s})$ contains weights $\vect{s}\in\mathbb{R}^N$ for all correspondences. As Eq.~\eqref{eq:LS} is differentiable w.\,r.\,t.\xspace $\delta\mathbf{v}'$, we obtain
\begin{equation}
\delta\mathbf{v}=\regPPC{\matr{A}, \vect{b},\mathbf{s}}=(\matr{A}_s^\intercal\matr{A}_s+N\cdot\lambda \matr{I})^{-1}\matr{A}_s^\intercal\vect{b}_s \enspace ,
\label{eq:LSClosedForm}
\end{equation}
where $\matr{I}\in\mathbb{R}^{6\times6}$ is the identity matrix.
After each iteration, the registration $\matr{T}\in\mathbb{R}^{4\times4}$ is updated as
\begin{equation}
\matr{T} =
\begin{pmatrix}
\cos(\alpha)\matr{I}+(1-\cos(\alpha)\vect{r}\vect{r}^\intercal)+\sin(\alpha)[\vect{r}]_\times & \delta{\boldsymbol{\nu}} \\ 0 & 1
\end{pmatrix}
\cdot \hat{\matr{T}}
\enspace ,
\label{eq:currReg}
\end{equation}
where $\alpha = \norm{\delta{\boldsymbol{\omega}}}$, $\vect{r} = \delta{\boldsymbol{\omega}}/\norm{\delta{\boldsymbol{\omega}}}$, $[\vect{r}]_\times\in\mathbb{R}^{3\times3}$ is a skew matrix which expresses the cross product with $\vect{r}$ as a matrix multiplication and $\hat{\matr{T}}\in\mathbb{R}^{4\times4}$ is the registration after the previous iteration~\cite{DRR17}.

\subsection{Registration Error Computation}
\label{sec:errorComputation}
In the training phase, the registration error is measured and minimized via our training objective function.
Different error metrics, such as the mean target registration error (mTRE) or the mean re-projection distance (mRPD) can be used. For more details on these metrics, see Sec.~\ref{sec:evalMetrics}.
In this work, we choose the projection error (PE)~\cite{GBD14}, as it directly corresponds to the visible misalignment in the images and therefore roughly correlates to the difficulty to find correspondences by patch matching for the next iteration of the registration method. The PE is computed as
\begin{equation}
e = \errorFunc{\matr{T}, \matr{T}^{\mathrm{GT}}}=\dfrac{1}{M} \sum_{j=1}^M \norm{\projection{\matr{T}}{\mathbf{q}_j}-\projection{\matr{T}^{\mathrm{GT}}}{\mathbf{q}_j}} \enspace ,
\label{eq:error}
\end{equation}
where a set of $M$ target points $\set{\mathbf{q}_j}$ is used and $j$ is the point index. $\projection{\matr{T}}{\cdot}$ is the projection onto the image plane under the currently estimated registration and $\projection{\matr{T}^{\mathrm{GT}}}{\cdot}$ the projection under the \mbox{ground-truth} registration matrix $\matr{T}^{\mathrm{GT}}\in\mathbb{R}^{4\times4}$. Corners of the bounding box of the point set $\set{\mathbf{w}_i}$ are used as $\set{\mathbf{q}_j}$.

\begin{figure}[t]
\centering
  \includegraphics[width=1.0\textwidth]{architecture.pdf}
     \caption{Modified PointNet~\cite{PND17} architecture used for correspondence weighting. Rectangles with dashed outlines indicate feature vectors (orange for local features, i.\,e.\xspace containing information from single correspondences, and red for global features, i.\,e.\xspace containing information from the entire set of correspondences). Sets of feature vectors (one feature vector per correspondence) are depicted as a column of feature vectors (three correspondences shown here).
MLP denotes a multi-layer perceptron, which is applied to each feature vector individually.
     }
     \label{fig:architecture}
   \end{figure}

\subsection{Network Architecture}
\label{sec:architecture}
We want to weight individual correspondences based on their geometrical properties as well as the image similarity, taking into account the global properties of the correspondence set.
For every correspondence, we define the features
\begin{equation}
\vect{f}_i =
\begin{pmatrix}
\mathbf{w}_i^\intercal & \vect{n}_i^\intercal & \distance{\mathbf{w}_i}{\mathrm{\Pi}_i} & \text{NGC}_i&
\end{pmatrix}^\intercal \enspace ,
\label{eq:features}
\end{equation}
where $\text{NGC}_i$ denotes the normalized gradient correlation for the correspondences, which is obtained in the patch matching step.

The goal is to learn the mapping from a set of feature vectors $\set{\vect{f}_i}$ representing all correspondences to the weight vector $\vect{s}$ containing weights for all correspondences, i.\,e.\xspace~the mapping
\begin{equation}
\text{M}_{{\boldsymbol{\theta}}}: \set{\vect{f}_i} \mapsto \vect{s} \enspace ,
\end{equation}
where $\text{M}_{{\boldsymbol{\theta}}}$ is our network, and ${\boldsymbol{\theta}}$ the network parameters.

To learn directly on correspondence sets, we use the PointNet~\cite{PND17} architecture and modify it to fit our task (see Fig.~\ref{fig:architecture}).
The basic idea behind PointNet is to process points individually and obtain global information by combining the points in a symmetric way, i.\,e.\xspace~independent of order in which the points appear in the input~\cite{PND17}.
In the simplest variant, the PointNet consists of a multi-layer perceptron (MLP) which is applied for each point, transforming the respective $\vect{f_i}$ into a \mbox{higher-dimensional} feature space and thereby obtaining a local point descriptor.
To describe the global properties of the point set, the resulting local descriptors are combined by max pooling over all points, i.\,e.\xspace~for each feature, the maximum activation over all points in the set is retained.
To obtain per-point outputs, the resulting global descriptor is concatenated to the local descriptors of each point.
The resulting descriptors, containing global as well as local information, are further processed for each point independently by a second MLP.
For our network, we choose MLPs with the size of $8\times64\times128$ and $256\times64\times1$, which are smaller than in the original network~\cite{PND17}.
We enforce the output to be in the range of $[0;1]$ by using a softsign activation function~\cite{elliott1993better} in the last layer of the second MLP and modify it to re-scale the output range from $(-1;1)$ to $(0;1)$.
Our modified softsign activation function $f(\cdot)$ is defined as
\begin{equation}
f(x) = \left(\dfrac{x}{1+|x|}+1\right)\cdot0.5 \enspace ,
\end{equation}
where $x$ is the state of the neuron.
Additionally, we introduce a global trainable weighting factor which is applied to all correspondences.
This allows for an automatic adjustment of the strength of the regularization in the motion estimation step.
Note that the network is able to process correspondence sets of variable size so that no fixed amount of correspondences is needed and all extracted correspondences can be utilized.

\subsection{Training Objective}
\label{sec:objective}
We now combine the motion estimation, PE computation and the modified PointNet to obtain the training objective function as
\begin{equation}
\boldsymbol{\theta}=\underset{\mathbf{{\boldsymbol{\theta}}'}}{\arg\min}\dfrac{1}{K}\sum_{k=1}^K\errorFunc{\regPPC{\matr{A}_k, \vect{b}_k, \text{M}_{{\boldsymbol{\theta}}'}(\set{\mathbf{f}_i}_k)},\matr{T}^{\mathrm{GT}}_k} \enspace ,
\label{eq:Objective}
\end{equation}
where $k$ is the training sample index and $K$ the overall number of samples. Equation \eqref{eq:LSClosedForm} is differentiable with respect to $\vect{s}$, Eq.~\eqref{eq:currReg} with respect to $\delta\mathbf{v}$ and Eq.~\eqref{eq:error} with respect to $\matr{T}$.
Therefore, gradient-based optimization can be performed on Eq.~\eqref{eq:Objective}.

Note that using Eq.~\eqref{eq:Objective}, we learn directly with the objective to minimize the registration error and no per-correspondence \mbox{ground-truth} weights are needed.
Instead, the PPC metric is used to implicitly assess the quality of the correspondences during the back-propagation step of the training and the weights are adjusted accordingly. In other words, the optimization of the weights is driven by the PPC metric.

\subsection{Training Procedure}
\label{sec:training}
To obtain training data, a set of volumes  $\setV$ is used, each with one or more \mbox{2-D}\xspace images $\set{I^\mathrm{FL}}$ and a known $\matr{T}^{\mathrm{GT}}$ (see Sec.~\ref{sec:data}). For each pair of images, 60 random initial transformations with an uniformly distributed mTRE are generated~\cite{SEM05}. For details on the computation of the mTRE and start positions, see Sec.~\ref{sec:evalMetrics}.

Estimation of correspondences at training time is computationally expensive.
Instead, the correspondence search is performed once and the precomputed correspondences are used during training.
Training is performed for one iteration of the registration method and start positions with a small initial error are assumed to be representative for subsequent registration iterations at test time.
For training, the number of correspondences is fixed to 1024 to enable efficient batch-wise computations. The subset of used correspondences is selected randomly for every training step. Data augmentation is performed on the correspondence sets by applying translations, \mbox{in-plane} rotations and horizontal flipping, i.\,e.\xspace reflection over the plane spanned by the vertical axis of the \mbox{2-D}\xspace image and the principal direction. For each resolution level, a separate model is trained.

\section{Experiments and Results}

\subsection{Data}
\label{sec:data}

\begin{figure}[t]
\centering
     \hfill
     \subfloat{%
       \includegraphics[width=0.32\textwidth]{S1Marked.jpg}
     }
     \hfill
     \subfloat{%
       \includegraphics[width=0.32\textwidth]{S2Marked.jpg}
     }
     \hfill
     \subfloat{%
       \includegraphics[width=0.32\textwidth]{S3Marked.jpg}
     }
     \hfill
     \\
     \hfill
     \subfloat{%
       \includegraphics[width=0.32\textwidth]{S13D.jpg}
     }
     \hfill
     \subfloat{%
       \includegraphics[width=0.32\textwidth]{S23D.jpg}
     }
     \hfill
     \subfloat{%
       \includegraphics[width=0.32\textwidth]{S33D.jpg}
     }
     \hfill
     \caption{Examples of \mbox{2-D}\xspace images used as $I^\mathrm{FL}$ (top row) and the corresponding \mbox{3-D}\xspace images used as $V$ (bottom row) in the registration evaluation. Evaluated vertebrae are marked by a yellow cross in the top row.}
     \label{fig:data}
   \end{figure}

We perform experiments for \mbox{single-view} registration of individual vertebrae.
Note that \mbox{single-vertebra} registration is challenging due to the small size of the target structure and the presence of neighbor vertebrae. Therefore, achieving a high robustness is challenging.
We use clinical C-arm CT acquisitions from the thoracic and pelvic regions of the spine for training and evaluation. Each acquisition consists of a sequence of \mbox{2-D}\xspace images acquired with a rotating C-arm. These images are used to reconstruct the \mbox{3-D}\xspace volume. To enable reconstruction, the C-arm geometry has to be calibrated with a high accuracy (the accuracy is $\leq 0.16$\,mm for the projection error at the iso-center in our case). We register the acquired \mbox{2-D}\xspace images to the respective reconstructed volume and therefore the ground truth registration is known within the accuracy of the calibration.
Vertebra are defined by an axis-aligned volume of interest (VOI) containing the whole vertebra. Only surface points inside the VOI are used for registration. We register the projection images (resolution of $616\times480$ pixels, pixel size of 0.62\,mm) to the reconstructed volumes (containing around 390 slices with slice resolution of $512\times512$ voxels and voxel size of 0.49\,mm).
To simulate realistic conditions, we add Poisson noise to all \mbox{2-D}\xspace images and rescale the intensities to better match fluoroscopic images.

The training set consists of  \mbox{19 acquisitions} with a total of  \mbox{77 vertebrae}.
For each vertebra,  \mbox{8 different} \mbox{2-D}\xspace images are used. An additional validation set of  \mbox{23 vertebrae} from  \mbox{6 acquisitions} is used to monitor the training process.
The registration is performed on a test set of 6 acquisitions. For each acquisition, \mbox{2 vertebrae} are evaluated and registration is performed independently for both the \mbox{anterior-posterior} and the lateral views.
Each set contains data from different patients, i.\,e.\xspace~no patient appears in two different sets. The sets were defined so that all sets are representative to the overall quality of the available images, i.\,e.\xspace~contain both pelvic and thoracic vertebrae, as well as images with more or less clearly visible vertebrae.
Examples of images used in the test set are shown in Fig.~\ref{fig:data}.

\subsection{Compared Methods}
\label{sec:comparedMethods}
We evaluate the performance of the registration using the PPC model in combination with the learned correspondence weighting strategy (PPC-L), which was trained using our proposed metric-driven learning method.
To show the effectiveness of the correspondence weighting, we compare PPC-L to the original PPC method. The compared methods differ in the computation of the correspondence weights $\vect{s}$ and the regularizer weight $\lambda$. For \mbox{PPC-L}\xspace, the correspondence weights $\vect{s}^\mathrm{L} = \text{M}_{{\boldsymbol{\theta}}}({\set{\mathbf{f}})}$ and $\lambda = 0.01$ are used. For PPC, we set $\lambda = 0$ and the used correspondence weights $\vect{s}^\mathrm{PPC}$ are the $\text{NGC}_i$ values of the found correspondences, where any value below $0.1$ is set to $0$, i.\,e.\xspace~the correspondence is rejected. Additionally, the MCCR is used in the PPC method only. The minimum resolution level has a scaling of 0.25 and the highest a scaling of 1.0. For the PPC method, registration is performed on the lowest resolution level without allowing motion in depth first, as this showed to increases the robustness of the method. To differentiate between the effect of the correspondence weighting and the regularized motion estimation, we also consider registration using regularized motion estimation. We use a variant where the global weighting factor, which is applied to all points, is matched to the regularizer weight automatically by using our objective function (\mbox{PPC-R}\xspace). For the different resolution levels, we obtained a data weight in the range of $[2.0 ; 2.1]$. Therefore, we use $\lambda = 0.01$ and $\vect{s}^\mathrm{R} = 2.0 \cdot \vect{s}^\mathrm{PPC}$. Additionally, we empirically set the correspondence weight to $\vect{s}^\mathrm{RM} = 0.25 \cdot \vect{s}^\mathrm{PPC}$, which increases the robustness of the registration while still allowing for a reasonable amount of motion (\mbox{PPC-RM}\xspace).

\subsection{Evaluation Metrics}
\label{sec:evalMetrics}
To evaluate the registration, we follow the standardized evaluation methodology~\cite{SEM05,ROC13}.
The following metrics are defined by van de Kraats~et~al.\xspace~\cite{SEM05}:
\begin{itemize}
\item{\it Mean Target Registration Error:}
The mTRE is defined as the mean distance of target points under $\matr{T}^{\mathrm{GT}}$ and the estimated registration $\matr{T}^\mathrm{est}\in\mathbb{R}^{4\times4}$.
\item{\it Mean Re-Projection Distance (mRPD):}
The mRPD is defined as the mean distance of target points under $\matr{T}^{\mathrm{GT}}$ and the \mbox{re-projection} rays of the points as projected under $\matr{T}^\mathrm{est}$.
\item{\it Success Rate (SR):}
The SR is the number of registrations with with a registration error below a given threshold.  As we are concerned with \mbox{single-view} registration, we define the success criterion as a mRPD $\leq$ 2\,mm.
\item{\it Capture Range (CR):}
The CR is defined as the maximum initial mTRE for which at least 95\% of registrations are successful.
\end{itemize}
Additionally, we compute the gross success rate (GSR)~\cite{DFM17} as well as a gross capture range (GCR) with a success criterion of a mRPD $\leq$ 10\,mm in order to further assess the robustness of the methods in case of a low accuracy.
We define target points as uniformly distributed points inside the VOI of the registered vertebra.
For the evaluation, we generate 600 random start transformations for each vertebra in a range \mbox{of 0\,mm - 30\,mm} initial mTRE using the methodology described by van de Kraats~et~al.\xspace~\cite{SEM05}.
We evaluate the accuracy using the mRPD and the robustness using the SR, CR GSR and GCR.

\subsection{Results and Discussion}

\subsubsection{Accuracy and Robustness}
The evaluation results for the compared methods are summarized in Tab. \ref{tab:resBase}. We observe that \mbox{PPC-L}\xspace achieves the best SR of 94.3\,\% and CR of 13\,mm. Compared to PPC (SR of 79.3\,\% and CR of 3\,mm), \mbox{PPC-R}\xspace also achieves a higher SR of 88.1\,\% and CR of 6\,mm. For the regularized motion estimation, the accuracy decreases for increasing regularizer influence (0.79$\pm${0.22}\,mm for \mbox{PPC-R}\xspace and 1.18$\pm${0.42}\,mm for \mbox{PPC-RM}\xspace), compared to PPC (0.75$\pm$0.21\,mm) and \mbox{PPC-L}\xspace (0.74$\pm$0.26\,mm). A sample registration result using \mbox{PPC-L}\xspace is shown in Fig.~\ref{fig:sample:res}.
\begin{table}[b]
\centering
\caption{Evaluation results for the compared methods. The mRPD is computed for the 2\,mm success criterion and is shown as mean\,$\pm$\,standard deviation.}
\label{tab:angioRes}
\begin{tabular}{l|c|c|c|c|c}
\hline
Method    & mRPD    {[}mm{]} & SR {[}\%{]} & CR {[}mm{]} & GSR {[}\%{]} & GCR {[}mm{]}\\
\hline
PPC				& 0.75$\pm$0.21				& 79.3 & 3			& 81.8		& 3	 	\\
\mbox{PPC-R}\xspace			& {0.79}$\pm${0.22}			& 88.1		& 6		 	& 90.72	& 6 	\\
\mbox{PPC-RM}\xspace			& {1.18}$\pm${0.42}			& 59.6		& 4	 		& 95.1		& 20	\\
\bf{\mbox{PPC-L}\xspace}		& {{0.74}}$\pm$0.26	 		& 94.3		& 13	 	& 96.3		& 22	\\
\hline
\end{tabular}
\label{tab:resBase}
\end{table}
\begin{figure}[t]
\centering
     \subfloat[\label{fig:sample:td}]{%
       \includegraphics[width=0.23\textwidth]{Im2D-3.jpg}
     }
     \hfill
     \subfloat[\label{fig:sample:NGC}]{%
       \includegraphics[width=0.23\textwidth]{ImNGC-3.jpg}
     }
     \hfill
     \subfloat[\label{fig:sample:W}]{%
       \includegraphics[width=0.23\textwidth]{ImW-3.jpg}
     }
     \hfill
     \subfloat[\label{fig:sample:res}]{%
       \includegraphics[width=0.23\textwidth]{ImRes-3.jpg}
     }
     \caption{Registration example: (a) shows $I^\mathrm{FL}$ with one marked vertebra to register. Red dots depict initially extracted (b,\,c) and final aligned (d) contour points. Green lines depict the same randomly selected subset of correspondences, whose intensities are determined by $\text{NGC}_i$ (b) and learned weights (c). Final \mbox{PPC-L}\xspace registration result overlaid in yellow (d). Also see video in the supplementary material.
     }
     \label{fig:sample}
   \end{figure}

For strongly regularized motion estimation, we observe a large difference between the GSR and the SR. While for \mbox{PPC-R}\xspace, the difference is relatively small \mbox{(88.1\% vs. 90.7\%)}, it is very high for \mbox{PPC-RM}\xspace. Here a GSR of 95.1\,\% is achieved, while the SR is 59.6\,\%. This indicates that while the method is robust, the accuracy is low. Compared to the CR, the GCR is increased for \mbox{PPC-L}\xspace (22\,mm vs. 13\,mm) and especially for \mbox{PPC-RM}\xspace (20\,mm vs. 4\,mm).
Overall, this shows that while some inaccurate registrations are present in \mbox{PPC-L}\xspace, they are very common for \mbox{PPC-RM}\xspace.

\subsubsection{Single Iteration Evaluation}
\begin{figure}[b]
\centering
     \subfloat[PPC]{%
       \includegraphics[width=0.32\textwidth]{BaseSingleIter.pdf}
     }
     \hfill
     \subfloat[\mbox{PPC-R}\xspace]{%
       \includegraphics[width=0.32\textwidth]{RegSingleIter.pdf}
     }
     \hfill
     \subfloat[\mbox{PPC-L}\xspace]{%
       \includegraphics[width=0.32\textwidth]{LearnedSingleIter.pdf}
     }
     \caption{Histograms showing initial and result projection error (PE) in pixels for a single iteration of registration on lowest resolution level (on validation set, 1024 correspondences per case). Motion estimation was performed using least squares for all methods. For PPC, no motion in depth is estimated (see Sec.~\ref{sec:comparedMethods}).}
     \label{fig:singleIter}
   \end{figure}
To better understand the effect of the correspondence weighting and regularization, we investigate the registration results after one iteration on the lowest resolution level. In Fig. \ref{fig:singleIter}, the PE in pixels (computed using $\set{\mathbf{q}_j}$ as target points) is shown for all cases in the validation set. As in training, 1024 correspondences are used per case for all methods. We observe that for PPC, the error has a high spread, where for some cases, it is decreased considerably, while for other cases, it is increased. For \mbox{PPC-R}\xspace, most cases are below the initial error. However, the error is decreased only marginally, as the regularization prevents large motions. For \mbox{PPC-L}\xspace, we observe that the error is drastically decreased for most cases. This shows that \mbox{PPC-L}\xspace is able to estimate motion efficiently. An example for correspondence weighting in \mbox{PPC-L}\xspace is shown in Fig.~\ref{fig:sample:W}, where we observe a set of consistent correspondences with high weights, while the remaining correspondences have low weights.

\subsubsection{Method Combinations}
\begin{figure}[t]
\centering
	 \subfloat[\mbox{PPC-RM+}\xspace]{%
       \includegraphics[width=0.49\textwidth]{RegMPAfterFirst.pdf}
     }
     \hfill
     \subfloat[\mbox{PPC-L+}\xspace]{%
       \includegraphics[width=0.49\textwidth]{LearnedPAfterFirst.pdf}
     }
     \caption{Box plots for distribution of resulting mRPD on the lowest resolution level for successful registrations for different initial mTRE intervalls.}
     \label{fig:boxFirstResLevel}
\end{figure}
We observed that while the \mbox{PPC-RM}\xspace method has a high robustness (GCR and GSR), it leads to low accuracy. For \mbox{PPC-L}\xspace, we observed an increased GCR compared to the CR. In both cases, this demonstrates that registrations are present with a mRPD between 2\,mm and 10\,mm. As the PPC works reliably for small initial errors, we combine these methods with PPC by performing PPC on the highest resolution level instead of the respective method. We denote the resulting methods as \mbox{PPC-RM+}\xspace and \mbox{PPC-L+}\xspace. We observe that \mbox{PPC-RM+}\xspace achieves an accuracy of 0.74$\pm$0.18\,mm, an SR of 94.6\,\% and a CR of 18\,mm, while \mbox{PPC-L+}\xspace achieves an accuracy of 0.74$\pm$0.19\,mm, an SR of 96.1\,\% and a CR of 19\,mm. While the results are similar, we note that for \mbox{PPC-RM+}\xspace a manual weight selection is necessary. Further investigations are needed to clarify the better performance of PPC compared to \mbox{PPC-L}\xspace on the highest resolution level. However, this result may also demonstrate the strength of MCCR for cases where the majority of correspondences are correct.
We evaluate the convergence behavior of \mbox{PPC-L+}\xspace and \mbox{PPC-RM+}\xspace by only considering cases which were successful. For these cases, we investigate the error distribution after the first resolution level. The results are shown in Fig. \ref{fig:boxFirstResLevel}. We observe that for \mbox{PPC-L+}\xspace, a mRPD of below 10\,mm is achieved for all cases, while for \mbox{PPC-RM+}\xspace, higher misalignment of around 20\,mm mRPD is present. The result for \mbox{PPC-L+}\xspace is achieved after an average of 7.6 iterations, while 11.8 iterations were performed on average for \mbox{PPC-RM+}\xspace using the stop criterion defined in~\cite{DRR17}. In combination, this further substantiates our findings from the single iteration evaluation and shows the efficiency of \mbox{PPC-L}\xspace and its potential for reducing the computational cost.

\section{Conclusion}
For \mbox{2-D/3-D}\xspace registration, we propose a method to learn the weighting of the local correspondences directly from the global criterion to minimize the registration error. We achieve this by incorporating the motion estimation and error computation steps into our training objective function. A modified PointNet network is trained to weight correspondences based on their geometrical properties and image similarity.
A large improvement in the registration robustness is demonstrated when using the \mbox{learning-based} correspondence weighting,
while maintaining the high accuracy. Although a high robustness can also be achieved by regularized motion estimation, registration using learned correspondence weighting has the following advantages: it is more efficient, does not need manual parameter tuning and achieves a high accuracy.
One direction of future work is to further improve the weighting strategy, e.\,g.\xspace~by including more information into the decision process and optimizing the objective function for robustness and/or accuracy depending on the stage of the registration, such as the current resolution level.
By regarding the motion estimation as part of the network and not the objective function, our model can also be understood in the framework of precision learning~\cite{PRT17} as a regression model for the motion, where we learn only the unknown component (weighting of correspondences), while employing prior knowledge to the known component (motion estimation).
Following the framework of precision learning, replacing further steps of the registration framework with learned counterparts can be investigated. One candidate is the correspondence estimation, as it is challenging to design an optimal correspondence estimation method by hand.

{\bf Disclaimer:} The concept and software presented in this paper are based on research and are not commercially available. Due to regulatory reasons its future availability cannot be guaranteed.

\bibliographystyle{splncs04}


\section{Introduction}
\label{sec:intro}

This paper stems from our research of finite simple connected tetravalent graphs that admit a group of automorphisms acting transitively on vertices and edges but not on the arcs of
the graph; such groups of automorphisms are said to be {\em half-arc-transitive}. Observe that the full automorphism group $\mathrm{Aut}(\Gamma)$ of such a graph $\Gamma$
is then either arc-transitive or itself half-arc-transitive. In the latter case the graph $\Gamma$ is called {\em half-arc-transitive}.

Tetravalent graphs admitting a half-arc-transitive group of automorphisms
are surprisingly rich combinatorial objects with connections to several other areas of mathematics (see, for example,
\cite{ConPotSpa15, MarNedMaps,MarNed3, MarPis99, MarSpa08, PotSpiVerBook,genlost}). One of the most fruitful tools for analysing the structure of a tetravalent graph $\Gamma$
admitting a half-arc-transitive group $G$ is to study a certain $G$-invariant decomposition of the edge set $E(\Gamma)$ of $\Gamma$ into the
{\em $G$-alternating cycles} of some even length $2r$; the parameter $r$ is then called the {\em $G$-radius} and denoted $\mathop{{\rm rad}}_G(\Gamma)$
 (see Section~\ref{sec:HAT} for more detailed definitions). Since $G$ is edge-transitive and the decomposition into $G$-alternating cycles
is $G$-invariant, any two intersecting $G$-alternating cycles meet in the same number of vertices; this number is then called the {\em attachment number}
and denoted $\mathop{{\rm att}}_G(\Gamma)$. When $G=\mathrm{Aut}(\Gamma)$
the subscript $G$ will be omitted in the above notation.

It is well known and easy to see that $\mathop{{\rm att}}_G(\Gamma)$ divides $2\mathop{{\rm rad}}_G(\Gamma)$.
However, for all known tetravalent half-arc-transitive graphs the attachment number in fact divides the radius.
This brings us to the following question that we would like to propose and address in this paper:

\begin{question}
\label{que:divides}
Is it true that the attachment number $\mathop{{\rm att}}(\Gamma)$ of an arbitrary tetravalent half-arc-transitive graph  $\Gamma$ divides the radius $\mathop{{\rm rad}}(\Gamma)$?
\end{question}

By checking the complete list of all tetravalent half-arc-transitive graphs on up to $1000$ vertices (see~\cite{PotSpiVer15}), we see the that answer to the above question is affirmative for the graphs in that range. Further, as was proved in \cite[Theorem~1.2]{MarWal00}, the question has an affirmative answer in the case $\mathop{{\rm att}}(\Gamma) = 2$. In Section~\ref{sec:AT}, we generalise this result by proving the following theorem.

\begin{theorem}
\label{the:AT}
Let $\Gamma$ be a tetravalent half-arc-transitive graph. If its radius $\mathop{{\rm rad}}(\Gamma)$ is odd, then $\mathop{{\rm att}}(\Gamma)$ divides $\mathop{{\rm rad}}(\Gamma)$. Consequently, if $\mathop{{\rm att}}(\Gamma)$ is not divisible by $4$, then $\mathop{{\rm att}}(\Gamma)$ divides $\mathop{{\rm rad}}(\Gamma)$.
\end{theorem}

As a consequence of our second main result (Theorem~\ref{the:main}) we see that, in contrast to Theorem~\ref{the:AT}, there exist infinitely many arc-transitive tetravalent graphs $\Gamma$ admitting a half-arc-transitive group $G$ with $\mathop{{\rm rad}}_G(\Gamma) = 3$ and $\mathop{{\rm att}}_G(\Gamma) = 2$. In fact, in Section~\ref{sec:HAT}, we characterise these graphs completely and prove the following theorem (see Section~\ref{subsec:Dart} for the definition of the dart graph).

\begin{theorem}
\label{the:main}
Let $\Gamma$ be a connected tetravalent graph. Then $\Gamma$ is $G$-half-arc-transitive for some $G \leq \mathrm{Aut}(\Gamma)$ with $\mathop{{\rm rad}}_G(\Gamma) = 3$ and $\mathop{{\rm att}}_G(\Gamma) = 2$ if and only if $\Gamma$ is the dart graph of some $2$-arc-transitive cubic graph.
\end{theorem}

The third main result of this paper, stemming from our analysis of the situation described by Theorem~\ref{the:main}, reveals a surprising connection to the theory of covering projections of graphs. This theory has become one of the central tools in the study of symmetries of graphs. A particularly
thrilling development started with the seminal work of Malni\v{c}, Nedela and \v{S}koviera \cite{MalNedSko} who analysed the condition under which a given automorphism group of the base graph lifts along the covering projection. Recently, the question of determining the structure of the lifted group received a lot of attention (see \cite{FenKutMalMar,MaPo16,MaPo??}).

To be more precise,
let $\wp \colon \tilde{\Gamma} \to \Gamma$ be a covering projection of connected graphs and let $\mathrm{CT}(\wp)$ be the corresponding group of covering transformations (see \cite{MalNedSko}, for example, for the definitions pertaining to the theory of graph covers).
 Furthermore, let $G \leq \mathrm{Aut}(\Gamma)$ be a subgroup that lifts along $\wp$. Then the lifted group $\tilde{G}$ is an extension of $\mathrm{CT}(\wp)$ by $G$.
 If this extension is split then the covering projection $\wp$ is called {\em $G$-split}. The most natural way in which this can occur is that there exists a complement $\bar{G}$ of $\mathrm{CT}(\wp)$ in
$\tilde{G}$ and a $\bar{G}$-invariant subset $S$ of $V(\tilde{\Gamma})$, that intersects each fibre of $\wp$ in exactly one vertex. In such a case we say that $S$ is a {\em section} for $\bar{G}$ and that $\bar{G}$ is a {\em sectional} complement of $\mathrm{CT}(\wp)$. Split covering projections without any sectional complement are called {\em non-sectional}. These turn out to be rather elusive and hard to analyse. To the best of our knowledge, the only known infinite family of non-sectional split covers was presented in~\cite[Section 4]{FenKutMalMar}. This family of non-sectional split covers involves cubic arc-transitive graphs of extremely large order.

In this paper we show that each connected tetravalent graph $\Gamma$ admitting a half-arc-transitive group $G$ of automorphisms such that $\mathop{{\rm att}}_G(\Gamma) = 2$ and $\mathop{{\rm rad}}_G(\Gamma) = 3$
is a $2$-fold cover of the line graph of a cubic $2$-arc-transitive graph, and that in the case when $\Gamma$ is not bipartite the corresponding covering projection is non-sectional.
This thus provides a new and rather simple infinite family of the somewhat mysterious case of non-sectional split covering projections (see Section~\ref{sec:ourcover} for more details).

\section{Half-arc-transitive group actions on graphs}
\label{sec:HAT}

In the next two paragraphs we briefly review some concepts and results pertaining half-arc-transitive group actions on tetravalent graphs that we shall need in the remainder of this section. For more details see~\cite{Mar98}, where most of these notions were introduced.

A tetravalent graph $\Gamma$ admitting a {\em half-arc-transitive} (that is vertex- and edge- but not arc-transitive) group of automorphisms $G$ is said to be {\em $G$-half-arc-transitive}. The action of $G$ induces two paired orientations of the edges of $\Gamma$ and for any one of them each vertex of $\Gamma$ is the head of two and the tail of the other two of its incident edges. (The fact that the edge $uv$ is oriented from $u$ to $v$ will be denoted by $u \to v$.) A cycle of $\Gamma$ for which every two consecutive edges either have a common head or common tail with respect to this orientation is called a {\em $G$-alternating cycle}. Since the action of $G$ is vertex- and edge-transitive all of the $G$-alternating cycles have the same even length $2\mathop{{\rm rad}}_G(\Gamma)$ and any two non-disjoint $G$-alternating cycles intersect in the same number $\mathop{{\rm att}}_G(\Gamma)$ of vertices. These intersections, called the {\em $G$-attachment sets}, form an imprimitivity block system for the group $G$. The numbers $\mathop{{\rm rad}}_G(\Gamma)$ and $\mathop{{\rm att}}_G(\Gamma)$ are called the {\em $G$-radius} and {\em $G$-attachment number} of $\Gamma$, respectively. If $G = \mathrm{Aut}(\Gamma)$ we suppress the prefix and subscript $\mathrm{Aut}(\Gamma)$ in all of the above definitions.

It was shown in~\cite[Proposition~2.4]{Mar98} that a tetravalent $G$-half-arc-transitive graph $\Gamma$ has at least three $G$-alternating cycles unless $\mathop{{\rm att}}_G(\Gamma) = 2\mathop{{\rm rad}}_G(\Gamma)$ in which case $\Gamma$ is isomorphic to a particular Cayley graph of a cyclic group (and is thus arc-transitive). Moreover, in the case that $\Gamma$ has at least three $G$-alternating cycles, $\mathop{{\rm att}}_G(\Gamma) \leq \mathop{{\rm rad}}_G(\Gamma)$ holds and $\mathop{{\rm att}}_G(\Gamma)$ divides $2\mathop{{\rm rad}}_G(\Gamma)$. In addition, the restriction of the action of $G$ to any $G$-alternating cycle is isomorphic to the dihedral group of order $2\mathop{{\rm rad}}_G(\Gamma)$ (or to the Klein 4-group in the case of $\mathop{{\rm rad}}_G(\Gamma) = 2$) with the cyclic subgroup of order $\mathop{{\rm rad}}_G(\Gamma)$ being the subgroup generated by a two-step rotation of the $G$-alternating cycle in question. In addition, if $C = (v_0, v_1, \ldots , v_{2r-1})$ is a $G$-alternating cycle of $\Gamma$ with $r = \mathop{{\rm rad}}_G(\Gamma)$ and $C'$ is the other $G$-alternating cycle of $\Gamma$ containing $v_0$ then $C \cap C' = \{v_{i\ell} \colon 0 \leq i < a\}$ where $a = \mathop{{\rm att}}_G(\Gamma)$ and $\ell = 2r/a$ (see \cite[Proposition~2.6]{Mar98} and \cite[Proposition~3.4]{MarPra99}).
\medskip

As mentioned in the Introduction one of the goals of this paper is to characterize the tetravalent $G$-half-arc-transitive graphs $\Gamma$ with $\mathop{{\rm rad}}_G(\Gamma) = 3$ and $\mathop{{\rm att}}_G(\Gamma) = 2$. The bijective correspondence between such graphs and $2$-arc-transitive cubic graphs (see Theorem~\ref{the:main}) is given via two pairwise inverse constructions: the {\em graph of alternating cycles} construction and the {\em dart graph} construction. We first define the former.

\subsection{The graph of alternating cycles}
\label{subsec:Alt}

Let $\Gamma$ be a tetravalent $G$-half-arc-transitive graph for some $G \leq \mathrm{Aut}(\Gamma)$. The {\em graph of $G$-alternating cycles} $\mathrm{Alt}_G(\Gamma)$ is the graph whose vertex set consists of all $G$-alternating cycles of $\Gamma$ with two of them being adjacent whenever they have at least one vertex in common. We record some basic properties of the graph $\mathrm{Alt}_G(\Gamma)$.

\begin{proposition}
\label{pro:gr_alt_cyc}
Let $\Gamma$ be a connected tetravalent $G$-half-arc-transitive graph for some $G \leq \mathrm{Aut}(\Gamma)$ having at least three $G$-alternating cycles. Then the graph $\mathrm{Alt}_G(\Gamma)$ is a regular graph of valence $2\mathop{{\rm rad}}_G(\Gamma)/\mathop{{\rm att}}_G(\Gamma)$ and the induced action of $G$ on  $\mathrm{Alt}_G(\Gamma)$ is vertex- and edge-transitive. Moreover, this action is arc-transitive if and only if $\mathop{{\rm rad}}_{G}(\Gamma)$ does not divide $\mathop{{\rm att}}_G(\Gamma)$.
\end{proposition}

\begin{proof}
To simplify notation, denote $r = \mathop{{\rm rad}}_G(\Gamma)$ and $a = \mathop{{\rm att}}_G(\Gamma)$. Since each vertex of $\Gamma$ lies on exactly two $G$-alternating cycles and the intersection of any two non-disjoint $G$-alternating cycles is of size $a$ it is clear that each $G$-alternating cycle is adjacent to $\ell = 2r/a$ other $G$-alternating cycles in $\mathrm{Alt}_G(\Gamma)$. Moreover, since $G$ acts edge-transitively on $\Gamma$ and each edge of $\Gamma$ is contained in a unique $G$-alternating cycle, the induced action of $G$ on $\mathrm{Alt}_G(\Gamma)$ is vertex-transitive. That this action is also edge-transitive follows from the fact that $G$ acts vertex-transitively on $\Gamma$ and that the edges of $\mathrm{Alt}_G(\Gamma)$ correspond to $G$-attachment sets of $\Gamma$.

For the rest of the proof fix one of the two paired orientations of $\Gamma$ given by the action of $G$, let $C = (v_0, v_1, \ldots , v_{2r-1})$ be a $G$-alternating cycle such that $v_0 \to v_1$ and let $C'$ be the other $G$-alternating cycle containing $v_0$, so that $C \cap C' = \{v_{i\ell}\colon 0 \leq i < a\}$. Since every other vertex of $C$ is the tail of the two edges of $C$ incident to it, the vertex $v_\ell$ is the tail of the two edges of $C$ incident to it if and only if $\ell$ is even (in which case each $v_{i\ell}$ has this property).

Now, if $\ell$ is odd, then each element of $G$, mapping $v_0$ to $v_\ell$ necessarily interchanges $C$ and $C'$, proving that in this case the induced action of $G$ on $\mathrm{Alt}_G(\Gamma)$ is in fact arc-transitive. We remark that this also follows from the fact, first observed by Tutte~\cite{Tutte66}, that a vertex- and edge-transitive group of automorphisms of a graph of odd valence is necessarily arc-transitive. To complete the proof we thus only need to show that the induced action of $G$ on $\mathrm{Alt}_G(\Gamma)$ is not arc-transitive when $\ell$ is even. Recall that in this case each vertex $v_{i\ell} \in C \cap C'$ is the tail of the two edges of $C$ incident to it. Therefore, since any element of $G$, mapping the pair $\{C, C'\}$ to itself of course preserves the intersection $C \cap C'$ it is clear that any such element fixes each of $C$ and $C'$ setwise, and so no element of $G$ can interchange $C$ and $C'$. This proves that the induced action of $G$ on $\mathrm{Alt}_G(\Gamma)$ is half-arc-transitive.
\end{proof}

\subsection{The dart graph and its relation to $\mathrm{Alt}_G(\Gamma)$}
\label{subsec:Dart}

The dart graph of a cubic graph was investigated in~\cite{HilWil12} (we remark that this construction can also be viewed as a special kind of the {\em arc graph} construction from~\cite{GR01book}). Of course the dart graph construction can be applied to arbitrary graphs but here, as in~\cite{HilWil12}, we are only interested in dart graphs of cubic graphs. We first recall the definition. Let $\Lambda$ be a cubic graph. Then its {\em dart graph} $\mathop{{\rm Dart}}(\Lambda)$ is the graph whose vertex set consists of all the arcs (called darts in~\cite{HilWil12}) of $\Lambda$ with $(u,v)$ adjacent to $(u', v')$ if and only if either $u' = v$ but $u \neq v'$, or $u = v'$ but $u' \neq v$. In other words, the edges of $\mathop{{\rm Dart}}(\Lambda)$ correspond to the $2$-arcs of $\Lambda$. Note that this enables a natural orientation of the edges of $\mathop{{\rm Dart}}(\Lambda)$ where the edge $(u,v)(v,w)$ is oriented from $(u,v)$ to $(v,w)$.

Clearly, $\mathrm{Aut}(\Lambda)$ can be viewed as a subgroup of $\mathrm{Aut}(\mathop{{\rm Dart}}(\Lambda))$ preserving the natural orientation. Furthermore, the permutation $\tau$ of $V(\mathop{{\rm Dart}}(\Lambda))$, exchanging each  $(u,v)$ with $(v,u)$, is an orientation reversing automorphism of $\mathop{{\rm Dart}}(\Lambda)$.
\medskip

We now establish the correspondence between the $2$-arc-transitive cubic graphs and the tetravalent graphs admitting a half-arc-transitive group of automorphisms with the corresponding radius $3$ and attachment number $2$. We do this in two steps.

\begin{proposition}
\label{pro:Dart_to_Alt}
Let $\Lambda$ be a connected cubic graph admitting a $2$-arc-transitive group of automorphisms $G$ and let $\Gamma = \mathop{{\rm Dart}}(\Lambda)$. Then $\Gamma$ is a tetravalent $G$-half-arc-transitive graph such that $\mathop{{\rm rad}}_G(\Gamma) = 3$ and $\mathop{{\rm att}}_G(\Gamma) = 2$ with $\mathrm{Alt}_G(\Gamma) \cong \Lambda$. Moreover, the natural orientation of $\Gamma$, viewed as $\mathop{{\rm Dart}}(\Lambda)$, coincides with one of the two paired orientations induced by the action of $G$.
\end{proposition}

\begin{proof}
That the natural action of $G$ on $\Gamma$ is half-arc-transitive can easily be verified (see also~\cite{HilWil12}). Now, fix an edge $(u,v)(v,w)$ of $\Gamma$ and choose the $G$-induced orientation of $\Gamma$ in such a way that $(u,v) \to (v,w)$. Since $G$ is $2$-arc-transitive on $\Lambda$, the other edge of $\Gamma$, for which $(u,v)$ is its tail, is $(u,v)(v,w')$, where $w'$ is the remaining neighbour of $v$ in $\Lambda$ (other than $u$ and $w$). It is now clear that for each pair of adjacent vertices $(x,y)$ and $(y,z)$ of $\Gamma$ the corresponding edge is oriented from $(x,y)$ to $(y,z)$, and so the chosen $G$-induced orientation of $\Gamma$ is the natural orientation of $\mathop{{\rm Dart}}(\Lambda)$.

Finally, let $v$ be a vertex of $\Lambda$ and let $u,u',u''$ be its three neighbours. The $G$-alternating cycle of $\Gamma$ containing the edge $(u,v)(v,u')$ is then clearly $C_v = ((u,v),(v,u'),(u'',v),(v,u),(u',v),(v,u''))$, implying that $\mathop{{\rm rad}}_G(\Gamma) = 3$. This also shows that the $G$-alternating cycles of $\Gamma$ naturally correspond to vertices of $\Lambda$. Since the three $G$-alternating cycles of $\Gamma$ that have a nonempty intersection with $C_v$ are the ones corresponding to the vertices $u$, $u'$ and $u''$, this correspondence in fact shows that $\mathrm{Alt}_G(\Gamma)$ and $\Lambda$ are isomorphic and that $\mathop{{\rm att}}_G(\Gamma) = 2$.
\end{proof}

\begin{proposition}
\label{pro:Alt_to_Dart}
Let $\Gamma$ be a connected tetravalent $G$-half-arc-transitive graph for some $G \leq \mathrm{Aut}(\Gamma)$ with $\mathop{{\rm rad}}_G(\Gamma) = 3$ and $\mathop{{\rm att}}_G(\Gamma) = 2$, and let $\Lambda = \mathrm{Alt}_G(\Gamma)$. Then the group $G$ induces a $2$-arc-transitive action on $\Lambda$ and $\mathop{{\rm Dart}}(\Lambda) \cong \Gamma$. In fact, an isomorphism $\Psi\colon \mathop{{\rm Dart}}(\Lambda) \to \Gamma$ exists which maps the natural orientation of $\mathop{{\rm Dart}}(\Lambda)$ to a $G$-induced orientation of $\Gamma$.
\end{proposition}

\begin{proof}
By Proposition~\ref{pro:gr_alt_cyc} the graph $\Lambda$ is cubic and the induced action of $G$ on it is arc-transitive. Since $\mathop{{\rm rad}}_G(\Gamma) = 3$ and $\mathop{{\rm att}}_G(\Gamma) = 2$ it is easy to see that $\Gamma$ and $\mathop{{\rm Dart}}(\Lambda)$ are of the same order.  Furthermore, let $C = (v_0, v_1, \ldots , v_5)$ be a $G$-alternating cycle of $\Gamma$ and $C', C'', C'''$ be the other $G$-alternating cycles of $\Gamma$ containing $v_0, v_1$ and $v_5$, respectively. Then $C \cap C' = \{v_0, v_3\}$, $C \cap C'' = \{v_1, v_4\}$ and $C \cap C''' = \{v_2, v_5\}$. It is thus clear that any element of $G$, fixing $v_0$ and mapping $v_1$ to $v_5$ (which exists since $C$ is $G$-alternating and $G$ is edge-transitive on $\Gamma$), fixes both $C$ and $C'$ but maps $C''$ to $C'''$. Therefore, the induced action of $G$ on $\Lambda$ is $2$-arc-transitive.

To complete the proof we exhibit a particular isomorphism $\Psi \colon \mathop{{\rm Dart}}(\Lambda) \to \Gamma$. Fix an orientation of the edges of $\Gamma$, induced by the action of $G$, and let $C$ and $C'$ be two $G$-alternating cycles of $\Gamma$ with a nonempty intersection. Then $(C,C')$ and $(C',C)$ are vertices of $\mathop{{\rm Dart}}(\Lambda)$. Let $C \cap C' = \{u,u'\}$ and observe that precisely one of $u$ and $u'$ is the head of both of the edges of $C$ incident to it. Without loss of generality assume it is $u$. Then of course $u'$ is the head of both of the edges of $C'$ incident to it. We then set $\Psi((C,C')) = u$ and $\Psi((C',C)) = u'$. Therefore, for non-disjoint $G$-alternating cycles $C$ and $C'$ of $\Gamma$ we map $(C,C')$ to the unique vertex in $C \cap C'$ which is the head of both of the edges of $C$ incident to it. Since each pair of non-disjoint $G$-alternating cycles meets in precisely two vertices and each vertex of $\Gamma$ belongs to two $G$-alternating cycles of $\Gamma$, this mapping is injective and thus also bijective. We now only need to show that it preserves adjacency and maps the natural orientation of $\mathop{{\rm Dart}}(\Lambda)$ to the chosen $G$-induced orientation of $\Gamma$. To this end let $C$, $C'$ and $C''$ be three $G$-alternating cycles of $\Gamma$ such that $C$ has a nonempty intersection with both $C'$ and $C''$. Recall that then the edge $(C',C)(C,C'')$ is oriented from $(C',C)$ to $(C,C'')$ in the natural orientation of $\mathop{{\rm Dart}}(\Lambda)$. Denote $C = (v_0,v_1, \ldots , v_5)$ and without loss of generality assume $C \cap C' = \{v_0,v_3\}$ and $C \cap C'' = \{v_1, v_4\}$.

Suppose first that $v_0 \to v_1$. Then $v_0$ is the head of both of the edges of $C'$ incident to it, and so $\Psi((C',C)) = v_0$. Similarly, $v_1$ is the head of both of the edges of $C$ incident to it, and so $\Psi((C,C'')) = v_1$. If on the other hand $v_1 \to v_0$, then $\Psi((C',C)) = v_3$ and $\Psi((C,C'')) = v_4$. In both cases, $\Psi$ maps the oriented edge $(C',C)(C,C'')$ to an oriented edge of $\Gamma$, proving that it is an isomorphism of graphs, mapping the the natural orientation of $\mathop{{\rm Dart}}(\Lambda)$ to the chosen $G$-induced orientation of $\Gamma$.
\end{proof}

Theorem ~\ref{the:main} now follows directly from Propositions~\ref{pro:Dart_to_Alt} and \ref{pro:Alt_to_Dart}.

\section{Partial answer to Question~\ref{que:divides} and proof of Theorem~\ref{the:AT}}
\label{sec:AT}

In this section we prove Theorem~\ref{the:AT} giving a partial answer to Question~\ref{que:divides}. We first prove an auxiliary result.

\begin{proposition}
\label{pro:transversal}
Let  $\Gamma$ be a tetravalent $G$-half-arc-transitive graph with $\mathop{{\rm att}}_G(\Gamma)$ even. Then for each vertex $v$ of $\Gamma$ and the two $G$-alternating cycles $C$ and $C'$, containing $v$, the antipodal vertex of $v$ on $C$ coincides with the antipodal vertex of $v$ on $C'$. Moreover, the involution $\tau$ interchanging each pair of antipodal vertices on all $G$-alternating cycles of $\Gamma$ is an automorphism of $\Gamma$ centralising $G$.
\end{proposition}

\begin{proof}
Denote $r = \mathop{{\rm rad}}_G(\Gamma)$ and $a = \mathop{{\rm att}}_G(\Gamma)$. Let $v$ be a vertex of $\Gamma$ and let $C$ and $C'$ be the two $G$-alternating cycles of $\Gamma$ containing $v$. Denote $C = (v_0, v_1, \ldots , v_{2r-1})$ with $v = v_0$. Recall that then $C \cap C' = \{v_{i\ell}\colon 0 \leq i < a\}$, where $\ell = 2r/a$. Since $a$ is even $v_r \in C \cap C'$. Now, take any element $g \in G_v$ interchanging $v_1$ with $v_{2r-1}$ as well as the other two neighbours of $v$ (which are of course neighbours of $v$ on $C'$). Then $g$ reflects both $C$ and $C'$ with respect to $v$. Since $v_r$ is antipodal to $v$ on $C$, it must be fixed by $g$, but since $v_r$ is also contained in $C'$, this implies that it is in fact also the antipodal vertex of $v$ on $C'$. This shows that for each $G$-alternating cycle $C$ and each vertex $v$ of $C$ the vertex $v$ and its antipodal counterpart on $C$ both belong to the same pair of $G$-alternating cycles (this implies that the $G$-transversals, as they were defined in~\cite{Mar98}, are of length $2$) and are also antipodal on the other $G$-alternating cycle containing them.

It is now clear that $\tau$ is a well defined involution on the vertex set of $\Gamma$. Since the antipodal vertex of a neighbor $v_1$ of $v = v_0$ on $C$ is the neighbor $v_{r+1}$ of the antipodal vertex $v_r$, it is clear that $\tau$ is in fact an automorphism of $\Gamma$. Since any element of $G$ maps $G$-alternating cycles to $G$-alternating cycles it is clear that $\tau$ centralises $G$.
\end{proof}

We are now ready to prove Theorem~\ref{the:AT}. Let $\Gamma$ be a tetravalent half-arc-transitive graph. Denote $r = \mathop{{\rm rad}}(\Gamma)$ and $a = \mathop{{\rm att}}(\Gamma)$, and assume $r$ is odd. Recall that $a$ divides $2r$. We thus only need to prove that $a$ is odd. Suppose to the contrary that $a$ is even, and so by assumption $a \equiv 2 \pmod{4}$. Then the graph $\Gamma$ admits the automorphism $\tau$ from Proposition~\ref{pro:transversal}. Now, fix one of the two paired orientations of the edges induced by the action of $\mathrm{Aut}(\Gamma)$ and let $C = (v_0, v_1, \ldots , v_{2r-1})$ be an alternating cycle of $\Gamma$ with $v_0$ being the tail of the edge $v_0 v_1$. Since $v_0^\tau = v_r$ and $v_1^\tau = v_{r+1}$ it follows that $v_r$ is the tail of the edge $v_rv_{r+1}$. But since $r$ is odd this contradicts the fact that every other vertex of $C$ is the tail of the two edges of $C$ incident to it. Thus $a$ is odd, as claimed.

To prove the second part of the theorem assume that $a$ is not divisible by $4$. If $r$ is even then the fact that $a$ divides $2r$ implies that $a$ divides $r$ as well. If however $r$ is odd, we can apply the first part of the theorem. This completes the proof.

\section{An infinite family of non-sectional split covers}
\label{sec:ourcover}

As announced in the introduction, tetravalent $G$-half-arc-transitive graphs $\Gamma$ with $\mathop{{\rm rad}}_G(\Gamma) =3$ and $\mathop{{\rm att}}_G(\Gamma)=2$
yield surprising examples of the elusive non-sectional split covers. In this section, we present this connection in some detail.

\begin{theorem}
\label{the:cover}
Let $\Gamma$ be a connected non-bipartite $G$-half-arc-transitive graph $\Gamma$ of order greater than $12$
with $\mathop{{\rm rad}}_G(\Gamma) =3$ and $\mathop{{\rm att}}_G(\Gamma)=2$. Then there exists a $2$-fold covering projection $\wp \colon \Gamma \to \Gamma'$
and an arc-transitive group $H\le \mathrm{Aut}(\Gamma')$ which lifts along $\wp$ in such a way that $\Gamma$ is a non-sectional $H$-split cover of $\Gamma'$.
\end{theorem}

\begin{proof}
Since $\mathop{{\rm att}}_G(\Gamma)=2$, each $G$-attachment set consists of a pair of antipodal vertices on a $G$-alternating cycle of $\Gamma$.
Let $\mathcal{B}$ be the set of all $G$-attachment sets in $\Gamma$.
By Proposition~\ref{pro:transversal}, there exists an automorphism $\tau$ of $\Gamma$ centralising $G$,
 which interchanges the two vertices in each element of $\mathcal{B}$. Let $\tilde{G} = \langle G, \tau\rangle$ and note that
 $\tilde{G}$ acts transitively on the arcs of $\Gamma$. Since $\tau$ is an involution centralising $G$
 not contained in $G$, we see that $\tilde{G} = G \times \langle \tau \rangle$.

 Let $\Gamma'$ be the quotient graph with respect to the group $\langle \tau \rangle$, that is, the graph whose vertices are the orbits of $\langle \tau \rangle$
 and with two such orbits adjacent whenever they are joined by an edge in $\Gamma$. Since $\tilde{G}$ is arc-transitive and $\langle \tau\rangle$ is normal in $\tilde{G}$, each $\langle \tau \rangle$-orbit is
 an independent set. Moreover, if two $\langle \tau \rangle$-orbits $B$ and $C$ are adjacent in $\Gamma'$, then the induced subgraph $\Gamma[B\cup C]$
 is clearly vertex- and arc-transitive and is thus either $K_{2,2}$ or $2K_2$. In the former case, it is easy to see that $\Gamma$ is isomorphic to
 the lexicographic product of a cycle with the edge-less graph on two vertices. Since $\mathop{{\rm rad}}_G(\Gamma) = 3$ and the orbits of $\langle \tau\rangle$ coincide with the elements of $\mathcal{B}$, this implies that $\Gamma$
 has only $6$ vertices, contradicting our assumption on the order of $\Gamma$. This contradiction implies that
  $\Gamma[B\cup C] \cong 2K_2$ for any pair of adjacent $\langle \tau \rangle$-orbits $B$ and $C$, and hence the quotient projection
  $\wp \colon \Gamma \to \Gamma'$ is a $2$-fold covering projection with $\langle \tau \rangle$ being its group of covering transformations.

  Since $\tau$ normalises $G$, the group $\tilde{G}$ projects along $\wp$ and the quotient group
  $H = \tilde{G}/ \langle \tau \rangle$ acts faithfully as an arc-transitive group of automorphisms on $\Gamma'$.
  In particular, since the group of covering projection $\langle \tau \rangle$ has a complement $G$ in $\tilde{G}$, the covering projection $\wp$ is
  $H$-split.

  By \cite[Proposition 3.3]{FenKutMalMar}, if $\wp$ had a sectional complement with respect to $H$, then $\Gamma$  would be a canonical double cover of $\Gamma'$, contradicting the assumption that $\Gamma$ is not bipartite.
  \end{proof}

  {\sc Remark.} In \cite[Proposition~9]{HilWil12} it was shown that a cubic graph $\Lambda$ is bipartite if and only if $\mathop{{\rm Dart}}(\Lambda)$ is bipartite. Since there exist infinitely many connected non-bipartite cubic $2$-arc-transitive graphs, Theorem~\ref{the:main} thus implies
  that there are indeed infinitely many connected non-bipartite $G$-half-arc-transitive graphs $\Gamma$ with $\mathop{{\rm rad}}_G(\Gamma) =3$ and $\mathop{{\rm att}}_G(\Gamma)=2$.
  In view of Theorem~\ref{the:cover}, these yield infinitely many non-sectional split covers, as announced in the introduction. Furthermore, note that
  the $G$-alternating $6$-cycles in the graph $\Gamma$ appearing in the proof of the above theorem
    project by $\wp$ to cycles of length $3$, implying that $\Gamma'$ is a tetravalent arc-transitive graph of girth $3$. Since
it  is assumed that the order of $\Gamma$ is larger than $12$ (and thus the order of $\Gamma'$ is larger than $6$), we may now use
  \cite[Theorem 5.1]{girth4} to conclude that $\Gamma'$ is isomorphic to the line graph of a $2$-arc-transitive cubic graph.
\bigskip

\noindent
{\bf Acknowledgment.} The first author was supported in part by Slovenian Research Agency, program P1-0294. The second author was supported in part by Slovenian Research Agency, program P1-0285 and projects N1-0038, J1-6720 and J1-7051.