  Relative state estimation using exteroceptive sensors suffers from
limitations of the field of view (FOV) and false detection, that the
proprioceptive sensor (IMU) data are usually engaged to compensate. Recently
ego-motion constraint obtained by Inertial measurement unit (IMU)
preintegration has been extensively used in simultaneous localization and
mapping (SLAM) to alleviate the computation burden. This paper introduces an
extended preintegration incorporating the IMU preintegration of two platforms
to formulate the motion constraint of relative state. One merit of this
analytic constraint is that it can be seamlessly integrated into the unified
graph optimization framework to implement the relative state estimation in a
high-performance real-time tracking thread, another point is a full smoother
design with this precise constraint to optimize the 3D coordinate and refine
the state for the refinement thread. We compare extensively in simulations the
proposed algorithms with two existing approaches to confirm our outperformance.
In the real virtual reality (VR) application design with the proposed
estimator, we properly realize the visual tracking of the six degrees of
freedom (6DoF) controller suitable for almost all scenarios, including the
challenging environment with missing features, light mutation, dynamic scenes,
etc. The demo video is at https://www.youtube.com/watch?v=0idb9Ls2iAM. For the
benefit of the community, we make the source code public.
