  Robotic manipulation requires accurate perception of the environment, which
poses a significant challenge due to its inherent complexity and constantly
changing nature. In this context, RGB image and point-cloud observations are
two commonly used modalities in visual-based robotic manipulation, but each of
these modalities have their own limitations. Commercial point-cloud
observations often suffer from issues like sparse sampling and noisy output due
to the limits of the emission-reception imaging principle. On the other hand,
RGB images, while rich in texture information, lack essential depth and 3D
information crucial for robotic manipulation. To mitigate these challenges, we
propose an image-only robotic manipulation framework that leverages an
eye-on-hand monocular camera installed on the robot's parallel gripper. By
moving with the robot gripper, this camera gains the ability to actively
perceive object from multiple perspectives during the manipulation process.
This enables the estimation of 6D object poses, which can be utilized for
manipulation. While, obtaining images from more and diverse viewpoints
typically improves pose estimation, it also increases the manipulation time. To
address this trade-off, we employ a reinforcement learning policy to
synchronize the manipulation strategy with active perception, achieving a
balance between 6D pose accuracy and manipulation efficiency. Our experimental
results in both simulated and real-world environments showcase the
state-of-the-art effectiveness of our approach. %, which, to the best of our
knowledge, is the first to achieve robust real-world robotic manipulation
through active pose estimation. We believe that our method will inspire further
research on real-world-oriented robotic manipulation.
