  Kinesthetic Teaching is a popular approach to collecting expert robotic
demonstrations of contact-rich tasks for imitation learning (IL), but it
typically only measures motion, ignoring the force placed on the environment by
the robot. Furthermore, contact-rich tasks require accurate sensing of both
reaching and touching, which can be difficult to provide with conventional
sensing modalities. We address these challenges with a See-Through-your-Skin
(STS) visuotactile sensor, using the sensor both (i) as a measurement tool to
improve kinesthetic teaching, and (ii) as a policy input in contact-rich door
manipulation tasks. An STS sensor can be switched between visual and tactile
modes by leveraging a semi-transparent surface and controllable lighting,
allowing for both pre-contact visual sensing and during-contact tactile sensing
with a single sensor. First, we propose tactile force matching, a methodology
that enables a robot to match forces read during kinesthetic teaching using
tactile signals. Second, we develop a policy that controls STS mode switching,
allowing a policy to learn the appropriate moment to switch an STS from its
visual to its tactile mode. Finally, we study multiple observation
configurations to compare and contrast the value of visual and tactile data
from an STS with visual data from a wrist-mounted eye-in-hand camera. With over
3,000 test episodes from real-world manipulation experiments, we find that the
inclusion of force matching raises average policy success rates by 62.5%, STS
mode switching by 30.3%, and STS data as a policy input by 42.5%. Our results
highlight the utility of see-through tactile sensing for IL, both for data
collection to allow force matching, and for policy execution to allow accurate
task feedback.
