  Equipping robots with the sense of touch is critical to emulating the
capabilities of humans in real world manipulation tasks. Visuotactile sensors
are a popular tactile sensing strategy due to data output compatible with
computer vision algorithms and accurate, high resolution estimates of local
object geometry. However, these sensors struggle to accommodate high
deformations of the sensing surface during object interactions, hindering more
informative contact with cm-scale objects frequently encountered in the real
world. The soft interfaces of visuotactile sensors are often made of
hyperelastic elastomers, which are difficult to simulate quickly and accurately
when extremely deformed for tactile information. Additionally, many
visuotactile sensors that rely on strict internal light conditions or pattern
tracking will fail if the surface is highly deformed. In this work, we propose
an algorithm that fuses proximity and visuotactile point clouds for contact
patch segmentation that is entirely independent from membrane mechanics. This
algorithm exploits the synchronous, high-res proximity and visuotactile
modalities enabled by an extremely deformable, selectively transmissive soft
membrane, which uses visible light for visuotactile sensing and infrared light
for proximity depth. We present the hardware design, membrane fabrication, and
evaluation of our contact patch algorithm in low (10%), medium (60%), and high
(100%+) membrane strain states. We compare our algorithm against three
baselines: proximity-only, tactile-only, and a membrane mechanics model. Our
proposed algorithm outperforms all baselines with an average RMSE under 2.8mm
of the contact patch geometry across all strain ranges. We demonstrate our
contact patch algorithm in four applications: varied stiffness membranes,
torque and shear-induced wrinkling, closed loop control for whole body
manipulation, and pose estimation.
