  Can robots mold soft plastic materials by shaping depth images? The short
answer is no: current day robots can't. In this article, we address the problem
of shaping plastic material with an anthropomorphic arm/hand robot, which
observes the material with a fixed depth camera. Robots capable of molding
could assist humans in many tasks, such as cooking, scooping or gardening. Yet,
the problem is complex, due to its high-dimensionality at both perception and
control levels. To address it, we design three alternative data-based methods
for predicting the effect of robot actions on the material. Then, the robot can
plan the sequence of actions and their positions, to mold the material into a
desired shape. To make the prediction problem tractable, we rely on two
original ideas. First, we prove that under reasonable assumptions, the shaping
problem can be mapped from point cloud to depth image space, with many benefits
(simpler processing, no need for registration, lower computation time and
memory requirements). Second, we design a novel, simple metric for quickly
measuring the distance between two depth images. The metric is based on the
inherent point cloud representation of depth images, which enables direct and
consistent comparison of image pairs through a non-uniform scaling approach,
and therefore opens promising perspectives for designing \textit{depth image --
based} robot controllers. We assess our approach in a series of unprecedented
experiments, where a robotic arm/hand molds flour from initial to final shapes,
either with its own dataset, or by transfer learning from a human dataset. We
conclude the article by discussing the limitations of our framework and those
of current day hardware, which make human-like robot molding a challenging open
research problem.
