  In this paper, we propose a real-world benchmark for studying robotic
learning in the context of functional manipulation: a robot needs to accomplish
complex long-horizon behaviors by composing individual manipulation skills in
functionally relevant ways. The core design principles of our Functional
Manipulation Benchmark (FMB) emphasize a harmonious balance between complexity
and accessibility. Tasks are deliberately scoped to be narrow, ensuring that
models and datasets of manageable scale can be utilized effectively to track
progress. Simultaneously, they are diverse enough to pose a significant
generalization challenge. Furthermore, the benchmark is designed to be easily
replicable, encompassing all essential hardware and software components. To
achieve this goal, FMB consists of a variety of 3D-printed objects designed for
easy and accurate replication by other researchers. The objects are
procedurally generated, providing a principled framework to study
generalization in a controlled fashion. We focus on fundamental manipulation
skills, including grasping, repositioning, and a range of assembly behaviors.
The FMB can be used to evaluate methods for acquiring individual skills, as
well as methods for combining and ordering such skills to solve complex,
multi-stage manipulation tasks. We also offer an imitation learning framework
that includes a suite of policies trained to solve the proposed tasks. This
enables researchers to utilize our tasks as a versatile toolkit for examining
various parts of the pipeline. For example, researchers could propose a better
design for a grasping controller and evaluate it in combination with our
baseline reorientation and assembly policies as part of a pipeline for solving
multi-stage tasks. Our dataset, object CAD files, code, and evaluation videos
can be found on our project website:
https://functional-manipulation-benchmark.github.io
