  Objective: Surgical activity recognition is a fundamental step in
computer-assisted interventions. This paper reviews the state-of-the-art in
methods for automatic recognition of fine-grained gestures in robotic surgery
focusing on recent data-driven approaches and outlines the open questions and
future research directions. Methods: An article search was performed on 5
bibliographic databases with the following search terms: robotic,
robot-assisted, JIGSAWS, surgery, surgical, gesture, fine-grained, surgeme,
action, trajectory, segmentation, recognition, parsing. Selected articles were
classified based on the level of supervision required for training and divided
into different groups representing major frameworks for time series analysis
and data modelling. Results: A total of 52 articles were reviewed. The research
field is showing rapid expansion, with the majority of articles published in
the last 4 years. Deep-learning-based temporal models with discriminative
feature extraction and multi-modal data integration have demonstrated promising
results on small surgical datasets. Currently, unsupervised methods perform
significantly less well than the supervised approaches. Conclusion: The
development of large and diverse open-source datasets of annotated
demonstrations is essential for development and validation of robust solutions
for surgical gesture recognition. While new strategies for discriminative
feature extraction and knowledge transfer, or unsupervised and semi-supervised
approaches, can mitigate the need for data and labels, they have not yet been
demonstrated to achieve comparable performance. Important future research
directions include detection and forecast of gesture-specific errors and
anomalies. Significance: This paper is a comprehensive and structured analysis
of surgical gesture recognition methods aiming to summarize the status of this
rapidly evolving field.
