text: In this work, we address a challenging problem of fine-grained and coarse-grained recognition of object manipulation actions. Due to the variations in geometrical and motion constraints, there are different manipulations actions possible to perform different sets of actions with an object. Also, there are subtle movements involved to complete most of object manipulation actions. This makes the task of object manipulation action recognition difficult with only just the motion information. We propose to use grasp and motion-constraints information to recognise and understand action intention with different objects. We also provide an extensive experimental evaluation on the recent Yale Human Grasping dataset consisting of large set of 455 manipulation actions. The evaluation involves a) Different contemporary multi-class classifiers, and binary classifiers with one-vs-one multi- class voting scheme, b) Differential comparisons results based on subsets of attributes involving information of grasp and motion-constraints, c) Fine-grained and Coarse-grained object manipulation action recognition based on fine-grained as well as coarse-grained grasp type information, and d) Comparison between Instance level and Sequence level modeling of object manipulation actions. Our results justifies the efficacy of grasp attributes for the task of fine-grained and coarse-grained object manipulation action recognition.
  @cite_29: This paper presents a dataset of human grasping behavior in unstructured environments. Wide-angle head-mounted camera video was recorded from two housekeepers and two machinists during their regular work activities, and the grasp types, objects, and tasks were analyzed and coded by study staff. The full dataset contains 27.7 hours of tagged video and represents a wide range of manipulative behaviors spanning much of the typical human hand usage. We provide the original videos, a spreadsheet including the tagged grasp type, object, and task parameters, time information for each successive grasp, and video screenshots for each instance. Example code is provided for MATLAB and R, demonstrating how to load in the dataset and produce simple plots.
summary: The important aspects of our work include: a) A compact representation of the grasp and motion-constraints using some popular and some contemporary schemes. b) Demonstrating the usefulness of information from coarse-grained and fine-grained grasp attributes as well as motion-constraints for fine-grained and coarse-grained action recognition. c) A differential experimental analysis involving subsets of grasp and motion-constraints features, to provide more insights on the usefulness of grasp information alone, motion-constraints information alone, and grasp and motion-constraints based information together for intended classification problem. d) Comparisons between Instance and Sequence level modeling of object manipulation actions using fine-grained grasp information. e) An extensive experimental evaluation using different contemporary multi-class and binary classifiers (with a multi-class voting strategy), which also serves as a useful comparative study of popular classifiers for the manipulation action recognition problem. This analysis also helps to demonstrate that different classification frameworks, largely arrive at a consensus with respect to our hypothesis about using grasp and motion-constraints for fine-grained action classification. We demonstrate our results on a large Yale Human Grasping dataset @cite_29 which involves various tasks on different objects.