text: In this paper we consider the problem of human pose estimation in real-world videos of swimmers. Swimming channels allow filming swimmers simultaneously above and below the water surface with a single stationary camera. These recordings can be used to quantitatively assess the athletes' performance. The quantitative evaluation, so far, requires manual annotations of body parts in each video frame. We therefore apply the concept of CNNs in order to automatically infer the required pose information. Starting with an off-the-shelf architecture, we develop extensions to leverage activity information – in our case the swimming style of an athlete – and the continuous nature of the video recordings. Our main contributions are threefold: (a) We apply and evaluate a fine-tuned Convolutional Pose Machine architecture as a baseline in our very challenging aquatic environment and discuss its error modes, (b) we propose an extension to input swimming style information into the fully convolutional architecture and (c) modify the architecture for continuous pose estimation in videos. With these additions we achieve reliable pose estimates with up to +16 more correct body joint detections compared to the baseline architecture.
  @cite_4: Human activity recognition is a fundamental problem in computer vision with many applications such as video retrieval, automatic visual surveillance and human computer interaction. Sports represent one of the most viewed content on digital tv and the web. Automatically collected statistics of team sports game play represent actionable information for many end users such as coaches and broadcast speakers. Many computer vision methods applied to sport activity classification are often based on multi-camera setups, player tracking and exploit information on the ground-plane. In this work we overcome this limitations and propose an approach that exploits the spatio-temporal structure of a video grouping local spatio-temporal features unsupervisedly. Our robust representation allows to measure video similarity making correspondences among arbitrary patterns. We tested our method on two dataset of Volleyball and Soccer actions outperforming previous results by a large margin. Finally we show how our representation allows to highlight discriminative regions for each action.
  @cite_8: Tracking players in sports videos presents numerous challenges due to weak distinguishing features and unpredictable motion. Considerable work has been done to track players in such videos using a combination of appearance and motion modeling, mostly in continuous streams of video. However, in a broadcast sports video, having advertisements, replays and intermittent change of camera view, it becomes a challenging task to keep track of players over an entire game. In this work, we solve a novel problem of tracking over a sequence of temporally disjoint soccer videos without the use of appearance cue, using a Graph based optimization approach. Each team is represented by a graph, in which the nodes correspond to player positions and the edge weights depend on spatial inter-player distance. We use team formation to associate tracks between clips and provide an end-to-end system that is able to perform statistical and tactical analysis of the game. We also introduce a new challenging dataset of an international soccer game.
  @cite_24: Tracking objects like a basketball from a monocular view is challenging due to its small size, potential to move at high velocities as well as the high frequency of occlusion. However, humans with a deep knowledge of a game like basketball can predict with high accuracy the location of the ball even without seeing it due to the location and motion of nearby objects, as well as information of where it was last seen. Learning from tracking data is problematic however, due to the high variance in player locations. In this paper, we show that by simply "permuting" the multi-agent data we obtain a compact role-ordered feature which accurately predict the ball owner. We also show that our formulation can incorporate other information sources such as a vision-based ball detector to improve prediction accuracy.
  @cite_19: Due to recent advances in technology, the recording and analysis of video data has become an increasingly common component of athlete training programmes. Today it is incredibly easy and affordable to set up a fixed camera and record athletes in a wide range of sports, such as diving, gymnastics, golf, tennis, etc. However, the manual analysis of the obtained footage is a time-consuming task which involves isolating actions of interest and categorizing them using domain-specific knowledge. In order to automate this kind of task, three challenging sub-problems are often encountered: 1) temporally cropping events actions of interest from continuous video; 2) tracking the object of interest; and 3) classifying the events actions of interest.,,,,,,Most previous work has focused on solving just one of the above sub-problems in isolation. In contrast, this paper provides a complete solution to the overall action monitoring task in the context of a challenging real-world exemplar. Specifically, we address the problem of diving classification. This is a challenging problem since the person (diver) of interest typically occupies fewer than 1 of the pixels in each frame. The model is required to learn the temporal boundaries of a dive, even though other divers and bystanders may be in view. Finally, the model must be sensitive to subtle changes in body pose over a large number of frames to determine the classification code. We provide effective solutions to each of the sub-problems which combine to provide a highly functional solution to the task as a whole. The techniques proposed can be easily generalized to video footage recorded from other sports.
  @cite_23: The first book of its kind devoted to this topic, this comprehensive text reference presents state-of-the-art research and reviews current challenges in the application of computer vision to problems in sports. Opening with a detailed introduction to the use of computer vision across the entire life-cycle of a sports event, the text then progresses to examine cutting-edge techniques for tracking the ball, obtaining the whereabouts and pose of the players, and identifying the sport being played from video footage. The work concludes by investigating a selection of systems for the automatic analysis and classification of sports play. The insights provided by this pioneering collection will be of great interest to researchers and practitioners involved in computer vision, sports analysis and media production.
  @cite_31: Analyzing joint movements of an athlete helps to improve the pose of the athlete. Human pose estimation (HPE) algorithms regress the locations of parts such as wrists, ankles and knees. In this paper, we propose a network that combines global and local information for HPE using a 2D image. Unlike previous works that have used global or local information separately, we use the combined information to enhance the performance of HPE. General information from a global network is used as an input to a local network to refine the location of a part using a variety of regions. The global network is based on ResNet-101 [6] and trained to regress a heatmap representing parts’ locations. The output features from the global network are used as input features for the local network. The local network learns spatial information using position sensitive score maps [11]. Through the end-to-end learning, the global network is affected by the local information. We demonstrate that the proposed HPE method is efficient on the LSP and UCF sports datasets.
  @cite_20: Human pose estimation from monocular video streams is a challenging problem. Much of the work on this problem has focused on developing inference algorithms and probabilistic prior models based on learned measurements. Such algorithms face challenges in generalization beyond the learned dataset. We propose an interactive model-based generative approach for estimating the human pose in 2D from uncalibrated monocular video in unconstrained sports TV footage without any prior learning on motion captured or annotated data. Belief-propagation over a spatio-temporal graph of candidate body part hypotheses is used to estimate a temporally consistent pose between key-frame constraints. Experimental results show that the proposed generative pose estimation framework is capable of estimating pose even in very challenging unconstrained scenarios.
  @cite_17: Automatic classification of activities in a sports arena is important in order to analyse and optimise the use of the arenas. In this work we classify five sports types based only on occupancy heatmaps produced from position data. Due to privacy issues we use thermal imaging for detecting people and then calculate their positions on the court using homography. Heatmaps are produced by summarising Gaussian distributions respresenting people over 10-minute periods. Before classification the heatmaps are projected to a low-dimensional discriminative space using the principle of Fisherfaces. Our result using two weeks of video are very promising with a correct classification of 90.76 .
summary: Computer vision has been adopted for various applications in the sports domain. Prominent tasks include sports type @cite_17 and activity recognition @cite_4 @cite_19 , tracking athletes and other objects of interest in videos @cite_8 @cite_24 and human pose estimation @cite_20 @cite_31 . @cite_23 offer an overview of a wide range of application.