text: Clinical observations indicate that during critical care at the hospitals, patients sleep positioning and motion affect recovery. Unfortunately, there is no formal medical protocol to record, quantify, and analyze patient motion. There is a small number of clinical studies, which use manual analysis of sleep poses and motion recordings to support medical benefits of patient positioning and motion monitoring. Manual processes are not scalable, are prone to human errors, and strain an already taxed healthcare workforce. This study introduces DECU (Deep Eye-CU): an autonomous mulitmodal multiview system, which addresses these issues by autonomously monitoring healthcare environments and enabling the recording and analysis of patient sleep poses and motion. DECU uses three RGB-D cameras to monitor patient motion in a medical Intensive Care Unit (ICU). The algorithms in DECU estimate pose direction at different temporal resolutions and use keyframes to efficiently represent pose transition dynamics. DECU combines deep features computed from the data with a modified version of Hidden Markov Model to more flexibly model sleep pose duration, analyze pose patterns, and summarize patient motion. Extensive experimental results are presented. The performance of DECU is evaluated in ideal (BC: Bright and Clear occlusion-free) and natural (DO: Dark and Occluded) scenarios at two motion resolutions in a mock-up and a real ICU. The results indicate that deep features allow DECU to match the classification performance of engineered features in BC scenes and increase the accuracy by up to 8 in DO scenes. In addition, the overall pose history summarization tracing accuracy shows an average detection rate of 85 in BC and of 76 in DO scenes. The proposed keyframe estimation algorithm allows DECU to reach an average 78 transition classification accuracy.
  @cite_16: We present AALO: a novel Activity recognition system for single person smart homes using Active Learning in the presence of Overlapped activities. AALO applies data mining techniques to cluster in-home sensor firings so that each cluster represents instances of the same activity. Users only need to label each cluster as an activity as opposed to labeling all instances of all activities. Once the clusters are associated to their corresponding activities, our system can recognize future activities. To improve the activity recognition accuracy, our system preprocesses raw sensor data by identifying overlapping activities. The evaluation of activity recognition performance on a 26-day dataset shows that compared to Naive Bayesian (NB), Hidden Markov Model (HMM), and Hidden Semi Markov Model (HSMM) based activity recognition systems, our average time slice error (24.15 ) is much lower than NB (53.04 ), and similar to HMM (29.97 ) and HSMM (26.29 ). Thus, our active learning based approach performs as good as the state of the art supervised techniques (HMM and HSMM).
  @cite_21: Recognizing activities in a home environment is challenging due to the variety of activities that can be performed at home and the complexity of the environment. Multiple cameras are usually needed to cover the whole observation area. This adds camera fusion as another challenge to activity recognition. We propose a hierarchical approach that recognizes both coarse-level and fine-level activities, in which different image features and learning methods are used for different activities based on their characteristics. The paper focuses on discussing the second-level of activity recognition with spatio-temporal features. Specifically, three fusion approaches for multiview activity recognition with spatio-temporal features are presented, including two decision fusion methods and one feature fusion method. They are comparatively analyzed in terms of their tradeoffs on assumptions on system setup, model transferability and recognition rate. Experiments show that challenging activities with subtle motions such as eating, cutting, scrambling, typing, reading etc. can be recognized with our approaches.
summary: Multi-sensor and multi-camera systems and methods have been applied to smart environments @cite_16 and @cite_21 . The systems require alterations to existing infrastructure making their deployment in a hospital logistically impossible. The methods are not designed to account for illumination variations and occlusions and do not account for non-sequential, subtle motion. Therefore, these systems and methods cannot be used to analyze patient motion in a real ICU where patients have limited or constrained mobility and the scenes have random occlusions and unpredictable illumination.