  Human activity recognition (HAR) is one of the core research themes in
ubiquitous and wearable computing. With the shift to deep learning (DL) based
analysis approaches, it has become possible to extract high-level features and
perform classification in an end-to-end manner. Despite their promising overall
capabilities, DL-based HAR may suffer from overfitting due to the notoriously
small, often inadequate, amounts of labeled sample data that are available for
typical HAR applications. In response to such challenges, we propose ConvBoost
-- a novel, three-layer, structured model architecture and boosting framework
for convolutional network based HAR. Our framework generates additional
training data from three different perspectives for improved HAR, aiming to
alleviate the shortness of labeled training data in the field. Specifically,
with the introduction of three conceptual layers--Sampling Layer, Data
Augmentation Layer, and Resilient Layer -- we develop three "boosters" --
R-Frame, Mix-up, and C-Drop -- to enrich the per-epoch training data by
dense-sampling, synthesizing, and simulating, respectively. These new
conceptual layers and boosters, that are universally applicable for any kind of
convolutional network, have been designed based on the characteristics of the
sensor data and the concept of frame-wise HAR. In our experimental evaluation
on three standard benchmarks (Opportunity, PAMAP2, GOTOV) we demonstrate the
effectiveness of our ConvBoost framework for HAR applications based on variants
of convolutional networks: vanilla CNN, ConvLSTM, and Attention Models. We
achieved substantial performance gains for all of them, which suggests that the
proposed approach is generic and can serve as a practical solution for boosting
the performance of existing ConvNet-based HAR models. This is an open-source
project, and the code can be found at https://github.com/sshao2013/ConvBoost
