  This paper presents a novel approach, called AirScript, for creating,
recognizing and visualizing documents in air. We present a novel algorithm,
called 2-DifViz, that converts the hand movements in air (captured by a
Myo-armband worn by a user) into a sequence of x, y coordinates on a 2D
Cartesian plane, and visualizes them on a canvas. Existing sensor-based
approaches either do not provide visual feedback or represent the recognized
characters using prefixed templates. In contrast, AirScript stands out by
giving freedom of movement to the user, as well as by providing a real-time
visual feedback of the written characters, making the interaction natural.
AirScript provides a recognition module to predict the content of the document
created in air. To do so, we present a novel approach based on deep learning,
which uses the sensor data and the visualizations created by 2-DifViz. The
recognition module consists of a Convolutional Neural Network (CNN) and two
Gated Recurrent Unit (GRU) Networks. The output from these three networks is
fused to get the final prediction about the characters written in air.
AirScript can be used in highly sophisticated environments like a smart
classroom, a smart factory or a smart laboratory, where it would enable people
to annotate pieces of texts wherever they want without any reference surface.
We have evaluated AirScript against various well-known learning models (HMM,
KNN, SVM, etc.) on the data of 12 participants. Evaluation results show that
the recognition module of AirScript largely outperforms all of these models by
achieving an accuracy of 91.7% in a person independent evaluation and a 96.7%
accuracy in a person dependent evaluation.
