  Screen recordings of mobile applications are easy to obtain and capture a
wealth of information pertinent to software developers (e.g., bugs or feature
requests), making them a popular mechanism for crowdsourced app feedback. Thus,
these videos are becoming a common artifact that developers must manage. In
light of unique mobile development constraints, including swift release cycles
and rapidly evolving platforms, automated techniques for analyzing all types of
rich software artifacts provide benefit to mobile developers. Unfortunately,
automatically analyzing screen recordings presents serious challenges, due to
their graphical nature, compared to other types of (textual) artifacts. To
address these challenges, this paper introduces V2S+, an automated approach for
translating video recordings of Android app usages into replayable scenarios.
V2S+ is based primarily on computer vision techniques and adapts recent
solutions for object detection and image classification to detect and classify
user gestures captured in a video, and convert these into a replayable test
scenario. Given that V2S+ takes a computer vision-based approach, it is
applicable to both hybrid and native Android applications. We performed an
extensive evaluation of V2S+ involving 243 videos depicting 4,028 GUI-based
actions collected from users exercising features and reproducing bugs from a
collection of over 90 popular native and hybrid Android apps. Our results
illustrate that V2S+ can accurately replay scenarios from screen recordings,
and is capable of reproducing $\approx$ 90.2% of sequential actions recorded in
native application scenarios on physical devices, and $\approx$ 83% of
sequential actions recorded in hybrid application scenarios on emulators, both
with low overhead. A case study with three industrial partners illustrates the
potential usefulness of V2S+ from the viewpoint of developers.
