  Modern noise-cancelling headphones have significantly improved users'
auditory experiences by removing unwanted background noise, but they can also
block out sounds that matter to users. Machine learning (ML) models for sound
event detection (SED) and speaker identification (SID) can enable headphones to
selectively pass through important sounds; however, implementing these models
for a user-centric experience presents several unique challenges. First, most
people spend limited time customizing their headphones, so the sound detection
should work reasonably well out of the box. Second, the models should be able
to learn over time the specific sounds that are important to users based on
their implicit and explicit interactions. Finally, such models should have a
small memory footprint to run on low-power headphones with limited on-chip
memory. In this paper, we propose addressing these challenges using HiSSNet
(Hierarchical SED and SID Network). HiSSNet is an SEID (SED and SID) model that
uses a hierarchical prototypical network to detect both general and specific
sounds of interest and characterize both alarm-like and speech sounds. We show
that HiSSNet outperforms an SEID model trained using non-hierarchical
prototypical networks by 6.9 - 8.6 percent. When compared to state-of-the-art
(SOTA) models trained specifically for SED or SID alone, HiSSNet achieves
similar or better performance while reducing the memory footprint required to
support multiple capabilities on-device.
