  Voice User Interfaces (VUIs) are increasingly popular and built into
smartphones, home assistants, and Internet of Things (IoT) devices. Despite
offering an always-on convenient user experience, VUIs raise new security and
privacy concerns for their users. In this paper, we focus on attribute
inference attacks in the speech domain, demonstrating the potential for an
attacker to accurately infer a target user's sensitive and private attributes
(e.g. their emotion, sex, or health status) from deep acoustic models. To
defend against this class of attacks, we design, implement, and evaluate a
user-configurable, privacy-aware framework for optimizing speech-related data
sharing mechanisms. Our objective is to enable primary tasks such as speech
recognition and user identification, while removing sensitive attributes in the
raw speech data before sharing it with a cloud service provider. We leverage
disentangled representation learning to explicitly learn independent factors in
the raw data. Based on a user's preferences, a supervision signal informs the
filtering out of invariant factors while retaining the factors reflected in the
selected preference. Our experimental evaluation over five datasets shows that
the proposed framework can effectively defend against attribute inference
attacks by reducing their success rates to approximately that of guessing at
random, while maintaining accuracy in excess of 99% for the tasks of interest.
We conclude that negotiable privacy settings enabled by disentangled
representations can bring new opportunities for privacy-preserving
applications.
