Hark: A deep learning system for navigating privacy feedback at scale

H Harkous, ST Peddinti, R Khandelwal… - … IEEE Symposium on …, 2022 - ieeexplore.ieee.org
2022 IEEE Symposium on Security and Privacy (SP), 2022ieeexplore.ieee.org
Integrating user feedback is one of the pillars for building successful products. However, this
feedback is generally collected in an unstructured free-text form, which is challenging to
understand at scale. This is particularly demanding in the privacy domain due to the
nuances associated with the concept and the limited existing solutions. In this work, we
present Hark 1, a system for discovering and summarizing privacy-related feedback at scale.
Hark automates the entire process of summarizing privacy feedback, starting from …
Integrating user feedback is one of the pillars for building successful products. However, this feedback is generally collected in an unstructured free-text form, which is challenging to understand at scale. This is particularly demanding in the privacy domain due to the nuances associated with the concept and the limited existing solutions. In this work, we present Hark 1 , a system for discovering and summarizing privacy-related feedback at scale. Hark automates the entire process of summarizing privacy feedback, starting from unstructured text and resulting in a hierarchy of high-level privacy themes and fine-grained issues within each theme, along with representative reviews for each issue. At the core of Hark is a set of new deep learning models trained on different tasks, such as privacy feedback classification, privacy issues generation, and high-level theme creation. We illustrate Hark’s efficacy on a corpus of 626 M Google Play reviews. Out of this corpus, our privacy feedback classifier extracts privacy-related reviews (with an AUC-ROC of 0.92). With three annotation studies, we show that Hark’s generated issues are of high accuracy and coverage and that the theme titles are of high quality. We illustrate Hark’s capabilities by presenting high-level insights from Android apps. 1 an English verb meaning to “pay close attention”
ieeexplore.ieee.org
Showing the best result for this search. See all results