Redwood: Using Collision Detection to Grow a Large-Scale Intent Classification Dataset

Stefan Larson, Kevin Leach


Abstract
Dialog systems must be capable of incorporating new skills via updates over time in order to reflect new use cases or deployment scenarios. Similarly, developers of such ML-driven systems need to be able to add new training data to an already-existing dataset to support these new skills. In intent classification systems, problems can arise if training data for a new skill’s intent overlaps semantically with an already-existing intent. We call such cases collisions. This paper introduces the task of intent collision detection between multiple datasets for the purposes of growing a system’s skillset. We introduce several methods for detecting collisions, and evaluate our methods on real datasets that exhibit collisions. To highlight the need for intent collision detection, we show that model performance suffers if new data is added in such a way that does not arbitrate colliding intents. Finally, we use collision detection to construct and benchmark a new dataset, Redwood, which is composed of 451 categories from 13 original intent classification datasets, making it the largest publicly available intent classification benchmark.
Anthology ID:
2022.sigdial-1.45
Volume:
Proceedings of the 23rd Annual Meeting of the Special Interest Group on Discourse and Dialogue
Month:
September
Year:
2022
Address:
Edinburgh, UK
Editors:
Oliver Lemon, Dilek Hakkani-Tur, Junyi Jessy Li, Arash Ashrafzadeh, Daniel Hernández Garcia, Malihe Alikhani, David Vandyke, Ondřej Dušek
Venue:
SIGDIAL
SIG:
SIGDIAL
Publisher:
Association for Computational Linguistics
Note:
Pages:
468–477
Language:
URL:
https://aclanthology.org/2022.sigdial-1.45
DOI:
10.18653/v1/2022.sigdial-1.45
Bibkey:
Cite (ACL):
Stefan Larson and Kevin Leach. 2022. Redwood: Using Collision Detection to Grow a Large-Scale Intent Classification Dataset. In Proceedings of the 23rd Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 468–477, Edinburgh, UK. Association for Computational Linguistics.
Cite (Informal):
Redwood: Using Collision Detection to Grow a Large-Scale Intent Classification Dataset (Larson & Leach, SIGDIAL 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.sigdial-1.45.pdf
Video:
 https://youtu.be/E5tCFZ_5r0Y
Code
 gxlarson/redwood
Data
ATISBANKING77CLINC150MTOPTalk2Car