short-paper

What Do You Mean Exactly?: Analyzing Clarification Questions in CQA

Authors:

Pavel Braslavski,

Denis Savenkov,

Eugene Agichtein,

Alina DubatovkaAuthors Info & Claims

CHIIR '17: Proceedings of the 2017 Conference on Conference Human Information Interaction and Retrieval

Pages 345 - 348

https://doi.org/10.1145/3020165.3022149

Published: 07 March 2017 Publication History

Get Access

Abstract

Search as a dialogue is an emerging paradigm that is fueled by the proliferation of mobile devices and technological advances, e.g. in speech recognition and natural language processing. Such an interface allows search systems to engage in a dialogue with users aimed at fulfilling their information needs. One key capability required to make such search dialogues effective is asking clarification questions (CLARQ) proactively, when a user's intent is not clear, which could help the system provide more useful responses. With this in mind, we explore the dialogues between the users on a community question answering (CQA) website as a rich repository of information-seeking interactions. In particular, we study the clarification questions asked by CQA users in two different domains, analyze their behavior, and the types of clarification questions asked. Our results suggest that the types of CLARQ are very diverse, while the questions themselves tend to be specific and require both domain- and general knowledge. However, focusing on popular CLARQ types and domains can be fruitful. As a first step towards automatic generation of clarification questions, we explore the problem of predicting the specific subject of a clarification question. Our findings can be useful for future improvements of intelligent dialog search and question answering systems.

References

[1]

J. Allan, B. Croft, A. Moffat, and M. Sanderson. Frontiers, challenges, and opportunities for information retrieval. SIGIR Forum, 46(1):2--32, 2012.

Digital Library

Google Scholar

[2]

A. Anderson et al. Discovering value from community activity on focused question answering sites: a case study of stack overflow. In KDD'2012.

Digital Library

Google Scholar

[3]

R. Gangadharaiah and B. Narayanaswamy. Natural language query refinement for problem resolution from crowdsourced semi-structured data. In IJCNLP'2013.

Google Scholar

[4]

M. A. Hearst. 'Natural' search user interfaces. CACM, 54(11), 2011.

Digital Library

Google Scholar

[5]

M. P. Kato, R. W. White, J. Teevan, and S. T. Dumais. Clarifications and question specificity in synchronous social Q&A. In CHI'2013 Extended Abstracts.

Digital Library

Google Scholar

[6]

A. Kotov and C. Zhai. Towards natural question guided search. In WWW'2010.

Digital Library

Google Scholar

[7]

C. D. Manning et al. The Stanford CoreNLP natural language processing toolkit. In ACL System Demonstrations'2014.

Google Scholar

[8]

S. Quarteroni and S. Manandhar. Designing an interactive open-domain question answering system. Natural Language Engineering, 15(01):73--95, 2009.

Digital Library

Google Scholar

[9]

F. Radlinski and N. Craswell. A theoretical framework for conversational search. In CHIIR'2017.

Digital Library

Google Scholar

[10]

H. Sajjad, P. Pantel, and M. Gamon. Underspecified query refinement via natural language question generation. In COLING'2012.

Google Scholar

[11]

S. Stoyanchev, A. Liu, and J. Hirschberg. Modelling human clarification strategies. In SIGDIAL'2013.

Google Scholar

[12]

Y. Tang, F. Bu, Z. Zheng, and X. Zhu. Towards interactive qa: suggesting refinement for questions. In SIGIR'2011 Workshop on "entertain me": Supporting Complex Search Tasks.

Google Scholar

Cited By

View all

Tavakoli LTrippas JZamani HScholer FSanderson M(2024)Online and Offline Evaluation in Search ClarificationACM Transactions on Information Systems10.1145/368178643:1(1-30)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3681786
Sekulić IAlinannejadi MCrestani F(2024)Analysing Utterances in LLM-Based User Simulation for Conversational SearchACM Transactions on Intelligent Systems and Technology10.1145/365004115:3(1-22)Online publication date: 5-Mar-2024
https://dl.acm.org/doi/10.1145/3650041
Zhao ZDou ZZhou YSerra ESpezzano F(2024)Generating Intent-aware Clarifying Questions in Conversational Information Retrieval SystemsProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679851(3384-3394)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679851
Show More Cited By

Index Terms

What Do You Mean Exactly?: Analyzing Clarification Questions in CQA
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms
      1. Natural language interfaces
2. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Question answering
    2. Users and interactive retrieval

Recommendations

Towards Conversationally Intelligent Dialog Systems
CHI EA '22: Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems

Spoken dialog systems, lacking the means to address the complex phenomena of spontaneous speech and conversational dynamics, force users into a constrained mode of dialog that resembles text-based interaction more closely than spoken conversation. Turn-...
Learning the Structure of Task-Driven Human–Human Dialogs

With the availability of large corpora of spoken dialog, it is now possible to use data-driven techniques to build and use models of task-oriented dialogs. In this paper, we use data-driven techniques to build task structures for individual dialogs, and ...
Conversational Semantic Search: Looking Beyond Web Search, Q&A and Dialog Systems
WSDM '18: Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining

User expectations of web search are changing. They are expecting search engines to answer questions, to be more conversational, and to offer means to complete tasks on their behalf. At the same time, to increase the breadth of tasks that personal digital ...

Comments

Information & Contributors

Information

Published In

CHIIR '17: Proceedings of the 2017 Conference on Conference Human Information Interaction and Retrieval

March 2017

454 pages

ISBN:9781450346771

DOI:10.1145/3020165

Conference Chairs:
Ragnar Nordlie
Oslo & Akershus University College of Applied Sciences, Norway
,
Nils Pharo
Oslo and Akershus University College of Applied Sciences, Norway
,
Program Chairs:
Luanne Freund
University of British Columbia, Canada
,
Birger Larsen
Aalborg University, Denmark
,
Dan Russel
Google, USA

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

In-Cooperation

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 March 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

CHIIR '17

Sponsor:

SIGIR

CHIIR '17: Conference on Human Information Interaction and Retrieval

March 7 - 11, 2017

Oslo, Norway

Acceptance Rates

CHIIR '17 Paper Acceptance Rate 10 of 48 submissions, 21%;

Overall Acceptance Rate 55 of 163 submissions, 34%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

39
Total Citations
View Citations
672
Total Downloads

Downloads (Last 12 months)57
Downloads (Last 6 weeks)3

Reflects downloads up to 05 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Tavakoli LTrippas JZamani HScholer FSanderson M(2024)Online and Offline Evaluation in Search ClarificationACM Transactions on Information Systems10.1145/368178643:1(1-30)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3681786
Sekulić IAlinannejadi MCrestani F(2024)Analysing Utterances in LLM-Based User Simulation for Conversational SearchACM Transactions on Intelligent Systems and Technology10.1145/365004115:3(1-22)Online publication date: 5-Mar-2024
https://dl.acm.org/doi/10.1145/3650041
Zhao ZDou ZZhou YSerra ESpezzano F(2024)Generating Intent-aware Clarifying Questions in Conversational Information Retrieval SystemsProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679851(3384-3394)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679851
Samarinas CZamani HHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)ProCIS: A Benchmark for Proactive Retrieval in ConversationsProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657869(830-840)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657869
Yuan YSiro CAliannejadi MRijke MLam WChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Asking Multimodal Clarifying Questions in Mixed-Initiative Conversational SearchProceedings of the ACM Web Conference 202410.1145/3589334.3645483(1474-1485)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645483
Hu JYang B(2024)Posts Quality Prediction for StackOverflow WebsiteIEEE Access10.1109/ACCESS.2024.344087912(135601-135615)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3440879
Muroň MDařena FProcházka DKern R(2024)User-Driven Toponym Disambiguation Using DialogueJournal of Map & Geography Libraries10.1080/15420353.2024.236321219:3(198-219)Online publication date: 17-Jun-2024
https://doi.org/10.1080/15420353.2024.2363212
Bedi NSekulić ICrestani F(2024)Clarifying Questions Generation for Conversational Search Based on “People Also Ask” FeatureSustainability and Empowerment in the Context of Digital Libraries10.1007/978-981-96-0865-2_20(246-260)Online publication date: 6-Dec-2024
https://doi.org/10.1007/978-981-96-0865-2_20
Mansouri BJahedibashiz ZYoshioka MKiseleva JAliannejadi M(2023)Clarifying Questions in Math Information RetrievalProceedings of the 2023 ACM SIGIR International Conference on Theory of Information Retrieval10.1145/3578337.3605123(149-158)Online publication date: 9-Aug-2023
https://dl.acm.org/doi/10.1145/3578337.3605123
Liu CZhou SLiu DLi JHuang ZMa X(2023)CoArgue : Fostering Lurkers’ Contribution to Collective Arguments in Community-based QA PlatformsProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580932(1-17)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580932
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Towards Conversationally Intelligent Dialog Systems

Learning the Structure of Task-Driven Human–Human Dialogs

Conversational Semantic Search: Looking Beyond Web Search, Q&A and Dialog Systems