article

Designing an interactive open-domain question answering system

Authors:

S. ManandharAuthors Info & Claims

Natural Language Engineering, Volume 15, Issue 1

Pages 73 - 95

https://doi.org/10.1017/S1351324908004919

Published: 01 January 2009 Publication History

Abstract

Interactive question answering (QA), where a dialogue interface enables follow-up and clarification questions, is a recent although long-advocated field of research. We report on the design and implementation of YourQA, our open-domain, interactive QA system. YourQA relies on a Web search engine to obtain answers to both fact-based and complex questions, such as descriptions and definitions. We describe the dialogue moves and management model making YourQA interactive, and discuss the architecture, implementation and evaluation of its chat-based dialogue interface. Our Wizard-of-Oz study and final evaluation results show how the designed architecture can effectively achieve open-domain, interactive QA.

References

[1]

Alexandersson, J., Reithinger, N., and Maier, E. 1997. Insights into the dialogue processing of VERBMOBIL. Technical Report 191, DFKI GmbH, Saarbrücken, Germany.

[2]

Allen, J., Ferguson, G., Ringger, E., Zollo, T. S., and Miller, B. 2000. Dialogue systems: from theory to practice in TRAINS-96. In R. Dale, H. Moisl, and H. Somers (eds.), Handbook on Natural Language Processing, Chapter 14, pp. 347-376.

[3]

Austin, J. L. 1962. How to do Things with Words. Oxford, UK: Oxford University Press.

[4]

Bertomeu, N., Uszkoreit, H., Frank, A., Krieger, H., and Joerg, B. 2006. Contextual phenomena and thematic relations in database QA dialogues: results from a Wizard-of-Oz experiment. In Proceedings of the Interactive Question Answering Workshop at HLT-NAACL 2006 (IQA'06), New York, USA.

[5]

Bos, J., Klein, E., Lemon, O., and Oka, T. 2003. Dipper: description and formalisation of an information-state update dialogue system architecture. In 4th SIGdial Workshop on Discourse and Dialogue, Sapporo, Japan.

[6]

Cahn, J. E., and Brennan, S. E. 1999. A psychological model of grounding and repair in dialog. In Working Notes: AAAI Fall Symposium on Psychological Models of Communication in Collaborative Systems, Seacliff, MI, pp. 25-33. Menlo Park, CA: AAAI press.

[7]

Churcher, G. E., Atwell, E. S., and Souter, C. 1997. Dialogue management systems: a survey and overview. Technical Report 97.06, School of Computer Studies, University of Leeds.

[8]

Cohen, P. 1996. Dialogue modeling. In Survey of the State of the Art in Human Language Technology, Chapter 6.3, pp. 192-197. Cambridge, UK: Cambridge University Press.

[9]

Core, M. G., and Allen, J. F. 1997. Coding dialogs with the DAMSL annotation scheme. In Working Notes: AAAI Fall Symposium on Communicative Action in Humans and Machines, Boston, MA, pp. 28-35. Menlo Park, CA: AAAI press.

[10]

Dahlbaeck, N., Jonsson, A., and Ahrenberg, L. 1993. Wizard of Oz studies: why and how. In Workshop on Intelligent User Interfaces, Orlando, FL, pp. 193-200, ACM Press.

[11]

De Boni, M., and Manandhar, S. 2005. Implementing clarification dialogue in open-domain question answering. Journal of Natural Language Engineering 11(4): 343-361.

Digital Library

[12]

Galibert, O., Illouz, G., and Rousset, S. 2005. Ritel: an open-domain, human-computer dialogue system. In INTERSPEECH'05.

[13]

Grosz, B. J., and Sidner, C. L. 1986. Attention, intentions, and the structure of discourse. Journal of Computational Linguistics 12(3): 175-204.

Digital Library

[14]

Hearst, M., Allen, J., Horvitz, E., and Guinn, C. 1999. Mixed-initiative interaction. IEEE Intelligent Systems 14(5): 14-23.

Digital Library

[15]

Hobbs, J. R. 2002. From question-answering to information-seeking dialogs. Project presentation. Available at http://www.ai.sri.com/aquaint/.

[16]

Jiang, J. J., and Conrath, D. W. 1997. Semantic similarity based on corpus statistics and lexical taxonomy. In Proceedings of ROCLING, Taiwan.

[17]

Jönsson, A. 1993. A method for development of dialogue managers for natural language interfaces. In Proceedings of AAAI'93, Washington, DC.

[18]

Jönsson, A., and Merkel, M. 2003. Some issues in dialogue-based question-answering. In Working Notes from AAAI Spring Symposium on New Directions in Question Answering, Stanford, pp. 45-48.

[19]

Kato, T., Fukumoto, J., Masui, F., and Kando, N. 2006. Woz simulation of interactive question answering. In Proceedings of the Interactive Question Answering Workshop at HLT-NAACL 2006 (IQA'06), New York, USA.

[20]

Kelly, D., Kantor, P., Morse, E., Scholtz, J., and Sun, Y. 2006. User-centered evaluation of interactive question answering systems. In Proceedings of the Interactive Question Answering Workshop at HLT-NAACL 2006 (IQA'06), New York, USA.

[21]

Kitano, H., and Van Ess-Dykema, C. 1991. Toward a plan-based understanding model for mixed-initiative dialogues. In Proceedings of ACL'91, East Stroudsburgh, MA, pp. 25-32.

[22]

Kwok, C. T., Etzioni, O., and Weld, D. S. 2001. Scaling question answering to the web. In Proceedings of the 10th International Conference on WWW, pp. 150-161.

[23]

Larsson, S. 1998. Using a type hierarchy to characterize reliability of coding schemas for dialogue moves. Gothenburg Papers in Computational Linguistics. New York: ACM.

[24]

Larsson, S., and Traum, D. R. 2000. Information state and dialogue management in the TRINDI dialogue move engine toolkit. Natural Language Engineering 6(3-4): 323-340.

Digital Library

[25]

Li, X., and Roth, D. 2005. Learning question classifiers: the role of semantic information. Journal of Natural Language Engineering 12(3): 229-249.

Digital Library

[26]

Moschitti, A., Quarteroni, S., Basili, R., and Manandhar, S. 2007. Exploiting syntactic and shallow semantic kernels for question/answer classification. In Proceedings of ACL'07, Prague, Czech Republic.

[27]

Munteanu, C., and Boldea, M. 2000. MDWOZ: a wizard of Oz environment for dialog systems development. In Proceedings of LREC'00, Athens, Greece.

[28]

Quarteroni, S., and Manandhar, S. 2006. User modelling for adaptive question answering and information retrieval. In Proceedings of FLAIRS'06, Melbourne Beach, Florida.

[29]

Quarteroni, S., and Manandhar, S. 2007. User modelling for personalized question answering. In Proceedings of AI*IA'07, Frascati, Italy.

[30]

Schegloff, E., and Sacks, H. 1973. Opening up closing. Semiotica 7: 289-327.

[31]

Sinclair, J. M., and Coulthard, R. M. 1975. Towards an Analysis of Discourse: The English Used by Teachers and Pupils. Oxford: Oxford University Press.

[32]

Steinberger, J., Kabadjov, M. A., Poesio, M., and Sanchez-Graillet, O. 2005. Improving LSA-based summarization with anaphora resolution. In Proceedings of HLT '05, East Stroudsburgh, MA, pp. 1-8.

[33]

Sutton, S. 1998. Universal speech tools: the cslu toolkit. In Proceedings of ICSLP'98, Sydney, Australia.

[34]

Teevan, J., Dumais, S. T., and Horvitz, E. 2005. Personalizing search via automated analysis of interests and activities. In Proceedings of SIGIR '05, pp. 449-456, NY: ACM Press.

[35]

Traum, D. 1996. Dialogue management in conversational agency: the TRAINS-93 dialogue manager. In Proceedings of the Twente Workshop on Langauge Technology: Dialogue Management in Natural Language Systems (TWLT 11), Netherlands, pp. 1-11.

[36]

Voorhees, E. M. 2003. Overview of the TREC 2003 question answering track. In Proceedings of TREC'03, Gaithersburg, MD.

[37]

Walker, M. A., Kamm, C., and Litman, D. 2000. Towards developing general models of usability with PARADISE. Journal of Natural Language Engineering, 6(3-4): 363-377 (Special Issue on Best Practice in Spoken Dialogue Systems).

Digital Library

[38]

Webb, N., and Strzalkowski, T. (eds.) 2006. Proceedings of the Interactive Question Answering Workshop at HLT-NAACL 2006 (IQA'06), New York, USA.

[39]

Xu, W., Xu, B., Huang, T., and Xia, H. 2002. Bridging the gap between dialogue management and dialogue models. In Proceedings of SIGDIAL 3, Philadelphia, PA, pp. 201-210.

[40]

Yang, F., Feng, Z., and Di Fabbrizio, G. 2006. A data driven approach to relevancy recognition for contextual question answering. In Proceedings of the Interactive Question Answering Workshop at HLT-NAACL 2006 (IQA'06), New York, USA.

[41]

Zhang, D., and Lee, W. 2003. Question classification using support vector machines. In Proceedings of SIGIR'03, Toronto, Canada. New York: ACM.

Cited By

Hernandez-Bocanegra DZiegler J(2023)Explaining Recommendations through Conversations: Dialog Model and the Effects of Interface Type and Degree of InteractivityACM Transactions on Interactive Intelligent Systems10.1145/357954113:2(1-47)Online publication date: 12-Apr-2023
https://dl.acm.org/doi/10.1145/3579541
Luo RHuang LKe N(2023)A Review of Opportunities and Challenges of Artificial Intelligence Chatbots in Second Language AcquisitionEmerging Technologies for Education10.1007/978-981-97-4246-2_6(70-77)Online publication date: 26-Nov-2023
https://dl.acm.org/doi/10.1007/978-981-97-4246-2_6
Feine JMorana SMaedche A(2022)Intrance: Designing an Interactive Enhancement System for the Development of QA ChatbotsProceedings of the ACM on Human-Computer Interaction10.1145/35551996:CSCW2(1-24)Online publication date: 11-Nov-2022
https://dl.acm.org/doi/10.1145/3555199
Show More Cited By

Designing an interactive open-domain question answering system

Recommendations

Implementing clarification dialogues in open domain question answering

We examine the implementation of clarification dialogues, a mechanism for ensuring that question answering systems take into account user goals by allowing them to ask series of related questions either by refining or expanding on previous questions ...
Dynamic Graph Reasoning for Conversational Open-Domain Question Answering
In recent years, conversational agents have provided a natural and convenient access to useful information in people’s daily life, along with a broad and new research topic, conversational question answering (QA). On the shoulders of conversational QA, we ...
Interactive Multi-modal Question-Answering

Comments

Information & Contributors

Information

Published In

cover image Natural Language Engineering

Natural Language Engineering Volume 15, Issue 1

January 2009

170 pages

ISSN:1351-3249

Issue’s Table of Contents

Publisher

Cambridge University Press

United States

Publication History

Published: 01 January 2009

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

20
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 20 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Hernandez-Bocanegra DZiegler J(2023)Explaining Recommendations through Conversations: Dialog Model and the Effects of Interface Type and Degree of InteractivityACM Transactions on Interactive Intelligent Systems10.1145/357954113:2(1-47)Online publication date: 12-Apr-2023
https://dl.acm.org/doi/10.1145/3579541
Luo RHuang LKe N(2023)A Review of Opportunities and Challenges of Artificial Intelligence Chatbots in Second Language AcquisitionEmerging Technologies for Education10.1007/978-981-97-4246-2_6(70-77)Online publication date: 26-Nov-2023
https://dl.acm.org/doi/10.1007/978-981-97-4246-2_6
Feine JMorana SMaedche A(2022)Intrance: Designing an Interactive Enhancement System for the Development of QA ChatbotsProceedings of the ACM on Human-Computer Interaction10.1145/35551996:CSCW2(1-24)Online publication date: 11-Nov-2022
https://dl.acm.org/doi/10.1145/3555199
Liu CZhou SZhang YLiu DPeng ZMa X(2022)Exploring the Effects of Self-Mockery to Improve Task-Oriented Chatbot’s Social IntelligenceProceedings of the 2022 ACM Designing Interactive Systems Conference10.1145/3532106.3533461(1315-1329)Online publication date: 13-Jun-2022
https://dl.acm.org/doi/10.1145/3532106.3533461
Pisařovic IDařena FProcházka DJaniš V(2022)Preprocessing of normative documents for interactive question answeringExpert Systems with Applications: An International Journal10.1016/j.eswa.2021.116314191:COnline publication date: 6-May-2022
https://dl.acm.org/doi/10.1016/j.eswa.2021.116314
Hernandez-Bocanegra DZiegler J(2021)Conversational review-based explanations for recommender systems: Exploring users’ query behaviorProceedings of the 3rd Conference on Conversational User Interfaces10.1145/3469595.3469596(1-11)Online publication date: 27-Jul-2021
https://dl.acm.org/doi/10.1145/3469595.3469596
Deng YMurari S(2021)When a Voice Assistant Asks for Feedback: An Empirical Study on Customer Experience with A/B Testing and Causal Inference MethodsCompanion Publication of the 2021 International Conference on Multimodal Interaction10.1145/3461615.3485403(183-191)Online publication date: 18-Oct-2021
https://dl.acm.org/doi/10.1145/3461615.3485403
Syed ZTrabelsi AHelbert EBailleau VMuths C(2021)Question Answering Chatbot for Troubleshooting Queries based on Transfer LearningProcedia Computer Science10.1016/j.procs.2021.08.097192:C(941-950)Online publication date: 1-Jan-2021
https://dl.acm.org/doi/10.1016/j.procs.2021.08.097
Borsci SMalizia ASchmettow Mvan der Velde FTariverdiyeva GBalaji DChamberlain A(2021)The Chatbot Usability Scale: the Design and Pilot of a Usability Scale for Interaction with AI-Based Conversational AgentsPersonal and Ubiquitous Computing10.1007/s00779-021-01582-926:1(95-119)Online publication date: 21-Jul-2021
https://dl.acm.org/doi/10.1007/s00779-021-01582-9
Hahn NChang JKittur AMandryk RHancock MPerry MCox A(2018)Bento BrowserProceedings of the 2018 CHI Conference on Human Factors in Computing Systems10.1145/3173574.3173825(1-12)Online publication date: 21-Apr-2018
https://dl.acm.org/doi/10.1145/3173574.3173825
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents