Article

User walkthrough of multimodal access to multidimensional databases

Authors:

M. P. van Esch-Bussemakers,

A. H. M. CremersAuthors Info & Claims

ICMI '04: Proceedings of the 6th international conference on Multimodal interfaces

Pages 220 - 226

https://doi.org/10.1145/1027933.1027970

Published: 13 October 2004 Publication History

Get Access

Abstract

This paper describes a user walkthrough that was conducted with an experimental multimodal dialogue system to access a multidimensional music database using a simulated mobile device (including a technically challenging four-PHANToM-setup). The main objectives of the user walkthrough were to assess user preferences for certain modalities (speech, graphical and haptic-tactile) to access and present certain types of information, and for certain search strategies when searching and browsing a multidimensional database. In addition, the project aimed at providing concrete recommendations for the experimental setup, multimodal user interface design and evaluation. The results show that recommendations can be formulated both on the use of modalities and search strategies, and on the experimental setup as a whole, including the user interface. In short, it is found that haptically enhanced buttons are preferred for navigating or selecting and speech is preferred for searching the database for an album or artist. A 'direct' search strategy indicating an album, artist or genre is favorable. It can be concluded that participants were able to look beyond the experimental setup and see the potential of the envisioned mobile device and its modalities. Therefore it was possible to formulate recommendations for future multimodal dialogue systems for multidimensional database access.

References

[1]

Akamatsu, M,. & MacKenzie, I.S. (1996). Movement characteristics using a mouse with tactile and force feedback. International Journal of Human-Computer Studies (45), 483--493.

Digital Library

Google Scholar

[2]

Bernsen, N.O. (2001). Multimodality in language and speech systems-from theory to design support tool. Chapter in Granström, B. (Ed.): Multimodality in Language and Speech Systems. Dordrecht: Kluwer Academic Publishers 2001.

Google Scholar

[3]

Brewster, S., Wright, P. & Edwards, A. (1995). Experimentally derived guidelines for the creation of earcons. In Adjunct Proceedings of BCS HCI '95, Huddersfield, UK, 115--159.

Google Scholar

[4]

Cremers, A.H.M, Esch, M.P. van (2002) Multidimensional Information Access using Multiple Modalities (MIAMM). WP 1: Human factors analysis and validation; D1.1: Task analysis and validation. (Report TM - 02 - D014). Soesterberg, The Netherlands: TNO Human Factors.

Google Scholar

[5]

Landragin, F., Bellalem, N, & Romary, L. (2002). Referring to objects with spoken and haptic modalities. In proceedings of the IEEE fourth ICMI 2002, Pittsburg, USA.

Digital Library

Google Scholar

[6]

MacLean, K. (2000). Designing with haptic feedback. Proceedings of the IEEE Robotics and Automation, San Francisco, CA.

Crossref

Google Scholar

[7]

Martin, J., Julia, L., & Cheyer, A. (1998). A theoretical framework for multimodal user studies. Proceedings of the CMC98, 104--110.

Google Scholar

[8]

Michelitsch, G, Veen, A.H.C. van, & Erp B.F. van (2002) Multi-finger haptic interaction within the MIAMM-project. In proceedings of the IEEE fourth EuroHaptics conference 2002, Pittsburg.

Google Scholar

[9]

Oviatt, S. (2002). Multimodal interfaces. In: J. Jacko & A. Sears (Eds.) Handbook of Human Computer Interaction, Lawrence Erlbaum: New Jersey.

Google Scholar

[10]

Reithinger, N., Lauer C., & Romary, L. (2002). MIAMM: Multimodal Information Access using Multiple Modalities. Presentation at the CLASS workshop, Copenhagen, 2002.

Google Scholar

[11]

Shneiderman, B. (1996). The eyes have it: A task by datatype taxonomy for information visualizations. Technical Report ISR-TR-96-66, HCI laboratory, University of Maryland.

Crossref

Google Scholar

[12]

Van Esch-Bussemakers, M., Cremers, A., Neerincx, M., & Van der Flier, A. (2003). Deriving the optimal modality combination for mobile search in multimedia databases. In Proceedings of IADIS/WWW/Internet 2003, Algarve, Portugal.

Google Scholar

Cited By

View all

Palmerius K(2011)Combining vibrotactile and kinaesthetic cues in haptic volume visualization2011 IEEE World Haptics Conference10.1109/WHC.2011.5945538(511-515)Online publication date: Jun-2011
https://doi.org/10.1109/WHC.2011.5945538
Paneels SRoberts J(2010)Review of Designs for Haptic Data VisualizationIEEE Transactions on Haptics10.1109/TOH.2009.443:2(119-137)Online publication date: 1-Apr-2010
https://dl.acm.org/doi/10.1109/TOH.2009.44
Reithinger NFedeler DKumar ALauer CPecourt ERomary L(2005)Miamm — A Multimodal Dialogue System Using HapticsAdvances in Natural Multimodal Dialogue Systems10.1007/1-4020-3933-6_14(307-332)Online publication date: 2005
https://doi.org/10.1007/1-4020-3933-6_14

Index Terms

User walkthrough of multimodal access to multidimensional databases
1. Hardware
  1. Communication hardware, interfaces and storage
    1. Sound-based input / output
2. Human-centered computing
  1. Human computer interaction (HCI)
  2. Interaction design
    1. Interaction design process and methods
      1. User centered design

Recommendations

A toolkit for multimodal interface design: an empirical investigation
HCI'07: Proceedings of the 12th international conference on Human-computer interaction: intelligent multimodal interaction environments

This paper introduces a comparative multi-group study carried out to investigate the use of multimodal interaction metaphors (visual, oral, and aural) for improving learnability (or usability from first time use) of interface-design environments. An ...
Building multimodal applications with EMMA
ICMI-MLMI '09: Proceedings of the 2009 international conference on Multimodal interfaces

Multimodal interfaces combining natural modalities such as speech and touch with dynamic graphical user interfaces can make it easier and more effective for users to interact with applications and services on mobile devices. However, building these ...
Spoken Dialogue Interfaces: Integrating Usability
USAB '09: Proceedings of the 5th Symposium of the Workgroup Human-Computer Interaction and Usability Engineering of the Austrian Computer Society on HCI and Usability for e-Inclusion

Usability is a fundamental requirement for natural language interfaces. Usability evaluation reflects the impact of the interface and the acceptance from the users. This work examines the potential of usability evaluation in terms of issues and ...

Comments

Information & Contributors

Information

Published In

ICMI '04: Proceedings of the 6th international conference on Multimodal interfaces

October 2004

368 pages

ISBN:1581139950

DOI:10.1145/1027933

General Chairs:
Rajeev Sharma
Advanced Interfaces
,
Trevor Darrell
Massachusetts Institute of Technology
,
Program Chairs:
Mary Harper
Purdue University, West Lafayette, IN
,
Gianni Lazzari
ITC-IRST
,
Matthew Turk
University of California, Santa Barbara, CA

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 October 2004

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

ICMI04

Sponsor:

ICMI04: Sixth International Conference on Multimodal Interfaces 2004

October 13 - 15, 2004

PA, State College, USA

Acceptance Rates

Overall Acceptance Rate 453 of 1,080 submissions, 42%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
323
Total Downloads

Downloads (Last 12 months)1
Downloads (Last 6 weeks)0

Reflects downloads up to 14 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Palmerius K(2011)Combining vibrotactile and kinaesthetic cues in haptic volume visualization2011 IEEE World Haptics Conference10.1109/WHC.2011.5945538(511-515)Online publication date: Jun-2011
https://doi.org/10.1109/WHC.2011.5945538
Paneels SRoberts J(2010)Review of Designs for Haptic Data VisualizationIEEE Transactions on Haptics10.1109/TOH.2009.443:2(119-137)Online publication date: 1-Apr-2010
https://dl.acm.org/doi/10.1109/TOH.2009.44
Reithinger NFedeler DKumar ALauer CPecourt ERomary L(2005)Miamm — A Multimodal Dialogue System Using HapticsAdvances in Natural Multimodal Dialogue Systems10.1007/1-4020-3933-6_14(307-332)Online publication date: 2005
https://doi.org/10.1007/1-4020-3933-6_14

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

A toolkit for multimodal interface design: an empirical investigation

Building multimodal applications with EMMA

Spoken Dialogue Interfaces: Integrating Usability

Comments

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Other Metrics

Article Metrics

Other Metrics

Cited By

Login options

Full Access

PDF

eReader

Abstract

References

Cited By

Index Terms

Recommendations

A toolkit for multimodal interface design: an empirical investigation

Building multimodal applications with EMMA

Spoken Dialogue Interfaces: Integrating Usability

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Get Access

Login options

Full Access

View options

PDF

eReader

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations