short-paper

Experiments in the wild: public evaluation of off-screen visualizations in the Android market

Authors:

Niels Henze,

Benjamin Poppinga,

Susanne BollAuthors Info & Claims

NordiCHI '10: Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries

Pages 675 - 678

https://doi.org/10.1145/1868914.1869002

Published: 16 October 2010 Publication History

Get Access

Abstract

Since the introduction of application stores for mobile devices there has been an increasing interest to use this distribution platform to collect user feedback. Mobile application stores can make research prototypes widely available and enable to conduct user studies "in the wild" with participants from all over the world. Previous work published research prototypes to collect qualitative feedback or to collect quantitative attributes of specific prototypes. In this paper we explore how to conduct a study that focuses on a specific task and tries to isolate cause and effect much like controlled experiments in the lab. We compare three visualization techniques for off-screen objects by publishing a game in the Android Market. e.g. we show that the performance of the visualization techniques depends on the number of objects. Using a more realistic task and feedback from a hundred times more participants than previous studies lead to much higher external validity. We conclude that public experiments are a viable tool to complement or replace lab studies.

References

[1]

P. Baudisch and R. Rosenholtz. Halo: a technique for visualizing off-screen objects. Proc. of CHI, 2003.

Digital Library

Google Scholar

[2]

S. Burigat, L. Chittaro, and S. Gabrielli. Visualizing locations of off-screen objects on mobile devices: a comparative evaluation of three approaches. Proc. of MobileHCI, 2006.

Digital Library

Google Scholar

[3]

S. Gustafson, P. Baudisch, C. Gutwin, and P. Irani. Wedge: clutter-free visualization of off-screen locations. Proc. of CHI, 2008.

Digital Library

Google Scholar

[4]

N. Henze and S. Boll. Push the study to the app store: Evaluating off-screen visualizations for maps in the android market. Proc. of MobileHCI, 2010.

Digital Library

Google Scholar

[5]

J. Kjeldskov and C. Graham. A review of mobile hci research methods. Proc. of Mobile HCI, 2003.

Crossref

Google Scholar

[6]

R. Kohavi, R. Henne, and D. Sommerfield. Practical guide to controlled experiments on the web: listen to your customers not to the hippo. Proc. of SIGKDD, 2007.

Digital Library

Google Scholar

[7]

J. Looser, A. Cockburn, J. Savage, and N. Christchurch. On the Validity of Using First-Person Shooters for Fitts' Law Studies. Proc. of British HCI, 2005.

Google Scholar

[8]

D. McMillan, A. Morrison, O. Brown, M. Hall, and M. Chalmers. Further into the Wild: Running Worldwide Trials of Mobile Systems. Proc. of Pervasive, 2010.

Digital Library

Google Scholar

[9]

C. M. Nielsen, M. Overgaard, M. B. Pedersen, J. Stage, and S. Stenild. It's worth the hassle!: the added value of evaluating the usability of mobile systems in the field. Proc. of NordiCHI, 2006.

Digital Library

Google Scholar

[10]

M. Pielot, B. Poppinga, and S. Boll. PocketNavigator: Vibro-Tactile Waypoint Navigation for Everyday Mobile Devices. Proc. of MobileHCI, 2010.

Digital Library

Google Scholar

[11]

P. T. Zellweger, J. D. Mackinlay, L. Good, M. Stefik, and P. Baudisch. City lights: contextual views in minimal space. Proc. of CHI, 2003.

Digital Library

Google Scholar

[12]

S. Zhai, P. Kristensson, P. Gong, M. Greiner, S. Peng, L. Liu, and A. Dunnigan. Shapewriter on the iPhone: from the laboratory to the real world. Proc. of CHI, 2009.

Digital Library

Google Scholar

Cited By

View all

Woodworth JKhokhar ARahman YKulshreshth ABorst C(2024)Attention Guidance In The Wild: An Experiment Testing Visual Guidance Cues for VR Field Trips at High Schools2024 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)10.1109/VRW62533.2024.00035(173-179)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VRW62533.2024.00035
Auda JGruenefeld USchneegass S(2022)If The Map Fits! Exploring Minimaps as Distractors from Non-Euclidean Spaces in Virtual RealityExtended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491101.3519621(1-6)Online publication date: 27-Apr-2022
https://dl.acm.org/doi/10.1145/3491101.3519621
Trepkowski CMarquardt AEibich TShikanai YMaiero JKiyokawa KKruijff ESchoning JKonig P(2022)Multisensory Proximity and Transition Cues for Improving Target Awareness in Narrow Field of View Augmented Reality DisplaysIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2021.311667328:2(1342-1362)Online publication date: 1-Feb-2022
https://doi.org/10.1109/TVCG.2021.3116673
Show More Cited By

Index Terms

Experiments in the wild: public evaluation of off-screen visualizations in the Android market
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms

Recommendations

Push the study to the App store: evaluating off-screen visualizations for maps in the android market
MobileHCI '10: Proceedings of the 12th international conference on Human computer interaction with mobile devices and services

The introduction of publicly available application stores for mobile devices enables to publish research prototypes to a wide audience. This distribution channel can be used to conduct studies with participants from all over the world and diverse ...
My App is an Experiment: Experience from User Studies in Mobile App Stores

Experiments are a cornerstone of HCI research. Mobile distribution channels such as Apple's App Store and Google's Android Market have created the opportunity to bring experiments to the end user. Hardly any experience exists on how to conduct such ...
Beyond Google Play: A Large-Scale Comparative Study of Chinese Android App Markets
IMC '18: Proceedings of the Internet Measurement Conference 2018

China is one of the largest Android markets in the world. As Chinese users cannot access Google Play to buy and install Android apps, a number of independent app stores have emerged and compete in the Chinese app market. Some of the Chinese app stores ...

Comments

Information & Contributors

Information

Published In

NordiCHI '10: Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries

October 2010

889 pages

ISBN:9781605589343

DOI:10.1145/1868914

General Chairs:
Ebba Þóra Hvannberg
University of Iceland, Iceland
,
Marta Kristín Lárusdóttir
Reykjavik University, Iceland
,
Program Chairs:
Ann Blandford
University College London
,
Jan Gulliksen
Royal Institute of Technology, Sweden

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 October 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

Conference

NordiCHI '10

Sponsor:

NordiCHI '10: 6th Nordic Conference on Human-Computer Interaction

October 16 - 20, 2010

Reykjavik, Iceland

Acceptance Rates

Overall Acceptance Rate 379 of 1,572 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

27
Total Citations
View Citations
318
Total Downloads

Downloads (Last 12 months)13
Downloads (Last 6 weeks)1

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Woodworth JKhokhar ARahman YKulshreshth ABorst C(2024)Attention Guidance In The Wild: An Experiment Testing Visual Guidance Cues for VR Field Trips at High Schools2024 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)10.1109/VRW62533.2024.00035(173-179)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VRW62533.2024.00035
Auda JGruenefeld USchneegass S(2022)If The Map Fits! Exploring Minimaps as Distractors from Non-Euclidean Spaces in Virtual RealityExtended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491101.3519621(1-6)Online publication date: 27-Apr-2022
https://dl.acm.org/doi/10.1145/3491101.3519621
Trepkowski CMarquardt AEibich TShikanai YMaiero JKiyokawa KKruijff ESchoning JKonig P(2022)Multisensory Proximity and Transition Cues for Improving Target Awareness in Narrow Field of View Augmented Reality DisplaysIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2021.311667328:2(1342-1362)Online publication date: 1-Feb-2022
https://doi.org/10.1109/TVCG.2021.3116673
Wieland JGarcia RReiterer HFeuchtner T(2022)Arrow, Bézier Curve, or Halos? – Comparing 3D Out-of-View Object Visualization Techniques for Handheld Augmented Reality2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR55827.2022.00098(797-806)Online publication date: Oct-2022
https://doi.org/10.1109/ISMAR55827.2022.00098
Binetti NWu LChen SKruijff EJulier SBrumby D(2021)Using visual and auditory cues to locate out-of-view objects in head-mounted augmented realityDisplays10.1016/j.displa.2021.10203269(102032)Online publication date: Sep-2021
https://doi.org/10.1016/j.displa.2021.102032
Evangelista AManghisi VLaera FGattullo MUva AFiorentino M(2021)CompassbAR: A Technique for Visualizing Out-of-View Objects in a Mixed Reality EnvironmentDesign Tools and Methods in Industrial Engineering II10.1007/978-3-030-91234-5_14(141-148)Online publication date: 1-Dec-2021
https://doi.org/10.1007/978-3-030-91234-5_14
Costa LAliannejadi MCrestani FO'Brien HFreund LArapakis IHoeber OLopatovska I(2020)A Tool for Conducting User Studies on Mobile DevicesProceedings of the 2020 Conference on Human Information Interaction and Retrieval10.1145/3343413.3377985(462-466)Online publication date: 14-Mar-2020
https://dl.acm.org/doi/10.1145/3343413.3377985
Lange DStratmann TGruenefeld UBoll SBernhaupt RMueller FVerweij DAndres JMcGrenere JCockburn AAvellino IGoguey ABjørn PZhao SSamson BKocielnik R(2020)HiveFive: Immersion Preserving Attention Guidance in Virtual RealityProceedings of the 2020 CHI Conference on Human Factors in Computing Systems10.1145/3313831.3376803(1-13)Online publication date: 21-Apr-2020
https://dl.acm.org/doi/10.1145/3313831.3376803
Wimmer CSchlögl RKappel KGrechenig TPaternò FJacucci GRohs MSantoro C(2019)Measuring mobile text entry performance and behaviour in the wild with a serious gameProceedings of the 18th International Conference on Mobile and Ubiquitous Multimedia10.1145/3365610.3365633(1-11)Online publication date: 26-Nov-2019
https://dl.acm.org/doi/10.1145/3365610.3365633
Gruenefeld UPrädel LHeuten WPaternò FJacucci GRohs MSantoro C(2019)Locating nearby physical objects in augmented realityProceedings of the 18th International Conference on Mobile and Ubiquitous Multimedia10.1145/3365610.3365620(1-10)Online publication date: 26-Nov-2019
https://dl.acm.org/doi/10.1145/3365610.3365620
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Push the study to the App store: evaluating off-screen visualizations for maps in the android market

My App is an Experiment: Experience from User Studies in Mobile App Stores

Beyond Google Play: A Large-Scale Comparative Study of Chinese Android App Markets

Comments

Information

Published In

Sponsors

In-Cooperation

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Funding Sources

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations