article

Experiments in Mobile Web Survey Design

Authors:

Craig A. HillAuthors Info & Claims

Social Science Computer Review, Volume 28, Issue 3

Pages 319 - 335

https://doi.org/10.1177/0894439309353037

Published: 01 August 2010 Publication History

Abstract

Self-administered surveys can be conducted on mobile web-capable devices, yet these devices have unique features that can affect response processes. Ninety-two adults were randomly selected and provided with mobile devices to complete weekly web surveys. Experiments were designed to address three main objectives. First, the authors test fundamental findings which have been found robust across other modes, but whose impact may be diminished in mobile web surveys (due largely to the device), by manipulating question order and scale frequencies. Second, the authors test findings from experiments in computer-administered web surveys, altering the presentation of images and the number of questions per page. Third, the authors experiment with the unique display, navigation, and input methods, through the need to scroll, the vertical versus horizontal orientation of scales, and the willingness to provide open-ended responses. Although most findings from other modes are upheld, the small screen and keyboard introduce undesirable differences in responses.

References

[1]

Aapor. (2008). Standard definitions: Final dispositions of case codes and outcome rates for surveys. Lenexa, Kansas: AAPOR.

[2]

Baker, R.P., Bradburn, N.M., & Johnson, R.A. (1995). Computer-assisted personal interviewing: An experimental evaluation of data quality and cost. Journal of Official Statistics, 11, 413-431.

[3]

Brick, J.M., Brick, P.D., Dipko, S., Presser, S., Tucker, C., & Yuan, Y. (2007). Cell phone survey feasibility in the U.S.: Sampling and calling cell numbers versus landline numbers. Public Opinion Quarterly, 71, 23-39.

[4]

Cannell, C., Miller, P., & Oksenberg, L. (1981). Research on interviewing techniques. Sociological Methodology, 12, 389-437.

[5]

Chae, M., & Kim, J. (2004). Do size and structure matter to mobile users? An empirical study of the effects of screen size, information structure, and task complexity on user activities with standard Web phones. Behaviour & Information Technology, 23, 165-181.

[6]

Chaiken, S. (1987). The heuristic model of persuasion. In M. P. Zanna, J. M. Olson, & C. P. Herman (Eds.), Social influence: The Ontario symposium (Vol. 5, pp. 3-39). Mahwah, NJ: Lawrence Erlbaum Associates.

[7]

Conrad, F.G., Couper, M.P., Tourangeau, R., & Peytchev, A. (2006). Use and non-use of clarification features in Web surveys . Journal of Official Statistics, 22, 245-269.

[8]

Couper, M.P. (2000). Usability evaluation of computer-assisted survey instruments . Social Science Computer Review, 18, 384-396.

Digital Library

[9]

Couper, M.P., Conrad, F.G., & Tourangeau, R. (2007). Visual context effects in Web surveys. Public Opinion Quarterly, 71, 623-634.

[10]

Couper, M.P., Hansen, S.E., & Sadosky, S.A. (1997). Evaluating interviewer use of CAPI technology. In L. Lyberg, P. Biemer, M. Collins, E. de Leeuw, C. Dippo, N. Schwarz, et al. (Eds.), Survey Measurement and Process Quality (pp. 267-286). New York: John Wiley & Sons, Inc.

[11]

Couper, M.P., Tourangeau, R., Conrad, F., & Crawford, S. (2004). What they see is what we get: Response options for Web surveys. Social Science Computer Review, 22, 111-127.

Digital Library

[12]

Couper, M.P., Tourangeau, R., & Kenyon, K. (2004). Picture this! An analysis of visual effects in Web surveys . Public Opinion Quarterly, 682, 255-266.

[13]

Couper, M.P., Traugott, M.W., & Lamias, M.J. (2001). Web survey design and administration. Public Opinion Quarterly, 65, 230-253.

[14]

DeRouvray, C., & Couper, M.P. (2002). Designing a strategy for reducing ``no opinion¿¿ responses in web-based surveys. Social Science Computer Review, 20, 3-9.

Digital Library

[15]

Dillman, D.A. (2000). Mail and Internet surveys: The tailored design method (2nd ed.). New York: Wiley.

[16]

Fuchs, M. (2001). Screen design in a Web survey. Paper presented at the Paper presented at the annual meeting of the American Association for Public Opinion Research, Montréal, Quebec.

[17]

Hardwick, M.E., Pulido, P.A., & Adelson, W.S. (2007). The use of handheld technology in nursing research and practice. Orthopaedic Nursing, 26, 251-255.

[18]

House, C.C., & & Nicholls II, W.L. (1988). Questionnaire design for CATI: Design objectives and methods. In R. M. Groves, P. P. Biemer, L. E. Lyberg, J. T. Massey, W. L. Nicholls II, et al. (Eds.), Telephone survey methodology (pp. 421-436). New York: John Wiley & Sons, Inc.

[19]

Hyman, H.H., & Sheatsley, P.B. (1950). The current status of American public opinion. In J. C. Payne (Ed.), The teaching of contemporary affairs: Twenty-first yearbook of the national council for the social studies. Washington, D.C: National Council for the Social Studies.

[20]

Jones, M., Marsden, G., Mohd-Nasir, N., Boone, K., & Buchanan, G. (1999). Improving Web interaction on small displays. Computer Networks-the International Journal of Computer and Telecommunications Networking, 31, 1129-1137.

Digital Library

[21]

McCabe, S.E. (2004). Comparison of Web and mail surveys in collecting illicit drug use data: A randomized experiment. Journal of Drug Education, 34, 61-72.

[22]

Mooney, G.M., Rogers, B., & Trunzo, D. (2003). Examining the effect of error prompting on item nonresponse and survey nonresponse in Web surveys. American Association for Public Opinion Research 58th Annual Conference, Nashville, Tennessee.

[23]

Nielsen Mobile. (2008). Critical Mass: The Worldwide State of the Mobile Web. Retrieved July 21, 2008, from http://www.nielsenmobile.com/documents/CriticalMass.pdf

[24]

Okazaki, S. (2007). Assessing mobile-based online surveys: Methodological considerations and pilot study in an advertising context. International Journal of Market Research, 49, 651-674.

[25]

Parush, A., & Yuviler-Gavish, N. (2004). Web navigation structures in cellular phones: The depth/breadth trade-off issue. International Journal of Human-Computer Studies, 60, 753-770.

[26]

Petty, R.E., & Cacioppo, J.T. (1986). Communication and persuasion: Central and peripheral routes to attitude change. New York : Springer-Verlag.

[27]

Peytchev, A., Couper, M.P., McCabe, S.E., & Crawford, S. (2006). Web survey design: Paging vs. scrolling. Public Opinion Quarterly, 70, 596-607.

[28]

Peytchev, A.A. (2007). Participation decisions and measurement error in Web surveys. Unpublished doctoral dissertation, University of Michigan, Ann Arbor.

[29]

Ramos, M., Sedivi, B.M., & Sweet, E.M. (1998). Computerized self-administered questionnaires. In M. P. Couper, R. P. Baker, J. Bethlehem, C. Z. F. Clark, J. Martin, W. L. Nicholls II, et al. (Eds.), Computer assisted survey information collection (pp. 389-408). New York: John Wiley & Sons, Inc.

[30]

Rugg, D., & Cantril, H. (1944). The wording of questions. In H. Cantril (Ed.), Gauging Public Opinion (pp. 23-50). Princeton University Press.

[31]

Schuman, H., Kalton, G., & Ludwig, J. (1983). Context and contiguity in survey questionnaires. Public Opinion Quarterly, 47, 112-115.

[32]

Schuman, H., & Ludwig, J. (1983). The norm of even-handedness in surveys as in Life. American Sociological Review, 48, 112-120.

[33]

Schuman, H., & Presser, S. (1981). Questions and answers in attitude surveys: Experiments on question form, wording, and context. New York: Academic Press.

[34]

Schwarz, N., & Hippler, H.J. (1995). Subsequent questions may influence answers to preceding questions in mail surveys. Public Opinion Quarterly, 59, 93-97.

[35]

Schwarz, N., Hippler, H.J., Deutsch, B., & Strack, F. (1985). Response scales--Effects of category range on reported behavior and comparative judgments. Public Opinion Quarterly, 49, 388-395.

[36]

Strack, F., & Martin, L.L. (1987). Thinking, judging and communicating: A process account of context effects in attitude surveys. In H. -J. Hippler, N. Schwarz & S. Sudman (Eds.). Social information processing and survey methodology. New York: Springer-Verlag.

[37]

Sudman, S., Bradburn, N.M., & Schwarz, N. (1996). Thinking about answers: The application of cognitive processes to survey methodology (1sted.). San Francisco: Jossey-Bass Publishers.

[38]

Sweeney, S., & Crestani, F. (2006). Effective search results summary size and device screen size: Is there a relationship? Information Processing & Management, 42, 1056-1074.

[39]

Thomas, R.K., Uldall, B., & Krosnick, J.A. (2004). How many are too many?: Number of Response Categories and Validity, American Association for Public Opinion Research, Phoenix, AZ.

[40]

Tourangeau, R. (1984). Cognitive science and cognitive methods. In T. Jabine, M. L. Straff, J. M. Tanur, & R. Tourangeau (Eds.), Cognitive aspects of survey methodology: Building a bridge between disciplines (pp. 73-100). Washington, DC: National Academy Press.

[41]

Tourangeau, R., Couper, M.P., & Conrad, F. (2004). Spacing, position, and order: Interpretive heuristics for visual features of survey questions. Public Opinion Quarterly, 68, 368-393.

[42]

Tourangeau, R., & Smith, T.W. (1996). Asking sensitive questions--The impact of data collection mode, question format, and question context. Public Opinion Quarterly, 60, 275-304.

[43]

van Schaik, P., & Ling, J. (2007). Design parameters of rating scales for Web sites. ACM Transactions on Computer-Human Interactions, 14, Article 4.

[44]

Watters, C., Duffy, J., & Duffy, K. (2003). Using large tables on small display devices. International Journal of Human-Computer Studies, 58, 21-37.

Digital Library

[45]

Weber, B.A., Yarandi, H., Rowe, M.A., & Weber, J.P. (2005). A comparison study: Paper-based versus Web-based data collection and management. Applied Nursing Research, 18, 182-185.

[46]

Witte, J.C., Pargas, R.P., Mobley, C., & Hawden, J. (2004). Instrument effects of images in Web surveys: A research note. Social Science Computer Review, 22, 363-369.

Digital Library

Cited By

Aldalur IPerez ALarrinaga FIllarramendi M(2024)A visual programming tool for mobile web augmentationKnowledge and Information Systems10.1007/s10115-023-02039-666:9(5631-5668)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1007/s10115-023-02039-6
Vehovar VCouper MČehovin G(2023)Alternative Layouts for Grid Questions in PC and Mobile Web SurveysSocial Science Computer Review10.1177/0894439322113264441:6(2122-2144)Online publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1177/08944393221132644
Eck ACórdova Cazar ACallegaro MBiemer PSchonlau MGweon HWenemark M(2021)Automatic Classification of Open-Ended QuestionsSocial Science Computer Review10.1177/089443931986921039:4(562-572)Online publication date: 1-Aug-2021
https://dl.acm.org/doi/10.1177/0894439319869210
Show More Cited By

Recommendations

Web Versus Mobile Web

Due to a rising mobile device penetration, Web surveys are increasingly accessed and completed on smartphones or tablets instead of desktop computers or laptops. Mobile Web surveys are also gaining popularity as an alternative self-administered data ...
Use of mobile phones by male and female Greek students

Mobile technology is a continuously growing domain, and research activities regarding its use are quite intensive. A questionnaire regarding the use of mobile devices was developed and distributed to 416 students in a Greek University. There were completed ...
Data Quality in PC and Mobile Web Surveys

The considerable growth in the number of smart mobile devices with a fast Internet connection provides new challenges for survey researchers. In this article, I compare the data quality between two survey modes: self-administered web surveys conducted ...

Comments

Information & Contributors

Information

Published In

Social Science Computer Review Volume 28, Issue 3

August 2010

129 pages

ISSN:0894-4393

Issue’s Table of Contents

Publisher

Sage Publications, Inc.

United States

Publication History

Published: 01 August 2010

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

17
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 29 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Aldalur IPerez ALarrinaga FIllarramendi M(2024)A visual programming tool for mobile web augmentationKnowledge and Information Systems10.1007/s10115-023-02039-666:9(5631-5668)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1007/s10115-023-02039-6
Vehovar VCouper MČehovin G(2023)Alternative Layouts for Grid Questions in PC and Mobile Web SurveysSocial Science Computer Review10.1177/0894439322113264441:6(2122-2144)Online publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1177/08944393221132644
Eck ACórdova Cazar ACallegaro MBiemer PSchonlau MGweon HWenemark M(2021)Automatic Classification of Open-Ended QuestionsSocial Science Computer Review10.1177/089443931986921039:4(562-572)Online publication date: 1-Aug-2021
https://dl.acm.org/doi/10.1177/0894439319869210
Wenz A(2021)Do Distractions During Web Survey Completion Affect Data Quality? Findings From a Laboratory ExperimentSocial Science Computer Review10.1177/089443931985150339:1(148-161)Online publication date: 25-Feb-2021
https://dl.acm.org/doi/10.1177/0894439319851503
Rahman PNandi AFu WPan SBrdiczka OChau PCalvary G(2019)TransformerProceedings of the 24th International Conference on Intelligent User Interfaces10.1145/3301275.3302269(485-496)Online publication date: 17-Mar-2019
https://dl.acm.org/doi/10.1145/3301275.3302269
Antoun CKatz JArgueta JWang L(2018)Design Heuristics for Effective Smartphone QuestionnairesSocial Science Computer Review10.1177/089443931772707236:5(557-574)Online publication date: 1-Oct-2018
https://dl.acm.org/doi/10.1177/0894439317727072
Tourangeau RSun HYan TMaitland ARivero GWilliams D(2018)Web Surveys by Smartphones and TabletsSocial Science Computer Review10.1177/089443931771943836:5(542-556)Online publication date: 1-Oct-2018
https://dl.acm.org/doi/10.1177/0894439317719438
Naylor JSanchez C(2018)Smartphone Display Size Influences Attitudes Toward Information Consumed on Small DevicesSocial Science Computer Review10.1177/089443931770416136:2(251-260)Online publication date: 1-Apr-2018
https://dl.acm.org/doi/10.1177/0894439317704161
Schlosser SMays A(2018)Mobile and DirtySocial Science Computer Review10.1177/089443931769843736:2(212-230)Online publication date: 1-Apr-2018
https://dl.acm.org/doi/10.1177/0894439317698437
Corner MLevine B(2018)MicroMobileProceedings of the 16th Annual International Conference on Mobile Systems, Applications, and Services10.1145/3210240.3210326(310-322)Online publication date: 10-Jun-2018
https://dl.acm.org/doi/10.1145/3210240.3210326
Show More Cited By

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents