Article

iTest: testing software with mobile crowdsourcing

Authors:

CrowdSoft 2014: Proceedings of the 1st International Workshop on Crowd-based Software Development Methods and Technologies

Pages 19 - 24

https://doi.org/10.1145/2666539.2666569

Published: 17 November 2014 Publication History

Get Access

Abstract

In recent years, a lot of crowdsourcing systems have emerged and lead to many successful crowdsourcing systems like Wiki-pedia, Amazon Mechanical Turk and Waze. In the field of software engineering, crowdtesting has acquired increased interest and adoption, especially among personal developers and smaller companies. In this paper, we present iTest which combines mobile crowdsourcing and software testing together to support the testing of mobile application and web services. iTest is a framework for software developers to submit their software and conveniently get the test results from the crowd testers. Firstly, we analyze the key problems need to be solved in a mobile crowdtesting platform; Secondly, we present the architecture of iTest framework; Thirdly, we introduce the workflow of testing web service in iTest and propose an algorithm for solving the tester selection problem mentioned in Section 2; Then the development kit to support testing mobile application is explained; Finally, we perform two experiments to illustrate that both the way to access network and tester's location influence the performance of web service.

References

[1]

A. J. Quinn and B. B. Bederson. Human computation: a survey and taxonomy of a growing field. In Proceedings of CHI 2011, 1403-1412.

Digital Library

Google Scholar

[2]

A. Doan, R. Ramakrishnan, and A. Y. Halevy. Crowdsourcing systems on the World-Wide Web. Commun. ACM 54, 4 (April 2011), 86-96.

Digital Library

Google Scholar

[3]

Crowdsourcing. http://en.wikipedia.org/wiki/Crowdsourcing

Google Scholar

[4]

R. Pham, L. Singer and Kurt Schneider. Building Test Suites in Social Coding Sites by Leveraging Drive-By Commits. 35th International Conference on Software Engineering, pp. 1209-1212, May, 2013.

Digital Library

Google Scholar

[5]

D. Liu, M. Lease, R. Kuipers and R. Bias. Crowdsourcing for Usability Testing. March 2012. arXiv 1203.1468. http://arxiv.org/abs/1203.1468

Google Scholar

[6]

Mobile Crowdsoucing, http://www.clickworker.com/ en/crowdsourcing-glossar/mobile-crowdsourcing/

Google Scholar

[7]

L. Guoliang. Crowdsourcing: Challenges and Opportunities. Tutorial on HotDB 2012. February, 2012.

Google Scholar

[8]

R. M. Karp. Reducibility Among Combinatorial Problems. Complexity of Computer Computations. New York: Plenum. pp. 85´lC103. 1972.

Google Scholar

[9]

V. V. Vazirani. Approximation Algorithms. Springer : New York, 2001. ISBN 3-540-65367-8

Digital Library

Google Scholar

[10]

Apache Log4j Project. http://logging.apache.org/log4j/2.x/

Google Scholar

[11]

Y. Liu, T. Alexandrova and T. Nakajima. Using stranger as sensors: temporal and geo-sensitive question answering via social media. In Proceedings of the 22nd international conference on World Wide Web (WWW ’13), pp. 803-814.

Digital Library

Google Scholar

[12]

Y. Fan, W. H. Lee, C. T. Iam, G. H. Syu, Indoor Place Name Annotations with Mobile Crowd, 2013 International Conference on Parallel and Distributed Systems (ICPADS) pp.546-551, 15-18 Dec. 2013

Digital Library

Google Scholar

[13]

uTest. White Paper: Crowdsourced Usability Testing. http://alexcrockett.com/wp-content/uploads/ downloads/Books/Crowdsourced_Usability_ Testing.pdf Introduction Problem Analysis Tester Management Tester Selection Incentive Mechanism Test Result Aggregation iTest Architecture Web Service Testing Testing Workflow Tester Selection Algorithm Mobile Application Testing Logging Tool Visual Info Collector Experiments Network Type Influence on Service Performance Location Influence on Service Performance Related Work Mobile Crowdsourcing Crowdsourced Testing Conclusion and Future Work Acknowledgment References

Google Scholar

Cited By

View all

Puranik A(2023)Crowd Wisdom vs in-House Expertise: A Comprehensive Analysis of Quality Assurance ApproachesInternational Journal of Information technology and Computer Engineering10.55529/ijitc.31.36.47(36-47)Online publication date: 29-Jan-2023
https://doi.org/10.55529/ijitc.31.36.47
Tang WChen RZhang JHuang LZheng SGuo S(2023)Optimization of Web Service Testing Task Assignment in Crowdtesting EnvironmentJournal of Computer Science and Technology10.1007/s11390-022-0824-738:2(455-470)Online publication date: 30-Mar-2023
https://doi.org/10.1007/s11390-022-0824-7
Ngo CPastore FBriand L(2022)Automated, Cost-effective, and Update-driven App TestingACM Transactions on Software Engineering and Methodology10.1145/350229731:4(1-51)Online publication date: 12-Jul-2022
https://dl.acm.org/doi/10.1145/3502297
Show More Cited By

Index Terms

iTest: testing software with mobile crowdsourcing
1. Human-centered computing
  1. Human computer interaction (HCI)
2. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging

Recommendations

Efficient Testing of Web Services with Mobile Crowdsourcing
Internetware '15: Proceedings of the 7th Asia-Pacific Symposium on Internetware

Nowadays, online Internet services are pervasive and can be invoked from diverse locations in anytime with multitudinous devices. Conventional testing approaches for online services like Web services are conducted by professional tester or developers ...
Mobile Compatibility Testing Using Multi-objective Genetic Algorithm
SOSE '15: Proceedings of the 2015 IEEE Symposium on Service-Oriented System Engineering

Mobile compatibility testing has been identified as one urgent and challenging issue. Mobile apps are expected to work on thousand kinds of mobile devices with diverse device features and mobile platforms. So mobile compatibility testing is complex and ...
Effective and ineffective software testing behaviors by novice programmers
ICER '13: Proceedings of the ninth annual international ACM conference on International computing education research

This data-driven paper quantitatively evaluates software testing behaviors that students exhibited in introductory computer science courses. The evaluation includes data collected over five years (10 semesters) from 49,980 programming assignment ...

Comments

Information & Contributors

Information

Published In

CrowdSoft 2014: Proceedings of the 1st International Workshop on Crowd-based Software Development Methods and Technologies

November 2014

66 pages

ISBN:9781450332248

DOI:10.1145/2666539

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 November 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

SIGSOFT/FSE'14

Sponsor:

SIGSOFT

SIGSOFT/FSE'14: 22nd ACM SIGSOFT Symposium on the Foundations of Software Engineering

November 17, 2014

Hong Kong, China

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

17
Total Citations
View Citations
417
Total Downloads

Downloads (Last 12 months)9
Downloads (Last 6 weeks)1

Reflects downloads up to 20 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Puranik A(2023)Crowd Wisdom vs in-House Expertise: A Comprehensive Analysis of Quality Assurance ApproachesInternational Journal of Information technology and Computer Engineering10.55529/ijitc.31.36.47(36-47)Online publication date: 29-Jan-2023
https://doi.org/10.55529/ijitc.31.36.47
Tang WChen RZhang JHuang LZheng SGuo S(2023)Optimization of Web Service Testing Task Assignment in Crowdtesting EnvironmentJournal of Computer Science and Technology10.1007/s11390-022-0824-738:2(455-470)Online publication date: 30-Mar-2023
https://doi.org/10.1007/s11390-022-0824-7
Ngo CPastore FBriand L(2022)Automated, Cost-effective, and Update-driven App TestingACM Transactions on Software Engineering and Methodology10.1145/350229731:4(1-51)Online publication date: 12-Jul-2022
https://dl.acm.org/doi/10.1145/3502297
Stengele OWestermeyer CHartenstein H(2022)Decentralized Review and Attestation of Software Attribute ClaimsIEEE Access10.1109/ACCESS.2022.318504610(66694-66710)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2022.3185046
Huang YNazir SWu JHussain Khoso FAli FKhan H(2021)An Efficient Decision Support System for the Selection of Appropriate Crowd in CrowdsourcingComplexity10.1155/2021/55188782021(1-11)Online publication date: 4-Jun-2021
https://doi.org/10.1155/2021/5518878
Yao YHuang SZong CLiu EChen N(2021)A Study on Testers’ Learning Curve in Crowdsourced Software TestingIEEE Access10.1109/ACCESS.2021.30815929(77127-77137)Online publication date: 2021
https://doi.org/10.1109/ACCESS.2021.3081592
Cao ZWang XYu SYun YFang CGrundy JLe Goues CLo D(2020)STIFAProceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering10.1145/3324884.3415300(1331-1335)Online publication date: 21-Dec-2020
https://dl.acm.org/doi/10.1145/3324884.3415300
Wang HHong TCao JZhang HNie CCheng BHe YKuang LGong DChen WShi YHuang D(2020)COSINE:a software development model integrating collective intelligence, service and ecosystem2020 IEEE World Congress on Services (SERVICES)10.1109/SERVICES48979.2020.00038(122-127)Online publication date: Oct-2020
https://doi.org/10.1109/SERVICES48979.2020.00038
Eberhart ZLeClair AMcMillan C(2020)Automatically Extracting Subroutine Summary Descriptions from Unstructured Comments2020 IEEE 27th International Conference on Software Analysis, Evolution and Reengineering (SANER)10.1109/SANER48275.2020.9054789(35-46)Online publication date: Feb-2020
https://doi.org/10.1109/SANER48275.2020.9054789
Kong XLiu XJedari BLi MWan LXia F(2019)Mobile Crowdsourcing in Smart Cities: Technologies, Applications, and Future ChallengesIEEE Internet of Things Journal10.1109/JIOT.2019.29218796:5(8095-8113)Online publication date: Oct-2019
https://doi.org/10.1109/JIOT.2019.2921879
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Efficient Testing of Web Services with Mobile Crowdsourcing

Mobile Compatibility Testing Using Multi-objective Genetic Algorithm

Effective and ineffective software testing behaviors by novice programmers

Comments

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Other Metrics

Article Metrics

Other Metrics

Cited By

Login options

Full Access

PDF

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Efficient Testing of Web Services with Mobile Crowdsourcing

Mobile Compatibility Testing Using Multi-objective Genetic Algorithm

Effective and ineffective software testing behaviors by novice programmers

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations