research-article

Designing Digital Peer Assessment for Second Language Learning in Low Resource Learning Settings

Authors:

Maletsabisa Molapo,

Chane Simone Moodley,

Ismail Yunus Akhalwaya,

Jay Kloppenberg,

Richard YoungAuthors Info & Claims

L@S '19: Proceedings of the Sixth (2019) ACM Conference on Learning @ Scale

Article No.: 14, Pages 1 - 13

https://doi.org/10.1145/3330430.3333626

Published: 24 June 2019 Publication History

Abstract

In low-resource, over-burdened schools and learning centres, peer assessment systems promise significant practical and pedagogical benefits. Many of the these benefits have been realised in contexts like massive open online courses (MOOCs) and university classrooms which share a specific trait with low-resource schools: high learner-teacher ratios. However, the constraints and considerations for designing and deploying peer assessment systems in low-resource classrooms have not been well-researched and understood, especially for high school. In this paper, we present the design of a peer assessment system for second language learning (English as a Second Language) for high school learners in South Africa. We report findings from multiple studies investigating qualitative and quantitative aspects of peer review, as well as the contextual factors that influence the viability of peer assessment systems in these contexts.

References

[1]

Luca de Alfaro and Michael Shavlovsky. 2013. CrowdGrader: Crowdsourcing the Evaluation of Homework Assignments.

Digital Library

[2]

Stephen Balfour. 2013. Assessing Writing in MOOCs: Automated Essay Scoring and Calibrated Peer Review (tm). Research & Practice in Assessment 8.

[3]

Edwin H. Blake and William D. Tucker. 2006. User interfaces for communication bridges across the digital divide. AI & SOCIETY 20, 2: 232--242.

Digital Library

[4]

Bridges.org. 2005. Real Access / Real Impact criteria | bridges.org.

[5]

Emma Brunskill, Sunil Garg, Clint Tseng, U Washington, and Leah Findlater. Evaluating an Adaptive Multi-User Educational Tool for Low-Resource Environments. Test.

[6]

Aoife Cahill. Deep Learning for Automated Scoring*. Retrieved October 17, 2018 from www.kaggle.com/c/asap-aes

[7]

Julia Cambre, Scott Klemmer, and Chinmay Kulkarni. 2018. Juxtapeer:Comparative Peer Review Yields Higher Quality Feedback and Promotes Deeper Reflection. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI '18, 1--13.

Digital Library

[8]

Nicola Capuano, Vincenzo Loia, and Francesco Orciuoli. 2017. A Fuzzy Group Decision Making Model for Ordinal Peer Assessment. IEEE Transactions on Learning Technologies 10, 2: 247--259.

Digital Library

[9]

Xi Chen, Paul N Bennett, Kevyn Collins-thompson, and Eric Horvitz. 2013. Pairwise Ranking Aggregation in a Crowdsourced Settingwsdm2013-preference-chen-et-al.pdf.

Digital Library

[10]

Kwangsu Cho and Charles MacArthur. 2010. Student revision with peer and expert reviewing. Learning and Instruction 20, 4: 328--338.

[11]

Kwangsu Cho and Christian D. Schunn. 2007. Scaffolded writing and rewriting in the discipline: A web-based reciprocal peer review system. Computers & Education 48, 3: 409--426.

Digital Library

[12]

Kwangsu Cho, Christian D. Schunn, and Davida Charney. 2006. Commenting on Writing: Typology and Perceived Helpfulness of Comments from Novice Peer Reviewers and Subject Matter Experts. Written Communication 23, 3: 260--294.

[13]

Kwangsu Cho, Christian D. Schunn, and Roy W. Wilson. 2006. Validity and reliability of scaffolded peer assessment of writing from instructor and student perspectives. Journal of Educational Psychology 98, 4: 891--901.

[14]

Shamil Chollampatt, Duc Tam Hoang, Hwee Tou Ng, and Hwee Tou Ng. 2016. Adapting Grammatical Error Correction Based on the Native Language of Writers with Neural Network Joint Models. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, 1901--1911. Retrieved October 19, 2018 from http://www.aclweb.org/anthology/D16-1195

[15]

Shamil Chollampatt and Hwee Tou Ng. 2018. A Multilayer Convolutional Encoder-Decoder Neural Network for Grammatical Error Correction. arXiv preprint arXiv:1801, 08831. Retrieved October 19, 2018 from http://arxiv.org/abs/1801.08831

[16]

Shamil Chollampatt, Kaveh Taghipour, and Hwee Tou Ng. 2016. Neural Network Translation Models for Grammatical Error Correction. arXiv preprint arXiv:1606, 00189. Retrieved October 19, 2018 from http://arxiv.org/abs/1606.00189

[17]

Scott A. Crossley and Danielle S. McNamara. 2012. Predicting second language writing proficiency: the roles of cohesion and linguistic sophistication. Journal of Research in Reading 35, 2: 115--135.

[18]

Scott A. Crossley, Tom Salsbury, and Danielle S. McNamara. 2012. Predicting the proficiency level of language learners using lexical indices. Language Testing 29, 2: 243--263.

[19]

Ronan Cummins and Marek Rei. 2018. Neural Multi-task Learning in Automated Assessment. arXiv preprint arXiv:1801, 06830. Retrieved October 17, 2018 from http://arxiv.org/abs/1801.06830

[20]

Nicola Dell and Neha Kumar. 2016. The Ins and Outs of HCI for Development. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems - CHI '16, 2220--2232.

Digital Library

[21]

Anders Ericsson, Ralf Krampe, and Clemens Tesch-Römer. 1993. The role of deliberate practice in the acquisition of expert performance. Psychological review 10, 1: 363. Retrieved April 16, 2019 from https://psycnet.apa.org/buy/1993-40718-001

[22]

KA Ericsson, RT Krampe, C Tesch-Römer -Psychological review, and undefined 1993. The role of deliberate practice in the acquisition of expert performance. doi.apa.org. Retrieved February 11, 2019 from http://doi.apa.org/journals/rev/100/3/363.html

[23]

Ilya M Goldin. 2012. Accounting for Peer Reviewer Bias with Bayesian Models. In Proceedings of the Workshop on Intelligent Support for Learning Groups at the 11th International Conference on Intelligent Tutoring Systems. Retrieved February 22, 2018 from https://pdfs.semanticscholar.org/d5a2/8c14e86afbc6f21a0bcc462404a529991cfe.pdf

[24]

Ilya M Goldin. 2012. Accounting for Peer Reviewer Bias with Bayesian Models. Proceedings of the Workshop on Intelligent Support for Learning Groups at the 11th International Conference on Intelligent Tutoring Systems.

[25]

Catherine M. Hicks, Vineet Pandey, C. Ailie Fraser, and Scott Klemmer. 2016. Framing Feedback: Choosing Review Environment Features that Support High Quality Peer Assessment. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems - CHI '16, 458--469.

Digital Library

[26]

Kevin G. Jamieson and Robert D. Nowak. 2011. Active Ranking using Pairwise Comparisons. 1: 1--9.

[27]

Rhelda Krugel and Elsa Fourie. 2014. Concerns for the Language Skills of South African Learners and Their Teachers. International Journal of Educational Sciences 7, 1: 219--228.

[28]

James A. Kulik and Chen-Lin C. Kulik. 1988. Timing of Feedback and Verbal Learning. Review of Educational Research 58, 1: 79--97.

[29]

Chinmay E. Kulkarni, Michael S. Bernstein, and Scott R. Klemmer. 2015. PeerStudio: Rapid Peer Feedback Emphasizes Revision and Improves Performance. In Proceedings of the Second (2015) ACM Conference on Learning @ Scale - L@S '15, 75--84.

Digital Library

[30]

Chinmay Kulkarni, Koh Pang Wei, Huy Le, Daniel Chia, Kathryn Papadopoulos, Justin Cheng, Daphne Koller, and Scott R. Klemmer. 2015. Peer and Self Assessment in Massive Online Classes. Design Thinking Research. Understanding Innovation.: 131--168.

[31]

Diane Litman. 2016. Natural Language Processing for Enhancing Teaching and Learning. In Proceedings of the 30th Conference on Artificial Intelligence (AAAI 2016), 4170--4176.

Digital Library

[32]

Kristi Lundstrom and Wendy Baker. 2009. To give is better than to receive: The benefits of peer review to the reviewer's own writing. Journal of Second Language Writing 18, 1: 30--43.

[33]

Wencan Luo. 2017. Automatic Summarization for Student Reflective Responses. University of Pittsburgh. Retrieved January 15, 2018 from http://dscholarship.pitt.edu/31455/

[34]

Gary Marsden, Andrew Maunder, and Munier Parker. 2008. People are people, but technology is not technology. Philosophical transactions. Series A, Mathematical, physical, and engineering sciences 366, 1881: 3795--804.

[35]

Andrew Maunder, Gary Marsden, Dominic Gruijters, and Edwin Blake. 2007. Designing interactive systems for the developing world -reflections on user-centred design. In 2007 International Conference on Information and Communication Technologies and Development, 18.

[36]

Chao Mbogo, Edwin Blake, and Hussein Suleman. 2016. Design and Use of Static Scaffolding Techniques to Support Java Programming on a Mobile Phone. In Proceedings of the 2016 ACM Conference on Innovation and Technology in Computer Science Education - ITiCSE '16, 314--319.

Digital Library

[37]

Fei Mi and Dit-Yan Yeung. 2015. Probabilistic Graphical Models for Boosting Cardinal and Ordinal Peer Grading in MOOCs. Proceedings of the 29th AAAI Conference on Artificial Intelligence (AAAI-15): 454--460.

Digital Library

[38]

M. Molapo, M. Densmore, and L. Morie. 2016. Designing with community health workers: Enabling productive participation through exploration. In ACM International Conference Proceeding Series.

Digital Library

[39]

M. Molapo, M. Densmore, and B. De Renzi. 2017. Video consumption patterns for first time smartphone users: Community health workers in Lesotho. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems.

Digital Library

[40]

Maletsabisa Molapo. 2015. Local Games, Local Content, Local Applications - On Mobile: ICTs for Informal Learning in Rural Africa. In Digitally Connected: Global Perspectives on Youth and Digital Media, Sandra Cortesi and Urs Gasser (eds.). Harvard Univeristy Berkman Center for Internet & Society, 54--58.

[41]

Maletsabisa Molapo and Melissa Densmore. 2015. How To Choose a Mobile Phone for an ICT4D Project. In Proceedings of the Seventh International Conference on Information and Communication Technologies and Development Notes - ICTD '15, 10--13.

Digital Library

[42]

Huy Nguyen, Wenting Xiong, and Diane Litman. 2017. Iterative Design and Classroom Evaluation of Automated Formative Feedback for Improving Peer Feedback Localization. International Journal of Artificial Intelligence in Education 27, 3: 582--622.

[43]

Benjamin D. Nye. 2015. Intelligent Tutoring Systems by and for the Developing World: A Review of Trends and Approaches for Educational Technology in a Global Context. International Journal of Artificial Intelligence in Education 25, 2: 177--203.

[44]

Chris Piech, Jonathan Huang, Zhenghao Chen, Chuong Do, Andrew Ng, and Daphne Koller. 2013. Tuned Models of Peer Assessment in MOOCs.

[45]

Chris Piech, Jonathan Huang, Zhenghao Chen, Chuong Do, Andrew Ng, and Daphne Koller. 2013. Tuned Models of Peer Assessment in MOOCs. Retrieved February 22, 2018 from http://arxiv.org/abs/1307.2579

[46]

Karthik Raman and Thorsten Joachims. 2014. Methods for ordinal peer grading. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining - KDD '14, 1037--1046.

Digital Library

[47]

Karthik Raman and Thorsten Joachims. 2014. Methods for Ordinal Peer Grading.

Digital Library

[48]

Marek Rei and Helen Yannakoudakis. 2016. Compositional Sequence Labeling Models for Error Detection in Learner Writing. arXiv preprint arXiv:1607, 06153.

[49]

Thomas Staubitz, Dominic Petrick, Matthias Bauer, Jan Renz, and Christoph Meinel. 2016. Improving the Peer Assessment Experience on MOOC Platforms. In Proceedings of the Third (2016) ACM Conference on Learning @ Scale - L@S '16, 389--398.

Digital Library

[50]

Kaveh Taghipour and Hwee Tou Ng. 2016. A Neural Approach to Automated Essay Scoring. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, 1882--1891. Retrieved October 17, 2018 from https://www.kaggle.com/c/asap-aes

[51]

Fatema Wali and Henk Huijser. 2018. Write to improve: exploring the impact of an online feedback tool on Bahraini learners of English. Learning and Teaching in Higher Education: Gulf Perspectives 15, 1.

[52]

Andrew E. Waters, David Tinapple, and Richard G. Baraniuk. 2015. BayesRank: A Bayesian Approach to Ranked Peer Grading. In Proceedings of the Second (2015) ACM Conference on Learning @ Scale - L@S '15, 177--183.

Digital Library

[53]

Andrew E. Waters, David Tinapple, and Richard G. Baraniuk. 2015. BayesRank: A Bayesian Approach to Ranked Peer Grading. Proceedings of the Second (2015) ACM Conference on Learning @ Scale -L@S '15: 177--183.

Digital Library

[54]

Wenting Xiong. 2014. Helfulness-Guided Review Summarization. University of Pittsburgh. Retrieved January 15, 2018 from https://dscholarship.pitt.edu/22509/1/WentingXiong_ETD_12042014_final.pdf

[55]

Helen Yannakoudakis, Øistein E Andersen, Ardeshir Geranpayeh, Ted Briscoe, and Diane Nicholls. 2018. Developing an automated writing placement system for ESL learners. Applied Measurement in Education 31, 3: 251--267.

Cited By

(2022)Learning, Marginalization, and Improving the Quality of Education in Low-income Countries10.11647/obp.0256Online publication date: Jan-2022
https://doi.org/10.11647/obp.0256
Álvarez Dorado MMora Rey-Sánchez BMartín Pérez JPérez González GMorón Fernández C(2021)Impacto del meta aprendizaje en el rendimiento académico y la motivación de alumnos primer curso de grado en el área de Expresión Gráfica. = Impact of meta learning on academic performance and motivation of first year bachelor students in descriptive geometry.Advances in Building Education10.20868/abe.2021.1.45705:1(67)Online publication date: 16-Apr-2021
https://doi.org/10.20868/abe.2021.1.4570
Ahn JNguyen HCampos FYoung WScheffel MDowell NJoksimovic SSiemens G(2021)Transforming Everyday Information into Practical Analytics with Crowdsourced Assessment TasksLAK21: 11th International Learning Analytics and Knowledge Conference10.1145/3448139.3448146(66-76)Online publication date: 12-Apr-2021
https://dl.acm.org/doi/10.1145/3448139.3448146

Index Terms

Designing Digital Peer Assessment for Second Language Learning in Low Resource Learning Settings
1. Human-centered computing

Recommendations

The Effectiveness of Peer Assessment in EFL Blended Learning Environments
CIPAE 2021: 2021 2nd International Conference on Computers, Information Processing and Advanced Education

There is an increasingly application of peer assessment (PA) in blended learning environments to motivate and evaluate learners. In spite of its popularity, there are few studies to be conducted to analyze and corroborate its impact on students’ ...
The impact of a two-round, mobile peer assessment on learning achievements, critical thinking skills, and meta-cognitive awareness

Educators and practitioners recognise peer assessment as an effective educational approach. However, designing the most effective peer assessment activities remains a major concern. In this study, a two-round peer assessment and a series of ...
Employing Wikibook project in a linguistics course to promote peer teaching and learning

Peer teaching and learning are learner-centred approaches with great potential for promoting effective learning, and the fast development of Web 2.0 technology has opened new doors for promoting peer teaching and learning. In this study, we aim to ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

L@S '19: Proceedings of the Sixth (2019) ACM Conference on Learning @ Scale

June 2019

386 pages

ISBN:9781450368049

DOI:10.1145/3330430

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 June 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

L@S '19

L@S '19: Sixth (2019) ACM Conference on Learning @ Scale

June 24 - 25, 2019

IL, Chicago, USA

Acceptance Rates

L@S '19 Paper Acceptance Rate 24 of 70 submissions, 34%;

Overall Acceptance Rate 117 of 440 submissions, 27%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
219
Total Downloads

Downloads (Last 12 months)28
Downloads (Last 6 weeks)9

Reflects downloads up to 22 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

(2022)Learning, Marginalization, and Improving the Quality of Education in Low-income Countries10.11647/obp.0256Online publication date: Jan-2022
https://doi.org/10.11647/obp.0256
Álvarez Dorado MMora Rey-Sánchez BMartín Pérez JPérez González GMorón Fernández C(2021)Impacto del meta aprendizaje en el rendimiento académico y la motivación de alumnos primer curso de grado en el área de Expresión Gráfica. = Impact of meta learning on academic performance and motivation of first year bachelor students in descriptive geometry.Advances in Building Education10.20868/abe.2021.1.45705:1(67)Online publication date: 16-Apr-2021
https://doi.org/10.20868/abe.2021.1.4570
Ahn JNguyen HCampos FYoung WScheffel MDowell NJoksimovic SSiemens G(2021)Transforming Everyday Information into Practical Analytics with Crowdsourced Assessment TasksLAK21: 11th International Learning Analytics and Knowledge Conference10.1145/3448139.3448146(66-76)Online publication date: 12-Apr-2021
https://dl.acm.org/doi/10.1145/3448139.3448146

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents