research-article

Improving Grading Fairness and Transparency with Decentralized Collaborative Peer Assessment

Authors:

Sharareh Alipour,

Sina Elahimanesh,

Soroush Jahanzad,

Iman Mohammadi,

Parimehr Morassafar,

Seyed Parsa Neshaei,

Mojtaba TefaghAuthors Info & Claims

Proceedings of the ACM on Human-Computer Interaction, Volume 8, Issue CSCW1

Article No.: 73, Pages 1 - 24

https://doi.org/10.1145/3637350

Published: 26 April 2024 Publication History

Abstract

Computer-assisted collaborative peer grading is a developing growth area in academic evaluation. However, peer assessment often needs help with problems such as the lack of reliability, transparency, fairness, grading speed, and motivation to participate among students. The literature suggests several principles that each partly address the said issues. We propose a novel decentralized approach to academic peer assessment, using blockchain as an underlying technology, to address the principal problems in traditional peer assessment. We also derive design concepts for a modern courseware (CW) application consisting of our method and apply them to implement our approach in a CW called Blockment. We test the effectiveness of our method and system by running quantitative and qualitative experiments, proving our claims of improving reliability, transparency, fairness, grading speed, and motivation of grades in peer assessment. The results suggest embedding our method and system in academic courses to improve conventional peer grading methods.

References

[1]

Nurzhan Zhumabekuly Aitzhan and Davor Svetinovic. 2016. Security and privacy in decentralized energy trading through multi-signatures, blockchain and anonymous messaging streams. IEEE Transactions on Dependable and Secure Computing 15, 5 (2016), 840--852.

[2]

Ali Alammary, Samah Alhazmi, Marwah Almasri, and Saira Gillani. 2019. Blockchain-based applications in education: A systematic review. Applied Sciences 9, 12 (2019), 2400.

[3]

Mustafa Cem Aldag. 2020. The use of blockchain technology in agriculture. Zeszyty Naukowe Uniwersytetu Ekonomicznego w Krakowie/Cracow Review of Economics and Management 4 (982) (2020), 7--17.

[4]

Sharareh Alipour, Sina Elahimanesh, Soroush Jahanzad, Parimehr Morassafar, and Seyed Parsa Neshaei. 2022. A Blockchain Approach to Academic Assessment. In CHI Conference on Human Factors in Computing Systems Extended Abstracts. 1--6.

Digital Library

[5]

Mohammed AlShamsi, Said A Salloum, Muhammad Alshurideh, and Sherief Abdallah. 2021. Artificial intelligence and blockchain for transparency in governance. In Artificial intelligence for sustainable development: Theory, practice and future applications. Springer, 219--230.

[6]

Rodelio Arenas and Proceso Fernandez. 2018. CredenceLedger: a permissioned blockchain for verifiable academic credentials. In 2018 IEEE International Conference on Engineering, Technology and Innovation (ICE/ITMC). IEEE, 1--6.

[7]

Rawia Bdiwi, Cyril De Runz, Sami Faiz, and Arab Ali Cherif. 2018. A blockchain based decentralized platform for ubiquitous learning environment. In 2018 IEEE 18th International Conference on Advanced Learning Technologies (ICALT). IEEE, 90--92.

[8]

Nelson Bore, Samuel Karumba, Juliet Mutahi, Shelby Solomon Darnell, Charity Wayua, and Komminist Weldemariam. 2017. Towards blockchain-enabled school information hub. In Proceedings of the Ninth International Conference on Information and Communication Technologies and Development. 1--4.

Digital Library

[9]

Steven Bradley. 2019. Addressing Bias to Improve Reliability in Peer Review of Programming Coursework. In Proceedings of the 19th Koli Calling International Conference on Computing Education Research (Koli, Finland) (Koli Calling '19). Association for Computing Machinery, New York, NY, USA, Article 19, 10 pages. https://doi.org/10.1145/3364510.3364523

Digital Library

[10]

Clare Brindley and Susan Scoffield. 1998. Peer Assessment in Undergraduate Programmes. Teaching in Higher Education 3, 1 (1998), 79--90. https://doi.org/10.1080/1356215980030106 arXiv:https://doi.org/10.1080/1356215980030106

[11]

Hou Pong Chan and Irwin King. 2017. Leveraging social connections to improve peer assessment in MOOCs. In Proceedings of the 26th International Conference on World Wide Web Companion. 341--349.

Digital Library

[12]

Elizabeth Charters. 2003. The use of think-aloud methods in qualitative research an introduction to think-aloud methods. Brock Education Journal 12, 2 (2003).

[13]

Jingchun Chen. 2021. The Effectiveness of Peer Assessment in EFL Blended Learning Environments. In 2021 2nd International Conference on Computers, Information Processing and Advanced Education (Ottawa, ON, Canada) (CIPAE 2021). Association for Computing Machinery, New York, NY, USA, 566--568. https://doi.org/10.1145/3456887.3457015

Digital Library

[14]

Winnie Cheng and Martin Warren. 1997. Having second thoughts: Student perceptions before and after a peer assessment exercise. Studies in Higher Education 22, 2 (1997), 233--239. https://doi.org/10.1080/03075079712331381064 arXiv:https://doi.org/10.1080/03075079712331381064

[15]

Winnie Cheng and Martin Warren. 1999. Peer and Teacher Assessment of the Oral and Written Tasks of a Group Project. Assessment & Evaluation in Higher Education 24, 3 (1999), 301--314. https://doi.org/10.1080/0260293990240304 arXiv:https://doi.org/10.1080/0260293990240304

[16]

Avi J. Cohen and Andrea L. Williams. 2019. Scalable, scaffolded writing assignments with online peer review in a large introductory economics course. The Journal of Economic Education 50, 4 (2019), 371--387. https://doi.org/10.1080/00220485.2019.1654951 arXiv:https://doi.org/10.1080/00220485.2019.1654951

[17]

Michael Crosby, Pradan Pattanayak, Sanjeev Verma, Vignesh Kalyanaraman, et al . 2016. Blockchain technology: Beyond bitcoin. Applied Innovation 2, 6--10 (2016), 71.

[18]

Phil Davies. 2000. Computerized Peer Assessment. Innovations in Education and Training International 37, 4 (2000), 346--355. https://doi.org/10.1080/135580000750052955 arXiv:https://doi.org/10.1080/135580000750052955

[19]

Tamara Denning, Michael Kelly, David Lindquist, Roshni Malani, William G. Griswold, and Beth Simon. 2007. Lightweight Preliminary Peer Review: Does in-Class Peer Review Make Sense?. In Proceedings of the 38th SIGCSE Technical Symposium on Computer Science Education (Covington, Kentucky, USA) (SIGCSE '07). Association for Computing Machinery, New York, NY, USA, 266--270. https://doi.org/10.1145/1227310.1227406

Digital Library

[20]

Joseph S Dumas, Joseph S Dumas, and Janice Redish. 1999. A practical guide to usability testing. Intellect books.

[21]

Joseph S Dumas and Jean E Fox. 2007. Usability testing: Current practice and future directions. In The human-computer interaction handbook. CRC Press, 1155--1176.

[22]

Chris Elsden, Arthi Manohar, Jo Briggs, Mike Harding, Chris Speed, and John Vines. 2018. Making Sense of Blockchain Applications: A Typology for HCI. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI '18). Association for Computing Machinery, New York, NY, USA, 1--14. https://doi.org/10.1145/3173574.3174032

Digital Library

[23]

Stephen Fallows and Balasubramanyan Chandramohan. 2001. Multiple Approaches to Assessment: Reflections on use of tutor, peer and self-assessment. Teaching in Higher Education 6, 2 (2001), 229--246. https://doi.org/10.1080/13562510120045212 arXiv:https://doi.org/10.1080/13562510120045212

[24]

George Filippakis, Katerina Georgouli, and Cleo Sgouropoulou. 2015. The Design of a Web-Based System to Support the Distribution of Assignments for Distance Peer Evaluation. In Proceedings of the 19th Panhellenic Conference on Informatics (Athens, Greece) (PCI '15). Association for Computing Machinery, New York, NY, USA, 400--405. https://doi.org/10.1145/2801948.2802014

Digital Library

[25]

Edward F. Gehringer. 2001. Electronic Peer Review and Peer Grading in Computer-Science Courses. In Proceedings of the Thirty-Second SIGCSE Technical Symposium on Computer Science Education (Charlotte, North Carolina, USA) (SIGCSE '01). Association for Computing Machinery, New York, NY, USA, 139--143. https://doi.org/10.1145/364447.364564

Digital Library

[26]

Edward F. Gehringer, Donald D. Chinn, Manuel A. Pérez-Quiñones, and Mark A. Ardis. 2005. Using Peer Review in Teaching Computing. In Proceedings of the 36th SIGCSE Technical Symposium on Computer Science Education (St. Louis, Missouri, USA) (SIGCSE '05). Association for Computing Machinery, New York, NY, USA, 321--322. https://doi.org/10.1145/1047344.1047455

Digital Library

[27]

Katerina Georgouli, C. Sgouropoulou, Ilias Skalkidis, and Charilaos Tsetsekas. 2012. Introducing a Collaborative Peer-Evaluation Learning Model in Higher Education Programming-Based Courses. Proceedings of the 2012 16th Panhellenic Conference on Informatics, PCI 2012, 399--404. https://doi.org/10.1109/PCi.2012.72

Digital Library

[28]

Hanna Halaburda. 2018. Blockchain Revolution without the Blockchain? Commun. ACM 61, 7 (jun 2018), 27--29. https://doi.org/10.1145/3225619

Digital Library

[29]

John Hamer, Catherine Kell, and Fiona Spence. 2007. Peer Assessment Using Aropä. In Proceedings of the Ninth Australasian Conference on Computing Education - Volume 66 (Ballarat, Victoria, Australia) (ACE '07). Australian Computer Society, Inc., AUS, 43--54.

[30]

Meng Han, Zhigang Li, Jing He, Dalei Wu, Ying Xie, and Asif Baba. 2018. A novel blockchain-based education records verification solution. In Proceedings of the 19th annual SIG conference on information technology education. 178--183.

Digital Library

[31]

Yu He, Xinying Hu, and Guangzhong Sun. 2019. A Cognitive Diagnosis Framework Based on Peer Assessment. In Proceedings of the ACM Turing Celebration Conference - China (Chengdu, China) (ACM TURC '19). Association for Computing Machinery, New York, NY, USA, Article 78, 6 pages. https://doi.org/10.1145/3321408.3322850

Digital Library

[32]

Thomas Hepp, Alexander Schoenhals, Christopher Gondek, and Bela Gipp. 2018. OriginStamp: A blockchain-backed system for decentralized trusted timestamping. it-Information Technology 60, 5--6 (2018), 273--281.

[33]

Janette R. Hill, Liyan Song, and Richard E. West. 2009. Social Learning Theory and Web-Based Learning Environments: A Review of Research and Discussion of Implications. American Journal of Distance Education 23, 2 (2009), 88--103. https://doi.org/10.1080/08923640902857713 arXiv:https://doi.org/10.1080/08923640902857713

[34]

Marko Hölbl, Aida Kamisalic, Muhamed Turkanovic, Marko Kompara, Bla? Podgorelec, and Marjan Hericko. 2018. EduCTX: an ecosystem for managing digital micro-credentials. In 2018 28th EAEEIE Annual Conference (EAEEIE). IEEE, 1--9.

[35]

Tani Hossain, Tasniah Mohiuddin, AM Hasan, Muhammad Nazrul Islam, and Syed Akhter Hossain. 2020. Designing and developing graphical user interface for the MultiChain blockchain: towards incorporating HCI in blockchain. In International Conference on Intelligent Systems Design and Applications. Springer, 446--456.

[36]

Jennifer L Hughes, Abigail A Camden, Tenzin Yangchen, et al . 2016. Rethinking and updating demographic questions: Guidance to improve descriptions of research samples. Psi Chi Journal of Psychological Research 21, 3 (2016), 138--151.

[37]

Suhan Jiang and Jie Wu. 2022. A reward response game in the blockchain-powered federated learning system. International Journal of Parallel, Emergent and Distributed Systems 37, 1 (2022), 68--90.

[38]

Bojana Koteska, Elena Karafiloski, and Anastas Mishev. 2017. Blockchain implementation quality challenges: a literature. In SQAMIA 2017: 6th workshop of software quality, analysis, monitoring, improvement, and applications, Vol. 1938. 8--8.

[39]

Chinmay Kulkarni, Koh Pang Wei, Huy Le, Daniel Chia, Kathryn Papadopoulos, Justin Cheng, Daphne Koller, and Scott R. Klemmer. 2013. Peer and Self Assessment in Massive Online Classes. ACM Trans. Comput.-Hum. Interact. 20, 6, Article 33 (dec 2013), 31 pages. https://doi.org/10.1145/2505057

Digital Library

[40]

Chinmay E. Kulkarni, Richard Socher, Michael S. Bernstein, and Scott R. Klemmer. 2014. Scaling Short-Answer Grading by Combining Peer Assessment with Algorithmic Scoring. In Proceedings of the First ACM Conference on Learning @ Scale Conference (Atlanta, Georgia, USA) (L@S '14). Association for Computing Machinery, New York, NY, USA, 99--108. https://doi.org/10.1145/2556325.2566238

Digital Library

[41]

Yi-Chieh Lee and Wai-Tat Fu. 2019. Supporting Peer Assessment in Education with Conversational Agents. In Proceedings of the 24th International Conference on Intelligent User Interfaces: Companion (Marina del Ray, California) (IUI '19). Association for Computing Machinery, New York, NY, USA, 7--8. https://doi.org/10.1145/3308557.3308695

Digital Library

[42]

Erno Lehtinen, Kai Hakkarainen, Lasse Lipponen, Marjaana Rahikainen, and Hanni Muukkonen. 1999. Computer supported collaborative learning: A review. The JHGI Giesbers reports on education 10 (1999), 1999.

[43]

Jinrong Li and Mimi Li. 2018. Turnitin and peer review in ESL academic writing classrooms. (2018).

[44]

Sunny San-Ju Lin, E. Z.-F. Liu, and Shyan-Ming Yuan. 2001. Web Based Peer Assessment: Attitude and Achievement. IEEE Trans. on Educ. 44, 2 (may 2001), 13 pp. https://doi.org/10.1109/13.925865

Digital Library

[45]

Yu-Pin Lin, Joy R Petway, Wan-Yu Lien, and Josef Settele. 2018. Blockchain with artificial intelligence to efficiently manage water use under climate change., 34 pages.

[46]

Eric Zhi-Feng Liu, S. S.J. Lin, Chi-Huang Chiu, and Shyan-Ming Yuan. 2001. Web-Based Peer Review: The Learner as Both Adapter and Reviewer. IEEE Trans. on Educ. 44, 3 (aug 2001), 246--251. https://doi.org/10.1109/13.940995

Digital Library

[47]

Qin Liu, Qingchen Guan, Xiaowen Yang, Hongming Zhu, Gill Green, and Shaohan Yin. 2018. Education-industry cooperative system based on blockchain. In 2018 1st IEEE international conference on hot information-centric networking (HotICN). IEEE, 207--211.

[48]

David Lizcano, Juan A Lara, Bebo White, and Shadi Aljawarneh. 2020. Blockchain-based approach to create a model of trust in open and ubiquitous higher education. Journal of Computing in Higher Education 32, 1 (2020), 109--134.

[49]

Yanxin Lu, Joe Warren, Christopher Jermaine, Swarat Chaudhuri, and Scott Rixner. 2015. Grading the graders: Motivating peer graders in a MOOC. In Proceedings of the 24th International Conference on World Wide Web. 680--690.

Digital Library

[50]

Zhenqiu Lu and Ke-Hai Yuan. 2010. Welch's t test. Encyclopedia of research design (2010), 1620--1623.

[51]

Heng Luo, Anthony Robinson, and Jae-Young Park. 2014. Peer grading in a MOOC: Reliability, validity, and perceived effects. Online Learning Journal 18, 2 (2014).

[52]

Douglas Magin and Phil Helmore. 2001. Peer and Teacher Assessments of Oral Presentation Skills: How reliable are they? Studies in Higher Education 26, 3 (2001), 287--298. https://doi.org/10.1080/03075070120076264 arXiv:https://doi.org/10.1080/03075070120076264

[53]

Roberto Mavilia and Roberta Pisani. 2022. Blockchain for agricultural sector: The case of South Africa. African Journal of Science, Technology, Innovation and Development 14, 3 (2022), 845--851.

[54]

Paola Mejia-Domenzain, Eva Laini, Seyed Parsa Neshaei, Thiemo Wambsganss, and Tanja Käser. 2023. Visualizing Self-Regulated Learner Profiles in Dashboards: Design Insights from Teachers. In Artificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners, Doctoral Consortium and Blue Sky, Ning Wang, Genaro Rebolledo-Mendez, Vania Dimitrova, Noboru Matsuda, and Olga C. Santos (Eds.). Springer Nature Switzerland, Cham, 619--624.

[55]

Alexander Mikroyannidis, John Domingue, Michelle Bachler, and Kevin Quick. 2018. A learner-centred approach for lifelong learning powered by the blockchain. In EdMedia innovate learning. Association for the Advancement of Computing in Education (AACE), 1388--1393.

[56]

Martijn Millecamp, Francisco Gutiérrez, Sven Charleer, Katrien Verbert, and Tinne De Laet. 2018. A qualitative evaluation of a learning dashboard to support advisor-student dialogues. In Proceedings of the 8th international conference on learning analytics and knowledge. 56--60.

Digital Library

[57]

Rolf Molich, Ann Damgaard Thomsen, Barbara Karyukina, Lars Schmidt, Meghan Ede, Wilma van Oel, and Meeta Arcuri. 1999. Comparative evaluation of usability tests. In CHI'99 extended abstracts on Human factors in computing systems. 83--84.

[58]

Raoul A Mulder, Jon M Pearce, and Chi Baik. 2014. Peer review in higher education: Student perceptions before and after participation. Active Learning in Higher Education 15, 2 (2014), 157--171. https://doi.org/10.1177/1469787414527391 arXiv:https://doi.org/10.1177/1469787414527391

[59]

Satoshi Nakamoto. 2008. Bitcoin: A peer-to-peer electronic cash system. Decentralized Business Review (2008), 21260.

[60]

Tricia J. Ngoon, Rachel Chen, Amit Deutsch, and Sean Lip. 2016. Oppia: A Community of Peer Learners to Make Conversational Learning Experiences. In Proceedings of the 19th ACM Conference on Computer Supported Cooperative Work and Social Computing Companion (San Francisco, California, USA) (CSCW '16 Companion). Association for Computing Machinery, New York, NY, USA, 73--76. https://doi.org/10.1145/2818052.2874328

Digital Library

[61]

Shunsuke Noguchi and Naomi Fujimura. 2015. Implementation and Experience of the Online Peer Grading System for Our Real Class. In Proceedings of the 2015 ACM SIGUCCS Annual Conference (St. Petersburg, Florida, USA) (SIGUCCS '15). Association for Computing Machinery, New York, NY, USA, 125--128. https://doi.org/10.1145/2815546.2815581

Digital Library

[62]

Joonsuk Park and Kimberley Williams. 2016. The Effects of Peer- and Self-Assessment on the Assessors. In Proceedings of the 47th ACM Technical Symposium on Computing Science Education (Memphis, Tennessee, USA) (SIGCSE '16). Association for Computing Machinery, New York, NY, USA, 249--254. https://doi.org/10.1145/2839509.2844602

Digital Library

[63]

Meghan Perdue and Jessica Sandland. 2022. Evaluating the impact of transparency on peer review quality in MOOCs. In 2022 IEEE Learning with MOOCS (LWMOOCS). 101--107. https://doi.org/10.1109/LWMOOCS53067.2022.9928020

[64]

Chris Phielix, Frans J. Prins, and Paul A. Kirschner. 2010. Awareness of group performance in a CSCL-environment: Effects of peer feedback and reflection. Computers in Human Behavior 26, 2 (2010), 151--161. https://doi.org/10.1016/j.chb.2009.10.011

Digital Library

[65]

Chris Piech, Jonathan Huang, Zhenghao Chen, Chuong Do, Andrew Ng, and Daphne Koller. 2013. Tuned models of peer assessment in MOOCs. arXiv preprint arXiv:1307.2579 (2013).

[66]

Aleksandra Prikhodko and Elvir Akhmetshin. 2022. Peer-to-Peer Grading as a Way to Improve Educational Process Efficiency for Certain Industrial Sectors under Digitalization of Education at Universities. In IV International Scientific and Practical Conference (St.Petersburg, Russian Federation) (DEFIN-2021). Association for Computing Machinery, New York, NY, USA, Article 22, 5 pages. https://doi.org/10.1145/3487757.3490860

Digital Library

[67]

Frans J. Prins, Dominique M. A. Sluijsmans, Paul A. Kirschner, and Jan-Willem Strijbos. 2005. Formative peer assessment in a CSCL environment: a case study. Assessment & Evaluation in Higher Education 30, 4 (2005), 417--444. https://doi.org/10.1080/02602930500099219 arXiv:https://doi.org/10.1080/02602930500099219

[68]

Ricardo Raimundo and Albérico Rosário. 2021. Blockchain system in the higher education. European Journal of Investigation in Health, Psychology and Education 11, 1 (2021), 276--293.

[69]

Ken Reily, Pam Ludford Finnerty, and Loren Terveen. 2009. Two Peers Are Better than One: Aggregating Peer Reviews for Computing Assignments is Surprisingly Accurate. In Proceedings of the ACM 2009 International Conference on Supporting Group Work (Sanibel Island, Florida, USA) (GROUP '09). Association for Computing Machinery, New York, NY, USA, 115--124. https://doi.org/10.1145/1531674.1531692

Digital Library

[70]

Jennifer M Robinson. 2002. In search of fairness: An application of multi-reviewer anonymous peer review in a large class. Journal of Further and Higher Education 26, 2 (2002), 183--192.

[71]

Philip M Sadler and Eddie Good. 2006. The impact of self-and peer-grading on student learning. Educational assessment 11, 1 (2006), 1--31.

[72]

Mike Sharples and John Domingue. 2016. The blockchain and kudos: A distributed system for educational record, reputation and reward. In European conference on technology enhanced learning. Springer, 490--496.

[73]

Jirarat Sitthiworachart and Mike Joy. 2004. Effective Peer Assessment for Learning Computer Programming. In Proceedings of the 9th Annual SIGCSE Conference on Innovation and Technology in Computer Science Education (Leeds, United Kingdom) (ITiCSE '04). Association for Computing Machinery, New York, NY, USA, 122--126. https://doi.org/10.1145/1007996.1008030

Digital Library

[74]

Holly Smith, Ali Cooper, and Les Lancaster. 2002. Improving the Quality of Undergraduate Peer Assessment: A Case for Student and Staff Development. Innovations in Education and Teaching International 39, 1 (2002), 71--81. https://doi.org/10.1080/13558000110102904 arXiv:https://doi.org/10.1080/13558000110102904

[75]

Harald Sondergaard. 2009. Learning from and with Peers: The Different Roles of Student Peer Reviewing. In Proceedings of the 14th Annual ACM SIGCSE Conference on Innovation and Technology in Computer Science Education (Paris, France) (ITiCSE '09). Association for Computing Machinery, New York, NY, USA, 31--35. https://doi.org/10.1145/1562877.1562893

Digital Library

[76]

Remya Stephen and Aneena Alex. 2018. A review on blockchain security. In IOP Conference Series: Materials Science and Engineering, Vol. 396. IOP Publishing, 012030.

[77]

Brent Strong, Mark Davis, and Val Hawks. 2004. Self-Grading In Large General Education Classes: A Case Study. College Teaching 52, 2 (2004), 52--57. https://doi.org/10.3200/CTCH.52.2.52--57 arXiv:https://doi.org/10.3200/CTCH.52.2.52--57

[78]

Xiaotian Su, Thiemo Wambsganss, Roman Rietsche, Seyed Parsa Neshaei, and Tanja Käser. 2023. Reviewriter: AI-Generated Instructions For Peer Review Writing. In Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023). Association for Computational Linguistics, Toronto, Canada, 57--71. https://aclanthology.org/2023.bea-1.5

[79]

Sarah L. Sullivan. 1994. Reciprocal Peer Reviews. In Proceedings of the Twenty-Fifth SIGCSE Symposium on Computer Science Education (Phoenix, Arizona, USA) (SIGCSE '94). Association for Computing Machinery, New York, NY, USA, 314--318. https://doi.org/10.1145/191029.191158

Digital Library

[80]

Huang-Chih Sung. 2018. When open source software encounters patents: blockchain as an example to explore the dilemma and solutions. J. Marshall Rev. Intell. Prop. L. 18 (2018), v.

[81]

Atima Tharatipyakul and Suporn Pongnumkul. 2021. User Interface of Blockchain-Based Agri-Food Traceability Applications: A Review. IEEE Access 9 (2021), 82909--82929. https://doi.org/10.1109/ACCESS.2021.3085982

[82]

Daniel Toll and Anna Wingkvist. 2017. How Tool Support and Peer Scoring Improved Our Students' Attitudes Toward Peer Reviews. In Proceedings of the 2017 ACM Conference on Innovation and Technology in Computer Science Education (Bologna, Italy) (ITiCSE '17). Association for Computing Machinery, New York, NY, USA, 311--316. https://doi.org/10.1145/3059009.3059059

Digital Library

[83]

Keith Topping. 1998. Peer Assessment Between Students in Colleges and Universities. Review of Educational Research 68, 3 (1998), 249--276. https://doi.org/10.3102/00346543068003249 arXiv:https://doi.org/10.3102/00346543068003249

[84]

A. Trivedi, Dulal C. Kar, and Holly Patterson-McNeill. 2003. Automatic Assignment Management and Peer Evaluation. J. Comput. Sci. Coll. 18, 4 (apr 2003), 30--37.

[85]

Tom Tullis, Stan Fleischman, Michelle McNulty, Carrie Cianchette, and Margaret Bergel. 2002. An empirical comparison of lab and remote usability testing of web sites. In Usability Professionals Association Conference.

[86]

Scott Turner, Manuel A. Pérez-Quiñones, Stephen Edwards, and Joseph Chase. 2011. Student Attitudes and Motivation for Peer Review in CS2. In Proceedings of the 42nd ACM Technical Symposium on Computer Science Education (Dallas, TX, USA) (SIGCSE '11). Association for Computing Machinery, New York, NY, USA, 347--352. https://doi.org/10.1145/1953163.1953268

Digital Library

[87]

Sarah Underwood. 2016. Blockchain beyond bitcoin. Commun. ACM 59, 11 (2016), 15--17.

Digital Library

[88]

I. Van den Berg, Wilfried Admiraal, and Albert Pilot. 2006. Design principles and outcomes of peer assessment in higher education. Studies in Higher Education 31 (06 2006), 341--356. https://doi.org/10.1080/03075070600680836

[89]

P Vinothiyalakshmi, C Muralidharan, Y Mohamed Sirajudeen, and R Anitha. 2022. Digitized Land Registration Using Blockchain Technology. In Blockchain Technology. CRC Press, 73--86.

[90]

Tim Vogelsang and Lara Ruppertz. 2015. On the Validity of Peer Grading and a Cloud Teaching Assistant System. In Proceedings of the Fifth International Conference on Learning Analytics And Knowledge (Poughkeepsie, New York) (LAK 15). Association for Computing Machinery, New York, NY, USA, 41--50. https://doi.org/10.1145/2723576.2723633

Digital Library

[91]

Thiemo Wambsganss, Xiaotian Su, Vinitra Swamy, Seyed Parsa Neshaei, Roman Rietsche, and Tanja Käser. 2023. Unraveling Downstream Gender Bias from Large Language Models: A Study on AI Educational Writing Assistance. In Findings of the Association for Computational Linguistics: EMNLP 2023, Houda Bouamor, Juan Pino, and Kalika Bali (Eds.). Association for Computational Linguistics, Singapore, 10275--10288. https://doi.org/10.18653/v1/2023.findings-emnlp.689

[92]

Huaqun Wang, Debiao He, Zhe Liu, and Rui Guo. 2019. Blockchain-based anonymous reporting scheme with anonymous rewarding. IEEE Transactions on Engineering Management 67, 4 (2019), 1514--1524.

[93]

Bin Wu and Yinsheng Li. 2018. Design of evaluation system for digital education operational skill competition based on blockchain. In 2018 IEEE 15th international conference on e-business engineering (ICEBE). IEEE, 102--109.

[94]

Junfeng Xie, F. Richard Yu, Tao Huang, Renchao Xie, Jiang Liu, and Yunjie Liu. 2019. A Survey on the Scalability of Blockchain Systems. IEEE Network 33, 5 (2019), 166--173. https://doi.org/10.1109/MNET.001.1800290

Digital Library

[95]

Nur Syafiqah Yaccob, Melor Md Yunus, and Harwati Hashim. 2022. The Integration of Global Competence Into Malaysian English as a Second Language Lessons for Quality Education (Fourth United Nations Sustainable Development Goal). Frontiers in Psychology 13 (2022).

[96]

Su-Fang Yeh, Meng-Hsin Wu, Tze-Yu Chen, Yen-Chun Lin, XiJing Chang, You-Hsuan Chiang, and Yung-Ju Chang. 2022. How to Guide Task-Oriented Chatbot Users, and When: A Mixed-Methods Study of Combinations of Chatbot Guidance Types and Timings. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI '22). Association for Computing Machinery, New York, NY, USA, Article 488, 16 pages. https://doi.org/10.1145/3491102.3501941

Digital Library

[97]

Willie Yip. 2004. Web-Based Support for Peer Tutoring. In Proceedings of the IEEE International Conference on Advanced Learning Technologies (ICALT '04). IEEE Computer Society, USA, 619--623.

[98]

Ahmed Mohamed Fahmy Yousef, Usman Wahid, Mohamed Amine Chatti, Ulrik Schroeder, and Marold Wosnitza. 2015. The Effect of Peer Assessment Rubrics on Learners' Satisfaction and Performance Within a Blended MOOC Environment. In Proceedings of the 7th International Conference on Computer Supported Education - Volume 2 (Lisbon, Portugal) (CSEDU 2015). SCITEPRESS - Science and Technology Publications, Lda, Setubal, PRT, 148--159. https://doi.org/10.5220/0005495501480159

Digital Library

[99]

Javad Zarrin, Hao Wen Phang, Lakshmi Babu Saheer, and Bahram Zarrin. 2021. Blockchain for decentralization of internet: prospects, trends, and challenges. Cluster Computing 24, 4 (2021), 2841--2866.

Digital Library

[100]

Li Zhang and Jianbo Xu. 2022. Blockchain-based anonymous authentication for traffic reporting in VANETs. Connection Science 34, 1 (2022), 1038--1065.

[101]

Jiemin Zhong, Haoran Xie, Di Zou, and Dickson KW Chui. 2018. A blockchain model for word-learning systems. In 2018 5th international conference on behavioral, economic, and socio-cultural computing (BESC). IEEE, 130--131.

Cited By

Morales-Alarcón CBodero-Poveda EVilla-Yánez HBuñay-Guisñan P(2024)Blockchain and Its Application in the Peer Review of Scientific Works: A Systematic ReviewPublications10.3390/publications1204004012:4(40)Online publication date: 8-Nov-2024
https://doi.org/10.3390/publications12040040

Index Terms

Improving Grading Fairness and Transparency with Decentralized Collaborative Peer Assessment
1. Human-centered computing
  1. Collaborative and social computing
    1. Collaborative and social computing systems and tools

Recommendations

How Tool Support and Peer Scoring Improved Our Students' Attitudes Toward Peer Reviews
ITiCSE '17: Proceedings of the 2017 ACM Conference on Innovation and Technology in Computer Science Education

We wanted to introduce peer reviews for the final report in a course on Software Testing. The students had however experienced issues with peer reviews in a previous course which made this a challenge. To get a better understanding of the situation, we ...
Peer Grading Without Protest: The SPARK Approach to Summative Peer Assessment
SIGCSE 2022: Proceedings of the 53rd ACM Technical Symposium on Computer Science Education - Volume 1

As Computer Science Professors, we strive to construct courses that maximally support and contribute to student learning through carefully crafted in-class and out-of-class activities. There is evidence that homework enhances student learning, and that ...
On the validity of peer grading and a cloud teaching assistant system
LAK '15: Proceedings of the Fifth International Conference on Learning Analytics And Knowledge

We introduce a new grading system, the Cloud Teaching Assistant System (CTAS), as an additional element to instructor grading, peer grading and automated validation in massive open online courses (MOOCs). The grading distributions of the different ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Human-Computer Interaction

Proceedings of the ACM on Human-Computer Interaction Volume 8, Issue CSCW1

CSCW

April 2024

6294 pages

EISSN:2573-0142

DOI:10.1145/3661497

Editor:
Jeff Nichols
Apple Inc., United States

Issue’s Table of Contents

Copyright © 2024 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 April 2024

Published in PACMHCI Volume 8, Issue CSCW1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
83
Total Downloads

Downloads (Last 12 months)83
Downloads (Last 6 weeks)12

Reflects downloads up to 21 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Morales-Alarcón CBodero-Poveda EVilla-Yánez HBuñay-Guisñan P(2024)Blockchain and Its Application in the Peer Review of Scientific Works: A Systematic ReviewPublications10.3390/publications1204004012:4(40)Online publication date: 8-Nov-2024
https://doi.org/10.3390/publications12040040

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents