research-article

Self-evaluation in advanced power searching and mapping with google MOOCs

Authors:

Julia Wilkowski,

Daniel M. Russell,

Amit DeutschAuthors Info & Claims

L@S '14: Proceedings of the first ACM conference on Learning @ scale conference

Pages 109 - 116

https://doi.org/10.1145/2556325.2566241

Published: 04 March 2014 Publication History

Get Access

Abstract

While there is a large amount of work on creating autograded massive open online courses (MOOCs), some kinds of complex, qualitative exam questions are still beyond the current state of the art. For MOOCs that need to deal with these kinds of questions, it is not possible for a small course staff to grade students' qualitative work. To test the efficacy of self-evaluation as a method for complex-question evaluation, students in two Google MOOCs have submitted projects and evaluated their own work. For both courses, teaching assistants graded a random sample of papers and compared their grades with self-evaluated student grades. We found that many of the submitted projects were of very high quality, and that a large majority of self-evaluated projects were accurately evaluated, scoring within just a few points of the gold standard grading.

References

[1]

Balfour, S.P. 2013. Assessing writing in MOOCS: Automated essay scoring and Calibrated Peer Review. Research & Practice in Assessment 8.1 (2013): 40--48.

Google Scholar

[2]

Black, P. & Wiliam, D. 1998. Assessment and classroom learning. Assessment in Education: Principles, Policy & Practice, 5(1), 7.

Crossref

Google Scholar

[3]

Eslinger, E., White, B., Frederiksen, J., & Brobst, J. 2008. Supporting Inquiry Processes with an Interactive Learning Environment: Inquiry Island. Journal of Science Education and Technology, 17(6), 610--617.

Crossref

Google Scholar

[4]

Kulkarni, C., and S. R. Klemmer. 2012. Learning design wisdom by augmenting physical studio critique with online self-assessment. Technical report, Stanford University.

Google Scholar

[5]

Kulkarni, C., Wei, K. P., Le, H., Chia, D., Papadopoulos, K., Koller, D., Klemmer, S.R. 2013. Scaling self and peer assessment to the global design classroom. Transactions on Computer-Human Interactions Journal, 20(6) in publication.

Digital Library

Google Scholar

[6]

Moskal, B. M. 2000. Scoring rubrics: what, when and how?. Practical Assessment, Research & Evaluation, 7(3).

Google Scholar

[7]

Piech, C., Huang, J., Chen, Z., Do, C., Ng, A., & Koller, D. (n.d.). Tuned Models of Peer Assessment in MOOCs. Retrieved (11/7/13) http://www.stanford.edu/~cpiech/bio/papers/tuningPeerGrading.pdf

Google Scholar

[8]

Rivers, W. P. 200 Autonomy at All Costs: An Ethnography of Metacognitive Self--Assessment and Self--Management among Experienced Language Learners. The modern language journal 85.2 (2001): 279--290.

Google Scholar

[9]

Sadler, P., & Good, E. 2006. The Impact of Self- and Peer-Grading on Student Learning. Educational Assessment, 11(1), 1--31.

Crossref

Google Scholar

[10]

Veenman, M., Van Hout-Wolters, B., & Affleerbach. P. 2006. Metacognition and learning: conceptual and methodological considerations. Metacognition and Learning, 1(1):3, (Apr 14, 2006).

Google Scholar

Cited By

View all

Duan TWu B(2023)The student self-assessment paradigm in MOOC: An example in Chinese higher educationComunicar10.3916/C75-2023-0931:75(115-128)Online publication date: 1-Apr-2023
https://doi.org/10.3916/C75-2023-09
Jonson Carlon MGayed JCross J(2021)Development of Open-Response Prompt-Based Metacognitive Tutor for Online Classrooms2021 IEEE International Conference on Engineering, Technology & Education (TALE)10.1109/TALE52509.2021.9678573(01-08)Online publication date: 5-Dec-2021
https://doi.org/10.1109/TALE52509.2021.9678573
Boudria ALafifi YBordjiba Y(2020)Collaborative Calibrated Peer Assessment in Massive Open Online CoursesLearning and Performance Assessment10.4018/978-1-7998-0420-8.ch065(1408-1434)Online publication date: 2020
https://doi.org/10.4018/978-1-7998-0420-8.ch065
Show More Cited By

Index Terms

Self-evaluation in advanced power searching and mapping with google MOOCs
1. Applied computing
  1. Education

Recommendations

Student skill and goal achievement in the mapping with google MOOC
L@S '14: Proceedings of the first ACM conference on Learning @ scale conference

Students who registered for the Mapping with Google massive open online course (MOOC) were asked several questions during the registration process to identify prior experience with eleven skills as well as their goals for registering for the course. ...
Teaching FEM software in formal and non-formal environment with MOOCs
TEEM'18: Proceedings of the Sixth International Conference on Technological Ecosystems for Enhancing Multiculturality

This paper presents a real case of using a MOOC course into an obligatory subject in a Master level with face-to-face lessons as a supplementary (and optional) material. The case presented is related to a MOOC course developed between April and May of ...
MOOCs and their impact on academics
Koli Calling '14: Proceedings of the 14th Koli Calling International Conference on Computing Education Research

Massive Open Online Courses (MOOCs) have recently become a hot topic in the academic world, launching a wide ranging discussion on a number of issues. In this research, we surveyed academics' awareness, attitudes, perceptions, and experiences of MOOCs. ...

Comments

Information & Contributors

Information

Published In

L@S '14: Proceedings of the first ACM conference on Learning @ scale conference

March 2014

234 pages

ISBN:9781450326698

DOI:10.1145/2556325

General Chair:
Mehran Sahami
Stanford University, USA
,
Program Chairs:
Armando Fox
University of California, Berkeley, USA
,
Marti A. Hearst
University of California, Berkeley, USA
,
Michelene T.H. Chi
Arizona State University, USA

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 March 2014

Check for updates

Author Tags

Qualifiers

Research-article

Conference

L@S 2014

Sponsor:

ACM Ed Board

L@S 2014: First (2014) ACM Conference on Learning @ Scale

March 4 - 5, 2014

Georgia, Atlanta, USA

Acceptance Rates

L@S '14 Paper Acceptance Rate 14 of 38 submissions, 37%;

Overall Acceptance Rate 117 of 440 submissions, 27%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

12
Total Citations
View Citations
353
Total Downloads

Downloads (Last 12 months)8
Downloads (Last 6 weeks)2

Reflects downloads up to 21 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Duan TWu B(2023)The student self-assessment paradigm in MOOC: An example in Chinese higher educationComunicar10.3916/C75-2023-0931:75(115-128)Online publication date: 1-Apr-2023
https://doi.org/10.3916/C75-2023-09
Jonson Carlon MGayed JCross J(2021)Development of Open-Response Prompt-Based Metacognitive Tutor for Online Classrooms2021 IEEE International Conference on Engineering, Technology & Education (TALE)10.1109/TALE52509.2021.9678573(01-08)Online publication date: 5-Dec-2021
https://doi.org/10.1109/TALE52509.2021.9678573
Boudria ALafifi YBordjiba Y(2020)Collaborative Calibrated Peer Assessment in Massive Open Online CoursesLearning and Performance Assessment10.4018/978-1-7998-0420-8.ch065(1408-1434)Online publication date: 2020
https://doi.org/10.4018/978-1-7998-0420-8.ch065
Kulkarni C(2019)Design Perspectives of Learning at ScaleProceedings of the Sixth (2019) ACM Conference on Learning @ Scale10.1145/3330430.3333620(1-11)Online publication date: 24-Jun-2019
https://dl.acm.org/doi/10.1145/3330430.3333620
Lafifi YBoudria ABordjiba Y(2018)Collaborative Calibrated Peer Assessment in Massive Open Online CoursesInternational Journal of Distance Education Technologies10.4018/IJDET.201801010516:1(76-102)Online publication date: 1-Jan-2018
https://dl.acm.org/doi/10.4018/IJDET.2018010105
Kross SGuo PLuckin RKlemmer SKoedinger K(2018)Students, systems, and interactionsProceedings of the Fifth Annual ACM Conference on Learning at Scale10.1145/3231644.3231662(1-10)Online publication date: 26-Jun-2018
https://dl.acm.org/doi/10.1145/3231644.3231662
Lepp MLuik PPalts TPapli KSuviste RSäde MHollo KVaherpuu VTõnisson E(2017)Self- and Automated Assessment in Programming MOOCsTechnology Enhanced Assessment10.1007/978-3-319-57744-9_7(72-85)Online publication date: 13-May-2017
https://doi.org/10.1007/978-3-319-57744-9_7
Yang SDomeniconi CRevelle MSweeney MGelman BBeckley CJohri AKiczales GRussell DWoolf B(2015)Uncovering Trajectories of Informal Learning in Large Online Communities of CreatorsProceedings of the Second (2015) ACM Conference on Learning @ Scale10.1145/2724660.2724674(131-140)Online publication date: 14-Mar-2015
https://dl.acm.org/doi/10.1145/2724660.2724674
Coetzee DLim SFox AHartmann BHearst MCosley DForte ACiolfi LMcDonald D(2015)Structuring Interactions for Large-Scale Synchronous Peer LearningProceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing10.1145/2675133.2675251(1139-1152)Online publication date: 28-Feb-2015
https://dl.acm.org/doi/10.1145/2675133.2675251
Papathoma TBlake CClow DScanlon E(2015)Investigating Learners’ Views of Assessment Types in Massive Open Online Courses (MOOCs)Design for Teaching and Learning in a Networked World10.1007/978-3-319-24258-3_72(617-621)Online publication date: 15-Sep-2015
https://dl.acm.org/doi/10.1007/978-3-319-24258-3_72
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Student skill and goal achievement in the mapping with google MOOC

Teaching FEM software in formal and non-formal environment with MOOCs

MOOCs and their impact on academics