research-article

Embedded questions in ebooks on programming: useful for a) summative assessment, b) formative assessment, or c) something else?

Authors:

Teemu SirkiäAuthors Info & Claims

Koli Calling '15: Proceedings of the 15th Koli Calling Conference on Computing Education Research

Pages 152 - 156

https://doi.org/10.1145/2828959.2828961

Published: 19 November 2015 Publication History

Abstract

We discuss uses of embedded computing questions (ECQs) in interactive electronic textbooks on programming, identifying a non-exhaustive list of three main categories of ECQs and nine subcategories. The main categories are: ECQs that introduce content, ECQs that reinforce learning, and ECQs that highlight content. We provide examples from an existing ebook, discuss how student perceptions may pose challenges to the use of ECQs, and invite the research community to debate ECQs and investigate them empirically.

References

[1]

I. Aedo, P. Díaz, C. Fernández, G. Muñoz Martín, and A. Berlanga. Assessing the utility of an interactive electronic book for learning the PASCAL programming language. IEEE Transactions on Education, 43(4):403--413, 2000.

Digital Library

[2]

L. W. Anderson, D. R. Krathwohl, P. W. Airasian, K. A. Cruikshank, R. E. Mayer, P. R. Pintrich, J. Raths, and M. C. Wittrock. A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives. Longman, 2001.

[3]

J. Biggs and C. Tang. Teaching for Quality Learning at University. McGraw-Hill, 3rd edition, 2007.

[4]

M. Birenbaum and R. A. Feldman. Relationships between learning patterns and attitudes towards two assessment formats. Educational Research, 40(1):90--98, 1998.

[5]

T. Buchanan. The efficacy of a World-Wide Web mediated formative assessment. Journal of Computer Assisted Learning, 16:193--200, 2000.

[6]

P. Denny, J. Hamer, A. Luxton-Reilly, and H. Purchase. PeerWise: Students sharing their multiple choice questions. In Proceedings of the Fourth International Workshop on Computing Education Research, ICER '08, pages 51--58. ACM, 2008.

Digital Library

[7]

P. Denny, A. Luxton-Reilly, and B. Simon. Evaluating a new exam question: Parsons problems. In Proceedings of the Fourth International Workshop on Computing Education Research, ICER '08, pages 113--124. ACM, 2008.

Digital Library

[8]

A. S. Donker, H. de Boer, D. Kostons, C. C. Dignath van Ewijk, and M. P. C. van der Werf. Effectiveness of learning strategy instruction on academic performance: A meta-analysis. Educational Research Review, 11:1--26, 2014.

[9]

B. Ericson, M. Guzdial, and B. Morrison. Analysis of interactive features designed to enhance learning in an ebook. In Proceedings of the Eleventh Annual Conference on International Computing Education Research, ICER '15. ACM, 2015.

Digital Library

[10]

J. B. Fenwick Jr, B. L. Kurtz, and P. Meznar. Developing a highly interactive ebook for CS instruction. In Proceedings of the 44th ACM Technical Symposium on Computer Science Education, SIGCSE '13, pages 135--140. ACM, 2013.

Digital Library

[11]

M. Guzdial and B. Ericson. Media Computation Teachers Website. http://www.mediacomputation.org/, 2015.

[12]

O. Hazzan, T. Lapidot, and N. Ragonis. A Guide to Teaching Computer Science. Springer, 2011.

Digital Library

[13]

A. Korhonen, T. L. Naps, C. R. Boisvert, P. Crescenzi, V. Karavirta, L. Mannila, B. N. Miller, B. B. Morrison, S. H. Rodger, R. J. Ross, and C. A. Shaffer. Requirements and design strategies for open source interactive computer science eBooks. In ITiCSE 2013 Working Group Reports, ITiCSE -WGR '13, pages 53--72. ACM, 2013.

Digital Library

[14]

D. Laurillard. Rethinking university teaching: A framework for the effective use of educational technology. Routledge, 2nd edition, 2001.

[15]

M. C. Linn, H.-Y. Chang, J. L. Chiu, Z. H. Zhang, and K. W. McElhaney. Can desirable difficulties overcome deceptive clarity in scientific visualizations? In A. S. Benjamin, editor, Successful Remembering and Successful Forgetting: A Festschrift in Honor of Robert A. Bjork, pages 235--258. Psychology Press, 2011.

[16]

R. Lister. Objectives and objective assessment in CS1. SIGCSE Bulletin, 33(1):292--296, 2001.

Digital Library

[17]

R. Martínez-Unanue, M. Paredes-Velasco, C. Pareja-Flores, J. Urquiza-Fuentes, and J. A. Velázquez-Iturbide. Electronic books for programming education: A review and future prospects. In Proceedings of the 7th Annual Conference on Innovation and Technology in Computer Science Education, ITiCSE '02, pages 34--38. ACM, 2002.

Digital Library

[18]

E. Mazur. Peer Instruction: A User's Manual. Pearson, 2014.

[19]

K. W. McElhaney, H.-Y. Chang, J. L. Chiu, and M. C. Linn. Evidence for effective uses of dynamic visualisations in science curriculum materials. Studies in Science Education, 51(1):49--85, 2015.

[20]

B. N. Miller and D. L. Ranum. Beyond PDF and ePub: Toward an interactive textbook. In Proceedings of the 17th ACM Annual Conference on Innovation and Technology in Computer Science Education, ITICSE '12, pages 150--155. ACM, 2012.

Digital Library

[21]

T. L. Naps, G. Rößling, V. Almstrum, W. Dann, R. Fleischer, C. Hundhausen, A. Korhonen, L. Malmi, M. McNally, S. Rodger, and J. A. Velázquez-Iturbide. Exploring the role of visualization and engagement in computer science education. SIGCSE Bulletin, 35(2):131--152, 2003.

Digital Library

[22]

D. Nicol. E-assessment by design: Using multiple-choice tests to good effect. Journal of Further and Higher Education, 31(1):53--64, 2007.

[23]

L. Porter, C. Bailey-Lee, and B. Simon. Halving fail rates using peer instruction: A study of four computer science courses. In Proceeding of the 44th ACM Technical Symposium on Computer Science Education, SIGCSE '13, pages 177--182. ACM, 2013.

Digital Library

[24]

D. Pritchard and T. Vasiga. CS Circles: An in-browser Python course for beginners. In Proceedings of the 44th ACM Technical Symposium on Computer Science Education, SIGCSE '13, pages 591--596. ACM, 2013.

Digital Library

[25]

T. S. Roberts. The use of multiple choice tests for formative and summative assessment. In D. Tolhurst and S. Mann, editors, Eighth Australasian Computing Education Conference (ACE '06), volume 52 of CRPIT, pages 175--180, 2006.

Digital Library

[26]

G. Rößling, J. Urquiza-Fuentes, J. A. Velázquez-Iturbide, T. Naps, M. S. Hall, V. Karavirta, A. Kerren, C. Leska, A. Moreno, R. Oechsle, and S. H. Rodger. Merging interactive visualizations with hypertextbooks and course management. SIGCSE Bulletin, 38(4):166--181, 2006.

Digital Library

[27]

G. Rößling and T. Vellaramkalayil. A visualization-based computer science hypertextbook prototype. ACM Transactions on Computing Education, 9(2):1--13, 2009.

Digital Library

[28]

K. Sanders, M. Ahmadzadeh, T. Clear, S. H. Edwards, M. Goldweber, C. Johnson, R. Lister, R. McCartney, E. Patitsas, and J. Spacco. The Canterbury QuestionBank: Building a repository of multiple-choice CS1 and CS2 questions. In ITICSE'13 Working Group Reports, ITiCSE WGR '13, pages 33--51, 2013.

Digital Library

[29]

K. Scouller. The influence of assessment method on students' learning approaches: Multiple choice question examination versus assignment essay. Higher Education, 35(4):453--472, 1998.

[30]

C. A. Shaffer, V. Karavirta, A. Korhonen, and T. L. Naps. OpenDSA: Beginning a community active-eBook project. In Proceedings of the 11th Koli Calling International Conference on Computing Education Research, Koli Calling '11, pages 112--117. ACM, 2011.

Digital Library

[31]

T. Sirkiä and J. Sorva. How do students use program visualizations within an interactive ebook? In Proceedings of the Eleventh Annual Conference on International Computing Education Research, ICER '15. ACM, 2015.

Digital Library

[32]

J. Sorva and O. Seppälä. Research-based design of the first weeks of CS1. In Proceedings of the 14th Koli Calling International Conference on Computing Education Research, Koli Calling '14, pages 71--80. ACM, 2014.

Digital Library

[33]

K. Woodford and P. Bancroft. Multiple choice questions not considered harmful. In A. Young and D. Tolhurst, editors, Seventh Australasian Computing Education Conference (ACE '05), volume 42 of CRPIT, pages 109--116, 2005.

Digital Library

[34]

L. Zhang, K. Vanlehn, S. Girard, W. Burleson, M. E. Chavez-Echeagaray, J. Gonzalez-Sanchez, and Y. Hidalgo-Pontet. Evaluation of a meta-tutor for constructing models of dynamic systems. Computers and Education, 75:196--217, 2014.

Cited By

Lehtinen TKoutcheme CHellas ARoychoudhury APaiva AAbreu RStorey MGama KSiegmund J(2024)Let's Ask AI About Their Programs: Exploring ChatGPT's Answers To Program Comprehension QuestionsProceedings of the 46th International Conference on Software Engineering: Software Engineering Education and Training10.1145/3639474.3640058(221-232)Online publication date: 14-Apr-2024
https://dl.acm.org/doi/10.1145/3639474.3640058
Thomas A(2023)A Consolidated Catalogue of Question Types for Programming CoursesICT Education10.1007/978-3-031-48536-7_9(119-133)Online publication date: 29-Nov-2023
https://doi.org/10.1007/978-3-031-48536-7_9
Shanley NMartin FHite NPerez-Quinones MAhlgrim-Delzell LPugalee DHart E(2022)Teaching Programming Online: Design, Facilitation and Assessment Strategies and Recommendations for High School TeachersTechTrends10.1007/s11528-022-00724-x66:3(483-494)Online publication date: 12-Apr-2022
https://doi.org/10.1007/s11528-022-00724-x
Show More Cited By

Index Terms

Embedded questions in ebooks on programming: useful for a) summative assessment, b) formative assessment, or c) something else?
1. Applied computing
  1. Education
    1. E-learning
    2. Interactive learning environments
2. Social and professional topics
  1. Professional topics
    1. Computing education
      1. Computing education programs
        Computer science education
        CS1
      2. Student assessment

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

Koli Calling '15: Proceedings of the 15th Koli Calling Conference on Computing Education Research

November 2015

183 pages

ISBN:9781450340205

DOI:10.1145/2828959

Program Chairs:
Päivi Kinnunen
Aalto University, Finland
,
Judy Sheard
Monash University, Australia

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Aalto University
Univ. of Turin: University of Turin
Monash University, Australia: Monash University, Australia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 November 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

Koli Calling '15

Sponsor:

Univ. of Turin
Monash University, Australia

Koli Calling '15: 15th Koli Calling International Conference on Computing Education Research

November 19 - 22, 2015

Koli, Finland

Acceptance Rates

Koli Calling '15 Paper Acceptance Rate 20 of 45 submissions, 44%;

Overall Acceptance Rate 80 of 182 submissions, 44%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
152
Total Downloads

Downloads (Last 12 months)5
Downloads (Last 6 weeks)0

Reflects downloads up to 01 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Lehtinen TKoutcheme CHellas ARoychoudhury APaiva AAbreu RStorey MGama KSiegmund J(2024)Let's Ask AI About Their Programs: Exploring ChatGPT's Answers To Program Comprehension QuestionsProceedings of the 46th International Conference on Software Engineering: Software Engineering Education and Training10.1145/3639474.3640058(221-232)Online publication date: 14-Apr-2024
https://dl.acm.org/doi/10.1145/3639474.3640058
Thomas A(2023)A Consolidated Catalogue of Question Types for Programming CoursesICT Education10.1007/978-3-031-48536-7_9(119-133)Online publication date: 29-Nov-2023
https://doi.org/10.1007/978-3-031-48536-7_9
Shanley NMartin FHite NPerez-Quinones MAhlgrim-Delzell LPugalee DHart E(2022)Teaching Programming Online: Design, Facilitation and Assessment Strategies and Recommendations for High School TeachersTechTrends10.1007/s11528-022-00724-x66:3(483-494)Online publication date: 12-Apr-2022
https://doi.org/10.1007/s11528-022-00724-x
Heinonen AHellas A(2020)Exploring the Instructional Efficiency of Representation and Engagement in Online Learning MaterialsUnited Kingdom & Ireland Computing Education Research conference.10.1145/3416465.3416470(38-44)Online publication date: 3-Sep-2020
https://dl.acm.org/doi/10.1145/3416465.3416470
Jamil H(2017)Automated Personalized Assessment of Computational Thinking MOOC Assignments2017 IEEE 17th International Conference on Advanced Learning Technologies (ICALT)10.1109/ICALT.2017.147(261-263)Online publication date: Jul-2017
https://doi.org/10.1109/ICALT.2017.147
Sorva JVihavainen A(2016)Break statement consideredACM Inroads10.1145/29500657:3(36-41)Online publication date: 17-Aug-2016
https://dl.acm.org/doi/10.1145/2950065
Baneres D(2016)Towards a Personalized Summative Model Based on Learner's Preferences2016 International Conference on Intelligent Networking and Collaborative Systems (INCoS)10.1109/INCoS.2016.31(339-344)Online publication date: Sep-2016
https://doi.org/10.1109/INCoS.2016.31

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents