research-article

The FCS1: a language independent assessment of CS1 knowledge

Authors:

Allison Elliott Tew,

Mark GuzdialAuthors Info & Claims

SIGCSE '11: Proceedings of the 42nd ACM technical symposium on Computer science education

Pages 111 - 116

https://doi.org/10.1145/1953163.1953200

Published: 09 March 2011 Publication History

Get Access

Abstract

A primary goal of many CS education projects is to determine the extent to which a given intervention has had an impact on student learning. However, computing lacks valid assessments for pedagogical or research purposes. Without such valid assessments, it is difficult to accurately measure student learning or establish a relationship between the instructional setting and learning outcomes.

We developed the Foundational CS1 (FCS1) Assessment instrument, the first assessment instrument for introductory computer science concepts that is applicable across a variety of current pedagogies and programming languages. We applied methods from educational and psychological test development, adapting them as necessary to fit the disciplinary context. We conducted a large scale empirical study to demonstrate that pseudo-code was an appropriate mechanism for achieving programming language independence. Finally, we established the validity of the assessment using a multi-faceted argument, combining interview data, statistical analysis of results on the assessment, and CS1 exam scores.

References

[1]

American Educational Research Association, American Psychological Association, and National Council on Measurement in Education. Standards for educational and psychological testing. American Educational Research Association, Washington, DC, 1999.

Google Scholar

[2]

J. Cohen. Statistical Power Analysis for the Behavioral Sciences. Lawrence Erlbaum Associates, Hillsdale, NJ, 2nd edition, 1988.

Google Scholar

[3]

A. M. Decker. How Students Measure Up: An Assessment Instrument for Introductory Computer Science. PhD thesis, University at Bu ffalo (SUNY), Buff alo, NY, 2007.

Google Scholar

[4]

K. Goldman, P. Gross, C. Heeren, G. Herman, L. Kaczmarczyk, M. C. Loui, and C. Zilles. Identifying important and difficult concepts in introductory computing courses using a Delphi process. In SIGCSE '08: Proceedings of the 39th ACM Technical Symposium on Computer Science Education, pages 256--260, 2008.

Digital Library

Google Scholar

[5]

R. K. Hambleton, H. Swaminathan, and H. J. Rogers. Fundamentals of item response theory. Sage Publications, Newbury Park, CA, 1991.

Google Scholar

[6]

D. Hestenes, M. Wells, and G. Swackhamer. Force concept inventory. The Physics Teacher, 30:141--158, March 1992.

Crossref

Google Scholar

[7]

L. C. Kaczmarczyk, E. R. Petrick, J. P. East, and G. L. Herman. Identifying student misconceptions of programming. In SIGCSE '10: Proceedings of the 41st ACM Technical Symposium on Computer Science Education, 2010.

Digital Library

Google Scholar

[8]

M. T. Kane. Validation. In R. L. Brennen, editor, Educational Measurement, pages 17--64. American Council on Education/Praeger Publishers, Westport, CT, 4th edition, 2006.

Google Scholar

[9]

J. C. Libarkin and S. Anderson. Assessment of learning in entry-level geoscience courses: Results from the geoscience concept inventory. Journal of Geoscience Education, 53:394--401, 2005.

Crossref

Google Scholar

[10]

M. D. Miller, R. L. Linn, and N. E. Gronlund. Validity. In Measurement and assessment in teaching, pages 80--99. Pearson Education, Upper Saddle River, NJ, 10th edition, 2009.

Google Scholar

[11]

P. A. Moss, B. J. Girard, and L. C. Haniford. Validity in Educational Assessment. Review of Research in Education, 30(1):109--162, 2006.

Google Scholar

[12]

A. E. Tew. Assessing fundamental introductory computing concept knowledge in a language independent manner. PhD thesis, Georgia Institute of Technology, Atlanta, GA, 2010.

Digital Library

Google Scholar

[13]

A. E. Tew and M. Guzdial. Developing a validated assessment of fundamental CS1 concepts. In SIGCSE '10: Proceedings of the 41st ACM Technical Symposium on Computer Science education, 2010.

Digital Library

Google Scholar

Cited By

View all

Lu KKrishnamurthi S(2024)Identifying and Correcting Programming Language Behavior MisconceptionsProceedings of the ACM on Programming Languages10.1145/36498238:OOPSLA1(334-361)Online publication date: 29-Apr-2024
https://dl.acm.org/doi/10.1145/3649823
Ali MRao PMai YXie B(2024)Using Benchmarking Infrastructure to Evaluate LLM Performance on CS Concept Inventories: Challenges, Opportunities, and CritiquesProceedings of the 2024 ACM Conference on International Computing Education Research - Volume 110.1145/3632620.3671097(452-468)Online publication date: 12-Aug-2024
https://dl.acm.org/doi/10.1145/3632620.3671097
Nitta CEiselt KStephenson BStone JBattestilli LRebelsky SShoop L(2024)The First Five Years of a Dual Track Programming SeriesProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 110.1145/3626252.3630947(965-971)Online publication date: 7-Mar-2024
https://dl.acm.org/doi/10.1145/3626252.3630947
Show More Cited By

Index Terms

The FCS1: a language independent assessment of CS1 knowledge
1. Social and professional topics
  1. Professional topics
    1. Computing education
      1. Computing education programs
        Computer science education
      2. Model curricula

Recommendations

Uses, Revisions, and the Future of Validated Assessments in Computing Education: A Case Study of the FCS1 and SCS1
ICER 2021: Proceedings of the 17th ACM Conference on International Computing Education Research

Validated instruments of knowledge are important for creating an accepted and shared measurement for the most important part of education – student learning. The Force Concept of Inventory from Physics Education Research has had a significant impact ...
Replication, Validation, and Use of a Language Independent CS1 Knowledge Assessment
ICER '16: Proceedings of the 2016 ACM Conference on International Computing Education Research

Computing education lags other discipline-based education research in the number and range of validated assessments available to the research community. Validated assessments are important for researchers to reduce experimental error due to flawed ...
Developing a validated assessment of fundamental CS1 concepts
SIGCSE '10: Proceedings of the 41st ACM technical symposium on Computer science education

Previous studies of student programming ability have raised questions about students' ability to problem solve, read and analyze code, and understand introductory computing concepts. However, it is unclear whether these results are the product of ...

Comments

Information & Contributors

Information

Published In

SIGCSE '11: Proceedings of the 42nd ACM technical symposium on Computer science education

March 2011

754 pages

ISBN:9781450305006

DOI:10.1145/1953163

General Chairs:
Thomas J. Cortina
Carnegie Mellon University
,
Ellen L. Walker
Hiram College
,
Program Chairs:
Laurie Smith King
College of the Holy Cross
,
David R. Musicant
Carleton College

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 March 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SIGCSE '11

Sponsor:

SIGCSE

SIGCSE '11: The 42nd ACM Technical Symposium on Computer Science Education

March 9 - 12, 2011

TX, Dallas, USA

Acceptance Rates

SIGCSE '11 Paper Acceptance Rate 107 of 315 submissions, 34%;

Overall Acceptance Rate 1,595 of 4,542 submissions, 35%

Upcoming Conference

SIGCSE TS 2025

Sponsor:
sigcse

The 56th ACM Technical Symposium on Computer Science Education

February 26 - March 1, 2025

Pittsburgh , PA , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
1,578
Total Downloads

Downloads (Last 12 months)89
Downloads (Last 6 weeks)14

Reflects downloads up to 23 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Lu KKrishnamurthi S(2024)Identifying and Correcting Programming Language Behavior MisconceptionsProceedings of the ACM on Programming Languages10.1145/36498238:OOPSLA1(334-361)Online publication date: 29-Apr-2024
https://dl.acm.org/doi/10.1145/3649823
Ali MRao PMai YXie B(2024)Using Benchmarking Infrastructure to Evaluate LLM Performance on CS Concept Inventories: Challenges, Opportunities, and CritiquesProceedings of the 2024 ACM Conference on International Computing Education Research - Volume 110.1145/3632620.3671097(452-468)Online publication date: 12-Aug-2024
https://dl.acm.org/doi/10.1145/3632620.3671097
Nitta CEiselt KStephenson BStone JBattestilli LRebelsky SShoop L(2024)The First Five Years of a Dual Track Programming SeriesProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 110.1145/3626252.3630947(965-971)Online publication date: 7-Mar-2024
https://dl.acm.org/doi/10.1145/3626252.3630947
Bahr T(2023)Developing a Computer Science Content Knowledge Test for 10th Grade StudentsProceedings of the 18th WiPSCE Conference on Primary and Secondary Computing Education Research10.1145/3605468.3609778(1-2)Online publication date: 27-Sep-2023
https://dl.acm.org/doi/10.1145/3605468.3609778
Jain ABockmon RBourke CCooper SChoppella VPhatak DLuxton-Reilly ACraig M(2023)Validating a Language-Independent CS1 Learning Outcomes AssessmentProceedings of the ACM Conference on Global Computing Education Vol 110.1145/3576882.3617910(78-83)Online publication date: 5-Dec-2023
https://dl.acm.org/doi/10.1145/3576882.3617910
Ali MGhosh SRao PDhegaskar RJawort SMedler AShi MDasgupta S(2023)Taking Stock of Concept Inventories in Computing Education: A Systematic Literature ReviewProceedings of the 2023 ACM Conference on International Computing Education Research - Volume 110.1145/3568813.3600120(397-415)Online publication date: 7-Aug-2023
https://dl.acm.org/doi/10.1145/3568813.3600120
Lapierre HCharland PLéger P(2023)Looking “Under the hood” of learning computer programming: the emotional and cognitive differences between novices and beginnersComputer Science Education10.1080/08993408.2023.221403334:3(331-352)Online publication date: 25-May-2023
https://doi.org/10.1080/08993408.2023.2214033
Layman LSong YGuinn CChang MLo DGamess E(2020)Toward Predicting Success and Failure in CS2Proceedings of the 2020 ACM Southeast Conference10.1145/3374135.3385277(218-225)Online publication date: 2-Apr-2020
https://dl.acm.org/doi/10.1145/3374135.3385277
Kennedy CMalmi LKorhonen AMcCartney RPetersen AMalmi LMcCartney R(2018)What are they thinking?Proceedings of the 2018 ACM Conference on International Computing Education Research10.1145/3230977.3231023(272-273)Online publication date: 8-Aug-2018
https://dl.acm.org/doi/10.1145/3230977.3231023
Nelson GKo AMalmi LKorhonen AMcCartney RPetersen AMalmi LMcCartney R(2018)On Use of Theory in Computing Education ResearchProceedings of the 2018 ACM Conference on International Computing Education Research10.1145/3230977.3230992(31-39)Online publication date: 8-Aug-2018
https://dl.acm.org/doi/10.1145/3230977.3230992

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Uses, Revisions, and the Future of Validated Assessments in Computing Education: A Case Study of the FCS1 and SCS1

Replication, Validation, and Use of a Language Independent CS1 Knowledge Assessment

Developing a validated assessment of fundamental CS1 concepts