Evaluating Exploratory Reading Groups for Supporting Undergraduate Research Pipelines in Computing
Authors: 
David M. Torres-Mendoza, Saba Kheirinejad, Mustafa Ajmal, Ashwin Chembu, 
Dustin Palea, Jim Whitehead, David T. LeeAuthors Info & Claims
David M. Torres-Mendoza, Saba Kheirinejad, Mustafa Ajmal, Ashwin Chembu, Dustin Palea, Jim Whitehead, David T. LeeAuthors Info & ClaimsPages 389 - 405
Abstract
This paper reports on a summative analysis of Exploratory Reading Groups (ERGs), a low time-commitment, relational, student-led reading group program designed to provide students from any background and year with a broad exploration of computing research. Since prior work, the program was institutionalized as a 1-credit course with a greater emphasis on strengthening pipelines into research labs. In analyzing 3 quarters of data from 136 participants, we found diverse indicators of impact. Surprisingly, despite the lightweight nature of the program (∼ 2 hours/week), we observed a statistically significant increase in satisfaction with their intellectual development at the university; confidence in reading, presenting, and communicating about their field; sense of belonging for women and minoritized ethnic groups; alignment with faculty goals in joining research labs (greater desire to make a research contribution and publish, decreased desire to join for the purpose of exploration); and engagement in the ‘reconsideration’ dimension of career identity formation. Over 70% of the participants continued on into group research projects for undergraduate students. The effectiveness of this scalable, lightweight initiative shows the promise of ERGs as a tool to support students in computing when connected to group research projects and points to future research directions on designing other lightweight, relational, scalable learning experiences.
References
[1]
2024. Exploratory Readings Groups Playbook. Retrieved June 12, 2024 from https://tech4good.soe.ucsc.edu/exploratory-reading-groups
[2]
Christine Alvarado, Alistair Gray, Diba Mirza, and Madeline Tjoa. 2021. The Role of Mentoring in a Dual-Mentored Scalable CS Research Program. In Proceedings of the 52nd ACM Technical Symposium on Computer Science Education (Virtual Event, USA) (SIGCSE ’21). Association for Computing Machinery, New York, NY, USA, 945–951. https://doi.org/10.1145/3408877.3432364
[3]
Christine Alvarado, Joe Hummel, Diba Mirza, Renata Revelo, and Lisa Yan. 2022. Scaling and Adapting a Program for Early Undergraduate Research in Computing. In Proceedings of the 53rd ACM Technical Symposium on Computer Science Education - Volume 1 (Providence, RI, USA) (SIGCSE 2022). Association for Computing Machinery, New York, NY, USA, 50–56. https://doi.org/10.1145/3478431.3499336
[4]
Christine Alvarado, Sergio Villazon, and Burcin Tamer. 2019. Evaluating a Scalable Program for Undergraduate CS Research. In Proceedings of the 2019 ACM Conference on International Computing Education Research (Toronto ON, Canada) (ICER ’19). Association for Computing Machinery, New York, NY, USA, 269–277. https://doi.org/10.1145/3291279.3339406
[5]
L. Barker. 2009. Student and Faculty Perceptions of Undergraduate Research Experiences in Computing. ACM Trans. Comput. Educ. 9, 1, Article 5 (mar 2009), 28 pages. https://doi.org/10.1145/1513593.1513598
[6]
Michael Barrow, Shelby Thomas, and Christine Alvarado. 2016. ERSP: A Structured CS Research Program for Early-College Students. In Proceedings of the 2016 ACM Conference on Innovation and Technology in Computer Science Education (Arequipa, Peru) (ITiCSE ’16). Association for Computing Machinery, New York, NY, USA, 148–153. https://doi.org/10.1145/2899415.2899436
[7]
Kristin P. Bennett, John S. Erickson, Amy Svirsky, and Josephine C. Seddon. 2022. A Mathematics Pipeline to Student Success in Data Analytics through Course-Based Undergraduate Research. The Mathematics Enthusiast 19, 3 (Dec. 2022), 730–750. https://doi.org/10.54870/1551-3440.1573
[8]
David Boud, Ruth Cohen, and Jane (All Of The University Of Tec Sampson (Eds.). 2014. Peer Learning in Higher Education (0 ed.). Routledge. https://doi.org/10.4324/9781315042565
[9]
Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology 3, 2 (2006), 77–101.
[10]
Amy M Buddie and Courtney L Collins. 2011. Faculty perceptions of undergraduate research. PURM: Perspectives on Mentoring Undergraduate Researchers 1, 1 (2011), 1–21.
[11]
Eileen D Bunderson and Mary Elizabeth Christensen. 1995. An analysis of retention problems for female students in university computer science programs. Journal of Research on Computing in Education 28, 1 (1995), 1–18.
[12]
Jamika D. Burge and Nancy M. Amato. 2016. CREU & DREU: Expanding the Impact of the Traditional REU. ACM Inroads 7, 4 (nov 2016), 81–83. https://doi.org/10.1145/2967500
[13]
Sapna Cheryan, Sianna A. Ziegler, Amanda K. Montoya, and Lily Jiang. 2017. Why are some STEM fields more gender balanced than others?Psychological Bulletin 143, 1 (Jan. 2017), 1–35. https://doi.org/10.1037/bul0000052
[14]
Jeffrey Scott Coker and Eric Davies. 2006. Ten Time‐Saving Tips for Undergraduate Research Mentors. Journal of Natural Resources and Life Sciences Education 35, 1 (Jan. 2006), 110–112. https://doi.org/10.2134/jnrlse2006.110s
[15]
Committee on Strengthening Research Experiences for Undergraduate STEM Students, Board on Science Education, Division of Behavioral and Social Sciences and Education, Board on Life Sciences, Division on Earth and Life Studies, and National Academies of Sciences, Engineering, and Medicine. 2017. Undergraduate Research Experiences for STEM Students: Successes, Challenges, and Opportunities. National Academies Press, Washington, D.C.https://doi.org/10.17226/24622 Pages: 24622.
[16]
Eileen L. Cooley, Amber L. Garcia, and Jennifer L. Hughes. 2008. Undergraduate research in psychology at liberal arts colleges: Reflections on mutual benefits for faculty and students.North American Journal of Psychology 10, 3 (2008), 463–471. Place: US Publisher: North American Journal Of Psychology.
[17]
Elisabetta Crocetti, Monica Rubini, Koen Luyckx, and Wim Meeus. 2008. Identity Formation in Early and Middle Adolescents From Various Ethnic Groups: From Three Dimensions to Five Statuses. Journal of Youth and Adolescence 37, 8 (Sept. 2008), 983–996. https://doi.org/10.1007/s10964-007-9222-2
[18]
Teresa Dahlberg, Tiffany Barnes, Audrey Rorrer, Eve Powell, and Lauren Cairco. 2008. Improving retention and graduate recruitment through immersive research experiences for undergraduates. ACM SIGCSE Bulletin 40, 1 (2008), 466–470.
[19]
Y. Deenadayalan, K. Grimmer‐Somers, M. Prior, and S. Kumar. 2008. How to run an effective journal club: a systematic review. Journal of Evaluation in Clinical Practice 14, 5 (Oct. 2008), 898–911. https://doi.org/10.1111/j.1365-2753.2008.01050.x
[20]
M. Kevin Eagan, Sylvia Hurtado, Mitchell J. Chang, Gina A. Garcia, Felisha A. Herrera, and Juan C. Garibay. 2013. Making a Difference in Science Education: The Impact of Undergraduate Research Programs. American Educational Research Journal 50, 4 (Aug. 2013), 683–713. https://doi.org/10.3102/0002831213482038
[21]
Marcus Fechheimer, Karen Webber, and Pamela B. Kleiber. 2011. How Well Do Undergraduate Research Programs Promote Engagement and Success of Students?CBE—Life Sciences Education 10, 2 (June 2011), 156–163. https://doi.org/10.1187/cbe.10-10-0130
[22]
Sara-Jane Finlay and Guy Faulkner. 2005. Tête à tête: Reading groups and peer learning. Active Learning in Higher Education 6, 1 (March 2005), 32–45. https://doi.org/10.1177/1469787405049945
[23]
Maria Lydia Fioravanti, Bruno Sena, Leo Natan Paschoal, Laíza R. Silva, Ana P. Allian, Elisa Y. Nakagawa, Simone R.S. Souza, Seiji Isotani, and Ellen F. Barbosa. 2018. Integrating Project Based Learning and Project Management for Software Engineering Teaching: An Experience Report. In Proceedings of the 49th ACM Technical Symposium on Computer Science Education (Baltimore, Maryland, USA) (SIGCSE ’18). Association for Computing Machinery, New York, NY, USA, 806–811. https://doi.org/10.1145/3159450.3159599
[24]
Allan Fisher and Jane Margolis. 2002. Unlocking the clubhouse: the Carnegie Mellon experience. ACM SIGCSE Bulletin 34, 2 (2002), 79–83.
[25]
Allan Fisher, Jane Margolis, and Faye Miller. 1997. Undergraduate women in computer science: experience, motivation and culture. In Proceedings of the twenty-eighth SIGCSE technical symposium on Computer science education. ACM, San Jose California USA, 106–110. https://doi.org/10.1145/268084.268127
[26]
Josephine Gasiewski, Minh C Tran, Felisha Herrera, Gina A Garcia, and Christopher B Newman. 2010. Barricades, bridges, and programmatic adaptation: A multi-campus case study of STEM undergraduate research programs. In Association for Institutional Research Annual Meeting, Chicago, IL. Citeseer.
[27]
Michail N Giannakos, Ilias O Pappas, Letizia Jaccheri, and Demetrios G Sampson. 2017. Understanding student retention in computer science education: The role of environment, gains, barriers and usefulness. Education and Information Technologies 22 (2017), 2365–2382.
[28]
Gregor Große-Bölting, Dietrich Gerstenberger, Lara Gildehaus, Andreas Mühling, and Carsten Schulte. 2023. Identity in higher computer education research: a systematic literature review. ACM Transactions on Computing Education 23, 3 (2023), 1–35.
[29]
Janet Harris, Karen Kearley, Carl Heneghan, Emma Meats, Nia Roberts, Rafael Perera, and Katharine Kearley-Shiers. 2011. Are journal clubs effective in supporting evidence-based decision making? A systematic review. BEME Guide No. 16.Medical Teacher 33, 1 (Jan. 2011), 9–23. https://doi.org/10.3109/0142159X.2011.530321
[30]
Paul R. Hernandez, P. Wesley Schultz, Mica Estrada, Anna Woodcock, and Randie C. Chance. 2013. Sustaining optimal motivation: A longitudinal analysis of interventions to broaden participation of underrepresented students in STEM.Journal of Educational Psychology 105, 1 (Feb. 2013), 89–107. https://doi.org/10.1037/a0029691
[31]
Sally G. Hoskins and Leslie M. Stevens. 2009. Learning our L.I.M.I.T.S.: less is more in teaching science. Advances in Physiology Education 33, 1 (2009), 17–20. https://doi.org/10.1152/advan.90184.2008 19261755.
[32]
Shouping Hu, Kathyrine Scheuch, Robert Schwartz, Joy Gaston Gayles, and Shaoqing Li. 2008. Reinventing Undergraduate Education: Engaging College Students in Research and Creative Activities. ASHE Higher Education Report, Volume 33, Number 4.ASHE Higher Education Report 33, 4 (2008), 1–103.
[33]
Danusha Jebanesan and Aarthi Ashok. 2016. "Reading groups" in an undergraduate biology course: A peer-based model to help students develop skills to evaluate primary literature. CourseSource 3 (2016). https://doi.org/10.24918/cs.2016.2
[34]
Amanpreet Kapoor and Christina Gardner-McCune. 2018. Understanding professional identities and goals of computer science undergraduate students. In Proceedings of the 49th ACM Technical Symposium on Computer Science Education. 191–196.
[35]
Amanpreet Kapoor and Christina Gardner-McCune. 2019. Understanding CS undergraduate students’ professional identity through the lens of their professional development. In Proceedings of the 2019 ACM conference on innovation and technology in computer science education. 9–15.
[36]
CarolAnne M. Kardash. 2000. Evaluation of undergraduate research experience: Perceptions of undergraduate interns and their faculty mentors.Journal of Educational Psychology 92, 1 (March 2000), 191–201. https://doi.org/10.1037/0022-0663.92.1.191
[37]
Robert W Lent, Steven D Brown, and Gail Hackett. 2008. Social cognitive career theory. In Career development and counseling: Putting theory and research to work. John Wiley & Sons, 255–278.
[38]
Jane Margolis and Allan Fisher. 2002. Unlocking the clubhouse: women in computing. The MIT Press, Cambridge, Massachusetts.
[39]
Allison Master, Sapna Cheryan, and Andrew N Meltzoff. 2016. Computing whether she belongs: Stereotypes undermine girls’ interest and sense of belonging in computer science.Journal of educational psychology 108, 3 (2016), 424.
[40]
Jeffrey Mervis. 2001. Student Research: What Is It Good For?Science 293, 5535 (Aug. 2001), 1614–1615. https://doi.org/10.1126/science.293.5535.1614
[41]
Janet A. Morrison, Nancy J. Berner, Jill M. Manske, Rebecca M. Jones, Shannon N. Davis, and Pamela W. Garner. 2018. Surveying Faculty Perspectives on Undergraduate Research, Scholarship, and Creative Activity: A Three-Institution Study. Council on Undergraduate Research Quarterly 2, 1 (Sept. 2018), 43–54. https://doi.org/10.18833/spur/2/1/1
[42]
Arnab Nandi and Meris Mandernach. 2016. Hackathons as an Informal Learning Platform. In Proceedings of the 47th ACM Technical Symposium on Computing Science Education (Memphis, Tennessee, USA) (SIGCSE ’16). Association for Computing Machinery, New York, NY, USA, 346–351. https://doi.org/10.1145/2839509.2844590
[43]
Joseph R. Nolan, Kelly S. McConville, Vittorio Addona, Nathan L. Tintle, and Dennis K. Pearl. 2020. Mentoring Undergraduate Research in Statistics: Reaping the Benefits and Overcoming the Barriers. Journal of Statistics Education 28, 2 (May 2020), 140–153. https://doi.org/10.1080/10691898.2020.1756542
[44]
Dustin Palea and David Lee. 2022. Lightweight, Scalable, and Relational Learning Experiences as an Approach to Overcoming System-Level Challenges in Education. In 2022 ASEE Annual Conference & Exposition.
[45]
Dustin Palea and David T Lee. 2021. Exploratory Reading Groups: A Scalable Approach to Creative, Relational, and Student-Driven Exploration in CS Education. In Proceedings of the 52nd ACM Technical Symposium on Computer Science Education. 837–843.
[46]
Ann M. Pendleton-Jullian and John Seely Brown. 2018. Design unbound: designing for emergence in a white water world. The MIT Press, Cambridge, Massachusetts.
[47]
Beatriz Pérez and Ángel L. Rubio. 2020. A Project-Based Learning Approach for Enhancing Learning Skills and Motivation in Software Engineering. In Proceedings of the 51st ACM Technical Symposium on Computer Science Education (Portland, OR, USA) (SIGCSE ’20). Association for Computing Machinery, New York, NY, USA, 309–315. https://doi.org/10.1145/3328778.3366891
[48]
John K. Petrella and Alan P. Jung. 2008. Undergraduate Research: Importance, Benefits, and Challenges. International Journal of Exercise Science 1, 3 (2008), 91–95.
[49]
Sophie Pierszalowski, Jana Bouwma-Gearhart, and Lindsay Marlow. 2021. A Systematic Review of Barriers to Accessing Undergraduate Research for STEM Students: Problematizing Under-Researched Factors for Students of Color. Social Sciences 10, 9 (Sept. 2021), 328. https://doi.org/10.3390/socsci10090328
[50]
Diane Railton and Paul Watson. 2005. Teaching autonomy: ‘Reading groups’ and the development of autonomous learning practices. Active Learning in Higher Education 6, 3 (Nov. 2005), 182–193. https://doi.org/10.1177/1469787405057665
[51]
Audrey Smith Rorrer, Joseph Allen, and Huifang Zuo. 2018. A National Study of Undergraduate Research Experiences in Computing: Implications for Culturally Relevant Pedagogy. In Proceedings of the 49th ACM Technical Symposium on Computer Science Education (Baltimore, Maryland, USA) (SIGCSE ’18). Association for Computing Machinery, New York, NY, USA, 604–609. https://doi.org/10.1145/3159450.3159510
[52]
Susan H. Russell, Mary P. Hancock, and James McCullough. 2007. Benefits of Undergraduate Research Experiences. Science 316, 5824 (April 2007), 548–549. https://doi.org/10.1126/science.1140384
[53]
Elaine Seymour, Anne‐Barrie Hunter, Sandra L. Laursen, and Tracee DeAntoni. 2004. Establishing the benefits of research experiences for undergraduates in the sciences: First findings from a three‐year study. Science Education 88, 4 (July 2004), 493–534. https://doi.org/10.1002/sce.10131
[54]
Susan E. Shadle, Anthony Marker, and Brittnee Earl. 2017. Faculty drivers and barriers: laying the groundwork for undergraduate STEM education reform in academic departments. International Journal of STEM Education 4, 1 (Dec. 2017), 8. https://doi.org/10.1186/s40594-017-0062-7
[55]
Rhea Sharma, Atira Nair, Ana Guo, Dustin Palea, and David T. Lee. 2022. “It’s usually not worth the effort unless you get really lucky”: Barriers to Undergraduate Research Experiences from the Perspective of Computing Faculty. In Proceedings of the 2022 ACM Conference on International Computing Education Research - Volume 1 (Lugano and Virtual Event, Switzerland) (ICER ’22). Association for Computing Machinery, New York, NY, USA, 149–163. https://doi.org/10.1145/3501385.3543976
[56]
Leonard Springer, Mary Elizabeth Stanne, and Samuel S. Donovan. 1999. Effects of Small-Group Learning on Undergraduates in Science, Mathematics, Engineering, and Technology: A Meta-Analysis. Review of Educational Research 69, 1 (March 1999), 21–51. https://doi.org/10.3102/00346543069001021
[57]
Jennifer S. Stanford, Suzanne E. Rocheleau, Kevin P.W. Smith, and Jaya Mohan. 2017. Early undergraduate research experiences lead to similar learning gains for STEM and Non-STEM undergraduates. Studies in Higher Education 42, 1 (Jan. 2017), 115–129. https://doi.org/10.1080/03075079.2015.1035248
[58]
Stuart A Tross, Jeffrey P Harper, Lewis W Osher, and Linda M Kneidinger. 2000. Not just the usual cast of characteristics: Using personality to predict college performance and retention.Journal of college student development (2000).
[59]
US National Science Foundation. 2023. Research Experiences for Undergraduates (REU). https://new.nsf.gov/funding/opportunities/research-experiences-undergraduates-reu Accessed: 2023-05-19.
[60]
Rajan Vaish, Snehalkumar (Neil) S. Gaikwad, Geza Kovacs, Andreas Veit, Ranjay Krishna, Imanol Arrieta Ibarra, Camelia Simoiu, Michael Wilber, Serge Belongie, Sharad Goel, James Davis, and Michael S. Bernstein. 2017. Crowd Research: Open and Scalable University Laboratories. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology (Québec City, QC, Canada) (UIST ’17). Association for Computing Machinery, New York, NY, USA, 829–843. https://doi.org/10.1145/3126594.3126648
[61]
Heidi A. Wayment and K. Laurie Dickson. 2008. Increasing Student Participation in Undergraduate Research Benefits Students, Faculty, and Department. Teaching of Psychology 35, 3 (July 2008), 194–197. https://doi.org/10.1080/00986280802189213
[62]
Jesse LM Wilkins. 2004. Mathematics and science self-concept: An international investigation. The journal of experimental education 72, 4 (2004), 331–346.
[63]
David S. Yeager, Marlone D. Henderson, David Paunesku, Gregory M. Walton, Sidney D’Mello, Brian J. Spitzer, and Angela Lee Duckworth. 2014. Boring but important: A self-transcendent purpose for learning fosters academic self-regulation.Journal of Personality and Social Psychology 107, 4 (Oct. 2014), 559–580. https://doi.org/10.1037/a0037637
[64]
Haoqi Zhang, Matthew W. Easterday, Elizabeth M. Gerber, Daniel Rees Lewis, and Leesha Maliakal. 2017. Agile Research Studios: Orchestrating Communities of Practice to Advance Research Training. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing. ACM, Portland Oregon USA, 220–232. https://doi.org/10.1145/2998181.2998199
Index Terms
- Evaluating Exploratory Reading Groups for Supporting Undergraduate Research Pipelines in Computing
Recommendations
Holistic Summer Undergraduate Research Program in High Performance Computing Research
PEARC '19: Practice and Experience in Advanced Research Computing 2019: Rise of the Machines (learning)This study investigated the effectiveness of a National Science Foundation-sponsored Research Experience for Undergraduates Site in High Performance Computing (HPC) at South Dakota State University. In the 2018 program, ten first- and second-year ...
A National Study of Undergraduate Research Experiences in Computing: Implications for Culturally Relevant Pedagogy
SIGCSE '18: Proceedings of the 49th ACM Technical Symposium on Computer Science EducationIn this paper, we describe a large-scale, long-term study of a National Science Foundation undergraduate research program via the Computer and Information Science and Engineering (CISE) Research Experiences for Undergraduates (REU) Evaluation Toolkit ...
Promoting undergraduate research: a classroom model
This paper describes a computer science course designed to develop and enhance undergraduates' research skills. The course pedagogy served to significantly reduce the faculty mentor's tedious task of building the mentee's research-related skill set, yet ...
Comments
Information & Contributors
Information
Published In
August 2024
539 pages
ISBN:9798400704758
DOI:10.1145/3632620
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 12 August 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
Conference
ICER 2024
Sponsor:
ICER 2024: ACM Conference on International Computing Education Research
August 13 - 15, 2024
VIC, Melbourne, Australia
Acceptance Rates
Overall Acceptance Rate 189 of 803 submissions, 24%
Upcoming Conference
ICER 2025
- Sponsor:
- sigcse
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 113Total Downloads
- Downloads (Last 12 months)113
- Downloads (Last 6 weeks)24
Reflects downloads up to 01 Jan 2025
Other Metrics
Citations
View Options
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML FormatLogin options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in