Digitally Based Assessment
The NAEP 2018 digitally based geography assessment was designed to continue reporting trends in student performance dating back to 1994, while keeping pace with the new generation of classroom environments in which digital technology has become an increasing part of students' learning. The 2018 assessment content was developed with the same geography framework used to develop the previous paper-based assessments.
Most of the content administered in the 2018 digitally based geography assessment was also used in the 2014 paper-based assessment. The previously used questions were adapted to fit a tablet screen. While the presentation of content changed, the content itself did not change. Of the seven DBA blocks administered at grade 8, two blocks were newly developed for 2018.
The assessment was administered on tablets supplied by NCES using a secure, local NAEP network. This allowed the NAEP administrators to create a stable assessment environment by bringing in their own equipment that would not be influenced by school-based equipment or school internet connectivity, thereby maintaining consistency across the assessed schools. Students were able to interact with the tablets via touchscreen, with an attached keyboard, or using a stylus provided by NAEP. The digitally based geography assessment provided students with interactive item types and online tools such as zooming capability, and different contrast settings for the visually impaired. See how the geography assessment was presented to students. At the beginning of the assessment session, students viewed an interactive tutorial that provided all the information needed to take the assessment on a tablet; for example, it explains how to progress through questions, and how to indicate answers for selected-response questions and use online tools.
To investigate potential differences in performance that might be due to the digital transition, the paper-based and digitally based versions of the assessment were administered to randomly equivalent groups of students, and each participating student took the assessment in only one mode.
After the administration of the assessment, the National Center for Education Statistics (NCES) conducted rigorous analyses of the data and aligned the 2018 results to previous assessment years using a two-step process.
- First, common item linking was used to calculate the trend line from 2014 to 2018 based on the paper-based assessment results. This kind of linking was possible because all the 2018 paper-based questions were also administered in 2014 and showed the same statistical properties.
- Second, common population linking was used to align the 2018 paper-based assessment results with the 2018 digitally based assessment results. This kind of linking was possible because the samples of students for each assessment mode were randomly equivalent, ensuring that the students' educational experiences and characteristics were equivalent.
Once the common population linking aligned the digital results to the paper results on the national level, the analyses evaluated whether the linking allowed for fair and meaningful comparisons for major reporting student groups. These evaluations supported making trend comparisons between the digital assessment and previous paper-based assessments for student groups.
These analyses—common item linking based on paper results and common population linking of paper results to digital results—enabled NCES to successfully maintain the geography trend line while transitioning to digital assessment. The 2018 geography assessment results in this report are based on the combined performance of students who took the assessment on paper and students who took the assessment on tablets.
Technical documentation on the transition to the 2018 digitally based geography assessment is forthcoming.
About the NAEP Geography Assessment
The National Assessment of Educational Progress (NAEP) in geography is designed to measure students' knowledge, understanding, and application of geography in the content areas of Space and Place, Environment and Society, and Spatial Dynamics and Connections. Students answer a series of selected-response and open-ended questions based on these content areas in geography. Performance results are reported for students in the nation and disaggregated by various student characteristics.
In 2018, the NAEP geography assessment transitioned from a paper-based assessment (PBA) to a digitally based assessment (DBA) at grade 8. A multi-step process was used for the transition from PBA to DBA, which involved administering the assessment in both formats to randomly equivalent groups of students in 2018. The transition was designed and implemented with careful intent to preserve trend lines that show student performance over time. Thus, the results from the 2018 geography assessment can be compared to results from previous years.