Assessment

Marxe Assessment Spotlight
Interview with NASPAA’s Chief Accreditation Officer, Ms. Martha Bohrt

At Marxe, a component of our academic assessment program includes the work of fostering a positive assessment culture (in addition to implementing our assessment cycles). This aspect of the work is critical to the overall success of assessment, as it aids in cultivating support and participation. As a result, we have incorporated assessment guides, presentations, workshops, a website, and now a newsletter to our initiatives.

Although tremendous strides have been made in terms of both the actual assessments as well as the efforts to ensure a supportive culture, there is always more that can be done to bridge the gap and ensure the faculty and staff have a stronger understanding of and relationship with NASPAA and its mission. As a result, we have invited Chief Accreditation Office, Martha Bohrt to speak directly from a NASPAA lens regarding key areas/questions of accreditation.

How does NASPAA accreditation serve as a quality assurance mechanism for students and other stakeholders? NASPAA’s accreditation process is driven by public service values, is mission- and outcomes-based, and is grounded in an in-depth self-evaluation. Programs seeking accreditation use the self-study process to support ongoing program improvement, strengthen their commitment to public service education, and showcase their accomplishments. NASPAA-accredited programs establish observable goals and outcomes, and use information about their performance to guide program improvement. They contribute to the knowledge, research, and practice of public service, practice truth in advertising and ensure their students achieve learning objectives in five domains essential to public service.

What role should faculty play in the process of demonstrating and maintaining NASPAA accreditation standards? How does the process of complying with the standards and maintaining accreditation benefit faculty directly? Program faculty in NASPAA-accredited programs participate in program delivery, program governance, and research and/or professional service related to the field of public affairs. These three areas are the ones where programs are expected to demonstrate conformance with the NASPAA Standards. In other words, faculty are at the front line of NASPAA accreditation, ensuring student learning, strategic program management, and the advancement of the field of public service. At the same time, faculty can use the NASPAA universal required competencies to determine learning objectives for their courses and the NASPAA accreditation standards to determine mechanisms to assess student learning and success. Additionally, Standard 3 requires programs to demonstrate support of the faculty body’s continuous professional development, diversity, and research agendas. In summary, the NASPAA accreditation process supports faculty in their continuous professional development both in teaching and research.

We understand that NASPAA is not prescriptive about how institutions engage in assessment of student learning. Can you speak about the relative importance of different aspects/phases of assessment (i.e., faculty engagement in regular reflection and revision of their goals and assessment instruments, faculty discussions of assessment results, the development of action items and curricular changes). Are they all equally important to NASPAA? If not, which aspect is most important, and why? All aspects that contribute to program improvement are of equal importance to the NASPAA accreditation process. Assessment is like a machine with many components that help each other create a product. One piece of the machine are the assessment instruments, another piece is analysis of assessment results, and the product is program improvement. Faculty are of course the oil. If the program has a series of tools to assess student learning, for example, but is not doing anything to understand the results, then needed modifications cannot be identified or, if they are, they may be the wrong ones, as we only have determined effect, but not cause. The same is true of what may happen without the oil: if the faculty, the frontline of assessment, don’t participate in the assessment processes, then all the parts are missing that first-hand information needed to create appropriate, quality tools.

How can institutions partner with NASPAA to better educate faculty about what improvement in teaching and learning means to NASPAA? Becoming a NASPAA site visitor is the easiest and most fun way to learn what the NASPAA accreditation standards are all about! Through the site visit, volunteers see the standards applied across a wide range of contexts. Over 90% of volunteers report each year that the site visit allows them to learn new practices that can be implemented at their own program. Additionally, NASPAA staff is available year-round to engage with faculty. NASPAA also hosts accreditation institutes throughout the year specifically designed for faculty to learn about the accreditation process. All inquiries about becoming a site visitor and for setting up appointments with staff can be sent to [email protected].

Assessment efforts are collaborative; the key to success is a combination of faculty expertise and the guidance and vision of specialized administration.

Learning Assessment Committee (LAC)

The Committee’s conduct shall be consistent with the standards for the assessment of the MPA program required for accreditation by the National Association of Schools of Public Affairs and Administration (NASPAA).

The LAC assists with the:

• establishment and maintenance of a process and policies for assessing and improving student learning
• development of program learning goals and objectives
• alignment of program learning goals in collaboration with the Curriculum Committee
• creation of assessment instruments as well as the collection of the assessments
• analysis and implementation of information for continual improvement.

View information on LAC members

Administering Assessment at Marxe

At Marxe, the Learning Assessment Committee (LAC) works in tandem with administration to establish a collaborative vision as well as design and implement assessment initiatives across programs. The assessment-focused administrative roles have continued to evolve as a demonstration of Marxe’s commitment to improving teaching and learning.

The Associate Director of Assessment
Positioned within the College’s Office of Assessment, Accreditation, and Institutional Effectiveness, the Associate Director focuses on managing and guiding Marxe assessment initiatives and each phase of the cycle, acting as a liaison between the accreditor and the School regarding student learning and assessment, building and maintaining a culture of assessment, and implementing best practices.

The Data Analyst
Assists in the documentation, collection, and analysis of assessment data. This individual is also responsible for managing the data for the entire School, ensuring the overall statistics regarding the programs are understood.

Guiding Principles of Assessment at Marxe

Assessment is successful when it is mission and value focused. Each aspect of the process should originate from this lens, ensuring that we not only examine what the program currently offers, but also establish what more we may want to provide, change, or enhance.

• Faculty-driven: Assessment initiatives are most effective when faculty-driven in combination with administrative leadership to provide vision and support (Kinzie, et al., 2014, p.2). Faculty involvement in every step of the process is essential for meaningful content and valid results; administrative guidance and expertise is also key in leading each step forward and providing an assessment lens to the process.
• Positive Culture: Assessment initiatives are most effective when a positive culture is cultivated and woven throughout all aspects of the academic environment. As conversations and awareness increase, assessment will become a common theme and a path toward improving teaching and learning. This will lead to increased support and involvement.
• Program Improvement: The goal of assessment is continual program improvement. This occurs based on programmatic/curricular actions that take place as a result of the assessment findings and inclusive faculty conversations around learning expectations.

Assessment Cycle

Assessment at Marxe is a deliberate and purposeful process. It includes the:

• development and revision of goals
• collaborative and inclusive pedagogical discourse
• careful design and implementation of methods
• evaluation of data and realization of action and improvement of content as a result of these various phases

Project-specific assessment cycles typically operate in two-year time frames and include the following phases:

Program and Course Learning Goals within a degree provide an understanding of the significant expectations of that program.

• Program Learning Goals (or outcomes) are overarching “promises” of learning that encompass the degree’s mission, values, and overall expectations.
  • Program learning goals speak to what a student should know and be able to demonstrate by graduation and are significantly addressed at various points throughout the curriculum.
  • Through academic assessment, the program learning goals are measured to understand the proficiency of student learning and quality of teaching.
  • Note: NASPAA Competencies at Marxe act as categories and are used to develop outcomes within specific areas of learning.

• Course goals, which are specific to the content of a particular class, are often derived from program learning goals and provide course expectations and connect to activities associated with achieving them.
  • When relevant program learning goals are emphasized in class, students can align course goals with the broader program expectations to draw curricular connections across the program. This allows students to understand the broader expectations and hold themselves accountable.

Developing Goals

Through various domains of learning (Bloom’s taxonomy), learning goals should address the why of what is occurring in the classroom.

^{*developed by Office of Assessment,
Accreditation, and Institutional Effectiveness}

Goals should:

• Use concrete action verbs that address observable performance
• Be clear, simple, and not compound
• Be measurable
• Connect with class work or assignments that develop that goal

Note: Traits such as appreciation, respect, integrity, compassion, etc. are difficult to quantify or define. These constructs are not only difficult to measure, but nearly impossible to teach in a classroom. We can aim to instill these traits by modeling behaviors and discussing experiences and history; however, it is important that goals remain realistic and measurable.

Categories of Cognitive Learning

Bloom’s Taxonomy (Bloom, 1956)

• Knowledge (know and remember specific facts, theories)
• Comprehension (understand, interpret, compare, contrast, and explain)
• Application (apply knowledge to new situations and solve problems using required knowledge or skills)
• Analysis (identify relationships, organizational structures, and principles)
• Synthesis (create something, to integrate ideas into a solution, propose an action plan, or formulate a new classification scheme)
• Evaluation (judge the quality of something based on its adequacy, value, logic, or use)

Affective (Attitudinal) Learning

Attitudes, values, and habits of mind.

Curriculum Map

Learning goals are addressed at different levels within different courses as well as within other aspects of the program, such as through recommended readings, events, and internships.

Levels of teaching material include Introduction (I), Reinforced (R), Mastered (M)

Courses should be mapped to the various program goals that they address noting the level in which they are addressed.

Example:

The goal of assessment is to enhance the curriculum by evaluating student learning. This is accurately achieved by collecting a valid sample of evidence in the form of student work, exams, surveys, and other relevant data throughout the program.

• Direct evidence: student work such as case studies, capstones, portfolios, and oral presentations
• Indirect evidence: methods that demonstrate perceived student learning, such as surveys, focus groups, and graduation rate
• Evidence is kept confidential and only used for assessment purposes

Rubric Design

A rubric is a scoring guide developed to evaluate student proficiency. The rubric is used to define the various elements of a program learning goal. Goals and their criteria remain consistent across the program where they are significantly addressed.

While standards or criteria clearly define the goal, descriptions are provided for each score to provide an understanding of how that score is achieved.

Program rubrics:

• Define and clarify each specific Program Learning Goal
• Should be goal-focused and universal, with criteria that remains applicable wherever the goal is significantly addressed
• Program rubrics should be developed prior to the assignments or tools used to test the students’ proficiency with a particular goal

Reliable Rubrics: New Innovative Template Design at Marxe

The rubrics designed for program assessment provide criteria that speak to the overarching nature of program learning goals and, as a result, should have a universal quality. This ensures that the rubric is a consistent and reliable scoring instrument that can measure a goal at different points across a program.

However, discipline-specific courses and assignments often comprise of more narrowly focused expectations. Assessing a program goal within a course using an embedded assignment means that the assignment must significantly address the broader program learning goal criteria. When variations occur across different courses and assignments used to measure the same goal, an additional insert is included with the more general rubric descriptions, addressing the assignment’s specificity as it is represented in the assessed class. This allows for further clarity when evaluating certain assignments.

The revised rubric model provides:

• Consistent, reliable universal descriptions addressing the meaning of the goal under each score
• Assignment-specific inserts that allow for distinct specificity and tailoring depending on the course or assignment used to assess the goal

^{*developed by Office of Assessment, Accreditation, and Institutional Effectiveness}

Recent Marxe Program Rubrics can be referenced below:

MPA Comp#1 A&B: Management Rubric

MPA Comp #1, A&B: Management Rubric with PAF 9163 customization

MPA Comp #2, Goal A: Policy Rubric

MPA Comp#2, Goal B: Policy Rubric

MPA Comp #3, Goal A: Critical Thinking/Analysis Rubric

MPA Comp #4, Goal A: Public Service Perspectives Rubric

MPA Comp #5, Goal A: Communication Rubric

HEA Goal #1 Rubric

HEA Goal #2 Rubric

Assessment occurs naturally in the ongoing engagement of teaching and learning; however, it is often beneficial to have additional information on best practices to guide future work. For additional resources, please explore the links below.

Writing Program and Course Goals

Baruch College Writing Learning Goals

Rubrics

AACU Value Rubrics and Resources

Assessment Journals/White Papers/Websites

NASPAA Publications

National Institute for Learning Outcomes Assessment

Research and Practice in Assessment

Association for the Assessment of Learning in Higher Education (AALHE)

Assessment Update

American Journal of Evaluation (AJE)

New Directions for Teaching and Learning

CUNY Assessment 101

CUNY Assessment 101 is a new, free, online course now available to all CUNY faculty and staff. This initiative was developed by the CUNY Assessment Council with the goal of providing a comprehensive assessment resource available to all. It is an asynchronous course that can be taken at your own pace.

The Marxe community has committed to building a strong foundation of values throughout our School and programs. From our mission, we continue this work through a comprehensive lens, including efforts within the curriculum. We expand our scope of inclusion, diversity, and equity to go beyond recruitment and retention and address what students should learn and be able to demonstrate in relation to these values.

As we prepare students for positions of responsibility and leadership in public service, it is essential that we provide them with skills to demonstrate accountability and results within their own organizations and that we model what we teach through our own systematic and thoughtful assessment processes.* (Rubaii & Calarusse, 2014)

As an element of our dedication, the Marxe community has collaborated on and finalized a policy statement that represents a collective definition of inclusion, diversity, and equity and how these perspectives and behaviors should be represented within academia. It is an important statement that supports our mission and advancement of work throughout all programs.

Marxe’s Inclusion, Equity, and Diversity in Academia (IDEA) Statement

*Rubaii, N., & Calarusse, C. (2014). Preparing Public Service Professionals for a Diverse and Changing Workforce and Citizenry: Evaluating the Progress of NASPAA Programs in Competency Assessment. Journal of Public Affairs Education 20(3), 285-304.

Marxe is one of three Schools within Baruch College and operates under Middle States Commission on Higher Education (MSCHE) Accreditation Standards. Baruch College’s next Accreditation Review is 2026-27.

Middle States Mission
Middle States Standards
Resources

The Master of Public Administration program degree receives program accreditation from the Network of Schools of Public Policy, Affairs, and Administration (NASPAA). Marxe’s last Accreditation Review was 2021-2022.

NASPAA Mission
NASPAA Standards
Resources

For additional information, the Office of Assessment, Accreditation, and Institutional Effectiveness at Baruch College provides comprehensive guidance and resources to support all aspects of assessment and accreditation across campus.

Marxe Program Learning Goals address what a student should know and be able to demonstrate by graduation:

• Broad statements that address higher order learning as well as encompass the degree’s mission, values, and overall expectations
• Overarching, broad expectations that are addressed in different places throughout the curriculum
• Teachable and measurable
• Students should be able to align their course goals with the broader program expectations in order to draw curricular connections across the program
• Ensuring program goals are clear and present throughout the curriculum allows students to understand their expectations and hold themselves accountable

Learning goals are developed from various cognitive frameworks including Bloom’s taxonomy. These frameworks are organized by levels based on the intricacy of skill and may be summarized into three categories:

• Knowledge and conceptual understanding
• Application, analysis, synthesis, and evaluation
• Attitudes, values, dispositions, and habits of mind

Below are the program learning goals listed for each of Marxe’s degree programs. Upon graduation, students within the Master of Public Administration (MPA) degree programs at Marxe are expected to obtain knowledge and demonstrate skills associated with the following Program Learning Goals approved by the faculty and fall under five Universal Competencies established by NASPAA.