The moving goal post of “proficiency” 

How the class of 2022 went from “aspirational cohort” to the new floor for student achievement 

State education officials are poised to reset the way state assessments are scored. The move is much more than a technical change: it marks the beginning of yet another overhaul to the way student success is measured in the state, and the continuously shifting goalpost of what it means to be “proficient”. It raises the question of whether Albany has, at any point, had an accurate picture of where the state’s 2.4 million students stand academically.

This reset takes the form of a change to scoring criteria on state assessments. New York State Education Department’s (NYSED) technical advisory committee announced last month that they will be adjusting the cut scores on grades 3-8 assessments to reflect what officials have called a “new normal” for student achievement expectations. However, this new normal is based in part on test scores from the 2021-22 school year — one of the lowest in New York’s history.

In test design, the cut score refers to a numerical range of test score outcomes that fall into a given performance level classification. Performance level classifications are typically nonnumeric labels like basic, proficient or advanced. The cut score is the point on the scale of 0-100% where each classification “cuts off.” Lowering the cut score for one performance indicator, such as proficient, would allow a wider range of test scores (and more students) to be labeled proficient.

NYSED and their technical advisory committee have not released a detailed methodology that outlines which quantitative data will inform where future cut scores are set. What we do know is that it will involve a group of educators coming together to provide input on how the “average” or “target” student might perform.

But what is the “average” student in New York? According to the technical advisory committee, the assessment results from school year 2021-22, one of the lowest in the state’s history, will set the bar—after years of insisting those results are not indicative of students’ ability or schools’ efficacy.

The saddest part? The class of 2022 was once set apart as New York’s “aspirational cohort.”

These students were meant to represent New York’s successful alignment of curriculum and assessments to federal common core standards, for all grade levels, as demonstrated by senior class performance.

As early as 2014, when NYSED was struggling to align with common core standards, the state Board of Regents committed to holding that year’s senior class to higher graduation standards, as a reflection of their cumulative curriculum and assessment efforts. These students would be the first class required to pass Regent’s examinations with a score in the “college and career readiness level” range of 75-80, rather than 65.

However, by 2017, long before the pandemic, NYSED already knew they were not on target, and announced they would not be moving forward.

That same group of students would eventually be granted such flexibility on their Regents’ exams that they could guess and still pass; in some cases, even with a failing course grade. Students experienced social promotion and unethical grading procedures at schools desperate to graduate more students, many of whom may have been struggling to reach proficiency in English and math since their own 3rd through 8th grade years, the state comptroller’s office found.

Their results are where NYSED plans to set the floor for future student achievement expectations — after a decade of shifting state standards and no significant improvement on measures of actual skill-mastery, like the Nation’s Report Card or even our own state assessments.

This cut score change is part of a larger initiative to implement the next round of educational norms in the state. New York’s “Next Generation” learning and assessment standards, which will fully replace the common core-aligned curriculum by 2025, represent a philosophical shift in how students learn and test in the state. This new approach places less emphasis on more objective measures of foundational knowledge, like traditional skills-based tests in literacy and numeracy. It instead focuses on project- and performance-based measures, such as presentations and portfolios, to assess “21st century skills” and close achievement gaps for underperforming groups.

The literature is mixed on whether performance-based assessments are a better measure of what students should know and be able to do. It is true that problem solving, critical thinking, and other higher-order cognitive abilities have not historically “fit” into traditional, skills-based assessment frameworks. On the other hand, foundational skills like literacy and numeracy are not always accurately measured by project- and performance-based tests due to the inherent level of subjectivity. Those like the National Association for Educational Progress (NAEP) and Program for International Assessment of Adult Competencies (PIAAC) still rely on a traditional, skills-based assessment framework to gain an accurate picture of where people stand on critical competencies like literacy and numeracy.

New York was not alone in its common core struggles, or its pandemic learning loss. Other states and countries have addressed the underlying challenge by leaning into increased rigor and foundational skills; promoting students based on mastery, not seat time; and setting statewide literacy standards with ambitious goals to reach on behalf of their students; like everyone reading on-level by 3rd grade, as measured by both internal and external assessment.

Many states have modernized their assessment frameworks to be more rigorous, accurate and transparent, and to take advantage of adaptability and diagnostic tools that can more intelligently assess and adapt to all students, especially those with learning differences or unidentified knowledge gaps.

Some states have committed to testing with a continuous measurement strategy, like the NAEP, which intentionally does not change over time. This allows for capture of meaningful, longitudinal data on where their students stand K-12 and post-graduation to better assess what is working and what isn’t. Others have increased accountability and transparency at the school-level through robust data portals and financial transparency websites. And officials have been thinking creatively about how their systems can innovate to better serve the unique needs and choices of families.

The Board of Regents’ fall-back has been status-quo solutions like blanket funding increases and moving goal posts with little continuity and transparency, rather than facing hard truths about where kids stand academically and demanding that students actually know what their report cards indicate they do. As a result, New York spends more money to educate its students than any other nation in the world, while measures of student achievement continue to decline, and many are slipping through the accountability framework.

Constant changes to the nature of assessments, performance standards and grading procedures, over the course of even one K-12 cohort, make it difficult to meaningfully measure where kids stand academically and what is working within schools. Combining that with outdated technology, incomplete data collection methods and inconsistent accountability practices, it becomes nearly impossible.

In fact, it can create the impression that student achievement is improving, while kids may not actually be mastering skills. For instance, proficiency rates may rise after an adjustment to cut scores, even while all other critical measures of student success (class grades, attendance, literacy and numeracy rates, etc.) decline.

The upcoming reset to scoring criteria, and the associated changes in the state’s learning standards, present an opportunity for New York to develop the country’s most rigorous curriculum and performance standards. Do the Regents want to?

You may also like

Don’t Tell The Grownups: NY Still Hiding State Test Scores

State education officials are refusing to release the results of federally required assessments in grades 3 through 8, deliberately keeping parents and taxpayers in the dark—not only about how New York’s public schools performed, but also about how that performance was measured. Read More

What You Should Know: NY’s changing graduation requirements

Months after lowering the scores to pass state assessment exams, New York education officials are considering eliminating the Regents diploma. Read More

State Drags Out Test Mess

The New York State Education Department (SED) was proud to announce this week that they have lifted the embargo on last year’s state test scores in reading and math. Does this mean the public gets to see them now?  Read More

Back to School: New York Style

Class is in session across the state, and things are messy (especially in New York City).  Read More

Where are New York’s Test Results?

For the second year in a row, New York parents will receive their back-to-school shopping lists before their students' results on state assessments. and Read More

NY school spending again led US, hitting all-time high in 2020-21

Public elementary and secondary school spending in New York rose to $26,571 per pupil in 2020-21, according to the latest Census Bureau data Read More

A Tale of Two Levies

New York school districts are getting record levels of state aid. But how many are using it to cut taxes? Read More

Blocking regs, court highlights role of parents 

A state Supreme Court judge yesterday voided the bulk of New York’s “substantial equivalence” regulations aimed at religious and other non-public schools Read More