Last Spring, students statewide in grades 3-8 sat for annual state assessments to measure proficiency in English language arts (ELA) and math. The results are of unusually high interest. Tests were cancelled in 2020 due to the COVID-19 pandemic, and 2021 tests were optional and featured low participation rates due to ongoing pandemic-related disruptions. Consequently, 2022 results mark the first meaningful dispensation of state assessments in three years.

Despite—or perhaps due to—unusually high public interest, the results were released only after an unusually long delay. Though individual school and districtwide results have historically been released in late summer, the State Education Department (SED) announced in June that it would forego releasing “preliminary” results this year and release only final results — but not until an unspecified date in the fall. Prompted by an Empire Center FOIL request and subsequent appeal of an initial response from SED, Commissioner Betty Rosa determined the results would be released on November 4.  Then, without warning, SED issued the results on October 24th, albeit in a format difficult for the public to access, and on the same day that National Association of Educational Progress (NAEP) state-level results were released.

Caution is warranted in using assessments to gauge how the performance of a single district has changed over time, as the conversion of raw score to performance level changes from year to year. The challenge is exacerbated by the fact that New York experienced an unprecedented exodus from traditional public schools and a surge in students enrolling in public charter schools and private schools, and in relocating with their families to other states. So, for example, 20 percent of third graders who were enrolled in a district in 2019 may have disenrolled by last spring and did  not factor into the district’s 2022 sixth grade assessments.

State assessments do not provide the first or best look at pandemic-related learning loss in New York overall. Results from the NAEP exam offer the best clue to the degree of learning loss in absolute terms and compared to other states, as the NAEP exam is distributed to a random sample of students in all 50 states and the test maintains the same standards through time. NAEP, however, only reports district-level results for 26 large districts across the country, so while it is useful for interstate comparison, it does not allow for intrastate comparison. Indeed, state assessments offer the best indication as to how districts within a state are performing relative to one another. Limitations notwithstanding, state assessments remain integral to parents to assess the performance of schools and districts.

Each state sets its own proficiency standards on its state assessments. Last year. researchers examined the relationship between proficiency on state tests and NAEP results and estimated how proficiency tracks with the NAEP exam and compares to other states. Overall, New York was estimated to have the 10th lowest standards in fourth grade math but middle-of-the-pack standards for fourth and eighth grade reading. An estimate could not be rendered for eighth grade math. In the three tests the researchers could assess, they determined that New York standards reached the threshold of “basic” performance on NAEP but did not meet the threshold of proficiency. In other words, set against the NAEP standard, New York features relatively low standards for what comprises proficiency.

Who Tested?

All states require public schools to administer math and ELA assessments between grades 3-8 in accordance with the federal Every Student Succeeds Act. Parents can opt out of the test, and in recent years as many as 1 in 5 New York parents have exercised that right. Consequently, New York has among the lowest participation rates in the country.

Overall, SED data indicate that 81.3 percent of students in public schools (including charter schools) participated in the 2020 ELA assessments and 81.5 percent participated in the math assessments. In 2019, 82.9 percent participated in ELA assessments and 84.6 in math, so statewide participation rates have essentially returned to a pre-COVID normal.

More granular inspection reveals notable differences in participation rates by region (see Table 1). Only 58.8 percent of eligible students participated in state assessments on Long Island, which though anomalously low, represents a 3.7 percentage point increase from 2019. No other region had a participation rate less than 80 percent in 2022.

Missing data can skew results when it is missing in a way that is not random.

Opt-out is particularly popular among politically progressive high income white families, which might explain in part low participation rates on Long Island. Consequently, results from Long Island and districts with low participation rates may not in fact be representative of the district.

New York City was the only region to experience a particularly notable change in participation rates compared to 2019, with an 8 percentage point drop to 88.4 percent. New York City has increased reliance on third party assessments to track student learning, so the drop-off there might be explained in part by testing fatigue.

Participation rates notably vary by school sector. In 2019, 94.8 percent of charter students participated in ELA assessments and 94.6 in math assessments. In 2022, 93.5 percent of charter students participated in ELA assessments and 92.9 percent in math assessments. In district schools, however, participation was already lower than charters in 2019 and dropped off even more in 2022. Whereas 84.6 percent of traditional public school students participated in ELA assessments and 82.9 percent in math assessments in 2019, those numbers dropped to 80.3 and 80.5 percent, respectively, in 2022. For many districts- disproportionately those on Long Island- state assessment results may not reflect the broad student population. In the Sachem Central School District, for example, only 31 percent of students participated in ELA assessments and 35 percent in math assessments.

English Language Arts

Statewide, the proportion of students whose scores reached the threshold of “proficient” in English ticked up from 45.4 percent in 2019 to 46.6 percent in 2022. Results track with NAEP scores in that younger grades fared worse, as basic literacy is still being cultivated and self-directed learning poses a greater challenge. Whereas proficiency dropped  from 45.9 percent to 41.8 percent in grades 3-5, it increased from 44.8 percent to 51.5 percent in grades 6-8.

Proficiency declined in charters from 54.0 to 51.6 percent but increased in district schools from 44.7 to 46.1 percent. At first blush the results qualify as a surprise given that New York experienced a surge in charter school enrollment and substantial decline in traditional public school enrollment during the pandemic, largely thanks to perceptions that charter schools more capably navigated COVID-related disruptions.

Given how parents voted with their feet, it would be imprudent to conclude that charters fared worse through the pandemic. Rather, possible explanations for the drop in charter proficiency but increase in traditional public school proficiency include changes in the composition of students who take the test, different priorities in charter schools versus traditional public schools, longer closures of charter schools, test score changes obfuscated by changes in proficiency counts, or that the observed differences are driven by a growing socioeconomic achievement gap, as charters are disproportionately located in urban areas and educate low-income and minority students.

Notably, charters that scored higher than the sector average (i.e. 54 percent) in 2019 declined by 10.3 percentage points while those that scored lower than the sector average gained 1.8 percentage points. Likely, the charters that have historically been most successful at breaking the link between poverty and student achievement simply could not sustain the same level of excellence through pandemic disruptions. The larger proficiency decline among high-performing charters might also reflect the limitations of proficiency as a performance metric. Specifically, score changes might be obfuscated in the lowest-achieving schools and districts, where a significant drop in raw scores might register as a modest change in proficiency rate (e.g. from 15 percent to 12 percent proficient).

Regional breakdowns reveal that the modest uptick in ELA proficiency was distributed evenly across the state, and in fact that all regions experienced a slight increase in ELA proficiency rates ranging from .2 percentage points (Long Island) to 2.4 percentage points (Southern Tier).

Student subgroup analysis similarly reveals that the modest increases in ELA proficiency were not consolidated to any particular groups of students but distributed across various economic and racial groups.

Math

In math, proficiency rates dropped sharply from 48.5 percent to 41.2 percent. Whereas NAEP results indicated that New York experienced a particularly notable drop in fourth grade math compared to eighth grade math, state assessments reveal dramatic decline across grade levels. Proficiency in grades 3-5 declined from 50.4 percent to 42.5 percent, while in grades 6-8 it declined from 42.4 percent to 33.8 percent.

Once again, while charter schools continued to exhibit higher proficiency rates than traditional public schools, they also experienced larger declines in proficiency. In charters, proficiency declined from 60.5 percent to 44.2 percent, whereas traditional public schools declined from 47.5 percent to 40.9 percent. Losses were greater among the historically highest performing charters. Among those charters that had higher than sector average math proficiency rates in 2019 (i.e. 60 percent or higher), proficiency declined by 20.7 percentage points. Among those charters where proficiency rates were less than the sector average in 2019,  proficiency declined by a comparatively modest 12.2 percentage points.

Math proficiency declined in all regions of the state, and nine of ten regions experienced a drop in math proficiency of at least 6 percentage points. The Finger Lakes region emerges as an outlier, as proficiency dropped by “only” 4.3 percentage points, a change that in a typical year would register as a concern rather than a cause for celebration. The Southern Tier profiled as the lowest-performing region in the state on math exams in 2019 but that ignoble distinction now belongs to the Mohawk Valley following a 10.3 percentage point decline. Whereas three regions had proficiency rates exceeding 50 percent in 2019, Long Island is now the only region where the proficiency rate exceeds 50 percent, and the Mid-Hudson is the second highest at only 42.2 percent proficiency.

Student subgroup analysis reveals that Asian students lost the least ground in math, as proficiency declined by a comparatively modest 6.2 percentage points. In other words, already large achievement gaps between Asian students and other students have grown considerably and are larger than ELA achievement gaps. For example, whereas the ELA proficiency rate of Asian students is nearly double that of Black students, the math proficiency rate of Asian students is nearly triple that of Black students.

Proficiency declines were identical (8.8 percentage points) among students eligible for free or reduced lunch compared to those not eligible. So, while racial achievement gaps intensified, economic achievement gaps did not.

Movers and Shakers

Districts that were notable outliers from the overall state trend deserve praise or scrutiny for their performance. On the positive side, among districts where at least 1,000 students tested, West Babylon UFSD experienced a 12-percentage point increase in ELA proficiency and Farmingdale UFSD and Eastport-South Manor CSD experienced increases of 11 percentage points. In math, Rye City SD (+6), Somers CSD (+3), and Herricks UFSD (+1) were the only districts of at least 1,000 students that didn’t experience a decline in proficiency.

On the negative side, among districts with at least 1,000 students, Sayville UFSD was a notable outlier, experiencing a 14-percentage point decline in ELA proficiency. Hauppauge UFSD, Commack UFSD, and Frontier CSD all experienced ELA proficiency declines of 10 percentage points. In math, Monroe-Woodbury CSD declined by 21 percentage points while Suffern CSD and Hamburg CSD each dropped by 19 percentage points.

Overall, among districts where at least 1,000 students tested, Rochester City (8 percent proficient), Poughkeepsie (10 percent), Syracuse (11 percent), and Schenectady (12 percent) profile as the lowest-performing in math while Scarsdale (87 percent), Jericho (87 percent), North Shore CSD (85 percent) and Herricks UFSD (85 percent) profile as the highest-performing. In ELA, Rochester City (13 percent proficient), Syracuse City (17 percent), and Schenectady (21 percent) profile as the lowest-performing while Jericho (88 percent), Scarsdale (85 percent) and Herricks UFSD (83 percent) are the highest-performing.

Despite its status as the lowest-performing district in the state, Rochester City spends $38,143 per student per year as of 2018-19, topping Herricks UFSD by more than $8,000 and Scarsdale by more than $4,500. It only narrowly trails Jericho and North Shore CSD, which spend $39,027 per student and $39,704, respectively.

Deck Reshuffled?

While NAEP results were down across the board, some states lost considerably more ground than others, and the overall picture constitutes a substantial reshuffling of the deck in terms of how states compare to one another. For example, in 2019 New York’s performance in fourth grade math was determined to be “significantly” worse than 25 states, but in 2022 “significantly” worse than 38 states. 

Examination of New York districts reveals that by and large high-performing districts in 2019 continue to lead the pack in 2022 and low-performing districts continue to trail it. In ELA, 86.4 percent of districts that were in the top quartile of proficiency rates in 2019 remained in the top quartile in 2022, and 76.8 percent of districts in the bottom quartile remained there. Only one district (Groton CSD) that was in the bottom quartile of ELA proficiency rates in 2019 moved into the top quartile in 2022, and none moved from the top to the bottom.

In math, 62.7 percent of districts in the top quartile in 2019 remained in the top quartile in 2022, and 75.8 percent of districts in the bottom quartile in 2019 remained there in 2022. No districts that were in the bottom quartile in 2019 moved to the top quartile in 2022, and no districts that were in the top quartile in 2019 moved to the bottom quartile in 2022.

Districts with above average math proficiency rates in 2019 lost only slightly more ground than those that registered as below average (9.1 percent decline versus 8.2 percent decline). The gap is larger but remains modest in ELA. Districts with below average proficiency rates in 2019 on average increased proficiency rates by 2.2 percentage points while districts that were above average lost 0.1 percentage points.

Overall, a few notable exceptions notwithstanding, historically high-performing districts remain high-performing relative to the rest of the state and low-performing districts remain low-performing, with both showing stagnation in ELA proficiency and substantial declines in math proficiency.