What difference did $190 billion make for student success coming out of the COVID-19 health crisis?

Not as much as you might think.

An ESSER spending analysis by Edunomics Lab at Georgetown University found some puzzling instances where funneling more money into a pandemic-worsened problem didn’t help schools recover.

The data ultimately points to no “silver bullet” in spending aimed at improving students’ academic performance since the pandemic, says Marguerite Roza, director of Edunomics Lab.

Return on Investment

A crunch of the numbers found that states varied widely when it came to the return on investment of their ESSER dollars. Both reading and math scores increased in districts in states like Mississippi, North Carolina and Tennessee, where the rate of ESSER spending per student was relatively high (over $1,000) from 2022 to 2023.

States like Nevada, California and South Dakota were also high spenders, but they saw some of the lowest gains in reading and math during the same time period.

Analysts said the difference likely came down to leadership in some states being “simply more effective at steering districts to focus on student learning” in the face of vague spending guidelines from the federal government. Leaders in Mississippi, North Carolina and Tennessee focused on setting clear goals and checking progress for reading and math performance.

Internal Competition

The analysis also found that ESSER spending led to a staff recruiting conundrum within school districts: In some cases the emergency funds were spent at higher rates at a district’s wealthier schools, despite similar needs at higher-poverty schools for additional staff.

Roza says that district leaders whom researchers interviewed about the problem reported staff vacancies were typically filled first at schools with lower poverty rates, sometimes with district employees looking to transfer out of schools with higher poverty levels.

That exacerbated the staffing issues at higher-poverty schools, which were then left trying to fill newly created ESSER-funded jobs and replace staff who moved across the district.

“Let’s say San Diego said, ‘I’m going to put a reading coach or a nurse or parent coordinator in every school,’” Roza says. “Immediately those positions got filled in their most wealthy schools first. We see this all the time where teachers migrate to less-poor schools, so you might have just actually created yet another opening in your high-poverty school.”

Not Looking Back

When it came to contracts for services like tutoring or edtech that were funded by ESSER, Roza says she and her fellow researchers found that school districts frequently renewed those contracts the following year without ever reviewing whether they were worth the money.

That was one of a litany of ways that contracted services were running ineffectively, according to the analysis.

“Let's say you had a restaurant or something like that. If they're going to spend money on a vendor product, they're gonna make sure they get the value out of it,” Roza says, “or they're going to cut that, because it matters for their bottom line. The market isn't functioning as well in the public education space.”

Roza says there’s no one person to blame in the system. An example of how money might go to waste through a contract, she offers, would be when a math coordinator requests a program for teachers that they end up not using much. But then the coordinator leaves their position for a promotion or a job outside the district, and their replacement renews the unused program without investigating whether it’s needed — it was simply part of the budget they inherited.

“[Districts] got all this new money, and some of them did indeed spend more money on the vendors that have good products across the board,” Roza says. “But they're not necessarily buying the best products, or getting what they need, or making the most of what they bought, or checking if it even worked. We hear this even from the vendors who are frustrated with this.”

Reading Outcomes

Edunomics Lab’s analysis found that when it came to improving reading scores, identifying more students with reading disabilities did not always lead to improvement in their reading ability.

Where did investment in reading pan out? Roza says districts were most successful when they first improved reading instruction for general education students — namely through instruction that relies on the science of reading. By the time a student immersed in that type of reading instruction is identified as needing special education services, Roza explains, they will already have a strong foundation on which to keep building.

“Fewer of them are even going to get referred to special ed for reading disabilities,” Roza says, “because having that kind of good core instruction from the beginning really helped them.”

Broken Budget Process

Similar to the problems with contract work, the analysis found that school districts tended to continue spending ESSER funds on programs for multiple years without reviewing their outcomes.

Part of the problem is that district budget cycles require them to finalize budgets for an upcoming year before getting standardized test results from the previous year, leaving little or no room to adjust spending based on student performance.

In one case, Roza says, a district leader reported having to finalize a budget an entire year in advance, locking it into spending that may or may not be ideal for students’ needs.

“In the first full year pandemic-relief funds were available from the American Rescue Plan, districts spent only 14 percent of the grant funds, largely because those monies were held up in district budget cycles that left no room for a more nimble, urgent response,” according to the analysis.

Ultimately, Roza says that while the averages in the data help researchers describe the relationship between ESSER spending and student outcomes, that doesn't mean the average reflects the reality of every school district. There are big differences between districts, she says, and some of them buck the trends.

“People keep wanting to say ‘the average district,’ and the average includes districts that have gotten great outcomes and some that have not,” Roza cautions, “so it's not great to try to apply our average findings to every individual district.”