It was announced by sweeping statements.

When the New York City Public Schools chancellor, David Banks, caused the largest district in the country to change how it taught students to read last year, it was with a sense of alarm. Statistics showed that many of the city’s students in third through eighth grades couldn’t read proficiently, which Banks blamed on the city embracing a “fundamentally flawed” approach to reading instruction. Per reporting from The New York Times, he told parents: “It’s not your fault. It’s not your child’s fault. It was our fault.” Reforming, Banks said, was “the beginning of a massive turnaround.”

The sentiment wasn’t isolated to New York, with almost all states having passed some legislation in the last few years to correct course on how reading is taught. These changes, called a “decisive victory” in the long-standing “reading wars,” have pitted education research favoring phonics-based instruction against other ways of teaching students to read, including word recognition. In the wake of the shift, a prominent curriculum group dissolved and the educational publisher Heinemann reportedly experienced sagging curriculum sales. Meanwhile, students still struggle to read.

But these recent education scraps in reading have also caused fresh uneasiness, as some observers begin hunting for the next education reform effort to go bust — perhaps in math next time.

There’s an unvoiced assumption behind this — that education is prone to “fads.” So where does this perception come from? And is it accurate?

The Reform Merry-Go-Round

Fad is the wrong word, says Larry Cuban, an emeritus professor at Stanford University who writes a blog about school reforms.

For Cuban, reform movements appear to be caught in a loop, attempting similar changes “again and again.” But it’s not that schools are constantly being burned by the latest craze. It’s that they’re suffering from deep structural problems, and they seem not to learn from the long history of school reforms.

The lesson? Public schools are particularly vulnerable to pressure, Cuban said on a call with EdSurge. That’s because national problems tend to become school ones, Cuban says. Schools have to walk a “tightrope,” striking a balance that is both stable for students and able to adapt to changes in the broader society, he says.

Pressure on schools to respond to new issues often ends up altering curricula or introducing new courses, because that’s the easiest part of the public education system to change, Cuban argues. But classrooms are isolated from the superintendent’s office, the school board and other “policy elites” who push change, he says.

For example, he adds, when it became known that teenage driving was causing road deaths, driving became part of public school curricula. When drugs became a national concern, schools added anti-drug curricula. “When the nation has a cold, schools sneeze,” Cuban says, adding that it’s an old cliche that turns out to be true.

That focus — the classroom, where abstract ideas about school meet real students — is a common sticking point, according to other observers as well.

It's not that specific reform ideas are fads, argues James Stigler, a distinguished professor of psychology at the University of California, Los Angeles. It's that schools seem susceptible to fads because people don't understand what it means to take an idea seriously, he says.

In reality, many ideas out there haven’t been properly tried out, because that would mean focusing largely on how they are put into practice in classrooms, he adds. There are probably a lot of ideas out there that are effective, he says — but nobody knows what they are.

To Ronald Gallimore, a distinguished professor emeritus at the University of California, Los Angeles, these efforts are sincere. Reform advocates believe they are on the cusp of something that will really work this time, he says. But they may not be aware of the history of instruction. It also doesn’t help that the U.S. has a highly decentralized school system, with schools being locally controlled, making it hard to make uniform sweeping changes to how students learn, he adds.

So how would teachers know if a proposed reform is effective?

Can You Prove It?

Evidence is the magic word, says Adrian Simpson, principal of St. Mary's College at Durham University in England and professor of mathematics education.

It’s also the source of part of the problem.

Those questing for evidence-based education approaches tend to rely on randomized controlled trials, a robust form of study widely used in medicine to establish causation, Simpson notes. In education, that can mean field experiments that show a practice worked in a particular context or laboratory experiments in cognitive science, he says.

“But what [these] tell you is very powerful, but very narrow,” Simpson says.

These studies are taken to show that certain approaches work. But, Simpson says, they only really establish that the sum of all the differences in interventions caused learning for some participants. Which specific intervention worked, and whether it would work for other students, is hard to determine, Simpson says.

That also puts pressure on how changes are carried out in the classroom.

Imagine the best teacher. How much time goes into designing his or her lessons, refining them, and adjusting to individual differences? asks Gallimore, the retired professor. That's what makes implementation of any reform effort so difficult, he says: to go from a general idea down into the details of making it work for a specific group of students, often across a range of different learning contexts.

So it’s tricky to translate the lessons of these experiments into learning.

Researchers also understand less about the mechanisms of how people think about, say, fractions than how kidneys function, according to Simpson, of St. Mary's College. So the evidence provided by experiments about specific practices in education is weaker than in other areas like medicine where it tends to be similar from person to person: “You can’t establish laws of the classroom that will apply everywhere,” Simpson says.

Ultimately, there’s no quick fix for the reform cycle, Simpson says. But he thinks teachers could learn from public health medicine, which is striving to make its interventions more attuned to personal peculiarities. Teachers should bring together insights from a number of sources — from research about memory capacity to tips from the teacher next door — to inform how they unlock learning for their students, he suggests. Rather than asking what they can do to make a student better with fractions, a teacher might ask: “What’s causing this child to handle fractions poorly?” That could provide an insight that isn’t solely focused on teacher interventions which could, nonetheless, help the student learn, Simpson says.

To Stigler, of UCLA, it’s hard to know what works in education right now.

Reform movements need to focus more on getting disciplined plans for moving from the idea phase to the implementation phase, he says. Teachers also need the time to make sure ideas have been effectively put in place, he adds.

Without that, Stigler says, nobody knows what’s truly effective.