[SoDu Reading Room] How Effective Are Education Innovations?

In the latest SoDu Reading Room installment, we check out an article shared by KQED that was written by Jill Barshay and produced by The Hechinger Report, a nonprofit independent news organization focused on inequality and innovation in education.

The short version: Federally funded education innovations don’t seem overwhelmingly effective for students, with about 26% of Investing in Innovation (i3) programs yielding any positive benefits and no harms.

Here are five takeaways from the piece:

1. Low Success Rate of Educational Innovations: Despite significant investment ($1.4 billion between 2010 and 2016), the i3 program had a low success rate, with only 26% of the initiatives demonstrating a positive impact on student outcomes without causing harm. This highlights the challenge of implementing effective educational changes at scale .
2. Challenges of Rigorous Testing: Many of the grants under the i3 program did not yield conclusive results, partly because conducting rigorous educational research is complex. It involves setting up control groups and managing numerous variables that can affect student performance, such as home environment and individual student needs .
3. Examples of Success and Failure: Some programs, like Building Assets, Reducing Risks (BARR), demonstrated success and scalability, showing the potential for certain types of innovations. Conversely, programs like Reading Recovery had initial success but failed in long-term impact, illustrating the difficulty in predicting long-term outcomes of educational interventions .
4. Importance of Larger Sample Sizes and Implementation Study: The findings suggest that larger sample sizes are necessary to produce statistically significant results and avoid outcomes that might appear as flukes. Also, studying the implementation process itself is crucial to understand and overcome barriers to success in real-world classroom settings .
5. Shift in Strategy in Successor Programs: Following the lessons learned from the i3 program, its successor, the Education Innovation and Research (EIR) program, focuses on smaller scale-up grants and practical adjustments to ensure innovations can be effectively replicated and sustained. This approach aims to refine the application of innovations rather than broad national scale-ups.

What do you think of the findings? What innovations are working in your schools?