Opportunity Culture Student Outcomes
Brookings-AIR Study Finds Large Academic Gains in Opportunity Culture
According to a rigorous third-party evaluation, students of teachers who served on teams led by multi-classroom leaders (MCLs) showed sizable, statistically significant academic gains. The team teachers were, on average, at the 50th percentile in the student learning gains they produced before joining a team led by an MCL. After joining the teams, they produced learning gains equivalent to those of teachers in the 75th to 85th percentiles in math, and, in six of seven statistical models, from the 66th to 72nd percentiles in reading. The median MCL team size was five teachers, and the average was six.
The effects appear to be both direct—with performance improving for individual students in MCL classrooms—and indirect, with whole schools’ growth rising when MCLs begin leading even part of a school. The study compared schools that had Opportunity Culture classrooms with schools that had none, as well as before-and-after results for schools that have implemented Opportunity Culture.
With these results year after year, students beginning kindergarten one year behind can catch up by the end of second grade. It’s a longer haul for reading, but the same kindergartners can still catch up before middle school. Then, teachers can help far more students leap to advanced levels.
The study, by Ben Backes of the American Institutes for Research and Michael Hansen of the Brookings Institution, covered about 15,000 students and 300 teachers. Three-quarters of Opportunity Culture schools in these districts were eligible for Title I funds in 2015–16. Researchers controlled for factors including student background and prior performance.
The Brookings-AIR Study & the Opportunity Culture Principles
What did we learn from this study about the five Opportunity Culture Principles, which Opportunity Culture districts and schools must follow?
- Reach more students with excellent teachers and their teams. Among multi-classroom leaders (MCLs) whose teaching effectiveness could be quantified through student learning growth data, all were in the top quartile of effectiveness before being selected for the role. On average, MCLs reached six teachers’ worth of students.
- Pay teachers more for extending their reach. The study did not examine the impact of pay on effectiveness. The district in the study with the highest pay supplements for MCL roles also achieved the highest level of math and reading gains by MCLs’ teams.
- Fund pay within regular budgets. All districts in the study funded OC pay supplements entirely within regular budgets.
- Provide protected in-school time and clarity about how to use it for planning, collaboration, and development. The study’s MCLs for whom data were available led a median of five and an average of six teachers. Small teams make it possible to schedule collaborative planning and development time for the whole team as well as one-on-one planning and feedback time for the MCL and each team teacher.
- Match authority and accountability to each person’s responsibilities. The two districts that assigned clear accountability to MCLs for all the students served by their teams had the strongest academic gains for MCL teams.
The study examined two ways teachers extended their reach: reaching more students directly, usually with extra paraprofessional support, and guiding teaching teams as multi-classroom leaders (MCLs). In contrast to the big gains seen under MCLs, the researchers found mixed results for teachers who reached more students directly without an MCL’s leadership. In addition, the study found, these interventions reached fewer students.
Specifically, expanded-impact teachers, whose roles were not designed to use extra digital instruction, produced gains similarly positive to MCL teams but more varied, and thus not statistically significant in half of the researcher’s models. Blended-learning teachers, whose roles were designed to use more digital instruction than was typical in their schools, produced gains that were much smaller or negative in the statistical models. None of these teachers were on MCL-led teams.
Interestingly, the study found positive, though smaller, benefits even for students whose teachers were not led by an MCL when at least one MCL was in the building. This situation typically occurred as schools were adding MCLs over a few years to cover the whole school. Some schools tried other turnaround strategies concurrently, but this appears to have been a minority of the schools in the study. The researchers acknowledged that the immediate schoolwide gains could indicate positive “spillover” effects of Opportunity Culture.
Indeed, interviews with teachers suggest spillovers from Opportunity Culture. Teachers reported that Multi-Classroom Leadership changed the culture of their entire schools: Collaboration, coaching, and support for everyone quickly became the norm. In schools that didn’t yet have MCLs for all teachers (such as those in early transition years), some MCLs reported that they were coaching teachers outside their teams. Schools also filled other non-OC teaching vacancies with strong applicants who had sought advanced OC roles but were not selected for them initially. Some or all of these factors undoubtedly affected student learning, even if the study could not tell which ones made a difference.
Importantly, the researchers found that among MCLs whose teaching effectiveness could be quantified through student learning growth data, all were in the top quartile of effectiveness before being selected for the role. The same was true of people selected for direct-reach roles; the difference in outcomes appears related to the roles, not the prior teaching effectiveness of people in the roles. These outcomes have led Public Impact to stress with all new Opportunity Culture districts the need to place all teachers on teams led by multi-classroom leaders, along with ensuring that the selection of those for advanced roles keeps prior effectiveness primary.
With a median team size of five teachers and an average of six, excellent teachers in the MCL role could reach all students and teachers nationally.
Long-Term Analysis of Impact on Student Learning Growth
Public Impact’s own annual data analyses of long-term outcomes through 2018–19 indicate that in the fourth year of implementation Opportunity Culture schools are more than twice as likely to make schoolwide high growth as those in their first year, and the odds of making high growth schoolwide are over 50 percent higher than other schools in the same states. Public Impact will continue annual and longitudinal data analyses in years when student learning growth data are available and encourage further third-party research.