Changing Instruction and Improving Student Learning: Lessons from Comprehensive School Reform

Brian Rowan
University of Michigan
2007

Prepared for the Carnegie-IAS Commission on Mathematics and Science Education

Perhaps the most frequent observation made about education reform in the United States is that it has failed, especially in producing deep or fundamental changes in teaching and learning in America’s K-12 schools. Looking just at mathematics and science education since the 1960s, for example, we’ve witnessed repeated efforts to push new curricular materials into schools, to upgrade the collegiate and in-service training of math and science teachers (as well as to increase the number of highly qualified teachers in these subject areas), and, since the late 1990s, to ratchet up education standards and test-based accountability systems in the 50 states. Over this same time period, NAEP scores in mathematics and science have risen (ever so slightly, and more for poor and minority students than for others). But most observers agree that K-12 schools continue to teach mathematics and science in ways that are far from ideal, and that as a result, students are learning less (and different things) than we want them to. It’s not that things have gotten worse in U.S. education as a result of decades of education reform. They haven’t. The big surprise is that things haven’t improved that much.

Making Reform More Productive

Despite these trends, I avoid the rhetoric of failure in describing the history of education reform in the United States. To be sure, scholars have developed a lot more arguments about why education reforms fail than they have about the conditions under which reforms succeed. What is remarkable, however, is how often these explanations blame the education system and/or the people who work in it, and how rarely they point to the nature of education reforms themselves as an explanation for failure.

What I want to do in this briefing paper is make a case that we actually know a lot about how to make fundamental changes in teaching and learning in K-12 schools. To build that case, I will first give some examples of prior education reform efforts, most of which, I will argue, only partially succeeded in changing instruction and student learning in K-12 schools. But I will also argue that these cases provide some important lessons about how to design more powerful reform efforts in the future. The argument proceeds historically. I’ll begin with some partial successes, and then move toward more successful approaches to education reform.

New textbooks. In the 1960s, policy makers were intent on reforming mathematics and science education through careful curriculum reform—usually as embedded in textbooks. A number of “new” math and science texts (and associated materials) were produced at the time, and although there was always some debate about the quality of the mathematics and science included in these texts, the texts were in fact widely adopted by schools. Thus, I find the case for using textbooks in any reform effort to be quite clear. Textbooks are the main tool of classroom teachers, and new texts (even when they are innovative, or perhaps even misguided) do find their way into classrooms. There is a hunger for new and better materials in the K-12 system.

As I tell it, the story of education reform in the United States is about designing better approaches to producing instructional change in schools. And, as I see it, we’ve learned a lot about this problem.

We should not, however, rely solely on textbooks as a reform strategy. We know from decades of research, for example, that teachers pick and choose what to teach from textbooks, and that they vary greatly in how they teach the content they choose. In addition, research suggests that the more novel the content being taught, the less prepared teachers are to teach it well. The usual recommendation, then, is to provide training to teachers to better prepare them for using new materials. As we shall see, that’s a good idea, but it requires far more intensive training and guidance than is provided by the usual summer institutes and one-shot workshops that typified textbook reforms in the 1960s.

Increased Graduation Requirements. Another strategy tried during the 1980s was to improve mathematics and science education by ratcheting high school graduation requirements. By many measures, this too was a successful reform initiative. We know from research, for example, that 33 states increased high school graduation requirements during the 1980s, and we also know that students who ended up taking more advanced math and science classes in high school scored higher on achievement tests. This is remarkable given the lack of any federal incentives to increase graduation requirements, and suggests a strong public demand for increased rigor in the schools. But there was a downside to this reform. In states where graduation requirements went up, so did dropout rates. Taken as whole, then, this reform produces both positive and negative lessons. A positive lesson is that students can learn new and more advanced content if they are exposed to it. A less positive lesson seems to be that efforts to ratchet up content coverage in schools probably should begin earlier (rather than later), or at least occur across the grades.

Standards Based Reform and Accountability. By the 1990s, this is just the route that education reformers took. At this point, school systems around the country began to develop new (and better) academic standards for all grades (not just high schools) and to tie these standards to tough assessment systems. Although federal efforts made this trend widespread by the 1990s, the nature of standards and assessments varied widely across states. But research suggests that new standards and accountability systems resulted in some real changes in instruction. One obvious effect involved a shift in instructional time away from non-tested subjects toward tested subjects. Another was the tendency for teaching to become more aligned to and consistent with state assessment formats. So, one obvious lesson from standards based reform is that teachers will teach the academic content that is tested, and they will mimic test formats in their instruction. When the tests and test formats are awful, teaching is awful. But when the tests and test formats are better, teaching gets somewhat better too. In addition, research on standards based reform suggests that in states where standards based reforms were more intensive, changes in instruction were more consequential. In California, for example, teachers who went to state-organized professional development workshops and actively used replacement curriculum units in their classrooms were more likely to teach in ways that were consistent with the aims of the reform mathematics framework the state developed. So, the more intensive the efforts were to support teacher change, the deeper the reform outcomes.

...successful instructional reform is intensive and multi-dimensional.

Comprehensive School Reform. The final initiative I want to discuss is Comprehensive School Reform. The origins of this reform effort lie in the New American Schools Development Corporation’s (NSDAC) funding of several “break the mold” whole school reform designs, which were in turn supported by the federal Comprehensive School Reform (CSR) program. This effort differed from most previous efforts by supporting not-for-profit “design teams” to develop academic programs for adoption by and use in schools. Over a ten-year period, hundreds of new “whole school” designs were invented, and these were adopted by somewhere between 10% and 20% of all elementary schools in the country. Research showed that there was a great deal of program-to-program variation in the extent to which these whole-school reform programs succeeded in changing teaching practices in elementary schools and in improving student achievement. But some clear lessons emerged from the research about which programs were effective and which weren’t. In general, the more effective programs:

• Were built around a coherent, scientifically-based curriculum at each grade level;

• Had clearly specified learning standards, along with embedded assessments;

• Provided very strong instructional guidance to teachers, both about what to teach and how to teach (indeed, many programs went so far as to provide teachers with “scripted” lesson plans that could be followed in sequence over the school year);

• Included extensive professional development and on-site coaching, typically over a period of three years; and

• Involved monitoring for fidelity of implementation both by local leaders and program staff.

Putting the Lessons Together

As I tell it, the story of education reform in the United States is about designing better approaches to producing instructional change in schools. And, as I see it, we’ve learned a lot about this problem. The main lesson I draw from studying four decades of education reform is that successful instructional reform is intensive and multi-dimensional. To fundamentally change instruction in schools and classrooms, it seems we need:

(a) clear academic standards to guide curriculum making;

(b) useful tools to frequently monitor students’ achievement;

(c) new curriculum materials aligned to these standards and tests that teachers can use, (d) systems of professional development that provide teachers with very intensive and explicit guidance about how to teach using these materials; and

(e) close monitoring of instructional processes to assure faithful implementation of the new instructional program. The best exemplars of this type of reform regimen are not the “weak” and “partial” reforms pursued in the U.S. for most of four decades. Rather, they are the highly intensive and specified kinds of reforms embodied in at least some Comprehensive School Reform (CSR) programs.

Coda

Am I optimistic that such intensive and specified reform is possible on a large scale in the United States? On a positive note, it seems that as a result of NCLB, the motivation to improve student achievement is stronger than ever. The key problem, it seems to me, is that federal, state and local government agencies lack the capacity to invent and implement the kinds of coherent, intensive instructional designs and support systems that research suggests are required to produce marked changes in classroom instruction and marked increases in student learning.

Where can we find such capacity? There are useful lessons from comprehensive school reform here. What we need, I think, are more “design teams.” These teams would, of course, include mathematicians and scientists, as well as education program developers. But this was the kind of team that designed the “new” math and science texts of the 1960s, which were used so variably in schools. So, in my view, in addition to scientists and program developers, we also need people who actively promote faithful program implementation inside schools. In my experience, this is a rare kind of team. Universities are good at teaming scientists and program developers, but they seldom have the capacity or interest to take newly developed programs to scale. By contrast, textbook producers have lots of incentive to take new “programs” to scale, but little incentive to do anything but provide the very least amount of research and development and on-site support for implementation, since efforts in that direction erode profit margins. So, what I am calling for is more “design teams” of the sort initially encouraged by NASDAC and the federal CSR program.

Thinking about how to create such designs, and bring them to scale, is a different sort of policy agenda than we have typically pursued. It is not an agenda that American government is well-positioned to take on (as it is in at least some other nations). Instead, it seems to call for greater public and private investment in the development of coherent instructional designs that are intended to operate at scale inside schools and classrooms; it calls for rethinking the kinds of organizations that are needed to develop and operate such programs; it calls for thinking about the kinds of incentives to be used to encourage local schools and school systems to adopt such coherent designs; and it requires developing new ways to regulate the market for school improvement so as to identify which programs work and which don’t. This is a big agenda—beyond the scope of this small briefing paper—but also a fruitful one. For if there is one lesson I draw from studying decades of education reform, it is that fundamental changes in teaching and learning occur most prominently when outside developers work intensively inside schools to implement strong and coherent designs for instructional practice.