The National Math and Science Initiative (NMSI): Designing for Scale through Replication of Existing Programs

The need for stronger math and science education is not confined to one district, one city, or even one state—rather, it is a challenge faced by schools and districts across the country. While many organizations create and introduce new programs into schools to meet this challenge, the National Math and Science Initiative (NMSI) takes a markedly different approach. NMSI identifies existing, proven programs and focuses attention on scaling those programs to a wider audience with the goal of producing strong outcomes for students. As Tom Luce, CEO of NMSI puts it, “You can choose to be one of a thousand pilot lights, or you can choose to light the furnace.” By identifying, refining, and scaling existing programs, NMSI has seen measurable improvements in teacher training and gains in student achievement in math and science across in the nation.

NMSI identifies proven projects related to math and science with the potential for wide-spread success, particularly those focused on producing strong math and science teachers, strengthening existing teachers’ skills, and expanding the pipeline of STEM-capable students. NMSI helps these programs overcome limitations to growth (time, money, national management) so they can survive beyond their early years and thrive. By identifying specific scaling strategies, the programs become long-term components of school districts or university systems across the country, where they can serve large numbers of students and teachers.

Program-at-a-Glance

NMSI was created in 2007 in response to the urgent rallying cry in the National Academies’ landmark report, “Rising Above the Gathering Storm,” which documented the need to improve STEM education and increase investment in research and development in the U.S. In its first year, NMSI targeted two programs—the Advanced Placement Training and Incentive Program (APTIP) and UTeach—for expansion. Both of these programs had proven records of success in Texas for nearly 10 years. NMSI has focused on bringing these programs to significantly greater numbers of schools and students across the country.

The first program, APTIP, increases the number of students in low-performing high schools who take and complete AP math, science, and English courses. This program brings college-level curriculum AP courses to students, which significantly increases their chances of completing a college education. The second program, UTeach, is an innovative, university-based program to produce strong high school math and science teachers by recruiting current math and science majors. Both programs are designed to respond to the pressing national need for well-prepared science and math students and teachers. 

The decision to invest in each of these programs demonstrates NMSI’s focus on both long- and short-term strategies for improving STEM education. APTIP serves students and their teachers with short-term goals for increasing student participation in AP programs each academic year, while UTeach serves prospective science and math teachers with a longer time line of five to seven years before the impact in classrooms is seen.

A key factor in the decision to support these two programs is their alignment with NMSI’s definition of “scalability.” By clearly defining indicators of successful replication and implementation, NMSI sets the bar high while ensuring the design of proposed pilots can successfully be introduced, evaluated, and adapted in new districts, cities, and states. NMSI evaluates projects’ potential for replication with the following measures:

1. Proof of success with documented results of at least five years of measurable success. Evaluation results must be based on objective measures (e.g., standardized test results, increased graduation rates, increased student acceptance into higher education), be easily understood, and show “significant” or “sustained” improvement.
2. Identifiable key components that make the program work and can be replicated. These components provide valuable benchmarks to determine faithful replication, and are tied to performance data (e.g., the number of students who sign up for AP classes in the spring prior to the next academic year).
3. “Milestones” tied to essential elements to enable measurement of successful replications. Marking milestones provides a critical path for new adopters to follow.
4. Clear understanding of program costs and cost-effectiveness. Many factors must be considered but, as a rule of thumb, if the program’s cost is less than 10 percent of the total per student cost in the public education system, then the program can potentially be replicated to serve larger numbers of students or teachers, especially if the program is designed to reach a certain student population.
5. The right entity to manage and implement the program. A credible entity should exist that has the capacity to manage the program for scale up; maintain replication of essential elements as a priority; provide expert support to successfully replicate the program; attract partners and influence policymakers to provide ongoing scale-up resources; and understand and establish an accountability system for meeting benchmarks and timelines.
6. Performance management and rigorous program monitoring. Examples include demonstrating local support for implementation, usually in the form of at least 25 percent matching funds; clear performance benchmarks tied to essential elements, with release of funds tied to reaching the benchmarks; changing of funders’ financial commitment over time as a replication site moves to self-sufficiency with local funds and other resources; and regular monitoring and progress reports.
7. Simple and effective collection and communication of evaluation data. Evaluation should not be too complicated and should focus on reasonable, objective data on performance that can lead to replication. Results should be easy to understand and communicate through charts and graphs with plain language, and also include personal narrative stories.
8. Strong local partnerships and advocacy. This includes local constituencies that are ready for change and can advocate for it. Such non-staff advocacy includes ensuring that those in decision-making positions know about the program, see it in action, receive periodic updates and performance reports, and even serve in partnership roles.

While APTIP’s and UTeach’s respective components differ substantially, they both reflect the essential elements outlined above, which is why they were selected by NMSI for replication. APTIP and UTeach incorporate these elements in the following ways:

APTIP

APTIP, an approach pioneered by AP Strategies in 1996 in Dallas, uses a comprehensive approach to increase teacher effectiveness and student achievement in rigorous AP math, science and English courses. NMSI is replicating the following components of APTIP:

• Teacher support, through AP and Pre-AP teacher training, AP Lead Teachers, and vertical integration team meetings
• Student support, through exam-fee payment, equipment and supplies, and tutoring and study sessions
• Program management, in the form of data collection and analysis, letters of agreement, and access to subject matter experts
• Awards to recognize the commitments of administrators, teachers, and students

UTeach 

NMSI’s UTeach Program, developed at The University of Texas at Austin in 1997, transforms the way colleges and universities recruit, prepare, and inspire new math and science teachers. For starters, they focus on those students who already have the skills and interest in math and science—they recruit existing math and science undergraduate majors to pursue teaching careers through a program that allows students to graduate in just four years with both a degree in math or science and a teaching certification. Jesse De La Huerta, a math teacher and UTeach alumnus, comments that “UTeach gives you the chance to see for yourself...UTeach prepares you so well to teach with different methods, you feel well-prepared from the start."

NMSI focuses on replicating the following elements of UTeach:

• Active recruitment of students by offering the first two courses free of charge
• A compact degree plan that allows students to graduate in four years with a degree in math or science and a teaching certification
• A focus on acquiring deep content knowledge in math and science as well as strong teaching skills that are based on current research
• Courses taught by faculty both in science and math and in education who are actively engaged in inquiry and project-based learning
• Early and intensive field experiences beginning in the first semester so that students are exposed to real classroom settings
• Personal attention and guidance from highly experienced master teachers, faculty, and successful public school teachers.

Outcomes

Thanks to NMSI’s financial support and guidance, APTIP programs have spread to 229 schools in 158 school districts and more than 6,000 AP and Pre-AP teachers in six states: Alabama, Arkansas, Connecticut, Kentucky, Massachusetts, and Virginia. In addition, the APTIP program is being delivered by internet through Learning Power in South Dakota, and it is being introduced in New Mexico and Colorado. As noted by Vinh Duc Nguyen, an APTIP student, “It is a great way to have the college experience while in high school and allows students to be developed and molded into college-ready adults.”

The program continues to deliver remarkable results. For example, results from two cohorts of students show the following: In the first cohort of NMSI APTIP program schools, from 2008–2010, there was a 97.7 percent increase in math, science, and English AP exams passed; a 154.6 percent increase in math, science, and English AP exams passed by African American and Hispanic students; and a 116.4 percent increase in math and science AP exams passed by females. In the second cohort of NMSI program schools, from 2009–2010, there was an 84.6 percent increase in math, science, and English AP exams passed; a 107.3 percent increase in math, science, and English AP exams passed by African American and Hispanic students; and a 91.5 percent increase in math and science AP exams passed by females.

Also through NMSI’s support, 22 universities are implementing the UTeach model, with projected enrollment of 3,862 students for fall of 2010. Since UTeach enrolled its first students in 1997, 92 percent of graduates have immediately gone on to teach math and science. Five years later, 82 percent were still in the classroom, compared with a national teacher retention rate below 65 percent.

While addressing the immediate need for improved math and science education, NMSI’s ultimate goal of scaling practices is focused on creating long-term, sustainable programs and results for students and teachers. To sustain long-term improvement in math and science education in their public schools, NMSI’s state affiliates have built extensive coalitions with strong support from leaders in business, state government, education, and their communities. NMSI’s APTIP and UTeach programs also have been featured as model programs by the business community’s national Change the Equation program and the President’s Council of Advisors on Science and Technology (in its September 2010 report to the President, “Prepare and Inspire: K-12 Education in Science, Technology, Engineering and Math (STEM) for American’s Future”). In addition, the UTeach model is consistent with the STEM training grant program included in the recent reauthorization of the American COMPETES Act of 2010. In the end, NMSI is working to ensure that all students, no matter what their financial circumstances, race, or gender, have access to careers in the promising fields of science, technology, and engineering.