Explain It Like I'm Nine: Low Substitute Fill Rates
Can’t find enough substitute teachers to cover teacher absences? Seems...
Last June, the RAND Corporation published research looking at the Bill & Melinda Gates Foundation’s Intensive Partnerships for Effective Teaching initiative. The initiative worked with three school districts and several charter management organizations to redesign teacher evaluations that factored in not only classroom observations, but also student growth measures.
The Gates Foundation’s goal? To improve teaching effectiveness and, as a result, student outcomes.
The results? The researchers found that while most teachers believed that the redesigned evaluation systems had a positive impact on their teaching, it failed to improve student performance and graduation rates.
The research brief goes into much more depth, of course, pointing to incomplete implementation, potentially conflicting goals and changes in budgets, statewide tests, leadership and other factors as possible reasons for this lack of success.
In a recent episode of Field Trip, Frontline’s podcast about leadership in education, we spoke with Dr. James Stronge, president of Stronge and Associates Educational Consulting, LLC, to get his thoughts on the report. Here are some of the highlights. (Note: the text below is from select portions of the interview, and has been edited for brevity and readability. To hear the entire interview, please listen to the podcast.)
I don’t think there’s much in the report to disagree with. They’re reporting the facts and then interpreting what they found. The place that I would disagree is on the causes. I doubt seriously that the design of the evaluations in those districts was the serious problem. I think instead it had to do with flaws in implementation.
“I doubt that the design of the evaluations was the serious problem. I think instead it had to do with flaws in implementation.” – Dr. James Stronge on the Intensive Partnerships for Effective Teaching Initiative
One of the problems that I see in the implementation in the Gates Foundation-funded projects is the emphasis that was placed on how data are collected and interpreted, what data were used. And in many of those districts, there was still a very heavy emphasis on classroom observation. Classroom observation is flawed as well. There’s been a great deal of writing and controversy around using value-added measures, but what people sometimes forget is that observation is flawed.
Were principals following the implementation procedures properly? The end result that you can’t argue with is that in the vast majority of cases, teachers are perfect. All teachers receive whatever the highest rating is. If it’s a five point scale, they get a five. If it’s a four point, they get a four.
Ultimately what happens in many schools is the evaluations get reduced to a one- or two-point scale. The districts think they have a four point summative scale, but they don’t. They never use the bottom two points. And consequently, the only distinction is between “effective” and “highly effective.” That’s a flaw in implementation.
Teachers have an enormous impact. We know that, and we know what good teachers do. The fallacy comes in how we assess that effectiveness. My worry is that because of reports like [the one released by] the RAND Corporation, and pushback that is constantly being felt across all the states, this initiative will be completely dropped, and we won’t return to it for another decade or so, and we’ll realize the mistake we’ve made.
There were mistakes made in implementation, mistakes made in rushing to design, but to throw all of that out and to say that, “Well, okay, evaluation is not worth doing. What’s next?” — that’s a huge mistake.
I don’t think that evaluation is the only reform that we need by any means. In fact, I have a bit of skepticism about evaluation being the best place to start. But I do think it’s an important reform, and quitting at this point will waste not hundreds of millions of dollars, it will waste billions of dollars.
“If we give up on distinguishing between effective, highly effective and less effective teaching, we’re never going to improve in our schools.” – Dr. James Stronge
And more importantly, it’s going to impact the lives of kids. If we give up on distinguishing between effective, highly effective and less effective teaching, we’re never going to improve in our schools. We’re going to be exactly where we were in the past and we’re going to remain there in the future, if we give up on that effort.
Hire and retain the absolute best principals. There is evidence that suggests it takes four or five years for a principal to begin to have a real footprint in that school, and if it’s an effective principal, that’s going to be a positive footprint.
How can schools maximize the impact that evaluations have on teaching? Dr. James Stronge: “Hire and retain the absolute best principals.”
We’ve known for a long time from research from the Dallas Public Schools that the quickest way to turn around a school, for good or bad, is to change the principal. Quality principals get quality results. And a quality principal will know good teaching. That person will be a medical practitioner, essentially, and be able to diagnose what’s effective and what’s not effective that’s occurring in a classroom, and then be able to prognose and say, “Here’s what you can do to get better and to improve.”
That good principal will follow up, give support, not just say, “You need to get better,” but will say precisely, “Here’s how you can get better, and we’re going to be there to support you in getting better.”
And then finally, he’ll follow up to verify that improvement has occurred. One additional thing we know from research on the best principals is that they will not condone poor teaching. They help teachers find another job if they don’t get better. Principals, while their direct influence on student achievement is in the range of five to ten percent of the total amount, they have a much greater influence. They hire teachers, they support teachers, they develop teachers, they keep teachers. If I could do anything immediately, that would be it.
James H. Stronge is President of Stronge and Associates Educational Consulting, LLC, an educational consulting company that focuses on teacher and leader effectiveness with projects internationally and in many U.S. states. Additionally, he is the Heritage Professor in the Educational Policy, Planning, and Leadership Area at the College of William and Mary, Williamsburg, Virginia. His research interests include policy and practice related to teacher quality and teacher and administrator evaluation. His work on teacher quality focuses on how to identify effective teachers and how to enhance teacher effectiveness. Dr. Stronge has presented his research at more than 350 regional, national, and international conferences and conducted workshops for educational organizations throughout the U.S. and internationally. Additionally, he has worked extensively with local school districts and states on issues related to teacher quality, teacher selection, and teacher and administrator evaluation. Dr. Stronge has been a teacher, counselor, and district-level administrator, and has authored, coauthored, or edited 30 books and more than 150 articles, chapters, and technical reports.
 Stecher, B. M., Holtzman, D. J., Garet, M. S., Hamilton, L. S., Engberg, J., Steiner, E. D., . . . Chambers, J. (2018, June 21). Intensive Partnerships for Effective Teaching Enhanced How Teachers Are Evaluated But Had Little Effect on Student Outcomes. Retrieved December 18, 2018, from https://www.rand.org/pubs/research_briefs/RB10009.html