Skip to content

Blog

7 Steps to Monitor Progress on Tier 1/Classroom Interventions

When I visit schools as an RTI/MTSS consultant and talk with teachers about Tier 1/classroom academic interventions, I often hear frustration over the difficulty of collecting and interpreting data to monitor student progress. Yet, the critical importance of data is that it ‘tells the story’ of the academic or behavioral intervention, revealing the answers to such central questions as:

  • what specific skills or behaviors does the student find challenging?
  • what is the student’s baseline or starting point?
  • what outcome goal would define success for this student?
  • has the student reached the goal?

If the information required to answer any of these questions is missing, the data story becomes garbled and teachers can find themselves unsure about the purpose and/or outcome of the intervention.

While following a guide does not eliminate all difficulties in tracking Tier 1/classroom interventions, these 7 steps will help the educators you work with ask the right questions, collect useful data and arrive at meaningful answers at Tier 1.

STEP 1: What skill or behavior is being measured?

The first step in setting up a plan to monitor a student is to choose the specific skill or behavior to measure. Your ‘problem-identification’ statement should define that skill or behavior in clear, specific terms.

Keep in mind that a clear problem definition is a necessary starting point for developing a monitoring plan[1]: “If you can’t name the problem, you can’t measure it.”

STEP 2: What data-collection method will best measure the target skill or behavior?

Next, select a valid, reliable and manageable way to collect data on the skill or behavior the instructor has targeted for intervention. Data sources used to track student progress on classroom interventions should be brief, valid measures of the target skill, and sensitive to short-term student gains.[2]

There are a range of teacher-friendly data-collection tools to choose from, such as rubrics, checklists, Daily Behavior Report Cards (DBRC), Curriculum-based Measures (CBMs), teacher logs and student work products.

STEP 3: How long will the intervention last?

When planning a classroom intervention, the teacher should choose an end-date when he/she will review the progress-monitoring data and decide whether the intervention is successful.

A good practice is to run an academic intervention for at least 6-8 instructional weeks before evaluating its effectiveness. Student data can vary significantly from day to day[3]: Allowing 6-8 weeks for data collection permits the teacher to collect sufficient data points to have greater confidence when judging the intervention’s impact.

[ctt template=”9″ link=”yebN9″ via=”yes” nofollow=”yes”]Run an #RTI / #MTSS intervention for at least 6-8 instructional weeks before evaluating effectiveness. Read why here [/ctt]

STEP 4: What is the student’s baseline performance?

Before launching the intervention, the teacher will use the selected data-collection tool to record baseline data reflecting the student’s current performance. Baseline data represents a starting point that allows the teacher to calculate precisely any progress the student makes during the intervention.

Because student data can be variable, the instructor should strive to collect at least 3 data points before starting the intervention and average them to calculate baseline.

STEP 5: What is the student’s outcome goal?

Next, the teacher sets a post-intervention outcome goal that defines the student’s expected performance on the target skill or behavior if the intervention is successful (e.g., after 6-8 weeks). Setting a specific outcome goal for the student is a critical step, as it allows educators to judge the intervention’s effectiveness.

Teachers can use several sources to calculate an outcome goal[4]:

  • When using academic CBMs with benchmark norms, those grade-level norms can help the instructor to set a goal for the student.
  • Classroom Norms. When measuring an academic skill for which no benchmark norms are available, the teacher might instead decide to compile classroom norms (i.e., sampling the entire class or a subgroup of the class) and use those group norms to set an outcome goal.
    Real-world Example:

    A teacher with a student who frequently writes incomplete sentences might collect writing samples from a small group of ‘typical’ student writers in the class, analyze those samples to calculate percentage of complete sentences, and use this peer norm (e.g., 90 percent complete sentences) to set a sentence-writing outcome goal for that struggling writer.

  • Teacher-defined Performance Goal (Criterion Mastery). Sometimes, the instructor must write an outcome goal — but will have access to neither benchmark norms nor classroom norms for the skill or behavior being measured. In this case, the teacher may be able to use his or her own judgment to define a meaningful outcome goal.
    Real-world Example:

    A math instructor wishes to teach a student to follow a 7-step procedural checklist when solving math word problems. The data source in this example is the checklist, and the teacher sets as the outcome goal that — when given a word problem — the student will independently follow all steps in the teacher-supplied checklist in the correct order.

TIP: For a student with a large academic deficit, the teacher may not be able to close that skill-gap entirely within one 6-8-week intervention cycle. In this instance, the instructor should instead set an ambitious ‘intermediate goal’ that, if accomplished, will demonstrate the student is clearly closing the academic gap with peers. It is not unusual for students with substantial academic delays to require several successive intervention-cycles with intermediate goals before they are able to close a skill-gap sufficiently to bring them up to meet their grade-level peers.

STEP 6: How often will data be collected?

The more frequently the teacher collects data, the more quickly she/he will be able to judge whether an intervention is effective.[5] This is because more data points make trends of improvement easier to spot and increase instructors’ confidence in the overall direction or ‘trend’ of the data.

Ideally, teachers should strive to collect data at least weekly for the duration of the intervention period. If that is not feasible, student progress should be monitored no less than twice per month.

STEP 7: How does the student’s actual performance compare with the outcome goal?

Once the teacher has created a progress-monitoring plan for the student, she/he puts that plan into action. At the end of the pre-determined intervention period (e.g., in 6 weeks), the teacher reviews the student’s cumulative progress-monitoring data, compares it to the outcome goal and judges the effectiveness of the intervention. Here are the decision rules:

  • Outcome goal met. If the student meets the outcome goal, the intervention is a success. The teacher may decide that the intervention is no longer necessary and discontinue. Or she/he may choose to continue the present intervention for an additional period because the student still appears to benefit from it.
  • Clear progress but outcome goal not met. If the student fails to meet the outcome goal, but the teacher sees clear signs that the student is making progress, that educator might decide that the intervention shows promise. In this case, the next step would be to alter the existing intervention in some way(s) to intensify its effect. For example, the teacher could meet more frequently with the student, meet for longer sessions, shrink the group size (if the intervention is group-based), etc.
  • Little or no progress observed. If the student fails to make meaningful progress on the intervention, the teacher’s logical next step will be to replace the current intervention plan with a new strategy. The instructor may also decide to refer the student to receive additional RTI/MTSS academic support.

Key Takeaway: Let Data Be Your Guide

The goal in monitoring any classroom intervention is to let the data guide you in understanding a learner’s unique story. When teachers can clearly define a student’s specific academic or behavioral challenge, collect data that accurately tracks progress, and calculate baseline level and outcome goal as points of reference to judge intervention success, the student’s story will be truly told.

Are your struggling learners in the right Tier? Collect the data you need to answer this question with Frontline’s RTI & MTSS Program Management software.

[1] Upah, K. R. F. (2008). Best practices in designing, implementing, and evaluating quality interventions. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology V (pp. 209-223). Bethesda, MD: National Association of School Psychologists.

[2] Howell, K. W., Hosp, J. L., & Kurns, S. (2008). Best practices in curriculum-based evaluation. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology V (pp.349-362). Bethesda, MD: National Association of School Psychologists.

[3] Hixson, M. D., Christ, T. J., & Bruni, T. (2014).  Best practices in the analysis of progress monitoring data and decision making in A. Thomas & Patti Harris (Eds.), Best Practices in School Psychology VI (pp. 343-354). Silver Springs, MD: National Association of School Psychologists.

[4] Shapiro, E. S. (2008). Best practices in setting progress-monitoring monitoring goals for academic skill improvement. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology V (pp. 141-157). Bethesda, MD: National Association of School Psychologists.

[5] Filderman, M. J., & Toste, J. R. (2018). Decisions, decisions, decisions: Using data to make instructional decisions for struggling readers. Teaching Exceptional Children, 50(3), 130-140.

Are Teacher Evaluations Worth It?

DR. JAMES STRONGE ON THE GATES FOUNDATION’S INTENSIVE PARTNERSHIPS FOR EFFECTIVE TEACHING INITIATIVE.

Last June, the RAND Corporation published research[1] looking at the Bill & Melinda Gates Foundation’s Intensive Partnerships for Effective Teaching initiative. The initiative worked with three school districts and several charter management organizations to redesign teacher evaluations that factored in not only classroom observations, but also student growth measures.

The Gates Foundation’s goal? To improve teaching effectiveness and, as a result, student outcomes.

The results? The researchers found that while most teachers believed that the redesigned evaluation systems had a positive impact on their teaching, it failed to improve student performance and graduation rates.

The research brief goes into much more depth, of course, pointing to incomplete implementation, potentially conflicting goals and changes in budgets, statewide tests, leadership and other factors as possible reasons for this lack of success.

In a recent episode of Field Trip, Frontline’s podcast about leadership in education, we spoke with Dr. James Stronge, president of Stronge and Associates Educational Consulting, LLC, to get his thoughts on the report. Here are some of the highlights. (Note: the text below is from select portions of the interview, and has been edited for brevity and readability. To hear the entire interview, please listen to the podcast.)

 

Do you agree with the findings of the RAND Corporation’s report?

I don’t think there’s much in the report to disagree with. They’re reporting the facts and then interpreting what they found. The place that I would disagree is on the causes. I doubt seriously that the design of the evaluations in those districts was the serious problem. I think instead it had to do with flaws in implementation.

[ctt template=”9″ link=”07G9g” via=”yes” nofollow=”yes”]“I doubt that the design of the evaluations was the serious problem. I think instead it had to do with flaws in implementation.” – Dr. James Stronge on the Intensive Partnerships for Effective Teaching Initiative [/ctt]

One of the problems that I see in the implementation in the Gates Foundation-funded projects is the emphasis that was placed on how data are collected and interpreted, what data were used. And in many of those districts, there was still a very heavy emphasis on classroom observation. Classroom observation is flawed as well. There’s been a great deal of writing and controversy around using value-added measures, but what people sometimes forget is that observation is flawed.

Were principals following the implementation procedures properly? The end result that you can’t argue with is that in the vast majority of cases, teachers are perfect. All teachers receive whatever the highest rating is. If it’s a five point scale, they get a five. If it’s a four point, they get a four.

Ultimately what happens in many schools is the evaluations get reduced to a one- or two-point scale. The districts think they have a four point summative scale, but they don’t. They never use the bottom two points. And consequently, the only distinction is between “effective” and “highly effective.” That’s a flaw in implementation.

Should schools abandon the focus on teacher evaluations?

Teachers have an enormous impact. We know that, and we know what good teachers do. The fallacy comes in how we assess that effectiveness. My worry is that because of reports like [the one released by] the RAND Corporation, and pushback that is constantly being felt across all the states, this initiative will be completely dropped, and we won’t return to it for another decade or so, and we’ll realize the mistake we’ve made.

There were mistakes made in implementation, mistakes made in rushing to design, but to throw all of that out and to say that, “Well, okay, evaluation is not worth doing. What’s next?” — that’s a huge mistake.

I don’t think that evaluation is the only reform that we need by any means. In fact, I have a bit of skepticism about evaluation being the best place to start. But I do think it’s an important reform, and quitting at this point will waste not hundreds of millions of dollars, it will waste billions of dollars.

[ctt template=”9″ link=”sU468″ via=”yes” nofollow=”yes”]“If we give up on distinguishing between effective, highly effective and less effective teaching, we’re never going to improve in our schools.” – Dr. James Stronge [/ctt]

And more importantly, it’s going to impact the lives of kids. If we give up on distinguishing between effective, highly effective and less effective teaching, we’re never going to improve in our schools. We’re going to be exactly where we were in the past and we’re going to remain there in the future, if we give up on that effort.

How can schools maximize the impact that evaluations have on teaching practice?

Hire and retain the absolute best principals. There is evidence that suggests it takes four or five years for a principal to begin to have a real footprint in that school, and if it’s an effective principal, that’s going to be a positive footprint.

[ctt template=”9″ link=”0yiao” via=”yes” nofollow=”yes”]How can schools maximize the impact that evaluations have on teaching? Dr. James Stronge: “Hire and retain the absolute best principals.” [/ctt]

We’ve known for a long time from research from the Dallas Public Schools that the quickest way to turn around a school, for good or bad, is to change the principal. Quality principals get quality results. And a quality principal will know good teaching. That person will be a medical practitioner, essentially, and be able to diagnose what’s effective and what’s not effective that’s occurring in a classroom, and then be able to prognose and say, “Here’s what you can do to get better and to improve.”

That good principal will follow up, give support, not just say, “You need to get better,” but will say precisely, “Here’s how you can get better, and we’re going to be there to support you in getting better.”

And then finally, he’ll follow up to verify that improvement has occurred. One additional thing we know from research on the best principals is that they will not condone poor teaching. They help teachers find another job if they don’t get better. Principals, while their direct influence on student achievement is in the range of five to ten percent of the total amount, they have a much greater influence. They hire teachers, they support teachers, they develop teachers, they keep teachers. If I could do anything immediately, that would be it.

How can you promote growth opportunities for your educators and administrators? Learn more about the research-based Stronge Effectiveness Performance Evaluation System, powered by Frontline Professional Growth, with specific components for teachers, educational specialists, principals, central office administrators and superintendents.

James H. Stronge is President of Stronge and Associates Educational Consulting, LLC, an educational consulting company that focuses on teacher and leader effectiveness with projects internationally and in many U.S. states. Additionally, he is the Heritage Professor in the Educational Policy, Planning, and Leadership Area at the College of William and Mary, Williamsburg, Virginia. His research interests include policy and practice related to teacher quality and teacher and administrator evaluation. His work on teacher quality focuses on how to identify effective teachers and how to enhance teacher effectiveness. Dr. Stronge has presented his research at more than 350 regional, national, and international conferences and conducted workshops for educational organizations throughout the U.S. and internationally. Additionally, he has worked extensively with local school districts and states on issues related to teacher quality, teacher selection, and teacher and administrator evaluation. Dr. Stronge has been a teacher, counselor, and district-level administrator, and has authored, coauthored, or edited 30 books and more than 150 articles, chapters, and technical reports.

[1] Stecher, B. M., Holtzman, D. J., Garet, M. S., Hamilton, L. S., Engberg, J., Steiner, E. D., . . . Chambers, J. (2018, June 21). Intensive Partnerships for Effective Teaching Enhanced How Teachers Are Evaluated But Had Little Effect on Student Outcomes. Retrieved December 18, 2018, from https://www.rand.org/pubs/research_briefs/RB10009.html

Better Early Hiring & Retaining Recruited Teachers

Is hiring at the top of your mind right now? If you want the most outstanding teachers in your classrooms, it should be.

Research shows that the strongest candidates are hired early, and are less likely to tolerate hiring delays. They prefer to withdraw from the hiring process in order to accept positions in districts with faster hiring processes. This leaves slower districts with a less-qualified applicant pool to choose from over the summer.

Candidate quality should be enough of a reason to hire earlier, but there are other advantages as well. You can avoid the last-minute hiring crunch, lessen the impact of teacher shortages and ensure that new hires have plenty of time to prepare for the first day of school.

How to Move to an Earlier Teacher Hiring Timeline

To be competitive, your district should hire all new teachers by May 1st at the latest. That’s an achievable goal, as long as you have the right processes in place and visibility into your district’s data. The most important number to know is how many teachers you will need to hire. To determine this, look closely at the following:

Historical Trends and Averages

How many vacancies have you had in the past few years? What’s your historical turnover rate? It’s unlikely that every teacher who plans on leaving will notify you before they’re committed to another job.  However, looking at retention and vacancy data from the past few years should give you an idea of how many positions you should plan on filling for the upcoming school year.

Teacher Retirement and Resignation Notifications

How many teachers are retiring or resigning? Keep track of how many teachers are eligible to retire by subject area and school building, and have teachers notify you as early as possible of their intent to retire or leave. Some districts offer an incentive to educators who submit retirement notifications by an early deadline, such as December 31.

Tip: it’s easier to collect retirement and resignation notices with an automated document management system.

New Staffing Needs

Do you have any new academic programs or schools that will need to be staffed? Also, consider if any new buildings opened over the past few years that might skew your historical data. Gather input from principals and other departments in the district (Academics and Business, for example.) By working closely with others across the district, you can be sure that you have a clear picture of any plans that may require additional staffing.

Beyond Early Hiring

The more confident you are that you have an accurate forecast of your staffing needs, the earlier you can begin to hire new teachers. To take it a step further, consider partnering with local colleges and universities to offer early commitments to education students in their final year of coursework, and student teachers who have shown aptitude. Some districts who take this approach have agreements with students in place as early as October.

This is not the same as offering the students a contract or any sort of binding agreement. If it were, it’d just be called early hiring. Instead, it’s putting a structure in place to build relationships with new teachers early, before they start looking for jobs in earnest. The important part is getting your district’s foot in the door and being the first to make an impression on teaching candidates. The early bird gets the worm (or, as the case may be, the best educators.) But if you can’t offer early commitments, hire earlier. And if you can’t hire earlier, hire faster.

[ctt template=”9″ link=”tTDd1″ via=”yes” nofollow=”yes”]The early bird gets the worm: if you can’t offer early commitments, hire earlier. And if you can’t hire earlier, hire faster.[/ctt]

The Most Important Part: Retaining Recruited Teachers

You know that employee engagement is key. An engaged educator is less likely to leave the district, and more likely to go above and beyond to ensure student success. But too often, engagement is seen as something to focus on after school is back in session, well after new hires have formed an impression of your district’s operations. This can leave a broad gap between the excitement of accepting an offer and the impact of any employee retention strategies in place — especially if they were hired months before, in March or April.

A fantastic orientation experience early on sets the stage for engagement as the school year goes on, and helps your new hires feel welcome. It starts all the way back at the application stage: making it easy for job-seekers to apply, so their enthusiasm for your district isn’t curbed by avoidable application-related frustrations. Communication is key as well — most hiring managers don’t have the time or means to communicate with applicants, so even an automated email or visibility into the status of their online application will have a positive impact. If there are any delays in the hiring process, it is crucial to continuously communicate in order to keep top prospects engaged. Plus, your district’s reputation will benefit, and candidates will tell other talented teachers in their network about their experience.

Onboarding, too, is a great opportunity to set the stage for lasting retention. If you can set up Welcome Days to help new hires meet their colleagues, the start of the school year will go much more smoothly. The same goes with mentoring or coaching relationships: starting over the summer shows that your district is committed to each teacher’s success, and they’ll have the chance to prepare more effectively for the first day of school.

Survey Results: How is RTI/MTSS Going in Your District?

ESSA requires districts to adopt a systematic approach for data-based decision-making to enable a rapid response to students’ needs. As a result, many district leaders are considering or re-validating RTI/MTSS models.

Yet, these frameworks are infamously complex with many stakeholders and moving parts, and can be tricky to implement – and maintain – with fidelity.

“MTSS not only focuses on meeting students’ academic needs, inherent within RTI, but in addition addresses the social, emotional and behavioral development of children,” notes Jo Ann Hanrahan, Director of the Frontline Research and Learning Institute.

Given these complexities and ESSA’s focus on agile, data-based student support, how do administrators and educators feel about current RTI/MTSS efforts in their districts? What changes need to be made?

Findings from a recent Frontline Education survey shed some light.

RTI/MTSS Frontline Education Survey Results

Graph showing breakdown of districts that use an RTI/MTSS model

Graph showing breakdown of how long districts have been using an RTI/MTSS model

Graph showing how effective RTI/MTSS programs are at identifying struggling learners

[ctt template=”9″ link=”TeVte” via=”yes” nofollow=”yes”]52% of survey respondents have their doubts that students are being placed in the right #RTI / #MTSS tier. How does this impact teacher effectiveness?[/ctt]

Graph showing how confident districts are that struggling learners are being place in appropriate Tier

Graph showing districts' confidence that their Data Analysis Teams enforce valid entry/exit criteria for Tiers 2/3

Graph showing the breakdown of RTI/MTSS staff training process

Graph showing staff understanding of districts' RTI/MTSS goals, processes and best practices

You may enjoy this hand-picked content:

Are State ESSA Plans Poised to Support Evidence-Based Interventions?

Graph showing the breakdown of what districts hope improve about their RTI/MTSS programs

Food for Thought

While results show that school staff are confident in their ability to identify struggling learners, 52% said they have their doubts or are not confident that students are being placed in the right Tier. And 59% reported they have their doubts or are not confident that their school’s Data Analysis Team consistently enforces valid entry and exit criteria for Tiers 2/3.

Could this lack of confidence be related to gaps in a district’s RTI/MTSS training procedures? And, if staff lack confidence in the process, how does that impact student outcomes?

Consider how RTI/MTSS stakeholders in your district would respond to these questions. If, like 63% of survey respondents, RTI/MTSS-focused trainings are only sporadic in your district, think about how integrating district-wide best practices can help raise staff buy-in of your RTI/MTSS efforts.

When navigating the complexities of RTI/MTSS, a little staff confidence goes a long way. Give your team  a common language and base of operations for assisting struggling learners.

11 Ways to Get Teacher Buy-in to Micro-credentials

 

Micro-credentials have been making waves in schools around the country for several years now. Maybe you’ve had the opportunity to earn them, or see them implemented into professional learning programs.

Maybe you’re thinking about taking the leap, offering competency-based learning for the teachers you serve.

Our world moves fast, and teachers can get inundated with new things to do: Chromebooks, and one-to-one initiatives, and iPads, and every other strategy or philosophy or piece of technology that promises to transform the classroom. How can school leaders who believe in the value of micro-credentials gain buy-in from teachers and see that the promise of competency-based learning is realized?

[ctt template=”9″ link=”S2jVh” via=”yes” nofollow=”yes”]How can school leaders who believe in the value of micro-credentials gain buy-in from teachers? [/ctt]

The Weilenmann School of Discovery, a charter school in Utah, has seen tremendous success with micro-credentials. When we spoke with several of their administrators and teachers, they offered some suggestions for getting teachers on board.

1. Prioritize new plans based on research. This helps to prevent Shiny Object Syndrome. “If you don’t make good research-based decisions, you’re bound to be kicked around by every fad,” said Cindy Phillips, Weilenmann’s Executive Director. “You look at the research on the efficiency to time of this kind of professional development, and you just can’t beat it.”

Welcome to weilenmann school of discovery sig

2. Set goals collaboratively with teachers. “We ask our teachers to set very specific goals that are aligned to our state standards of professionalism. And we identify the micro-credential that goes with that goal.”

3. Offer choice. “Instead of assigning a micro-credential, we asked them to choose,” said Kat Mitchell, Lower School Director. “And when we asked them to choose, we had far better success in them wanting to do it.”

4. Incentive. Perhaps this isn’t appropriate for every organization, but Weilenmann offers teachers a small monetary incentive for every micro-credential completed, up to three. Incentives don’t need to be financial, however — recognition and career pathing can be motivating as well.

5. Connect people together for support. Perhaps multiple teachers are pursuing the same micro-credential at the same time. If so, grease the wheels for collaboration.

6. Take strategic advantage of the school’s schedule. Is winter break coming up? That might be a great time to encourage teachers to pursue a micro-credential. Or perhaps include them as part of summer PD.

7. Ask administrators to complete micro-credentials, too. At Weilenmann, administrators are teachers as well, and they take part in the same professional learning that the rest of the certified staff does. Cindy said this helps keep them aware of what teachers need on an ongoing basis.

Outside photo of weilenmann school of discovery

8. Have fun. “Students will always do something that they perceive as fun, and that they see that there’s something meaningful about it,” said Cindy. “One of the best ways to get buy-in before the first year even starts is to have a fantastically fun moment with your teachers, where you’ve previewed all kinds of great curriculum, assigned it out, and have the teachers demonstrate how they’re going to use it the very first day, so that your buy-in is almost immediate because it’s fun and meaningful. The same way you would hook your students in.”

9. Give teachers freedom to use what they’ve learned. “There is no point in developing qualities of great teaching, or leadership abilities or whatever it may be, and having it hidden away in some corner of the school. You need to not only allow your teachers the autonomy to innovate and to utilize new skills in new ways, even if it wasn’t exactly what you had planned, because they’ve now learned something — they’ve grown beyond what they were doing before and want to try it out. As an administrator, if you snuff that out, you have completely undermined the credibility of all the premises on which you say that your school is based.”

[ctt template=”9″ link=”b8JIw” via=”yes” nofollow=”yes”]Allow teachers the autonomy to innovate and utilize new skills in new ways, even if it wasn’t exactly what you had planned — they’ve grown beyond what they were doing before and want to try it out.[/ctt]

10. Ask teachers to share success stories. Micro-credentials provide a rich opportunity for conversations about teaching, said Steve Williams, Weilenmann’s Middle School Dean. “One thing I’ve seen is that teachers are talking about things they’ve learned. They would talk about things that they would improve… I have seen that among all of our teachers, and I think they want to do this. I think 90% of the teachers that I have talked with are interested in doing more micro-credentials.”

11. Be open to feedback. “I don’t think anyone appreciates a push-down initiative,” said Steve. Communicating early and being up front with faculty about the what, why and how of micro-credentials, and giving teachers a chance to discuss, ask questions and react to them is important. “That there’s opportunity to talk about it, and it’s not just something you have to put your head down and do. And I think that gives people strength and a sense that they’re a part of this, and that their feedback really does matter in the process.”

Photo of 2nd grade teacher from Weilenmann school Kacey

Like anything worth doing, enhancing professional development with micro-credentials takes work. But Kacey Warburton, a 2nd grade teacher at Weilenmann, says it’s worth it. “If you are thinking about introducing micro-credentials to your staff, I say you should definitely do it. The professional development is more collaborative, more relevant, and more effective for each specific teacher than ‘regular’ professional development.”

Weilenmann’s journey in competency-based learning is fascinating — we created an entire podcast episode about it. Here’s their story, and how they got started.

How “Cultural Fit” Could Be Masking Unintentional Bias in Teacher Hiring

According to a new report from the Frontline Research & Learning Institute, hiring managers believe cultural fit is one of the most influential characteristics in deciding which teachers to hire — even above experience or training. But here’s the problem. Few districts have any clear, systematic way of defining or measuring cultural fit throughout the hiring process.

From the Institute report:

“While over 80% of respondents indicated that their school or district had standardized the hiring process…, only 70% of those with a standardized process indicated that their school or district uses a specific protocol to ensure consistency… and fewer than 5% of respondents reported using a quality protocol.”

Both the research and popular literature on cultural fit strongly suggest that the fundamental ambiguity in the term “cultural fit” has made it a shroud for unspoken, and perhaps unrecognized, hiring biases. As a result, teachers who weren’t the best fit may be less likely to stay in the district long-term, and talented educators may never even be seriously considered as candidates.

Undefined Cultural Fit and Bias Often Go Hand-in-Hand

Let’s be clear. It isn’t wrong to consider how a candidate will fit in your school culture — you want to hire teachers who believe in your school’s mission and share your commitment to helping students succeed. Cultural fit is not the boogeyman. But unconscious bias is, and it thrives in the gray space (“gut feeling”) that exists when cultural fit is not clearly defined and measured.

[ctt template=”9″ link=”8fd4d” via=”yes” nofollow=”yes”]Unconscious hiring bias thrives in the gray space that exists when cultural fit is not clearly defined and measured. #edchat #K12[/ctt]

Without a clear definition of cultural fit, hiring decisions may be inadvertently driven by unconscious bias and justified by nebulous rationale that a candidate is (or isn’t) the perfect cultural fit.

For example, when it comes to hiring, studies have shown that interviewers are significantly more likely to hire candidates who are similar to themselves or are very likeable. It’s surely not a conscious decision but as the very term “unconscious bias” suggests, it’s completely unintentional, and people don’t realize when they’re falling victim to it. That’s what makes it such a sneaky, pervasive problem. Unconscious biases, judgments and assumptions roll up into a gut feeling that this candidate is the right one.

Is There Room for Cultural Fit in Teacher Hiring?

In a nutshell, yes. But it can’t be based on gut feelings that could leave the door open for unconscious bias to flourish. Instead, define what cultural fit truly means for your district:

  • What values should new hires have in order to support the school’s mission and vision?
  • How can you objectively measure those values during the selection process?
  • How can you standardize your hiring process using research-based, quality protocols to ensure that all candidates are being equally evaluated on the same indicators?
You may enjoy this hand-picked content:

5 Steps to Choosing an Applicant Screening Tool

For example, let’s say a district has a strong belief that its educators should share their commitment to continual improvement. So, they have candidates give a sample lesson and a post-lesson reflection in order to help the hiring committee understand if the candidate possesses a growth mindset. In this way, they are both articulating the value they are looking for in a new hire (a growth mindset) and standardizing their evaluation of that value (via the post-lesson reflection.)

By defining cultural fit and establishing clear, consistent methods for evaluating candidates’ fit, you’re empowered to find talented teachers who align to your school’s values, without sacrificing diversity.

Consider Cultural Adds

Perhaps instead of looking for cultural fit, your district may be better served by looking for cultural adds — new hires who bring innovative ideas to the school and positively contribute to building a stronger school culture, rather than merely fitting in.

Either way, standardizing the hiring process and setting clear guidelines for how candidates should be evaluated should be a priority for district leaders. Standardized selection protocols can reduce the risk of unconscious bias, protect the district from discrimination claims and ensure that the most qualified candidates are hired to work in your schools.

“A commitment to cultural fit without a clear definition or evaluation scheme is simply a commitment to likeability bias.” – Repairing the Leak: How “Cultural Fit” Rusts the Teacher Pipeline

 

Why Do Special Education Teachers Quit?

Why did you choose a career in special education? is one of my favorite questions to ask when I interview teachers because it usually prompts an immediate brightening of the eyes and a smile. And, let’s face it, that kind of enthusiasm is catchy!

Yet, more and more, special education teachers are leaving the field. The annual attrition rate for special educators is 13%, twice that of general educators. The 3-year attrition rate is about 25%, with an additional 20% transferring to general education or to another role in special education each year.1

So, an equally important question to ask special education teachers might be: have you ever thought about stepping down from your role as special education teacher? And, if so, why?

Well, we’ve got some possible answers to the “why” part. We recently asked some special educators and administrators about their biggest challenges: here’s what they shared.

What Are the Biggest Challenges in Special Education?

The special education teacher shortage in the U.S. is intensifying. Fewer teachers in classrooms puts added stress on those who stay, and stress contributes to burnout. As many as 40% of teachers experience burnout ― which negatively impacts IEP goals.2 And research consistently shows that burnout is a major contributor to teacher attrition. So, which work-related activities should school leaders monitor to decrease burnout in their special educators?

Unrelenting Paperwork

[ctt template=”9″ link=”lSf5e” via=”yes” nofollow=”yes”]As many as 40% of teachers experience burnout, and burnout negatively impacts #IEP goals[/ctt]

Special education teachers spend less than half of their day teaching. And one study showed that almost 50% of a special educator’s workday is spent on paperwork.3 For educators wanting to make a difference for students, spending up to half of your time on paperwork is disheartening, even when you know it’s a key part of student support.

 

Rebecca Cole, a behavioral specialist, discusses the challenge of staying on top of special education paperwork.

Behavior and Discipline Issues

Teacher retention is directly related to managing student behavior.4 Yet, many students with behavioral disorders present serious obstacles to successful classroom management because of their disruptive behaviors. Consistently managing students with behavioral issues can cause higher levels of emotional exhaustion in teachers, which has been shown to decrease student engagement and, in turn, lead to poorer IEP outcomes.5

 

Debbie Roybal, an executive director of special education, discusses the challenge of managing students with behavior and discipline issues.

All the Regulation Changes

Changes to state and federal regulations are a familiar part of life ― but that doesn’t mean they’re not disruptive. Tight deadlines from these changes can cause stress, even when you know to expect the unexpected. Stress directly and indirectly influences student learning outcomes and teaching quality.6

 

Debbie Gazaway, a director of special education, describes the challenge of navigating the many regulation changes in special education.

Key Takeaway: Focus on What Inspires You

So how do you support your special education team through these challenges? First and foremost, remember the impact your team is having on students’ experiences and attitudes. Take time to reminisce about happy memories, shared accomplishments and what inspires you.

Which stories from your time working with students bring a smile to your face?

 

Sandra Dixon, an educational diagnostician, shares an inspiring story of helping students with disabilities succeed.

[ctt template=”9″ link=”O9IGv” via=”yes” nofollow=”yes”]20% of special educators transfer to general education or to another role in special education each year. How can you turn the tide?[/ctt]

Take some pressure off of your special educators with one intuitive software solution for IEP and special education management. Watch the Video.

References

[1] Wong, V. W., Ruble, L. A., Yu, Y., McGraw, J. H. (2017). Too Stressed to Teach? Teaching Quality, Student Engagement, and IEP Outcomes. Exceptional Children, 83(4), 412-427. Retrieved from http://journals.sagepub.com/doi/abs/10.1177/0014402917690729.

[2] Wong, V. W., Ruble, L. A., Yu, Y., McGraw, J. H. (2017). Too Stressed to Teach? Teaching Quality, Student Engagement, and IEP Outcomes. Exceptional Children, 83(4), 412-427. Retrieved from http://journals.sagepub.com/doi/abs/10.1177/0014402917690729.

[3] Vannest, Kimberly J.; Hagan-Burke, Shanna (2010). Remedial and Special Education 31(2):126-142. Retrieved from http://journals.sagepub.com/doi/10.1177/0741932508327459.

[4] Sabornie, Edward J., “Classroom and Behavioral Management of Students Who Are At-Risk” (2017). National Youth-At-Risk Conference Savannah. Retrieved from: https://digitalcommons.georgiasouthern.edu/nyar_savannah/2017/2017/97

[5] Wong, V. W., Ruble, L. A., Yu, Y., McGraw, J. H. (2017). Too Stressed to Teach? Teaching Quality, Student Engagement, and IEP Outcomes. Exceptional Children, 83(4), 412-427. Retrieved from http://journals.sagepub.com/doi/abs/10.1177/0014402917690729.

[6] Wong, V. W., Ruble, L. A., Yu, Y., McGraw, J. H. (2017). Too Stressed to Teach? Teaching Quality, Student Engagement, and IEP Outcomes. Exceptional Children, 83(4), 412-427. Retrieved from http://journals.sagepub.com/doi/abs/10.1177/0014402917690729.

How to Create Effective Reports for Communicating Professional Development Program Evaluation Results

Wouldn’t it be great if we knew when our professional learning programs were successful? What if we knew more than just the fact that teachers liked the presenter, were comfortable in the room or learned something new?

Wouldn’t it be better to know that teachers made meaningful changes in teaching practice that resulted in increased student learning?

I posed these questions in the first of this 7-part series on conducting program evaluation of professional development for teachers. Let’s say you have moved through the first four of the five phases of program evaluation:

  • You’ve engaged stakeholders and promoted a deep understanding of the program.
  • You’ve posed a set of evaluation questions.
  • You’ve collected data based on the evaluation questions.
  • You’ve analyzed all the data.

Undergo the 5 stages of program evaluation for professional development.

At this point, you have a solid understanding of program outcomes. You have a perspective on what teachers learned, if they’re using what they learned in the classroom, and perhaps even how students are responding to changes in teaching practice. Now what? Most likely, you’re not the only one in your district who needs this information. How do you share evaluation results and with whom?

The fifth phase in the cycle of program evaluation is reporting and use of results. In this phase, consider the following:

  • Who needs to know the results of your professional development program evaluation?
  • How will results be used to make decisions?
  • What information should be shared in a report?
  • What format should a report take (e.g., document, presentation)?
  • How and when should reports be shared?

Most importantly though, consider why you will create and share evaluation reports. The answer to this and the above questions form your communication plan.

Kylie Hutchinson, author of A Short Primer on Innovative Evaluation Reporting offers this insight:

Reporting your results is one of the most important tasks of an evaluation, but doing it in an engaging way that inspires action is often neglected…There is rarely uptake of recommendations, no lessons learned, and most importantly, no program improvement (2017, p. 11).

The reason to report and disseminate results is tied to key decisions that need to be made about professional development programming. To be meaningful, evaluation reports need to be used to determine whether a program should be continued, expanded, eliminated or changed in specific ways to improve outcomes.

What belongs in a professional development program evaluation report?

The most comprehensive form of an evaluation report might include all the details:

While this list may appear logical and sequential, the order also makes for a less engaging report. To ensure the use of evaluation results, many evaluators now encourage beginning reports with the exciting part — the findings and conclusions, and actionable recommendations.

[ctt template=”9″ link=”QyH4d” via=”yes” nofollow=”yes”]If you take only one lesson from this article, let it be this: Match your report to your audience. Here’s how and why. @SheilaBRobinson [/ctt]

Match the report to the audience.

If you take only one lesson from this article, let it be this: Match your report to your audience. Consider who needs to know the answers to your evaluation questions and understand your findings from data analysis. Who needs to use the information you share to make key decisions about professional learning? Who might be interested in results because they are in a position to support professional learning programs?

Creating evaluation reports to meet the needs of specific audiences involves three key steps:

  • Identify your audiences. Are they administrators? Teachers? Board of Education members? Parents? Community members?
  • Understand their information needs. What is important to them with regard to professional learning in general or the specific topic? How does the professional learning program connect to their work and responsibilities?
  • Know what actions they will take with the information in your report. Are they decision-makers? Do they sit at the table with decision-makers? Are they likely to share the information with others? Are they potential supporters or detractors?

Consider multiple forms of a report.

On the surface, it may sound like a lot of work to create multiple reports, but with careful planning it’s quite manageable. Creating different versions of reports for different audiences can be an enjoyable and rewarding part of the evaluation process and contributes to deepening your own learning as you dive into the data and help others make sense of it. Think about what would hold the most appeal for your stakeholders.

Do you have an audience who needs:

  • A 1-page overview with a few highlights?
  • A 3-5 page summary of key findings?
  • A 15-minute live presentation?
  • A comprehensive written report with all the details?

Choose the audience who needs the highest level of detail and create that report first. Then, work to strip away details the other audiences don’t need. You can always make the more comprehensive forms of the report available if they want access to them.

You may enjoy this hand-picked content:

Podcast: Building a Culture of Professional Learning — The big questions that shape professional learning at Jenks Public Schools in Oklahoma, how they iterate and improve, and what it looks like to measure impact.

Beware TL;DR.

Few people I know love spending endless hours writing long reports. But, if you’re one of those people, here is another reason to carefully consider your audience and their information needs. “TL;DR” is internet slang for “too long: didn’t read.” Part of the problem isn’t necessarily the length of some reports, but the length combined with a report that isn’t visually appealing. It just doesn’t draw the reader in and keep them there.

Fortunately, there are many ways to avoid TL;DR in evaluation reporting by creating different versions for different audiences, using creative or innovative formats, and embedding visual elements.

Think outside of the document.

A written report is far from the only way to communicate evaluation results, and it’s perfectly OK to think flexibly and creatively here. I’m not necessarily suggesting a song and dance routine, but believe me, it has been done!

Here are just a few alternatives to written reports or presentations:

  • An infographic
  • A brief video posted to a website
  • An interactive website
  • Social media sharing

A Short Primer on Innovative Evaluation Reporting features many more creative ideas.

Make it visual.

No matter the style, size or length of your report, be sure to include visuals to engage your audience. Use relevant photos, icons, or illustrations along with charts or graphs to draw the audience’s attention to the main points. There are many, many websites where you can find free stock photography, but consider taking your own photos. It isn’t that difficult, requires nothing more than a smartphone and brings a stronger sense of ownership and connection to the report and to the program. Your audiences will see your teachers in your classrooms doing the real work involved in professional learning, and that is more likely to inspire engagement with the report.

Most audiences also want to see data. They want to quickly and easily understand key findings. Charts or graphs can be efficient and powerful ways to communicate data, and they don’t need to be sophisticated or complex to have impact. Simple bar, line, or pie graphs can communicate meaningful data. I’ve been actively honing my data visualization skills in my spare time by simply reading blogs and books and experimenting. Little by little, I acquire new skills and attach them to prior knowledge to build a robust toolbox and solid repertoire of visualizations I can now use create to communicate program evaluation results.

Program evaluation is essential.

Professional development remains a critical component of school success. It is essential that we continue to create and implement high quality professional learning programs within the constant constraints of budgets and time. A rigorous program evaluation process helps us deeply understand how programs are performing in our school environments and is key to educator professional growth and the continuous improvement of our schools.

[ctt template=”9″ link=”b65Oa” via=”yes” nofollow=”yes”]Evaluating our professional development programs is key to educator professional growth and continuous improvement of our schools. – @SheilaBRobinson[/ctt]

 

Dyslexia Awareness: Preparing New Educators

When I began working with individuals with dyslexia in 1988, I was so green. I only had three years of teaching experience under my belt, and my college classes had not prepared me to find solutions for struggling readers.

So, I took it upon myself to learn as much about dyslexia as possible. I became a Certified Academic Language Therapist (CALT). I obsessed over the intervention process, trying method after method to perfect my skills in the classroom.

Now, as Assessment Specialist for my district, I have the opportunity to better equip new dyslexia and classroom educators to work with struggling readers. Based on my experience as a teacher and administrator, here are the seven biggest challenges, along with strategies for overcoming them.

1. Recognize the Characteristics of Dyslexia

Do you work with students with dyslexia? If you are a teacher, you almost certainly do. According to the International Dyslexia Association, as many as fifteen to twenty percent of school-age children have dyslexia. [1] So, your classroom of twenty-five students may have three or more students with dyslexia.

It is characterized by deficits in decoding, sight word reading, reading fluency and spelling. These reading difficulties must be the result of problems with phonological awareness because dyslexia is caused by the way the brain processes sounds in spoken language.

Dyslexia can be identified in students as early as kindergarten and early identification leads to students who are more successful throughout school. [2]

[ctt template=”9″ link=”v9sTS” via=”no” ]As many as 15 – 20% of school-age children have #dyslexia. Are you helping them learn to self-advocate?[/ctt]

2. Keep it Legal: State & Federal Dyslexia Law

In the U.S., we have laws that ensure students with disabilities are protected and that their individual needs are met. In some states, students with dyslexia are under the umbrella of IDEA and are served through special education. In other states, students with dyslexia are protected through general education under Section 504 of the Rehabilitation Act of 1973.

Staying up to date on the many regulations designed to help educators working with children with special needs can be intense. However, there are free resources to keep you informed of your state’s regulations. And don’t be afraid to reach out to fellow educators with questions.

You may enjoy this hand-picked content:

Webinar: Learn Core Concepts of Section 504.

3. Individualize Support

Students with dyslexia struggle to learn to read and their educators often struggle as well. In some cases, student success only occurs when teachers are innovative, flexible and supportive. There is no “one-size-fits-all” approach when it comes to teaching individuals ― even seasoned educators need help sorting through the available options.

To avoid frustration or research fatigue, find a few reliable sources you can turn to. Campus IEP and 504 committee members are excellent sources for ideas, programs and tips about what is and isn’t working for individual students.

4. Gather the Right Data

Monitoring students at every stage will help you identify those who are struggling, gauge their progress, plan lessons that are appropriate and provide feedback to parents and learners.

Administrators can help with gathering and analyzing classroom data by:

  • Presenting teachers with clear expectations
  • Identifying district and campus norms that teachers can use to pinpoint areas of struggle
  • Providing a system for sharing data with other decision makers

Classroom teachers must have enough knowledge about dyslexia to use the available data to know which students may need additional help.

5. Encourage Self-Advocation

How do you help students learn to help themselves? This is a years-long process that begins with modeling positive attitudes about dyslexia in front of students and parents.

I provide students with information and practice, so they feel comfortable responding if teased about being dyslexic. The goal is for the student to handle difficult situations without assistance.

Teach students all about dyslexia. Talk about the characteristics, share the research about interventions, post photos of famous dyslexics in your classroom and role-play situations where students can teach others about the disability. Turn a difference into an asset.

[ctt template=”9″ link=”3c5Yd” via=”no” ]When teachers advocate for students, individuals with dyslexia are gathering tools to learn to self-advocate. #DyslexiaAwarenessMonth[/ctt]

6. Challenge Misconceptions

Educators share humorous stories with me of things that students, parents and teachers believe about dyslexia. These are often based on ideas that were disproven years ago but will not die. Here are some common misconceptions:

  • Dyslexic individuals see backwards (dyslexia is a phonological disorder)
  • Students outgrow dyslexia (it is lifelong)
  • Dyslexia is characterized by letter reversals (it is usually a visual-motor or developmental difficulty)

One parent even reported to me that her child “caught” dyslexia. As an educator, arm yourself with up-to-date, correct information and current research about this disability to share in situations like these.

7. Plan Data-based Accommodations & Progress Monitoring

Most students with dyslexia have protections under either IDEA or Section 504. This likely includes an accommodation plan to ensure the student has equal access to the school-wide curriculum. The only appropriate accommodation for a student with dyslexia is the one that meets his or her individual needs. As a result, the classroom teacher is best equipped to gather data about what the individual student needs and to systematically try interventions to see which are successful.

So, keep meticulous records to help the IEP or 504 committee find effective strategies. Once the plan is in place, consistent recordkeeping and data sharing with other teachers and committee members will gauge the success or failure of a student’s plan.

In Summary: Success Takes Perseverance

Individuals with dyslexia can have very positive experiences in school, especially with the appropriate interventions. They have learned perseverance from watching those who have tirelessly worked with and for them. Most pass state assessments, take the SAT or ACT and go on to have successful careers. Their teachers modeled steadfastness and work ethic, and the importance of an end goal. Working through the struggles that occur in school, along with access to caring advocates and a steady support system, can transform a learner with no confidence into a school and community leader.

I am not going to say teaching kids with dyslexia is an easy job. Teaching students with dyslexia is hard and requires long hours, extra steps and more data collection than seems possible. Despite these challenges, I still love my job. I live for the texts I get from students about the novels they’re reading in class. I dance on air after a 504 or IEP meeting where a student has completed the dyslexia program and is on grade level, ready to soar. I sign college and job reference letters for former students with jubilation.

When teachers advocate for students, dyslexic individuals are gathering tools to learn to self-advocate. They can speak up and educate others about dyslexia. Watching students grow and bloom and overcome disability is worth all of the challenges involved.

Whether using a 504 Plan or an IEP to support a student with dyslexia, Frontline makes it easier to plan data-driven accommodations and measure their efficacy. Learn how.

[1] International Dyslexia Association. (2017). Retrieved from https://dyslexiaida.org/dyslexia-basics.

[2] Texas Education Agency. (2014). Procedures Concerning Dyslexia and Related Disorders. Retrieved from https://www.region10.org/r10website/assets/File/DHBwithtabs10214.pdf.

Effective Strategies for Analyzing Professional Development Data: It’s Not As Hard As You Think!

I remember struggling through my college statistics class. Just the term “data analysis” made me cringe. After all, I was going to be a teacher, not a scientist! Now that I’ve spent years collecting and analyzing data I’ve learned, I don’t need to be a statistician to do professional development program evaluation.

Whether you love or hate the idea of analyzing data, you probably don’t have loads of time on your hands — but you still need answers. You need actionable knowledge in order to report out results that inform smart decisions about professional learning. Good news! In this article, I’ll share strategies for analyzing and interpreting data in a painless way that doesn’t require unlimited time or advanced skills.

What is Data Analysis?

Data analysis and interpretation is about taking raw data and turning it into something meaningful and useful, much in the same way you turn sugar, flour, eggs, oil and chocolate into a cake! Analyzing data in service to answering your evaluation questions will give you the actionable insights you need. It’s important to remember that these questions drive data collection in the first place.

Since you’re generally not running experiments with randomly sampled study participants and control groups, you don’t need advanced statistical calculations or models to learn from professional development data. You mainly need to analyze basic survey data, and to do that, you will look at descriptive statistics.

You may enjoy this hand-picked content:

Podcast: Building a Culture of Professional Learning — Mary Kathryn Moeller and her team facilitate professional learning at Jenks Public Schools in Oklahoma. In this interview, she discusses the big questions that shape their program, how they iterate and improve, and what it looks like to measure impact.

Summarizing the Data

Raw data rarely yields insights. It’s simply too overwhelming to scan rows and rows of numbers or lines and lines of text and make meaning of it without reducing it somehow. People analyze data in order to detect patterns and glean key insights from it.

First, it’s helpful to understand the proportion of professional learning participants who complete a survey. This is your survey response rate. Your response rate is simply the number of people who completed the survey divided by the total number eligible to take the survey.

Example:

Effective Strategies

Next, use descriptive statistics including percentages, frequencies and measures of central tendency to summarize the data. Measures of central tendency — the mean, median, mode, and range — are summary statistics that describe the center of a given set of values.

Definitions and statistics

Choosing the Right Statistic to Report

How do you know when to use mean vs. median? When you know you have outliers, use the median. Here’s an example of how these measures can differ greatly in the same dataset. Let’s say you want to describe a group of 16 professional learning participants in terms of how much teaching experience they have. Here are the values and measures of central tendency:

Choosing the right statistic

Which summary statistic best describes this population of participants? The mean can be very sensitive to outliers, while the median is not. The mean of this dataset is 9 years, but the median is only 3. This means that half of participants have 3 or fewer years’ experience. In this case, knowing that half of participants were novice teachers may give you greater insight and better inform future programmatic decisions than knowing the average number of years of teaching experience of the group.

Example Insight: Half of the participants in this professional learning activity were novice teachers, which can be used to inform future decisions about professional development.

Next, you may want to cross-tabulate results. Cross-tabulating means looking at your dataset by subgroup to compare how different groups answered the questions. For example:

  • Were high school teachers more satisfied than elementary teachers?
  • Did more veteran teachers report great learning gains than novice teachers?
  • Did more teachers from one school express an intent to try a new instructional strategy?

Most online survey programs make cross-tabulation easy with built-in features, but you can also use pivot tables if your dataset is in a spreadsheet.

Descriptive Statistics Have Limits

Caution: when participants haven’t been randomly assigned and required to respond to feedback surveys, these types of analyses cannot be used to generalize to all teachers who participated in the professional learning. It’s always a possibility that more satisfied participants completed the survey and that more dissatisfied participants did not.

Descriptive statistics are helpful for telling what happened, but they can’t determine causality. They can’t tell you why something happened. You may know that 87% of participants feel they learned a great deal from participating in professional learning, but you won’t know what caused them to learn. That’s where qualitative data can help fill in the blanks.

Qualitative Data Analysis

Surveys may include some open-ended questions, or you may have conducted individual interviews or focus groups as part of professional development program evaluation. Crafting these questions carefully can help you understand why people experienced professional learning the way they did.

But what do you do with all of these answers, the words people write or say in response to these open-ended questions? Rigorous qualitative data analysis involves significant study to develop the needed skills, but you can still take a few easy steps to make sense of qualitative data in a credible way that will give you insight into participants’ experiences in professional learning.

Step 1: Begin by becoming very familiar with the data – just reading and rereading survey responses, interview transcripts or focus group notes. Try not to get caught up in the very positive or the very negative attention-grabbing comments at this stage.

Step 2: Revisit your evaluation questions to refresh what you need to know for the program evaluation.

Step 3: Start looking for patterns as you read and reread. Assign “codes” to chunks of text. If a participant talks about wishing there was more time to learn and practice what was learned, the code might be “time.” If another comment has to do with concern about administrative support, the code might be “support.” As you progress through the data, attempt to reduce as much as you can to codes that you generate from your reading. Write each code next to the data as you go along.

Step 4: Once you’ve coded all data, look for patterns in the codes. Are there sets of codes that are related and could become categories?

Interpreting the Data

Interpreting data is attaching meaning to it. For example, let’s say that 37% of professional learning participants indicated they learned something new. At first glance, that doesn’t sound like a particularly good outcome, does it? Too often, people view raw data like this in either a positive or negative light without taking the time to fully understand what’s really going on. What if I told you this was a refresher course for people who had already learned the material? In that case you might then interpret it as a positive outcome that more than one third picked up new learning.

[ctt template=”9″ link=”abjnc” via=”no” ]Too often in professional development program evaluation, people assign meaning to raw data without taking the time to fully understand what’s going on. -@SheilaBRobinson [/ctt]

Numbers don’t have inherent meaning. It’s up to us to put them in context to make sense of them.

What About Statistical Significance?

People like to ask about this, and most likely, what they’re really asking is, “Are the results you’re reporting on important? Are the differences we are seeing meaningful to us in any way?” Statistical significance is a technical term that has to do with whether results of an experiment are true, or are more likely due to chance. In evaluating professional learning programs, you are not likely to use the statistical analyses that result in statistical significance.

What If I Have a Small Sample?

You may be wondering, “I only surveyed 20 people — is that really enough data to give me an accurate picture of what’s really going on?” Absolutely! Remember, it’s about answering your evaluation questions to inform future professional learning programs. Even with what might seem like low response rates, you can still gain valuable insights, and make smart decisions for your school or district.

Next in the series, we’ll turn analyses and interpretation into a usable report!

No Substitute for Customer Service

As a former substitute classroom aide, Barbara Valencia is uniquely positioned to explain what exactly makes being a sub so difficult. How hard it can be to walk into an unfamiliar classroom full of unfamiliar students and try to teach an unfamiliar lesson plan. How daunting it can feel to have to treat every day as if it were your very first on the job; often, because that’s precisely what it is.

Barbara recently appeared on Frontline Education’s K-12 conversations podcast, Field Trip, where she shared some of the often-uncomfortable thoughts that run through the minds of classroom-wary substitutes:

“It was like being on a job interview every single day,” Barbara said. “Everything was new. I didn’t know where to go, who to talk to, where to get lunch, where my classroom was.”

Barbara Valencia

Now working in Human Resources at Spring Grove Area School District, Barbara has discovered a creative way to marry her experience as a substitute aide — coupled with her experience working the phones at a local electric company — with the needs of the district.

With the team at Spring Grove, she is focusing on something that’s key to substitute engagement, but rarely brought up in discussions regarding K-12: customer service for substitute teachers.

By focusing on the thoughts, feelings and overall fit of Spring Grove’s subs, Barbara has implemented a substitute management system that places the emphasis on the “customer” — the substitutes. They’re making a concerted effort to keep subs engaged as they support students — and that, Barbara says, is what keeps substitutes coming back to Spring Grove.

“When you have good customer service, people tend to return. It’s no different with subs.”

Barbara Valencia

Putting People First

The truth is that substitutes  have a choice. If one district doesn’t communicate well, or if another refuses to rid themselves of archaic substitute management processes, substitute teachers can simply choose to work elsewhere.

That’s why Spring Grove has focused on customer service — putting people first — to ensure that substitutes choose to work in their classrooms.

Administrators at Spring Grove know that phenomenal customer service isn’t based on a single interaction. They start by ensuring that substitutes feel welcomed from the moment they enter a school, meeting them when they arrive in the morning.

Leaders then take it a step further — like the time Barbara baked a plate full of cookies as a thank you to a sub after putting them in a less-than-ideal situation. From cc’ing subs on emails regarding important administrative changes, to sending cards on their birthdays, it’s all about substitute engagement and communication.

“It’s more of a team effort and they’re not just a sub or just a number or just a breathing person to walk into the room. They actually are part of our team.”

Barbara Valencia

“It’s more of a team effort and they’re not just a sub or just a number or just a breathing person to walk into the room. They actually are part of our team.”

But that’s not the only change Spring Grove has made.

The district holds an open house every summer, aimed at updating interested substitutes on any new processes. They provide training for all online software systems the substitutes may have to use, and they even distribute a booklet on the district, including information on other teachers and student highlights. And when a new substitute does start in the district, they make sure to provide context for everything, ensuring that person is in-the-loop.

“Once you’re a Spring Grove employee, you’re getting retirement benefits, you’re getting all the same benefits our part-time employees do, so it’s no different to be a substitute than it is to be a Spring Grove part-time employee.”

Barbara Valencia

Substitutes at Spring Grove are even offered the same benefits as part-time employees, just another example of Spring Grove’s commitment to its most vital resource — its people. Ultimately, what Barbara wants Spring Grove substitutes to know that working with the district isn’t a one-time, one-off experience — substitutes are part of the team.

[ctt template=”9″ link=”kL5Rh” via=”no” ]Customer service for substitute teachers means treating them like family. Get more sub engagement tips #k12[/ctt]

Seeing Positive Results

What Barbara and Spring Grove have managed to accomplish with a simple adherence to a singularly important factor — placing the emphasis on people — is significant.

  • Spring Grove is on track for a 94-99% fill rate. That’s up from 84-93% before they started using Frontline Absence & Time, their employee absence management software.
  • When Barbara first joined the district in 2015, they had about 50 substitutes who worked regularly. Now? 155 regular substitutes.
  • Perhaps more importantly, Barbara regularly hears that substitutes love teaching at Spring Grove, largely because they feel appreciated and supported.

99%

Increase in Fill Rate from a low of 84%

150

Increased Substitute pool from about 50 substitutes in the pool to 150 who regularly cover classes in the district.

100%

Feedback from substitutes is that they feel increasingly happy, appreciated, and supported at Spring Grove.

Looking Forward

“I had one substitute who made sure that she caught me at the end of an event that we were having at one of the schools and took me aside and said, ‘I can’t believe that I was so scared to substitute because I found that, when I was in the classroom, I was enjoying the kids so much that we got through everything we needed to and were able to accomplish all the teacher’s goals for the day and it wasn’t like it was work.’”

Barbara Valencia

That’s a far cry from a self-conscious substitute, tossed into a classroom they don’t recognize, struggling just to get their bearings.

Barbara credits a long-term view of the district’s future, coupled with their newfound commitment to customer service, for the improvements in Spring Grove. But none of it would have been possible without her early career experiences as a sub.

“I knew, first-hand, what it felt like to be that substitute, brand new in the system,” Barbara says. “So, I wanted to change the culture and the feeling substitutes would have when they come to Spring Grove.”

Barbara Valencia

Through a focus on substitute customer service, Barbara and Spring Grove have managed to build a reputation for being a great place to work—a reputation that has served, and continues to serve, the district well in filling unexpected teacher vacancies, hiring the best candidates for the district and improving outcomes for Spring Grove students.

5 Ways to Reduce Burnout for Special Education Teachers

When I became principal for a specialized school with various sites throughout a borough in New York City, one of my first priorities was to develop a plan of action to better support my special education staff.

Here’s why:

At the time, I was struggling in my newly assigned school. I had just hired two new administrators, opened a new inclusion site and hired twenty-one staff members, including teachers, paraprofessionals and related service providers.

Just as I thought the year was settling in, one of my first-year, highly qualified special education teachers came to me in tears. Her dream was to be a teacher. She loved children and was so thrilled to be an educator in the field of special education. She kept telling herself that things would get easier ― however, the sleepless nights and spending all of her free time planning lessons and materials for her students was just too much for her.

Although we did have curricula that offered tiered level materials, this teacher needed to break down instruction even further to meet the needs of her students. And there were other challenges she had to conquer:

  • Training her paraprofessionals to support instruction and collect data daily
  • Scheduling instructional days to meet mandates around related services
  • Navigating a myriad of emotional challenges in her classroom

She was spiraling downhill trying to coordinate her own emotions and keep up with paperwork, and it was taking a toll on her physical and mental health.

“She was spiraling downhill trying to coordinate her own emotions and keep up with paperwork, and it was taking a toll on her physical and mental health.”

Though special education teachers are often passionate and resilient individuals, they are twice as likely to leave the profession as their general education colleagues.[1] One of the reasons they leave is because they simply burn out.

Unique Challenges Faced by Special Education Teachers

With the national push to take students with disabilities out of self-contained environments and provide them with opportunities to spend part of or all of their days in general education classrooms, the ability to effectively teach students in special education is more critical today than ever before.

Each student brings with them their blueprint, as documented in their Individual Education Plan (IEP), which may include mandates to provide a variety of related services to help support students to progress. Many students have multiple services and classifications and are often years behind academically. Some of these students are mandated for mainstreaming or inclusion services in general education classes. This means even one small, self-contained class can have students within a three-year age span, with multiple individual needs, learning at different levels and with different styles. What a daunting task for any educator!

I needed to create an inclusive culture and climate that was beneficial to adult and student learning.

5 Ways School Leaders Can Reduce Burnout in Special Educators

As an administrator and an educator for over 35 years, I have been through the changing mandates of teaching students with disabilities, and I have experienced how these changes affect the passionate educators with whom I work.

My goal as a school leader was to support and empower teachers to collaborate, and to provide coherent, rigorous instructional and social-emotional support to the school community. This new kind of school environment improved the emotional state of many teachers, including special educators, and allowed me and my administrative team to focus on improving teacher practices and student achievement.

Here are five strategies we used to turn the tide in our school.

1. Emotional Intelligence

Creating a bond of trust gives insight into what others may be feeling or thinking; it helped me, as a principal, understand how and why members of my special education staff were reacting to situations in certain ways.

As a leader, it was necessary for me to have self-awareness and manage my emotions, to move the school community to a desired, cohesive vision by making informed decisions on how to communicate and support teachers and provide staff development.

2. Teacher Surveys

Through teacher surveys, we were able to consistently offer differentiated professional development opportunities and provide meaningful feedback to our special education teachers. We sought input from all members of the school community through regular meetings. Teachers are not superheroes. They can’t possibly be effective without the support and collaboration of a school-based team. Our school motto was, “Teamwork in the best interest of children.”

[ctt template=”9″ link=”35zCN” via=”no” ]“This new kind of school environment improved the emotional state of many teachers, including special educators.”[/ctt]

3. Examining Data

Teacher inquiry teams through distributive leadership — in collaboration with the instructional cabinet — examined trends and targeted student and teacher supports needed through the use of a wide range of formative assessments. The teams provided opportunities for veteran special education teachers to mentor new teachers to look at data and identify trends.

Data revealed an overwhelming pattern of challenging behaviors that would require intervention across all of my sites in order for teachers to effectively manage their classrooms and ensure successful instructional outcomes for students. It was important that the entire school community implement both a coherent school-wide, positive-behavior system and best instructional practices.

“Data revealed an overwhelming pattern of challenging behaviors that would require intervention across all of my sites in order for teachers to successfully manage their classrooms.”

4. Proactive Teamwork

We established two academic teams, one for Alternately Assessed Students and one for Standardized Assessed Students. Staff worked together to create a lesson-plan template that embedded instructional shifts, coherent instruction and best teaching practices that reflected a clear set of beliefs. This helped ensure all students had entry into learning that demonstrated high levels of thinking, ownership, participation and behavior expectation.

Working as a team provided support and opportunities for special education staff to look at student performance data and calibrate the results. Team members researched best practices together to implement strategies and support instruction throughout the day, promoting positive student outcomes.

[ctt template=”9″ link=”aOXe7″ via=”yes” nofollow=”yes”]”Working as a team provided support and opportunities for #specialeducation staff to look at student performance data and calibrate the results.”[/ctt]

Innovative scheduling was implemented as a necessary step to providing individual or small group support for students during class instruction, rather than students being pulled out for a related service one-to-one session – which would have resulted in instructional time loss. This scheduling technique involved assigning related services staff to specific classes and delivering services within the classroom, as long as the sessions were not therapeutic and could be carried out in a classroom setting.

5. Role-based Training for Paraprofessionals

It was also essential that special education teachers were given additional time to meet specifically with paraprofessionals to train and plan for the instructional day. Training paraprofessionals to implement instruction and adapt materials to support students was a needed component to assist in daily instruction. This training would allow the teacher to have more time to teach without stopping in the middle of a lesson to direct paraprofessionals on how to support students.

Key Learning: An Inclusive Work Environment & Clear Expectations Reduce Burnout in Special Education Teachers

Monitoring, adapting and modifying instruction for students with various classifications on multi-grade levels every day is exhausting — there is no one-size-fits-all approach. Providing tools to empower staff to work together to create a coherent, positive school environment with clear expectations for both students and staff reduces frustration for special educators.

Will it always be easy? Absolutely not. However, teachers don’t have to do it all alone. School leadership can use creative scheduling and teamwork to help support teachers in identifying entry points of learning — and assist in closing the achievement gap for students. Collaboration lets staff take ownership of researching academic and emotional strategies that can be embedded into instructional best practices.

With the help of your staff and the use of round-table decision-making and emotional intelligence strategies, it is possible to create a great support system for your special educators using existing school resources!

Want to reduce the likelihood that your special educators will leave their posts due to burnout? Learn how Frontline Special Ed & Interventions helps them spend less energy on paperwork and re-engage in delivering innovative student support.