Blog

Professional Development Program Evaluation: Using Surveys, Interviews, Focus Groups and Observations

In the previous article in this series on evaluating teacher professional development, I shared that evaluation questions drive data collection and asked, “How would we know what data to collect, and from whom, if we haven’t settled on the questions we’re asking?” Once we have settled on a small set (about 1-3) of evaluation questions, we set our sights on how to collect data to answer them.

There are a multitude of ways to collect data to answer evaluation questions. Surveys (aka questionnaires), interviews, focus groups and observation are the most commonly used, and each has distinct advantages and disadvantages.

You’ll choose data collection strategies based on these along with which align best with your evaluation questions.

Let’s take a quick look at each strategy:

Surveys

A survey is “an instrument or tool used for data collection composed of a series of questions administered to a group of people either in person, through the mail, over the phone, or online” (Robinson & Leonard, 2019, p. xiii). Surveys tend to have mostly closed-ended items — questions that have a question stem or statement, and a set of pre-determined response options (answer choices) or a rating scale. However, many surveys also include one or more open-ended questions that allow respondents — the people taking the survey — the opportunity to write in their own answers.

Many surveys are still administered on paper, but they’re conducted more frequently now in online environments. Professional development management systems, such as Frontline Professional Growth, allow feedback forms to be attached to professional development courses, and also feature the ability to construct and administer follow-up surveys.

The main advantage of a survey is that it can reach a large number of respondents. With electronic platforms, one click can send a survey to hundreds or thousands of respondents. Survey data is also relatively easy to analyze, and allows for easy comparison of answers across groups (such as elementary vs high school teachers, or different cohorts of participants). The main disadvantage is that we lack the opportunity to ask respondents follow-up questions, and quantitative survey data isn’t often as rich and detailed as data that result from interviews and focus groups.

[ctt template=”9″ link=”T21cb” via=”yes” nofollow=”yes”]4 ways to collect data when evaluating your professional development program: surveys, interviews, focus groups and observations.[/ctt]

Interviews

An interview is a set of questions asked in person or over the phone to one individual at a time. It’s essentially a conversation between interviewer and respondent. In contrast to surveys, interviews are largely composed of open-ended questions with the interviewer taking notes or recording respondents’ answers for later analysis. Interviewers can use “probes” to elicit more detailed information from respondents. Probes are specific follow-up questions based on how a respondent answers, or they can be more generic, such as, “Can you say more about that?”

An interview’s main advantage is that it allows us to deeply understand a respondent’s perspective and experience. An interview can give us a strong sense of how someone experienced new learning from professional development, and how that learning plays out in their teaching practice. The main disadvantage is that we usually don’t have time to interview more than a handful of people, unlike the hundreds of responses we can collect with surveys. Interview data is also qualitative, and thus a bit time-consuming to analyze.

Focus Groups

A focus group is simply a group interview. Typically a small group of people (ideally about 6-8) are brought together and asked a set of questions as a group. While one focus group member may answer a question first, others then chime in and offer their own answers, react to what others have said, agree, disagree, etc. The focus group functions like a discussion. It’s best to have both an interviewer and a notetaker and to video record for later review and analysis.

The main advantage of a focus group is that when people respond to questions in a group setting, they build off each others’ answers. Often, the conversation inspires respondents to think of something they may not have remembered otherwise. Also, focus groups allow us to interview more people than individual interviews. The main disadvantage is the same as with interviews — we can still reach only a small number of people, and since the resulting data is qualitative, it can take time to analyze.

Observations

Observing teachers and students in action can be one of the best ways to capture rich data about how teacher professional learning plays out in the classroom. Typically, observers use a protocol informed by the evaluation questions that outlines what the observer is looking for and what data to collect during the classroom visit.

The main advantage of observations is in witnessing first-hand how curriculum is being implemented, how instructional strategies are being used and how students are responding. The main disadvantage is in the potential for conflict, especially if positive relationships and trust aren’t a strong part of the school culture. While many teachers willingly invite observers into their classrooms, there can be tensions among colleagues and with unions who want to ensure that program evaluation does not influence teacher evaluation. It is critical to clearly communicate that data collected for professional development program evaluation is not to be used for teacher evaluation.

[ctt template=”9″ link=”XHiz8″ via=”yes” nofollow=”yes”]It’s critical to communicate to teachers & staff that data collected for professional development program evaluation is not to be used for teacher evaluation. — @SheilaBRobinson[/ctt]

A Few Recommended Practices

Do engage stakeholders, such as teachers and administrators, in all phases of program evaluation. Be transparent in letting people know why they are participating in data collection, why program evaluation is important to the department or district and what potential decisions may rest on the outcome. If people understand why program evaluation is being conducted and the role it plays in the organization, they will be much more likely to participate.

Don’t collect data you don’t need. For example, if you don’t need to compare how the program worked for 3rd grade vs 4th grade teachers, don’t ask them to provide their grade level. If you don’t plan to compare male and female teachers, or newer vs veteran teachers, don’t ask these questions.

Do keep the data collection brief. Have you ever known a teacher or administrator with loads of extra time in their schedule? Whether it’s an interview, focus group or survey, keep it brief by asking only the questions you need answers to.

Do incentivize responses to maximize the number of responses you receive. If possible, have light refreshments available for focus groups (minding rules for spending grants or general funds). Offer survey respondents raffle tickets for a good education book, a bag of school supplies, a gift card, etc. There are ways to keep survey responses anonymous while knowing who completed them for these types of incentives. Offer an extra planning period (coverage for a class or release from an administrative assignment) to interviewees.

Do find ways of working data collection into professional learning program activities — e.g., participant journals, pre-post assessments, logs (teachers might log how often they use a strategy or resource, and comment on how it worked with students), etc. The less people have to do outside of the professional learning program, the better.

Do think creatively about data collection. Student work samples, photos and videos are legitimate forms of data that can be analyzed to look for patterns that help to answer evaluation questions.

[ctt template=”9″ link=”bOK9x” via=”yes” nofollow=”yes”]In collecting data for #PD program evaluation, DO keep it brief, engage stakeholders, incentivize responses and be creative, but DON’T waste time collecting data you don’t need! — @SheilaBRobinson[/ctt]

Next up in the series is what to do with all the data you collect: analysis and interpretation.

Reference:
Robinson, S.B. & Leonard, K.F. (2019). Designing Quality Survey Questions. Thousand Oaks, CA: Sage.

Next Up In the Series:

In Part 6, we dive deeper into Effective Strategies for Analyzing Professional Development Data.

Using Data to Set & Reach Your Teacher Hiring Goals

Across your district, teachers and principals are using data to set goals and evaluate their performance. It should be natural that data-driven decision-making and strategic goal-setting are par for the course in the central office as well. When it comes to some of the most essential work in the district — recruiting and hiring exceptional educators — there’s often a lack of actionable goals and strategic use of data.

It’s understandable. Administrators working in Human Resources face an avalanche of tasks which often involve short deadlines and a dizzying amount of back-and forth. But by setting aside time to establish goals, you’ll have the structure you need to develop an effective strategy for continual improvement.

So, what goals should you set for recruiting and hiring? It all depends on your district’s unique situation. Let’s take a walk through the steps you need to take in order to set actionable (and attainable) goals for recruiting and hiring.

Step 1: Gather Hiring Data

The most actionable goals are based on a thorough review of your district’s data. So, you will need to pull together your recruiting and hiring data. If you have the right software, this is a breeze. But if not, this could be the hardest part of goal-setting — if your hiring process is based on paper, you will have some difficulty getting the full picture of your district’s trends. Or, if you manage your hiring through an inefficient HRIS system, you may need to get in touch with your IT department to export the data.

In any case, gather as much recruiting and hiring data as possible. Without this step, you might be able to come up with objectives, but you’ll be ill-equipped to track your progress toward them.

You may want to take a few minutes to reflect on this data-gathering process. If you had any difficulty accessing the data you need, or find any inaccuracies or gaps, consider setting a goal to make your data more visible and easier to use.

Step 2: Ask the Right Questions

Look closely at your district’s data and look for patterns. It may be most helpful to look for changes over the past several years and year-over-year trends, if the data is available. If not, it’s perfectly fine to start with the past year.

Ask yourself: Which data points are surprising to you? Why do you think the data shows one trend or another? What improvements would you like to see? Don’t forget to recognize and celebrate where you are doing well already.

A few specific areas to consider, and questions to ask:

New Hires & Applicant Pools

Look closely at who your district has hired. What do you notice about the population of new teachers? Is this what you would expect to see? Were you able to fill all of your open positions, or are still some waiting for the right candidate to come along?

Next, compare your new hire data to your applicant pool and note any differences you see. You may want to examine whether new hires disproportionately come from any particular recruitment channel, or if they tend to have a certain experience in common. For example, do word-of-mouth referrals represent a disproportionately large percentage of new hires? Or do you tend to hire heavily from your substitute teacher pool, or educators who started out as student teachers in your district?

You may also want to look at the volume of high-quality candidates, particularly by subject areas. Did you have enough high-quality applicants to select from overall? What about for shortage-prone positions like Special Education, STEM or speech pathology? And are you confident that the best candidates were hired, or is it possible that some may have slipped through the cracks?

For even better insight, take a look at retention rates for last year’s new hires, benchmarked against your overall retention rate. This will give you a better idea of how well your hiring process selects candidates who fit into the school culture.

The Hiring Timeline

The next step is exploring metrics like time-to-hire, and when most candidates were hired. By looking at how long it takes to fill most vacancies, you may be able to identify inefficient processes that hold up the hiring process. And by identifying which positions take the longest to fill, you’ll know where to put more of your recruitment resources.

Plus, you’ll be able to determine how early in the year you should start hiring to have positions filled. Remember that hiring late in the year means less-qualified applicant pools, so if most of your jobs are filled in late July and August, you could be missing out on the most talented candidates.

Dig into what could be causing jobs to not be posted until late summer. How can you predict staffing needs more accurately?

The Hiring Experience

Finally, you may want to gather input from individuals involved in hiring, from your new hires to principals. Consider sending out a survey asking new hires for feedback on the application and hiring process. For principals, consider sending a survey asking for their opinion on the quality of the applicant pool and the hiring process itself — were they satisfied with the level of support they received from Human Resources? Did they notice any inefficiencies that could be addressed?

Gathering feedback from those involved in hiring across the district allows you to go beyond hiring data alone and understand how others perceive the process. Plus, it fosters collaboration and engagement by showing that you care about their experience. And in the end, it will help you provide a more positive experience for applicants and administrators alike.

Step 3: Set Preliminary Goals and Actions

After working your way through the data, determine what your district needs to focus on for the upcoming year and set objective, clearly-defined goals that are aligned to those areas of improvement. These should be realistic — it’s good to have a few “stretch” goals, but ultimately, all of your objectives should be achievable. If you currently receive 40 applications a year for hard-to-staff positions across the district, don’t set a goal of having 250 next year.

A reasonable number of feasible goals will give you the structure you need to develop effective, targeted strategies, without overwhelming yourself or your colleagues.

Lay out your recruiting and hiring goals for the coming year and build a framework for a strategy to help you improve on those metrics and meet your goals. The more integrated that strategy is, the more effectively you’ll be able to improve your hiring across the board with less effort. But don’t stop there — to really ensure success, think about your goals forward and backward.

[ctt template=”9″ link=”oPd68″ via=”no” ]To ensure success, it’s essential to think about your hiring goals forward and backward. [/ctt]

Thinking forward comes naturally: I want X, so I will implement Y strategy to get there. Thinking backwards — or inverting the problem — means asking yourself, What would prevent me from reaching my goal? What should I do to make sure those things don’t happen? You might realize that your district is inadvertently doing something that holds your recruitment and hiring processes back. Then, you can make a change and ensure that nothing stands between you and your goals.

For example, if your goal is to find more applicants, you could “think forward” by allocating more time and money to attending more job fairs, both locally and out-of-state. But by “thinking backward” you might realize that plenty more job-seekers might apply to your district, if only your application process were more applicant-friendly. By no longer requiring job-seekers to apply in-person or send a thick packet through snail mail, you may find that more qualified educators apply to work in the district.

Step 4: Track Your Progress

After implementing any changes to your hiring process or strategy, the next step is to continually monitor your data. Don’t let yourself be surprised at the end of the year — you’re more likely to meet your goals when you can track your progress toward them and adjust course as needed.

This step shouldn’t be a burden, either. Just like you expect educators to embed data into their daily work, set aside 5-10 minutes once a week to check in on your progress. Even if the rest of your day is consumed by putting out fires and you don’t have time to immediately act on the data, you’ll be in a better position to stay on top of what’s happening in the district.

By setting goals based on your district’s hiring trends and tracking your progress against them, you’ll be able to strategically improve your teacher recruiting and hiring strategies.

With the Insights Dashboard in Frontline Recruiting & Hiring, your recruiting and hiring data is at your fingertips. You can monitor trends, plan ahead and maximize your recruiting efforts, without needing to dig through paperwork or spreadsheets.

6 Best Practices to Elevate IEPs

The individualized education program (IEP) is the seminal planning document for teaching students with disabilities, and the Present Levels of Academic Achievement and Functional Performance (PLAAFP) section should be its heart and soul. When done well, the PLAAFP describes a multi-dimensional student — one with strengths, interests, needs and aspirations. I’ve provided suggestions on how the PLAAFP can tell “the right story” for each student — let’s review some of these ideas and connect the dots so your PLAAFPs can set the stage for compliant, individualized and relevant IEPs.

Here are 6 best practices to help guide your IEP process.

1. Use Multiple Sources of Data to Build Comprehensive IEPs

The foundation of a compliant and relevant IEP is built upon the analysis and synthesis of multiple sources of data. Information obtained in the initial and 3-year re-evaluations is essential — however, don’t overlook the range of other data readily available or easily obtained on each student.

Collect direct feedback from students, parents and teachers about the student’s strengths and needs. Use progress monitoring data and progress reports to supplement present levels each year. And don’t overlook the information on functional performance you can get during formal observations.

2. Focus on Student Strengths, Interests and Preferences

Framing student strengths, interests and preferences (S-I-P) sets a positive tone in the PLAAFP and acknowledges that a student is so much more than his or her disability. As important, S-I-Ps are essential to good instructional planning. Identifying each student’s S-I-Ps should result in actionable information for educators.

For example, consider this question: “Now that I have this information, how can I use it to improve instruction for this student?” S-I-Ps, in essence, are instructional tools — they inform the selection of materials, strategies and reinforcers that result in engaging and accessible instruction.

You may enjoy this hand-picked content:

IEP Checklist for Teachers

3. Increase Parent Participation in the IEP Process

It’s important for parents to feel confident that school personnel know their child as an individual. Prior to the IEP meeting, reach out to obtain parental insights about their child and ideas they have for improving his or her school experience.

Help parents prepare for the meeting by providing them with some sentence starters. For example, “My child is really good at ______,” “My child is easily frustrated when ______,” “This would be a successful school year if my child ______.”

Seeing evidence of their input included in their child’s IEP can go a long way to support parents and increase their participation in the IEP process. Similarly, make sure parents have all the same information other team members have prior to the meeting. Prepared parents can be more engaged parents.

4. Describe How the Student’s Disability Affects Involvement and Progress in the General Education Curriculum

The essential question answered by the PLAAFP is, “How does the student’s disability affect involvement and progress in the general education curriculum?” Special education eligibility is conditioned on the student’s disability adversely affecting involvement and progress in the general education curriculum. The PLAAFP requires a description of HOW progress is affected.

[ctt template=”9″ link=”Q12aI” via=”no” ]The #IEP is the baton that gets passed from year to year; teacher to teacher.[/ctt]

For each PLAAFP statement, ask yourself, does this statement pass the “stranger test?” Could someone who does not know the student visualize the disability-related characteristics this student manifests as they function in the demands of the school environment?

When the student’s characteristics are out of alignment with the demands, there will be an adverse impact on the student’s involvement and progress. By aligning the student’s disability-related characteristics with authentic school demands, the student’s needs become apparent to the IEP Team.

5. Write Specific Statements on Academic and Functional Needs

This section of the IEP provides additional details to identify the student’s academic and functional needs. Describe the skills and behaviors the student demonstrates (“can do”) and compare them to the overall expectations (“expected to do”) with the curriculum. This gap analysis helps prioritize the student’s most urgent needs and provides the rationale for subsequent IEP goals, supports and services.

Academic needs are typically those associated with basic skill acquisition, reading, writing, oral communication and math.
Functional needs are those that affect a student’s ability to apply skills/knowledge to a range of domains, including communication, motor daily living, organization and more.

“Gap analysis helps prioritize the student’s most urgent needs and provides the rationale for subsequent IEP goals, supports and services.”

6. Establish Baseline Data to Take a Student’s Performance Temperature

With the student’s most urgent needs identified, the stage is set to select goals. Goals must be measurable and targeted — stating a specific and observable skill or behavior the student will do by the end of the IEP cycle. In other words, you’re asking yourself, “If the proposed interventions are successful, what will I see the student do in a year?”

Baseline data is important because it measures how well the student currently performs that skill before the intervention. Not to be confused with the most recent evaluations, think of baseline data as the “present tense” of the subsequent goals.

For example, if the goal is to increase a student’s time on task during independent seatwork, the baseline is how long the student is currently able to remain on task during independent seatwork. This information will help with IEP goal tracking. With this information, the team can determine the degree of change necessary for the student to reach a challenging and attainable goal.

Connecting the Dots for Each IEP

Teachers get to know their students well over time — but a well-written PLAAFP accelerates basic knowledge of a student on day 1. The IEP is the baton that gets passed from year to year; teacher to teacher. The PLAAFP paints a picture of the student so anyone receiving the baton can plan meaningful instruction for the student. It will provide teachers with the vital information they need to begin instruction and implement accommodations. Additionally, a well-written PLAAFP will show parents their child is known and valued as an individual.

Through a collaborative process, each student’s PLAAFP can elevate the IEP document from a compliant document to one that tells the right story about who each student is and what he or she needs to have a successful school year.

Looking to make the entire IEP process, including IEP goal tracking, more intuitive and less paperwork intensive for teachers in your organization? Dial down the stress levels with Frontline Special Ed & Interventions.

Dr. Jim Knight: Videotaping Lessons as Professional Development for Teachers

“Video is a central part of my own personal growth, because you need feedback to grow. You need to see where you are, you need to see where you’re trying to get to. So it’s a key part of what I do, not just professionally but personally.”

Dr. Jim Knight is widely known in the world of instructional coaching. He is president of the Instructional Coaching Group, and author of such books as The Impact Cycle: What Instructional Coaches Should Do to Foster Powerful Improvements in Teaching and High-Impact Instruction: A Framework for Great Teaching.

In 2014, he also wrote a book called Focus on Teaching: Using Video for High-Impact Instruction. Recently, he joined me for an interview about how teachers, principals, coaches and teams can all use self-produced videos of classroom lessons to advance teaching practice.

[Note: this interview has been edited for brevity and clarity.]

FRONTLINE EDUCATION: Dr. Knight, you’ve written and spoken about the use of video for high-impact instruction quite a bit. We really have to start with thinking about video itself. Why is video important for educators to consider?

JIM KNIGHT: We’ve seen the power of video in lots of other fields. You wouldn’t have a football team in America that doesn’t watch themselves on video, and lots of performing artists watch themselves on video. The reason why I think video is so important is that it provides you with a picture of reality you can’t get while you’re actually doing the job.

A hockey player, for example, might not know how out of position he is unless he sees the video, and when he sees the video he goes, “Holy smokes, I had no idea.” That’s why athletes, whether they’re in a middle school or in a university or whether they’re professional athletes watch themselves on video all the time. It’s the same thing with teachers, we’ve found…. Most people don’t have a very clear picture of what it looks like when they have conversations or when they do their work.

FRONTLINE EDUCATION: I can imagine someone saying, “I don’t think my teachers are going to be comfortable with it.”

JIM KNIGHT: Well the first thing I’d say is that they’re probably right. It is hard to watch yourself on video. We don’t like the way we look, and often we’re a little disappointed with our practice. But the way forward, the way to get better isn’t by avoiding reality, it’s by looking reality full-on. Video gives you a clearer picture of reality, and after a few times, you get used to it, and then it just becomes a tool you use.

The first time is kind of like hearing your voice for the first time on a recording to the power of ten. The way you move, what your voice sounds like — all kinds of things are a little disconcerting, but once you get used to it, you’re good.

We see it varies more by school than it does by person. In other words, if you wanted to introduce video in a school, likely either almost everybody would do it, or hardly anybody would do it. It’s not really an individual thing, it’s a culture thing. I think the issue is, if people feel psychologically safe, and they feel they can trust the people they work with, then they’re good. But if people don’t feel psychologically safe, they’re not going to open themselves up to a judgmental situation.

FRONTLINE EDUCATION: Your book discusses a number of ways that video can be used to improve teaching, by teachers themselves, by coaches, by principals and by teams. If I’m a coach, I can already observe teachers in the classroom, so what else does video bring to the table?

JIM KNIGHT: Well the trouble is that people often don’t have clear picture of current reality because of a number of different perceptual errors. I’m not just talking about teachers. We have a tendency to look for data — what’s called “confirmation bias” — that reinforces our perceptions of things. We also get used to stuff over time, what’s called “habituation.” What we think is happening and what’s really happening are often quite a bit different.

[ctt template=”9″ link=”kIZbE” via=”no” ]Videos of classroom lessons can help teachers curb habituation and confirmation bias. -Jim Knight [/ctt]

Video also allows us a chance to see things we might not see. It doesn’t have to be negative, necessarily. Sometimes a coach will video record a class, and the teacher will say, “I heard my kids talking and I couldn’t believe how supportive and encouraging they were, it was really a wonderful thing to see.” Or, “I realize when I watched the video, the kids actually understood the activity even before I started.” So sometimes they’ll see things that are good, not necessarily bad. It provides a bigger picture.

FRONTLINE EDUCATION: As you’ve spoken with teachers who have done this, what are the kinds of things that they’ve said to you that they’ve seen or come away with?

JIM KNIGHT: Sharon Thomas, who was a teacher in Maryland, she now works with us. She said when she watches it, it’s like the MacGuffin effect in Hitchcock movies. She always sits down expecting to see one thing and she’s looking for that one thing, but then as she watches the video, it always ends up being something else than what she thought. And I’ve heard that from other people too, that their expectation of what they’re going to see in the video, and what they really see, is quite a bit different.

FRONTLINE EDUCATION: What are the biggest hurdles that schools and districts face in using video to support improvements in practice?

JIM KNIGHT: The biggest hurdle is the culture of the school…. If [as a teacher] I know they’ve got my best interest at heart, it’s going to be pretty easy for me to agree, but if I’m not sure of that, I’m going to hesitate. I’d say it’s really important to create a culture where that’s going to happen.

A second thing is not just psychological safety. Sometimes there’s a culture of talk versus a culture of action — we do workshops all the time, and we talk about evaluation, and we do school improvement but nothing really changes. When you use video, the moment you push the red button on your phone or tablet, you move from a culture of talk to a culture of action. Once you look at the video, something has to happen, and in some systems it’s counter-cultural to actually be working on really doing things.

[ctt template=”9″ link=”daK8b” via=”no” ]Jim Knight: “When you use video [for teachers to self-reflect on classroom lessons], you move from a culture of talk to a culture of action.” [/ctt]

FRONTLINE EDUCATION: Is there anything else you’d add?

JIM KNIGHT: There’s a kind of a paradox at the heart of all of this: to live a fulfilling life, you have to be getting better. If you just stay the same all the time, something kind of shrivels up inside you. You impoverish your life if you don’t grow and learn. That’s why there are so many self-help books. To get better you have to face reality, and that can be painful, so the initial experience of getting better doesn’t seem like it’s nourishing your well-being at all. But you’re not going to get to the point of feeling like you’re really improving and growing unless you look at reality. So ironically, to get better you have to feel worse first. You have to learn where you are.

Right now, I’m trying to get in shape so I can run more, and that means I have to lose weight and I have to look at the scales every day and go, “Oh, I never should’ve had those chips and salsa.” I have to see reality every day and maybe it’s a little painful, but to get better you have to see it, and that’s a really interesting dynamic about the use of video.

Sometimes people would rather not see reality because it hurts too much to look at reality. But in the long run, to really feel fulfilled they have to be getting better.

This was just a small excerpt from a longer interview. Listen to the entire interview above, or subscribe to the podcast..

The Value of Providing Feedback for Substitutes

With each new school year you work hard to ensure that everyone is ready, from bus drivers to teachers. But we’d like to suggest one more thing to consider: committing to providing feedback to substitute teachers throughout the school year.. You’ll help your substitutes become more effective in the classroom and gain more visibility into your sub pool’s performance across the district.

We’ve written a lot about the value of feedback , particularly as it relates to teachers’ professional growth. But substitutes are often left out of these discussions, even though they’re essential to your district’s operations. Plus, remember the impact they have on student learning: students spend the equivalent of about two-thirds of a school year with substitute teachers over the course of their K-12 journey. That’s a lot of instructional time! So, it’s important to help your substitutes make the most of their time in the classroom by having teachers provide regular, honest feedback.

How to Provide Feedback for Substitute Teachers

Have your teachers get in the habit of documenting substitute performance where possible, and make sure that it’s made available for the substitute to see.

Try not to have substitute feedback communicated over email — you won’t have much visibility as an administrator into which substitutes are doing well and which aren’t. Instead, consider using an electronic system where teachers can provide feedback online. Teachers’ comments will be more consistent across the district, and substitutes will be better informed as to how they can better improve their skills as educators. It’s a good way to engage with substitutes and show that you see them as part of the educational community.

Even more importantly, by managing substitute feedback online, you’ll gain insight into your substitute pool’s effectiveness. For example, you’ll be able to:

See which substitutes may benefit from additional support or training
See which may not be the right fit for your district.
Address any possible issues that may arise, before they become a problem for the district.
Identify high-performing substitutes to recognize — whether on social media, at district events or at a Substitute Appreciation Tea.

Regular Substitute Teacher Feedback is Crucial

To get the full picture of substitute effectiveness, regular feedback is crucial. If teachers only leave comments from time to time, you won’t have context as to how substitutes perform on a daily basis. So, encourage teachers to leave feedback for substitutes after every absence. An electronic system like Frontline Absence & Time can automatically remind employees to leave feedback for substitutes, so it doesn’t fall through the cracks.

Make sure that teachers have a set of questions to answer about their experience with the substitute. If you leave it completely open-ended, you won’t have consistent reviews and may not get the information you’re looking for. By default, Frontline’s system asks the following questions, although they can be edited or customized to your heart’s desire:

Was classroom work collected?
Was the room left as neat and clean as it was found?
Was classroom work explained satisfactorily?
Did students report that they were treated fairly and consistently?
Were any disciplinary issues reported?
General notes/comments

By collecting and offering regular feedback, you can help your substitute pool be as effective as possible — which ultimately benefits students. So, make sure that your substitute management plan for the year includes a strategy to collect and act on teacher feedback.

Learn more about how Frontline Absence & Time can support your district.

Evaluating Professional Development for Teachers? Evaluation Questions Are the Linchpin.

Edwards Deming, famous American management consultant, once quipped, “If you do not know how to ask the right question, you discover nothing.”

Evaluation questions form the cornerstone of professional development program evaluation. They both frame and focus the work, pointing us in the right direction for data collection. After all, how would we know what data to collect, and from whom, if we haven’t settled on the questions we’re asking? Crafting the right questions for a particular evaluation project is critical to an effective evaluation effort.

What are evaluation questions?

Think of evaluation questions as research questions. They are broad questions that usually cannot be answered with a simple yes or no, and require collecting and analyzing data to answer. Most importantly, these are not the individual questions we would ask someone on a survey (we’ll get to those in the future!).

Evaluation questions are big picture questions that get at program characteristics and are evaluative. That is, the answers to these questions will help us understand the importance, the quality or the value of our programs.

[ctt template=”9″ link=”O728M” via=”no” ]Well-conceived evaluation questions help us to understand the importance, quality and value of our professional development programs in K-12. [/ctt]

Imagine you are evaluating a professional development program. What do you need to investigate? To answer this, let’s take a quick step back. The previous article in this series showed how to engage stakeholders and generate shared understanding of how our professional learning programs work by creating program descriptions, logic models, and a program theory. Now, we’ll see what you can do with these products to focus your evaluation!

Identifying information needs and decisions

First, consider what you need to know about your professional learning program. This may depend on what decisions you (or others) have to make about it. Do you need to decide whether to continue offering the program? Offer it to an expanded audience or at multiple sites? Eliminate it altogether? Try a different program to address the problem at hand (e.g. improving middle school writing skills)?

Once you’ve identified decisions that need to be made, revisit those three products — program description, logic model and program theory. What are the implicit or explicit assumptions being made about the program? For example, does the program theory state that the professional learning will change teachers’ thinking? Encourage them to use new strategies or resources in their teaching practice? Does the logic model identify certain expected outcomes for students?

Determining the questions

You may be thinking at this point, “Well, we just need to know if our program is working.” To that I would ask, “What do you mean by working?”

“Well,” you might say, “We want to know if the program is effective.” And I would answer with another question: “What do you mean by effective?”

And so it would go until you can define and describe exactly what you are looking for.

Again, revisit your three documents. As you review the program description and program theory, what clues do you have about what it should look like if the program is working or effective? Try to describe this scenario in as much detail as possible. Here’s an example:

If our professional learning program on writing instruction for middle school teachers is working (or effective, or successful…) we would hear teachers saying that they’ve tried the new strategies they learned, and are now using them in their daily practice. They would be able to show us how they are now teaching writing using the new templates. They would be able to show us “before” and “after” examples of student writing, and be able to describe in specific ways how students’ writing has improved.

Once we have defined success, we can turn these ideas into evaluation questions. One of my favorite tricks for doing this is to ask “To what extent…” questions. For the example above, these questions might look like this:

To what extent are teachers using the new writing instruction strategies?
To what extent are teachers using the writing template?
To what extent are teachers able to identify and show specific improvements in students’ writing?

Posing these questions may also inspire some sub-questions:

To what extent are teachers using the new writing instruction strategies?
- How many teachers have tried the strategies at least once?
- How many teachers are using the strategies twice per week or more often?
To what extent are teachers using the writing template?
- How many teachers are using the writing template at least once per week?
- To what extent are teachers using the template as given, or modifying it for their classrooms?
To what extent are teachers able to identify and show specific improvements in students’ writing?
- What specific improvements are teachers identifying?
- To what extent do the improvements we are seeing match the writing deficits we identified when we started the program?

As you can see, your list of evaluation questions can grow quite long! In fact, you may be able to identify dozens of potential evaluation questions. To keep the evaluation feasible, prioritize these and settle on perhaps just 1-2 big questions, especially if each has a couple of sub-questions.

You may enjoy this hand-picked content:

White Paper: 18,000 Reasons to Rethink Professional Development

Need more inspiration?

Here are generic examples of the types of evaluation questions you may need to ask. Some questions might be formative in nature. That is, they may be used to inform potential changes in the program. Think of these as process questions or implementation questions.

Is the program reaching the right people? (In other words, are the people we want to be enrolling in the program doing so?)
Is the program being delivered (implemented) as intended?
In what ways (if any) does the implementation of the program differ from the program description?

Other questions might be summative in nature. These questions ask about outcomes, impacts or changes that occur that we believe can be attributed to the program.

To what extent is the program producing the expected outcomes (i.e., achieving its goals)?
How do the effects of the program vary across participants? (i.e., are different teachers experiencing different results?)
What evidence do we have that the program is effective in meeting its stated goals and objectives?
How can the program be improved?
Is the program worth the resources expended (i.e., the costs)?

Using a checklist may be helpful in determining whether your questions are appropriate and will be effective.

[ctt template=”9″ link=”AwUf5″ via=”no” ]It’s critical to craft the right questions when evaluating #professionaldevelopment for teachers. [/ctt]

Evaluation questions lead to data collection

As we have learned previously, we can conceive of program evaluation as occurring in five phases: Engagement and Understanding, Evaluation Questions, Data Collection, Data Analysis and Interpretation, and Reporting and Use.

As you can see from the above examples, evaluation questions point us to where and from whom to collect data. If our question is, “To what extent are teachers using the new resources?” then we know we need to collect data from teachers. If our question is, “Are students’ writing skills improving?” we know we will likely need student work samples as evidence.

In each case, we will have to determine the feasibility of using different data collection strategies such as surveys, interviews, focus groups, observations or artifact reviews (e.g., looking at lesson plans or student work samples). Each of these strategies features a set of distinct advantages and disadvantages and requires different resources.

Next Up In the Series:

In Part 5, we dive deeper into Data Collection.

Micro-Credentials — Frequently Asked Questions

MICRO-CREDENTIALS HELP YOU FOCUS PROFESSIONAL LEARNING ON OUTCOMES

There’s nothing wrong with mandating a certain amount of professional learning for teachers throughout the year. But when meeting a minimum hourly requirement becomes the goal in itself, rather than the means to improving practice, the effectiveness of PD plummets.

That’s why it’s so important to focus on impact and outcomes, not punching a clock. That can be a bit of a holy grail: easy to talk about, difficult to achieve. And that’s one reason micro-credentials are growing in popularity in schools and districts.

Micro-credentials are an effective way for leaders to be sure their educators have the skills needed to be successful in the classroom. They’re also a great way for educators to be recognized for the skills they possess or develop through professional learning.

Q: What impact do micro-credentials have on professional learning?

Micro-credentials make the professional learning experience far more effective. They do this by:

Requiring job-embedded evidence to earn the micro-credential, so you know the educator can apply the skills they’ve learned in the classroom.
Allowing educators to focus on developing the skills they lack, without spending a lot of time on the skills they already have.

Some professional development systems that offer micro-credentials also enable a collaborative-based approach to assessment, which allows you to leverage the strength of experts in your district.

You may enjoy this hand-picked content:

It’s All About the Outcome: Unraveling the confusion around micro-credentials.”

Q: How do micro-credentials meet teachers’ needs?

While the need to develop skills and demonstrate competency has not dwindled, traditional ways of delivering and measuring professional development are becoming obsolete. In fact, 71% of teachers aren’t satisfied with their current PD offerings^[i] and only 40% feel that PD activities are a good use of their time.^[ii]

Every teacher has unique skills and areas for growth — so why should professional learning be one-size-fits-all? And among other criteria, the Every Student Succeeds Act calls for professional learning to be sustained, intensive and classroom-focused. In other words, the need is for professional development that reaches individual teachers right where they are — and equips them to make genuine strides in their classroom practice.

Micro-credentials provide exactly that. They allow learners to progress through their PD goals as they demonstrate mastery of skills — regardless of time, pace or place of learning.

[ctt template=”9″ link=”6ddbw” via=”no” ]Micro-credentials allow learners to progress through their PD goals as they demonstrate mastery of skills, regardless of time, pace or place of learning.[/ctt]

Q: Why should we use micro-credentials as part of our professional development strategy?

Glad you asked! Depending on the system you’re using, there are several advantages micro-credentials offer:

Take a growth-based approach. Micro-credentials redefine the professional learning experience, shifting the emphasis from seat time to demonstrated competence.
Support development around targeted needs. Allow instructional leaders to recommend micro-credentials that match an educator’s specific growth goals.
Provide ongoing feedback. Educators can submit pieces of required evidence — one at a time — enabling district-assigned assessors to provide incremental feedback.
Enhance all your PD resources. Micro-credentials have the potential to make all the PD tools and resources your educators access — from virtual PLCs to online tools to coaches and mentors — more relevant and meaningful.

Micro-credentials in Frontline Professional Growth make the professional learning experience far more effective. They clearly show what an educator needs to know to complete the micro-credential; how they can develop the skill and if they need more knowledge or training; and how they can demonstrate skill mastery and apply it in a classroom or other professional setting. Check them out.

^[i]Teachers Know Best: Teachers’ View on Professional Development. (Boston Consulting Group and Bill & Melinda Gates Foundation, December 2014). https://s3.amazonaws.com/edtech-production/reports/Gates-PDMarketResearch-Dec5.pdf

^[ii]The Mirage: Confronting the Hard Truth About Our Quest for Teacher Development. (The New Teacher Project, August 2015). http://tntp.org/assets/documents/TNTP-Mirage_2015.pdf

3 Trends to Know When Recruiting New Teachers

Every time a position opens up in your district, whether from attrition or increased student enrollment, you need to find a qualified, talented candidate to fill the role. It’s easy to get into reaction mode, where you go from filling one position onto the next, but part of strategic recruitment is looking ahead and planning accordingly.

So, what trends should you be aware of in K-12 recruiting and hiring? We looked at data from Frontline Recruiting & Hiring, the U.S. Department of Education and the National Center of Education Statistics to put together a comprehensive picture of what’s on the horizon.

The numbers don’t lie — there really is a teacher shortage, and it’s bound to only get worse.

The applicant pool is relatively inexperienced.

First up: an insight from the Frontline Research & Learning Institute. One-third of the national applicant pool consists of new teachers with fewer than three years of experience.

Less-experienced teachers may make up the bulk of your applicant list, and it’s important to have a way to highlight those with great potential, even if they don’t have years of teaching to back it up. And once they’re hired, you’ll want to have robust mentoring and professional development programs ready to help newer teachers hone their instructional skills.

Not only are many teaching candidates relatively inexperienced, they’re also fairly mobile. This is good news when recruiting (there’s a larger pool of job-seekers) but can be a challenge when it comes to your own district’s retention rate. New graduates don’t seem to be staying in one place for long.

Most states are seeing decreased enrollment in teacher prep programs — but not all.

If applicant pools are comprised in large part by newer educators, the logical next step is to look at the pipeline of new teachers set to enter the workforce in the next several years. Nationally, there’s certainly a shortage of teachers that’s bound to get worse. According to the U.S. Department of Education, the number of new teachers coming out of teacher preparation programs fell by 65% from 2009 to 2015.

[ctt template=”9″ link=”pA62H” via=”no” ]New teachers coming out of teacher prep programs down by 65%, according to DOE. #K12 #teachershortage[/ctt]

But what’s happening on a national level isn’t always reflective of what’s going in your own state, as demonstrated by the map below, which tracks changes in the number of educators completing teacher preparation programs by state.

Source: U.S. Department of Education, 2016 Title II Reports: National Teacher Preparation Data. https://title2.ed.gov/

Six states have defied the national decrease in the number of new teachers: Massachusetts, North Dakota, Utah, Virginia, Arizona and West Virginia. Of those, only Arizona and Utah continue to see increased enrollment in teacher preparation programs, suggesting that they will continue to produce more educators. And although New Hampshire, Washington and Vermont have had fewer teachers graduating from teacher prep programs, enrollment has recently ticked upward — so hopefully, those states will see an influx of new educators within the next couple of years.

New education graduates tend to be from large public universities in the West, or smaller private colleges in the East.

Finally, there are a few geographic trends when it comes to the most prolific teacher preparation programs.

Source: National Center for Education Statistics. IPEDS: Integrated Postsecondary Education Data System. https://nces.ed.gov/ipeds

There are far more programs east of the Mississippi River, with many educators coming from many smaller colleges. Meanwhile, the West tends to see a large number of new teachers from a relatively limited number of universities.

We also see a surprisingly large number of education degrees conferred by online programs headquartered in western states, such as Ashford University (California) and Western Governors (Utah). Note that these programs’ students are likely located across the country, even though their degree is recorded under the institution’s home state.

So, what does this mean for the school district administrator? If you recruit out-of-state, consider tailoring your recruitment strategy to target some of the larger universities, even if they aren’t nearby. Consider attending a job fair in the area, if you’re up for a road trip. Just make sure to bring plenty of recruitment materials that show why your town or city is a wonderful place to live.

If you’re not up for a road trip, or travel expenses aren’t in the budget, it may be worth reaching out to large universities’ career services or schools of education to ask if they’d spread the word about open positions in your district.

Looking for more insights into the new teacher pipeline? We’ve got you covered. See more trends and data here.

3 Surprising Ways to Engage Stakeholders in Evaluating Professional Development (and Ensuring a Quality Evaluation!)

At Greece Central School District in Rochester, NY, we hired certified trainers to facilitate the 8-day Cognitive Coaching® seminar for all of our teacher leaders. We followed up with monthly collegial circle meetings for them to reflect on their learning, share how coaching sessions were going and practice scenarios with one another to refine their skills. We invited the trainers back for yearly refresher sessions. This program was designed to influence changes in teacher practice and ultimately impact student learning.

But, ask each of our teacher leaders to describe what changes in practice might look like and how coaching would impact student learning, and you’ll get almost as many answers as participants.

How do we evaluate a professional learning program if everyone has a different idea of what the program does, who it serves or what success looks like? How might we generate the right questions to ask, identify appropriate expected outcomes and determine what to measure?

Part of evaluating a program is understanding the program and what we expect it to do. And part of a successful evaluation effort is getting stakeholders — teacher participants, principals, district office administrators, Board of Education members, etc. — on board to support the work.

In my previous article, I outlined five phases of program evaluation with the first being engagement and understanding. Here, I’ll describe three evaluation-related practices:

developing a program description,
creating a logic model, and
articulating a program theory.

These can be used to engage stakeholders, build a common understanding of professional learning programs and set up for a successful program evaluation.

[ctt template=”9″ link=”I67mB” via=”no” ]3 key practices in evaluating #ProfessionalLearning: 1) develop a program description, 2) create a logic model, and 3) articulate a program theory.[/ctt]

Why spend time on crafting these elements? There’s nothing worse in program evaluation than collecting and analyzing data only to realize that the results aren’t useful. They don’t help you answer questions, or inform decisions you have to make about the program. Let’s take a look at these three elements, and how they lay the foundation for successful program evaluation.

Program Description

Why is a program description so important to program evaluation? A program description promotes clarity and contributes to a shared and comprehensive understanding of what the program is, who it is intended to reach and what it expects to accomplish.

A thorough description also identifies why the program was developed. What is the need or problem the program addresses? It’s worth gathering a group of key people to craft a few brief paragraphs to answer these questions, even when the program has been developed by someone else.

It’s OK if your description isn’t the same as another district might come up with. For example, maybe your district held the Cognitive Coaching® seminar for administrators, not teachers, and for a different reason than my district did. Our descriptions of need, target audiences and expected outcomes will look different, even when the program itself may be delivered identically in both places.

Logic Model

A logic model is a graphic representation — a concept map of sorts — of a program’s inputs, activities, outputs and outcomes.

Inputs are what we need to run the program. What resources do we need? Funding for professional learning facilitators, curriculum materials, space to hold courses?
What activities comprise the program? Workshop sessions, follow-up coaching, action research and examination of student work are just a few of the possible professional learning formats.
Outputs are produced by the program activities – that is, the number of sessions held, the number of participants who attended, products such as action research findings, lesson or unit plans, or other curricular resources that were developed. Outputs are generally easy to report, but don’t speak to the quality or effectiveness of the program.
Outcomes describe the expected changes in program participants. With professional learning, we may expect that teachers learn and apply new content, use new resources or instructional strategies, or change their teaching practice. And of course, expected outcomes may include increases in student achievement (or other student-related metrics such as discipline or attendance) as a result of teacher learning and change in practice.

Outputs and outcomes are easily confused. Just remember that outputs are program data, and outcomes are people data! The Tearless Logic Model describes an interactive, collaborative (even fun!) process for creating a logic model that is certain to appeal to educators.

[ctt template=”9″ link=”gc7Cv” via=”no” ]In professional learning, it’s easy to confuse outputs and outcomes. Outputs are program data. Outcomes are people data.[/ctt]

Program Theory

When we create professional learning programs, purchase professional learning curriculum or hire consultants to facilitate learning, we think we have high quality professional development, but how do we really know? Programs may meet certain characteristics that make them likely to be high quality (e.g., ongoing, job-embedded, data-driven). But how can we connect the dots between what the teachers are learning and how their students will benefit?

Recently, I led a collegial book study on Culturally Responsive Teaching and the Brain by Zaretta Hammond. We had teachers read the chapters and participate in online discussions. But how did we expect that teachers reading a book and writing about their thoughts would lead to improvement for students?

This is where program theory comes in. Program theory describes how the program is supposed to work. Some might call this “theory of change.” The program theory blends elements from the program description and information outlined in the logic model. Most importantly, a program theory articulates the linkages among the components of the logic model.

A key reflective question for articulating a program theory is this: What makes us think that this program, the way it is designed, and these particular program activities will lead to those expected outcomes we identified? A simple program theory for my book study might start like this:

If teachers read, reflect on and discuss with colleagues the material in Culturally Responsive Teaching and the Brain, they will deepen their understanding of culturally responsive teaching (CRT).
If teachers deepen their understanding of CRT, they will begin to think about how it connects to other work we do in the district around equity and social emotional learning.
If teachers deepen their understanding of CRT, they will also begin to recognize the cultural capital and capabilities students bring to the classroom, and will be able to use these tools to create engaging instruction.
If teachers learn about implicit bias, they will develop an awareness of how it can be a barrier to positive student-teacher relationships.

[ctt template=”9″ link=”60PLz” via=”no” ]Reflection question when evaluating PD: Why do we believe that a professional learning program will lead to a given set of expected outcomes?[/ctt]

A few bullet points later, we might articulate how teachers will change their practice, and eventually, there will be a connection to the specific areas of student learning and achievement we want to improve.

The idea is that we’re identifying how we expect our professional learning programs to work. Once we do this, we can identify where we want to ask questions for the evaluation. Do we want to know if teachers are in fact, deepening their learning? Or do we want to investigate whether teacher learning is resulting in change in practice? A program theory helps us to know where to look and what to look for in a program evaluation.

Creating a program description, developing a logic model and articulating a program theory need not take a great deal of time. The investment, however, is sure to result in more clarity around our professional learning programs and shared understandings of how our professional learning programs are expected to produce results. They lay the foundation for us to identify relevant evaluation questions and set us up to collect the right data for our program evaluation.

Next Up In the Series:

In Part 4, we’ll learn about evaluation questions and how they point directly to the data we need to inform key decisions about professional learning.

The Truth Is… We Could Not Run Our Schools Without Our Substitutes

On any given Friday in May, my school district is struggling to fill our substitute vacancies. Many of our highly regarded substitutes are retired teachers, and by May they have worked all of the days that our state’s retirement system will allow. In addition, May brings college graduations and weddings and, well, a wide array of reasons for our teachers to be absent from work. And yet… learning must continue.

Thus, we find ourselves using creative solutions to plug those holes. A substitute may arrive at one school to step in for a third grade teacher, only to be told that we’ve covered that vacancy another way, and that he/she is instead needed at another school across town. Or, a substitute may come to a middle school teaching assignment expecting to have planning time as part of the day (definitely not guaranteed for substitutes, but a nice perk!), and will instead be told that he/she will be covering other classes during that time.

When we ask our substitutes to do more and more, I’m glad that we have a few constructs in place to thank them for their commitment to our district:

A Loyalty Rate for Loyal Substitutes

Like many districts, we have different pay rates — a daily rate, an increased rate for those who have worked a certain number of days, and a long-term rate. A couple of years ago, we also added a Loyalty Rate to honor those who have, indeed, shown loyalty to our school district.

In order to be eligible for this rate, a substitute must meet these qualifications:

Have worked in our district for at least 5 years as a substitute and/or in a full-time position.
Have worked on at least 90 days as a substitute in our school district in the previous year.

The Loyalty Rate resets each fall, and thus each spring we review our data to identify which substitutes qualify for the rate in the next school year. Full disclosure: the Loyalty Rate is only $3.00 per day more than the rate used for those who have worked at least 60 days. However, we know that our substitutes may also work in surrounding districts, and we created this rate to both encourage them to keep our district at the top of their list, and to acknowledge their consistent work and longevity with our district.

Substitute Appreciation Tea

Each June, we host a Substitute Appreciation Tea. We schedule it during the window between the school year ending and summer school starting so that as many substitutes as possible might be available. We work with our food service provider to create a real tea, complete with finger sandwiches, pastries, lemonade, and of course, tea!

The agenda is simple:

Very brief speeches from a couple of grateful administrators.
Time to mingle (many of our substitutes have known each other for years, and love the chance to catch up).
An opportunity for substitutes to provide feedback to us via a brief form which they complete in small groups.

Our Tea gives us the chance to let our substitutes know how much we really appreciate them while also getting feedback from them in a positive setting.

A few photos from our last Substitute Appreciation Tea:

Substitute Teacher Training

We are proud of the training that we provide to our substitutes and believe that putting time and effort into offering our substitutes training shows them that we appreciate and value the work that they do for us. We make it very clear to them that every day counts for our students, that gone are the days when a teacher might have the substitute put on a movie for the students. Our substitutes need to be ready to teach new skills and reinforce what has already been taught, and thus our school district helps them to hone their craft.

All of our substitutes are required to attend a half-day training prior to working for us. Even if they have worked in this role in other places, we want to make sure that they have full awareness of the expectations in our district. We try to keep our training interactive and fun while we provide important information.

During this training, we cover:

HR basics such as securing assignments and understanding pay structure
Information about our district
Crisis plans and health training (such as practice with EPI Pens)
Classroom management expectations and tips

In addition, we offer our substitutes optional training opportunities throughout the year. We take advantage of times when we know our substitutes would not be working for us, such as after school or during School Improvement Days when students are not in attendance, and then provide in-services run by our specialists. For example, training about working with students who receive special education services was provided by our Director of Student Services, and updated information about math instruction was given by our Math Coordinator. In this way, we help our substitutes to remain current in their teaching practices, and we honor their work as educators who are important to our system.

Of course, the best appreciation comes at the school level. I will never forget the joy and pride I felt when teachers told me that they had created goodie bags for all of the substitutes who were present at their school on a particular day. I also regularly remind our principals and building secretaries to greet and thank substitutes, and to treat them as the professionals that they are.

The truth is, we could not run our schools without our substitutes. It seems only right that we let them know that.

This post also appears on the author’s blog.

8 Tips on Listening for the Truth in Reference Calls

The reference call – it’s often the final check in a lengthy process of searching for the best educator to hire for the school district. These calls can either confirm the hiring of a great candidate – or prevent a terrible decision. However, to make the most of reference calls, administrators and hiring managers have to know how to listen for the truth.

Lynn Glickman, the Assistant Superintendent for Human Resources at Community Consolidated School District 21 in Wheeling, Illinois, has learned a few things in her years of hiring and both making and participating in reference calls. Get her eight tips for listening for the truth.

This post originally appeared on Dr. Lynn Glickman’s blog: Reference Calls — Listen for the Truth (8 Tips).

I was a newer principal, and I had recently dismissed a pre-tenured teacher. She wasn’t terrible, just not good enough for my students. I had written her a reference letter, the kind you write for just this type of situation — strengths, as everyone has them, with lots of holes that I hoped other interviewing administrators would find. Then I took a call from a principal in a nearby district. We knew each other by association, but that’s all. He was eager to hire her, and this call was his final stop. I answered his questions and thought to myself, “I hope he listened carefully, and realized that I wasn’t really recommending her.”

Years later, this principal and I landed together in a new district. Early in our professional relationship, he told me, “You know, I had to let that teacher go after one year. I couldn’t believe you had recommended her to me — I was always so annoyed about that!”

“What?” I said. “I didn’t recommend her! I was trying to tell you not to hire her!” It was a perfect example of what happens over and over again. He was listening only to confirm his decision to hire her, and I was being positive, hoping he could read my mind.

I share this story often in my current role as Assistant Superintendent for Human Resources. Administrators (and hiring executives in other organizations, I am sure) work diligently to find the very best person for their opening. They carefully comb through applications and resumes, conduct screening interviews, 1:1 interviews and often team interviews. By the time they get to the reference checks — well, that’s what they are often doing — checking a box. This is the last step in the process of finding the Perfect Person. They are already sold on the candidate, and if they brought in a team to interview, so is the team! Who wants bad news at this point? And so, like my principal friend from many years ago, they miss what a reference might be trying to convey in an oh-so-indirect way.

Hiring is one of our most important administrative tasks, particularly when hiring teachers and principals. This is the time when we really need to get it right, because getting it wrong can make for a very, very bad year. (And depending on a whole bunch of variables, one mistake can be prolonged for much longer than a year.)

So, how do we get it right? Here are a few tips:

Before starting to make reference calls, promise yourself that you will be open to hearing bad news. Honestly, that’s what is most important. In fact, when I interview with a team, I always talk about how important reference calls are, and let team members know that sometimes we have to go another way after we’ve made those calls.

Experienced interview teams know that the person who came to the interview is not always the person who “shows up” at work, even if it is the same person! When I remind my team of that, they are understanding if we end up going with our #2 choice, or even if we have to interview more candidates.
Try to establish rapport with your reference at the start of the call. Yes, everyone is busy, but find a way to connect with the reference as a person. It will be harder for your reference to evade your questions or straight out lie to you if he/she likes you on the phone — even a little bit!
Do you have flexibility with your questions? If so, make sure to ask, “Would you re-hire this person?” Or, if you are speaking with a colleague, you may get useful information asking, “Why do you think this person would be interested in coming to a new school/district/organization?” If the first answer you get seems bogus, find a way to dig a little.
Even if you have to use a standard set of questions, surely you can ask follow-ups. When I sense that someone is choosing words very carefully, or is trying to brush past a question with a pat answer, I may ask, “Can you tell me more about that?” Often, they will.
Pay attention to the reference’s tone of voice and listen for pauses. Your reference may be trying to figure out how to answer a question in a way that is semi-truthful without hurting the person’s chances for getting the job. This is the perfect place for those follow-up questions.
What about those references who ask you to simply email them the questions, and they’ll email you the answers? Yes, they might just be really busy, but they might be trying to control the situation by carefully wording their answers.

You can’t see pauses in an email, you can’t hear tone of voice and asking follow-up questions is then even more time-consuming.

I try very hard to get people to talk to me on the phone. When they won’t, I don’t necessarily decide that there is a problem with the candidate, but I don’t put much stock in that particular reference.
Think carefully about who you are calling. For the most part, avoid calling your candidates’ colleagues, or only do so if you are also calling a supervisor or two.

A teacher’s teammate is rarely going to tell you that his/her colleague had classroom management problems or didn’t communicate well with parents, and an administrator’s lateral coworker is unlikely to tell you that projects were not finished on time.

I won’t hire anyone without speaking with his/her current supervisor, although certainly I will agree to make that call last, after I’m satisfied with the results of other calls. Also, you may be able to glean quite a lot of information from a supervisor from 2-3 years ago. Sometimes, the supervisor won’t remember much — but sometimes he/she will remember a lot, and will be ready to talk when no longer in regular contact with the candidate.
Finally, use your resources. The best reference call is with someone you know. Even if you don’t know any of your candidate’s references, someone else who is part of the hiring process might! If so, hand that call over to your colleague to get the most truthful information.

I still enjoy a friendship with that principal colleague who hired that marginal teacher. We still talk about this. We both learned. We are both still learning.

10 Best Practices for Improving and Expanding Social, Emotional and Behavioral Supports

While there is much debate about why an increasing number of children come to school with significant social, emotional and behavioral (S, E & B) needs, nearly all districts report the number of children with these challenges is on the rise.

In order for students to meet developmental milestones, learn, grow and lead productive lives, it is critical that their social, emotional and behavioral issues be addressed. Research indicates that children and youth with mental health problems have lower educational achievement and greater involvement with the criminal justice system [1]. Improving and expanding S, E & B supports not only helps the students who have these challenges but can benefit nearly every student and adult in a school.

[ctt template=”9″ link=”4S5P3″ via=”yes” nofollow=”yes”]75% to 80% of children and youth in need of mental health services do not receive them.[/ctt]

All schools — urban, suburban and rural; large and small; and regardless of socioeconomics — have students with social, emotional and behavioral challenges. However, in some of these communities, students receive the counseling they need, classroom routines promote positive behavior, and most strikingly, students with problematic behavior are able to stay in class and seldom disrupt their peers. What is the difference between these schools and typical schools? The distinctions can be hard to notice because the difference isn’t in the amount they spend, the programs they bought, or the dedication of their staff. The people, tools and talents themselves aren’t all that different but the way in which staff work and deliver intervention is different — the more effective districts have created a coherent, collaborative plan grounded in a systems-thinking approach and incorporating best practices.

Here, we focus on 10 key, interconnected best practices to help you and your team effectively and comprehensively create a system to meet the social, emotional and behavioral needs of students. These practices fall into three major categories: Leveraging the talents of current staff, focusing on prevention and supporting local partnerships.

Leverage the Talents of Current Staff

1. Streamline meetings and paperwork to increase the time staff can spend with students.

Process mapping, reviewing who attends which meetings and setting guidelines for desired time with students can often significantly increase the services provided to students by current staff.

2. Allow staff to play to their strengths; assign roles based on strengths, not titles.

Identify staff’s unique skills and match job responsibilities to these areas of expertise. For example, some psychologists may have expertise in behavior management while others may have expertise in assessment and case management.

3. Facilitate teamwork with common planning time.

A wide array of people in a variety of roles are often involved in supporting the social, emotional and behavioral needs of students. Allow them to come together weekly to review student progress and adjust support strategies.

4. Support classroom teachers with in-the-classroom support from staff skilled in behavior management.

In-the-moment coaching, in-the-classroom observations and specific recommendations from behavior specialists can help classroom teachers meet the needs of their students.

Focus on Prevention

5. Focus on prevention by identifying and managing behavioral triggers.

Identify why a student acts out and develop specific strategies for averting these triggers to prevent outbursts before they happen.

6. Increase access to staff with expertise in behavior management.

To effectively focus on prevention, schools need access to experts trained in identifying and reducing behavioral triggers. Given tight budgets, seek to hire staff with expertise in behavior management when doing replacement hiring and/or seek to build a centralized behavior team that can provide support across many schools.

7. Align discipline policies to support a commitment to prevention.

It is important that the discipline code has the flexibility to support a focus on prevention, that loss of learning time is minimized, that suspensions are avoided for nonviolent infractions and that unconscious bias is mitigated.

8. Stay focused on academic achievement.

Many “behavior programs” seem to undervalue the importance of academic learning and student achievement. Core content is often taught by special education teachers instead of subject expert teachers, and curriculum is sometimes watered down; lowered expectations can exacerbate troubling behaviors.

Support Local Partnerships

9. Seek local partnerships.

Often, local mental health agencies, nearby nonprofit counseling services, universities and sometimes even for-profit practitioners can provide social and emotional services at little or no out-of-pocket costs to students or the district.

10. Actively support local partnerships.

Local partners can provide much-needed services, so it is worth making an investment in managing and facilitating these relationships to ensure their success.

Working Together to Improve Your School’s Behavioral Climate

With social, emotional and behavioral issues posing a growing challenge for schools, and with budgets tight for the foreseeable future, schools will need a new and comprehensive approach to meet the needs of students. While neither easy nor quick, these best practices can help to better serve students. This work, however, will need leadership from the top, systems thinking, support for teachers and principals and perseverance. If parents, staff, school leadership and district leaders work and plan together, much progress can be made in addressing this difficult challenge.

Read the full District Management Journal article “10 Best Practices for Improving and Expanding Social, Emotional, and Behavioral Supports.”

1 “Children’s Mental Health: Facts for Policymakers,” National Center for Children in Poverty, November 2006, http://www.nccp.org/publications/pub_687.html

Explore by ROLE

frontline resources

MEET FRONTLINE

Surveys

Interviews

Focus Groups

Observations

A Few Recommended Practices

Next Up In the Series:

Step 1: Gather Hiring Data

Step 2: Ask the Right Questions

New Hires & Applicant Pools

The Hiring Timeline

The Hiring Experience

Step 3: Set Preliminary Goals and Actions

Step 4: Track Your Progress

1. Use Multiple Sources of Data to Build Comprehensive IEPs

2. Focus on Student Strengths, Interests and Preferences

You may enjoy this hand-picked content:

3. Increase Parent Participation in the IEP Process

4. Describe How the Student’s Disability Affects Involvement and Progress in the General Education Curriculum

5. Write Specific Statements on Academic and Functional Needs

6. Establish Baseline Data to Take a Student’s Performance Temperature

Connecting the Dots for Each IEP

“Video is a central part of my own personal growth, because you need feedback to grow. You need to see where you are, you need to see where you’re trying to get to. So it’s a key part of what I do, not just professionally but personally.”

FRONTLINE EDUCATION: Dr. Knight, you’ve written and spoken about the use of video for high-impact instruction quite a bit. We really have to start with thinking about video itself. Why is video important for educators to consider?

FRONTLINE EDUCATION: I can imagine someone saying, “I don’t think my teachers are going to be comfortable with it.”

FRONTLINE EDUCATION: Your book discusses a number of ways that video can be used to improve teaching, by teachers themselves, by coaches, by principals and by teams. If I’m a coach, I can already observe teachers in the classroom, so what else does video bring to the table?

FRONTLINE EDUCATION: As you’ve spoken with teachers who have done this, what are the kinds of things that they’ve said to you that they’ve seen or come away with?

FRONTLINE EDUCATION: What are the biggest hurdles that schools and districts face in using video to support improvements in practice?

FRONTLINE EDUCATION: Is there anything else you’d add?

How to Provide Feedback for Substitute Teachers

Regular Substitute Teacher Feedback is Crucial

What are evaluation questions?

Identifying information needs and decisions

Determining the questions

You may enjoy this hand-picked content:

Need more inspiration?

Evaluation questions lead to data collection

Next Up In the Series:

MICRO-CREDENTIALS HELP YOU FOCUS PROFESSIONAL LEARNING ON OUTCOMES

Q: What impact do micro-credentials have on professional learning?

You may enjoy this hand-picked content:

Q: How do micro-credentials meet teachers’ needs?

Q: Why should we use micro-credentials as part of our professional development strategy?

The applicant pool is relatively inexperienced.

Most states are seeing decreased enrollment in teacher prep programs — but not all.

New education graduates tend to be from large public universities in the West, or smaller private colleges in the East.

Program Description

Logic Model

Program Theory

Next Up In the Series:

A Loyalty Rate for Loyal Substitutes

Substitute Appreciation Tea

Substitute Teacher Training

So, how do we get it right? Here are a few tips:

Leverage the Talents of Current Staff

1. Streamline meetings and paperwork to increase the time staff can spend with students.

2. Allow staff to play to their strengths; assign roles based on strengths, not titles.

3. Facilitate teamwork with common planning time.

4. Support classroom teachers with in-the-classroom support from staff skilled in behavior management.

Focus on Prevention

5. Focus on prevention by identifying and managing behavioral triggers.

6. Increase access to staff with expertise in behavior management.

7. Align discipline policies to support a commitment to prevention.

8. Stay focused on academic achievement.

Support Local Partnerships

9. Seek local partnerships.

10. Actively support local partnerships.

Working Together to Improve Your School’s Behavioral Climate