Blog

3 Surprising Ways to Engage Stakeholders in Evaluating Professional Development (and Ensuring a Quality Evaluation!)

At Greece Central School District in Rochester, NY, we hired certified trainers to facilitate the 8-day Cognitive Coaching® seminar for all of our teacher leaders. We followed up with monthly collegial circle meetings for them to reflect on their learning, share how coaching sessions were going and practice scenarios with one another to refine their skills. We invited the trainers back for yearly refresher sessions. This program was designed to influence changes in teacher practice and ultimately impact student learning.

But, ask each of our teacher leaders to describe what changes in practice might look like and how coaching would impact student learning, and you’ll get almost as many answers as participants.

How do we evaluate a professional learning program if everyone has a different idea of what the program does, who it serves or what success looks like? How might we generate the right questions to ask, identify appropriate expected outcomes and determine what to measure?

Part of evaluating a program is understanding the program and what we expect it to do. And part of a successful evaluation effort is getting stakeholders — teacher participants, principals, district office administrators, Board of Education members, etc. — on board to support the work.

In my previous article, I outlined five phases of program evaluation with the first being engagement and understanding. Here, I’ll describe three evaluation-related practices:

developing a program description,
creating a logic model, and
articulating a program theory.

These can be used to engage stakeholders, build a common understanding of professional learning programs and set up for a successful program evaluation.

3 key practices in evaluating #ProfessionalLearning: 1) develop a program description, 2) create a logic model, and 3) articulate a program theory.

Click To Tweet

Why spend time on crafting these elements? There’s nothing worse in program evaluation than collecting and analyzing data only to realize that the results aren’t useful. They don’t help you answer questions, or inform decisions you have to make about the program. Let’s take a look at these three elements, and how they lay the foundation for successful program evaluation.

Program Description

Why is a program description so important to program evaluation? A program description promotes clarity and contributes to a shared and comprehensive understanding of what the program is, who it is intended to reach and what it expects to accomplish.

A thorough description also identifies why the program was developed. What is the need or problem the program addresses? It’s worth gathering a group of key people to craft a few brief paragraphs to answer these questions, even when the program has been developed by someone else.

It’s OK if your description isn’t the same as another district might come up with. For example, maybe your district held the Cognitive Coaching® seminar for administrators, not teachers, and for a different reason than my district did. Our descriptions of need, target audiences and expected outcomes will look different, even when the program itself may be delivered identically in both places.

Logic Model

A logic model is a graphic representation — a concept map of sorts — of a program’s inputs, activities, outputs and outcomes.

Inputs are what we need to run the program. What resources do we need? Funding for professional learning facilitators, curriculum materials, space to hold courses?
What activities comprise the program? Workshop sessions, follow-up coaching, action research and examination of student work are just a few of the possible professional learning formats.
Outputs are produced by the program activities – that is, the number of sessions held, the number of participants who attended, products such as action research findings, lesson or unit plans, or other curricular resources that were developed. Outputs are generally easy to report, but don’t speak to the quality or effectiveness of the program.
Outcomes describe the expected changes in program participants. With professional learning, we may expect that teachers learn and apply new content, use new resources or instructional strategies, or change their teaching practice. And of course, expected outcomes may include increases in student achievement (or other student-related metrics such as discipline or attendance) as a result of teacher learning and change in practice.

Outputs and outcomes are easily confused. Just remember that outputs are program data, and outcomes are people data! The Tearless Logic Model describes an interactive, collaborative (even fun!) process for creating a logic model that is certain to appeal to educators.

In professional learning, it’s easy to confuse outputs and outcomes. Outputs are program data. Outcomes are people data.

Click To Tweet

Program Theory

When we create professional learning programs, purchase professional learning curriculum or hire consultants to facilitate learning, we think we have high quality professional development, but how do we really know? Programs may meet certain characteristics that make them likely to be high quality (e.g., ongoing, job-embedded, data-driven). But how can we connect the dots between what the teachers are learning and how their students will benefit?

Recently, I led a collegial book study on Culturally Responsive Teaching and the Brain by Zaretta Hammond. We had teachers read the chapters and participate in online discussions. But how did we expect that teachers reading a book and writing about their thoughts would lead to improvement for students?

This is where program theory comes in. Program theory describes how the program is supposed to work. Some might call this “theory of change.” The program theory blends elements from the program description and information outlined in the logic model. Most importantly, a program theory articulates the linkages among the components of the logic model.

A key reflective question for articulating a program theory is this: What makes us think that this program, the way it is designed, and these particular program activities will lead to those expected outcomes we identified? A simple program theory for my book study might start like this:

If teachers read, reflect on and discuss with colleagues the material in Culturally Responsive Teaching and the Brain, they will deepen their understanding of culturally responsive teaching (CRT).
If teachers deepen their understanding of CRT, they will begin to think about how it connects to other work we do in the district around equity and social emotional learning.
If teachers deepen their understanding of CRT, they will also begin to recognize the cultural capital and capabilities students bring to the classroom, and will be able to use these tools to create engaging instruction.
If teachers learn about implicit bias, they will develop an awareness of how it can be a barrier to positive student-teacher relationships.

Reflection question when evaluating PD: Why do we believe that a professional learning program will lead to a given set of expected outcomes?

Click To Tweet

A few bullet points later, we might articulate how teachers will change their practice, and eventually, there will be a connection to the specific areas of student learning and achievement we want to improve.

The idea is that we’re identifying how we expect our professional learning programs to work. Once we do this, we can identify where we want to ask questions for the evaluation. Do we want to know if teachers are in fact, deepening their learning? Or do we want to investigate whether teacher learning is resulting in change in practice? A program theory helps us to know where to look and what to look for in a program evaluation.

Creating a program description, developing a logic model and articulating a program theory need not take a great deal of time. The investment, however, is sure to result in more clarity around our professional learning programs and shared understandings of how our professional learning programs are expected to produce results. They lay the foundation for us to identify relevant evaluation questions and set us up to collect the right data for our program evaluation.

Next Up In the Series:

In Part 4, we’ll learn about evaluation questions and how they point directly to the data we need to inform key decisions about professional learning.

The Truth Is… We Could Not Run Our Schools Without Our Substitutes

On any given Friday in May, my school district is struggling to fill our substitute vacancies. Many of our highly regarded substitutes are retired teachers, and by May they have worked all of the days that our state’s retirement system will allow. In addition, May brings college graduations and weddings and, well, a wide array of reasons for our teachers to be absent from work. And yet… learning must continue.

Thus, we find ourselves using creative solutions to plug those holes. A substitute may arrive at one school to step in for a third grade teacher, only to be told that we’ve covered that vacancy another way, and that he/she is instead needed at another school across town. Or, a substitute may come to a middle school teaching assignment expecting to have planning time as part of the day (definitely not guaranteed for substitutes, but a nice perk!), and will instead be told that he/she will be covering other classes during that time.

When we ask our substitutes to do more and more, I’m glad that we have a few constructs in place to thank them for their commitment to our district:

A Loyalty Rate for Loyal Substitutes

Like many districts, we have different pay rates — a daily rate, an increased rate for those who have worked a certain number of days, and a long-term rate. A couple of years ago, we also added a Loyalty Rate to honor those who have, indeed, shown loyalty to our school district.

In order to be eligible for this rate, a substitute must meet these qualifications:

Have worked in our district for at least 5 years as a substitute and/or in a full-time position.
Have worked on at least 90 days as a substitute in our school district in the previous year.

The Loyalty Rate resets each fall, and thus each spring we review our data to identify which substitutes qualify for the rate in the next school year. Full disclosure: the Loyalty Rate is only $3.00 per day more than the rate used for those who have worked at least 60 days. However, we know that our substitutes may also work in surrounding districts, and we created this rate to both encourage them to keep our district at the top of their list, and to acknowledge their consistent work and longevity with our district.

Substitute Appreciation Tea

Each June, we host a Substitute Appreciation Tea. We schedule it during the window between the school year ending and summer school starting so that as many substitutes as possible might be available. We work with our food service provider to create a real tea, complete with finger sandwiches, pastries, lemonade, and of course, tea!

The agenda is simple:

Very brief speeches from a couple of grateful administrators.
Time to mingle (many of our substitutes have known each other for years, and love the chance to catch up).
An opportunity for substitutes to provide feedback to us via a brief form which they complete in small groups.

Our Tea gives us the chance to let our substitutes know how much we really appreciate them while also getting feedback from them in a positive setting.

A few photos from our last Substitute Appreciation Tea:

Substitute Teacher Training

We are proud of the training that we provide to our substitutes and believe that putting time and effort into offering our substitutes training shows them that we appreciate and value the work that they do for us. We make it very clear to them that every day counts for our students, that gone are the days when a teacher might have the substitute put on a movie for the students. Our substitutes need to be ready to teach new skills and reinforce what has already been taught, and thus our school district helps them to hone their craft.

All of our substitutes are required to attend a half-day training prior to working for us. Even if they have worked in this role in other places, we want to make sure that they have full awareness of the expectations in our district. We try to keep our training interactive and fun while we provide important information.

During this training, we cover:

HR basics such as securing assignments and understanding pay structure
Information about our district
Crisis plans and health training (such as practice with EPI Pens)
Classroom management expectations and tips

In addition, we offer our substitutes optional training opportunities throughout the year. We take advantage of times when we know our substitutes would not be working for us, such as after school or during School Improvement Days when students are not in attendance, and then provide in-services run by our specialists. For example, training about working with students who receive special education services was provided by our Director of Student Services, and updated information about math instruction was given by our Math Coordinator. In this way, we help our substitutes to remain current in their teaching practices, and we honor their work as educators who are important to our system.

Of course, the best appreciation comes at the school level. I will never forget the joy and pride I felt when teachers told me that they had created goodie bags for all of the substitutes who were present at their school on a particular day. I also regularly remind our principals and building secretaries to greet and thank substitutes, and to treat them as the professionals that they are.

The truth is, we could not run our schools without our substitutes. It seems only right that we let them know that.

This post also appears on the author’s blog.

8 Tips on Listening for the Truth in Reference Calls

The reference call – it’s often the final check in a lengthy process of searching for the best educator to hire for the school district. These calls can either confirm the hiring of a great candidate – or prevent a terrible decision. However, to make the most of reference calls, administrators and hiring managers have to know how to listen for the truth.

Lynn Glickman, the Assistant Superintendent for Human Resources at Community Consolidated School District 21 in Wheeling, Illinois, has learned a few things in her years of hiring and both making and participating in reference calls. Get her eight tips for listening for the truth.

This post originally appeared on Dr. Lynn Glickman’s blog: Reference Calls — Listen for the Truth (8 Tips).

I was a newer principal, and I had recently dismissed a pre-tenured teacher. She wasn’t terrible, just not good enough for my students. I had written her a reference letter, the kind you write for just this type of situation — strengths, as everyone has them, with lots of holes that I hoped other interviewing administrators would find. Then I took a call from a principal in a nearby district. We knew each other by association, but that’s all. He was eager to hire her, and this call was his final stop. I answered his questions and thought to myself, “I hope he listened carefully, and realized that I wasn’t really recommending her.”

Years later, this principal and I landed together in a new district. Early in our professional relationship, he told me, “You know, I had to let that teacher go after one year. I couldn’t believe you had recommended her to me — I was always so annoyed about that!”

“What?” I said. “I didn’t recommend her! I was trying to tell you not to hire her!” It was a perfect example of what happens over and over again. He was listening only to confirm his decision to hire her, and I was being positive, hoping he could read my mind.

I share this story often in my current role as Assistant Superintendent for Human Resources. Administrators (and hiring executives in other organizations, I am sure) work diligently to find the very best person for their opening. They carefully comb through applications and resumes, conduct screening interviews, 1:1 interviews and often team interviews. By the time they get to the reference checks — well, that’s what they are often doing — checking a box. This is the last step in the process of finding the Perfect Person. They are already sold on the candidate, and if they brought in a team to interview, so is the team! Who wants bad news at this point? And so, like my principal friend from many years ago, they miss what a reference might be trying to convey in an oh-so-indirect way.

Hiring is one of our most important administrative tasks, particularly when hiring teachers and principals. This is the time when we really need to get it right, because getting it wrong can make for a very, very bad year. (And depending on a whole bunch of variables, one mistake can be prolonged for much longer than a year.)

So, how do we get it right? Here are a few tips:

Before starting to make reference calls, promise yourself that you will be open to hearing bad news. Honestly, that’s what is most important. In fact, when I interview with a team, I always talk about how important reference calls are, and let team members know that sometimes we have to go another way after we’ve made those calls.

Experienced interview teams know that the person who came to the interview is not always the person who “shows up” at work, even if it is the same person! When I remind my team of that, they are understanding if we end up going with our #2 choice, or even if we have to interview more candidates.
Try to establish rapport with your reference at the start of the call. Yes, everyone is busy, but find a way to connect with the reference as a person. It will be harder for your reference to evade your questions or straight out lie to you if he/she likes you on the phone — even a little bit!
Do you have flexibility with your questions? If so, make sure to ask, “Would you re-hire this person?” Or, if you are speaking with a colleague, you may get useful information asking, “Why do you think this person would be interested in coming to a new school/district/organization?” If the first answer you get seems bogus, find a way to dig a little.
Even if you have to use a standard set of questions, surely you can ask follow-ups. When I sense that someone is choosing words very carefully, or is trying to brush past a question with a pat answer, I may ask, “Can you tell me more about that?” Often, they will.
Pay attention to the reference’s tone of voice and listen for pauses. Your reference may be trying to figure out how to answer a question in a way that is semi-truthful without hurting the person’s chances for getting the job. This is the perfect place for those follow-up questions.
What about those references who ask you to simply email them the questions, and they’ll email you the answers? Yes, they might just be really busy, but they might be trying to control the situation by carefully wording their answers.

You can’t see pauses in an email, you can’t hear tone of voice and asking follow-up questions is then even more time-consuming.

I try very hard to get people to talk to me on the phone. When they won’t, I don’t necessarily decide that there is a problem with the candidate, but I don’t put much stock in that particular reference.
Think carefully about who you are calling. For the most part, avoid calling your candidates’ colleagues, or only do so if you are also calling a supervisor or two.

A teacher’s teammate is rarely going to tell you that his/her colleague had classroom management problems or didn’t communicate well with parents, and an administrator’s lateral coworker is unlikely to tell you that projects were not finished on time.

I won’t hire anyone without speaking with his/her current supervisor, although certainly I will agree to make that call last, after I’m satisfied with the results of other calls. Also, you may be able to glean quite a lot of information from a supervisor from 2-3 years ago. Sometimes, the supervisor won’t remember much — but sometimes he/she will remember a lot, and will be ready to talk when no longer in regular contact with the candidate.
Finally, use your resources. The best reference call is with someone you know. Even if you don’t know any of your candidate’s references, someone else who is part of the hiring process might! If so, hand that call over to your colleague to get the most truthful information.

I still enjoy a friendship with that principal colleague who hired that marginal teacher. We still talk about this. We both learned. We are both still learning.

10 Best Practices for Improving and Expanding Social, Emotional and Behavioral Supports

While there is much debate about why an increasing number of children come to school with significant social, emotional and behavioral (S, E & B) needs, nearly all districts report the number of children with these challenges is on the rise.

In order for students to meet developmental milestones, learn, grow and lead productive lives, it is critical that their social, emotional and behavioral issues be addressed. Research indicates that children and youth with mental health problems have lower educational achievement and greater involvement with the criminal justice system [1]. Improving and expanding S, E & B supports not only helps the students who have these challenges but can benefit nearly every student and adult in a school.

75% to 80% of children and youth in need of mental health services do not receive them.

Click To Tweet

All schools — urban, suburban and rural; large and small; and regardless of socioeconomics — have students with social, emotional and behavioral challenges. However, in some of these communities, students receive the counseling they need, classroom routines promote positive behavior, and most strikingly, students with problematic behavior are able to stay in class and seldom disrupt their peers. What is the difference between these schools and typical schools? The distinctions can be hard to notice because the difference isn’t in the amount they spend, the programs they bought, or the dedication of their staff. The people, tools and talents themselves aren’t all that different but the way in which staff work and deliver intervention is different — the more effective districts have created a coherent, collaborative plan grounded in a systems-thinking approach and incorporating best practices.

Here, we focus on 10 key, interconnected best practices to help you and your team effectively and comprehensively create a system to meet the social, emotional and behavioral needs of students. These practices fall into three major categories: Leveraging the talents of current staff, focusing on prevention and supporting local partnerships.

Leverage the Talents of Current Staff

1. Streamline meetings and paperwork to increase the time staff can spend with students.

Process mapping, reviewing who attends which meetings and setting guidelines for desired time with students can often significantly increase the services provided to students by current staff.

2. Allow staff to play to their strengths; assign roles based on strengths, not titles.

Identify staff’s unique skills and match job responsibilities to these areas of expertise. For example, some psychologists may have expertise in behavior management while others may have expertise in assessment and case management.

3. Facilitate teamwork with common planning time.

A wide array of people in a variety of roles are often involved in supporting the social, emotional and behavioral needs of students. Allow them to come together weekly to review student progress and adjust support strategies.

4. Support classroom teachers with in-the-classroom support from staff skilled in behavior management.

In-the-moment coaching, in-the-classroom observations and specific recommendations from behavior specialists can help classroom teachers meet the needs of their students.

Focus on Prevention

5. Focus on prevention by identifying and managing behavioral triggers.

Identify why a student acts out and develop specific strategies for averting these triggers to prevent outbursts before they happen.

6. Increase access to staff with expertise in behavior management.

To effectively focus on prevention, schools need access to experts trained in identifying and reducing behavioral triggers. Given tight budgets, seek to hire staff with expertise in behavior management when doing replacement hiring and/or seek to build a centralized behavior team that can provide support across many schools.

7. Align discipline policies to support a commitment to prevention.

It is important that the discipline code has the flexibility to support a focus on prevention, that loss of learning time is minimized, that suspensions are avoided for nonviolent infractions and that unconscious bias is mitigated.

8. Stay focused on academic achievement.

Many “behavior programs” seem to undervalue the importance of academic learning and student achievement. Core content is often taught by special education teachers instead of subject expert teachers, and curriculum is sometimes watered down; lowered expectations can exacerbate troubling behaviors.

Support Local Partnerships

9. Seek local partnerships.

Often, local mental health agencies, nearby nonprofit counseling services, universities and sometimes even for-profit practitioners can provide social and emotional services at little or no out-of-pocket costs to students or the district.

10. Actively support local partnerships.

Local partners can provide much-needed services, so it is worth making an investment in managing and facilitating these relationships to ensure their success.

Working Together to Improve Your School’s Behavioral Climate

With social, emotional and behavioral issues posing a growing challenge for schools, and with budgets tight for the foreseeable future, schools will need a new and comprehensive approach to meet the needs of students. While neither easy nor quick, these best practices can help to better serve students. This work, however, will need leadership from the top, systems thinking, support for teachers and principals and perseverance. If parents, staff, school leadership and district leaders work and plan together, much progress can be made in addressing this difficult challenge.

Read the full District Management Journal article “10 Best Practices for Improving and Expanding Social, Emotional, and Behavioral Supports.”

1 “Children’s Mental Health: Facts for Policymakers,” National Center for Children in Poverty, November 2006, http://www.nccp.org/publications/pub_687.html

What’s Involved in Evaluating a Professional Development Program?

Today’s article, focused on the five phases of program evaluation, is Part 2 of a seven-part series on evaluating professional development. In the rest of the series, you’ll learn how you can apply these phases to evaluate your own programs.

What is Program Evaluation?

In Part 1, we learned that program evaluation is applying systematic methods to collect, analyze, interpret and communicate data about a program to understand its design, implementation, outcomes, or impacts. In the field, evaluators talk about evaluation as a way to determine the merit, worth and significance of a program. Simply put, it’s about understanding the quality of our professional learning programs, how valuable they are to our schools or districts and how important they are to what we are trying to accomplish with teachers and students.

PD program evaluation: using data to understand the quality of our programs, value to our schools and importance to our goals.

Click To Tweet

Program Evaluation Has 5 Key Phases

Evaluating a program consists of five interdependent phases that don’t necessarily occur in a perfectly linear fashion.

Phase 1: Engagement and Understanding

Program evaluation is most successful when stakeholders are involved and work collaboratively. Who should be involved? Administrators and teachers connected to the program, along with instructors and participants, will likely be called upon to offer or collect data, and should be included in planning the evaluation.

Think of a professional learning program in your school or district. Is there a clearly articulated description of the program? Are there stated and measurable outcomes? Does everyone involved with the program know what participants are expected to learn, how they might change their practice, and what student outcomes are expected as a result? Does everyone agree on these? Don’t worry! It’s quite common to answer “no” to one or all of these questions.

In a future post, you will learn about the importance of program descriptions and logic models. I’ll share how these tools can be easily created to promote shared understanding of our professional learning programs and how this sets us up to conduct high quality evaluation.

Phase 2: Questions

Developing evaluation questions is foundational to effective program evaluation. Evaluation questions form the basis for the evaluation plan and drive data collection.

Conducting evaluation is much like conducting a research study. Every research study starts with one or a few broad questions to investigate. These questions inform how and from whom we collect data. The following are examples of the types of questions we might pursue in evaluating our professional learning programs:

To what extent is the program changing teacher practice?
What evidence do we have (if any) of student learning that may be attributable to the program?
How might the program be improved?

Phase 3: Data Collection

We collect data on professional learning programs to answer our evaluation questions, and all decisions about data collection strategies to use rest squarely on these.

Most people are familiar with surveys (also called questionnaires; check out my book on designing surveys), interviews, or focus group interviews, but data collection can go far beyond asking people questions.
Observations of a professional learning program in action can yield important insights into how the program is going and whether or not it appears to be on track to achieving its objectives.
Classroom observations can help us understand if and how well teachers are implementing new practices, whether there are barriers to implementation, and what might be getting in the way.
Teachers journaling about their learning, or creating lesson plans or other artifacts, can also demonstrate whether a professional learning program is working well.
And of course, student data — achievement, attendance, discipline, work samples, etc. — can also serve to help answer the evaluation questions.

Later on in this series we’ll offer a more in-depth look at the advantages and disadvantages of specific data collection strategies, along with ideas for exploring more innovative data sources.

Phase 4: Data Analysis

This is the phase that scares people the most. People often think they need to understand statistics or have advanced spreadsheet skills to do data analysis. They worry when their datasets aren’t perfect or whether they have collected data in a “scientific” enough way. They are concerned about whether their data is reliable and valid, especially if it is qualitative and perceptual data, such as answers to open-ended questions from surveys or interviews.

You don’t need a Ph.D. in statistics to understand and analyze data around your professional learning program.

Click To Tweet

These concerns are understandable, but in truth, there’s no need to get worked up. In a future post, we will put to rest all of these fears!

Given the types of data we use to evaluate professional learning programs, we rarely need statistics beyond simple frequencies and averages. And datasets are seldom perfect. When we conduct surveys, for example, we find that some people don’t answer some questions. Others misinterpret questions, or it’s clear they make mistakes answering them.

On one feedback form after a very well-received workshop, a participant checked “Strongly disagree” for every statement when it was clear that “strongly agree” was the intended answer. How did I know this? Between the statements were comment boxes filled with glowing praise about how much the participant enjoyed the workshop, valued the materials and loved the instructor. It was clear the person mistook “Strongly disagree” for “Strongly agree” based on the location of those responses on the sheet.

Phase 5: Reporting and Use

Evaluation should be conducted with an emphasis on use. Once we interpret the data and glean insights that can inform future decisions about a program, we need to consider how to report new learning to key stakeholders. The formula for effective reporting includes:

identifying appropriate audiences for evaluation reports,
understanding their information needs, and
knowing how they consume information.

Are you reporting to a Board of Education? A superintendent? A group of administrators and teachers? Do they need all the details, just a few key data points, or a brief summary of results? Knowing our audience and how to engage them informs how we create reports, and reports can come in a wide variety of formats. Here are just a few examples:

Presentations
Documents
Infographics
Podcasts
Webpages

Evaluation as an Iterative Process

Earlier, I mentioned that these phases aren’t necessarily linear. In the graphic, you see them as a cycle where Reporting and Use points back to Engagement and Understanding. As we complete an evaluation for a program and make decisions about its future, we may enter another evaluation cycle. Also, as we collect data, we may analyze and report on it even as the evaluation work continues, thus revisiting Phases 3, 4 and 5 multiple times in one evaluation cycle.

Next up in the series

In Part 3, we go deep into ensuring that everyone has a shared understanding of how our professional learning programs are designed to influence change in teaching practice and student learning.

Professional Development for Principals

How Franklin Public Schools Used In-District Expertise to Promote Effective Observations

When you talk about employee evaluations in K-12, the first thing most people think is “teachers.” That’s true at Franklin Public Schools in Wisconsin, too. But as they work to build the capacity of teachers in the district, they also place a strong emphasis on developing principals — especially on equipping them to be strong instructional leaders.

This spring, we spoke with Christopher Reuter, Director of Teaching & Learning, and Erin King, principal at Forest Park Middle School, to find out what that looks like.

Here’s what Chris said about the role of the principal:

“I think the principalship, at least in the state of Wisconsin right now, is the hardest job in public education. You’re pulled in every direction. You open up any newspaper, go to any website or social media, and you see the many things that happen daily in our schools. It’s the principal who has to carry that load, as well as connect the ever-moving parts together to make sure that everyone’s moving forward.”
Christopher Reuter, Director of Teaching & Learning, Forest Park Middle School

It’s the principal who has to connect the ever-moving parts together to make sure everyone’s moving forward. – @chris_reuter

Click To Tweet

Erin has put a lot of work and thought into the kinds of feedback she provides her teachers. For starters, she said it needs to be provided soon after the observation:

“I had a big ‘aha’ moment during my own observation cycle that I wasn’t providing [teachers] timely feedback. I was trying to have everything look perfect, have it all done, tied up in a bow with my feedback entered into it, and I was waiting too long to have those conversations with teachers and focused too much on that one observation period versus instructional practice overall.”
Erin King, Principal, Forest Park Middle School

How does she make sure those are more than just one-way conversations? How does she build trust?

“I ask [teachers] for feedback, and I’m very honest. I put my cards on the table, and my staff has seen that who I am is who I am every day. They see that honesty, and that transparency and vulnerability….They see that I am learning and growing as well, which has been part of the feedback teachers have given me: that they really appreciate that they see that I’m learning, and that I’m growing, and that I’m dedicated to that process, which is very similar to the process that they’re going through.”
Erin King, Principal, Forest Park Middle School

Chris asked Erin if she’d be willing to conduct a post-observation conversation with a teacher in a fishbowl setting for other principals to observe.

“As administrators we don’t see each other engage in post-observation conversations, so we assume that we’re probably all doing it in the same way. I was told that I’m not necessarily doing it in the same way as others. Chris approached me to see if I’d be willing to model a post-observation conversation in front of other administrators.”

I approached a teacher that is a real growth-mindsetted professional and asked her if she would be willing to put herself out there, be vulnerable with me and model a post-observation conversation. [The day after her classroom observation] we engaged in the post-observation in front of all the other principals, directors and superintendent within the district.

We were in the middle of the room with tables around us in a U-shape, but I think that that the most important thing was that I had a document camera displaying my notes from our pre-observation conversation, and then the notes that I took along the way….The other principals were asking, ‘How did you circle to this feedback? Did you know what you wanted her to learn and where it was going to end?’”
Erin King, Principal, Forest Park Middle School

“As administrators we don’t see each other engage in post-observation conversations, so we assume that we’re probably all doing it in the same way.” – @erinkking

Click To Tweet

Since then, Chris and the other directors at the district have continued to observe principals as they conduct these conversations.

“It’s caught on. Look at the work that Erin has done over the last year, and how that has developed her and the feedback she’s received. I and two other directors have all observed at least two or three principals in the venue and provided feedback.

We as a director team meet every Monday, and one of our standing agenda items is to calibrate around our observation of principals and share what we saw, talking about the post-observation conversation. We’re trying to model and do what Erin does with her teachers, with our principals as well — whether it be one-on-one or in small groups.”
Christopher Reuter, Director of Teaching & Learning, Forest Park Middle School

Chris and Erin both emphasized that these open conversations, rooted in trust between principals and teachers, or directors and principals, are vital to their growth efforts — and ultimately, to student achievement.

You can listen to the entire interview above, or — better yet! — subscribe to Field Trip and get new episodes every other Friday.

You may enjoy this hand-picked content:

White Paper: “Feedback That Matters: Using formative feedback and meaningful conversations to grow teaching practice and foster a collaborative educational environment.”

RTI/MTSS and End of School Year: 7 Tips to Reflect and Recalibrate

School leaders who implement RTI/MTSS have a big responsibility ― to deploy a school’s full array of intervention resources to find and help struggling students. To meet this goal, periodic checkups are needed to ensure that schools align their current practices with RTI/MTSS best practices. The close of the school year offers staff the ideal time for an RTI/MTSS checkup ― now is your chance to tidy up loose ends in record-keeping, use data to improve classroom instruction, identify gaps between intended and actual service delivery and look ahead to the next phase in RTI/MTSS program roll-out.

As summer approaches, here are 7 steps to firm up your procedures, ensure they are carried out with integrity and prepare staff for the 2018-19 school year.

Steps to Making the Most of Spring RTI/MTSS Data

1. Archive RTI/MTSS information

Schools should remind all staff responsible for keeping track of RTI/MTSS information that they should complete their records for the current school year before summer break. Staff should also be given a deadline date for finishing record entries. Having a district or school-wide RTI/MTSS program management system that all stakeholders can access helps keep data organized and archived for future use. Once that deadline is past, school staff can spot-check student entries in the RTI/MTSS system to verify that records are indeed complete.

2. Evaluate Effectiveness of Core Instruction

RTI/MTSS schools typically collect building-wide academic screening data at fall, winter and spring checkpoints. These data-sets are invaluable, as they allow a school to judge the effectiveness of its core instruction and, if needed, provide guidance to teachers on strengthening their instructional practices.

It is a widely accepted rule of thumb that classroom instruction across a school can be considered adequate if at least 80% of students meet or exceed a screener’s performance cut-points. The close of the school year is an ideal time for administrators to meet with grade-level teams to review screening data and brainstorm future instructional ideas to boost students’ collective academic performance.

For real-world examples of how to use this best practice, download the free eBook RTI/MTSS and End of School Year.

3. Analyze Data to Uncover Performance ‘Pockets’

As schools build a strong RTI/MTSS model, they collect troves of data monitoring student performance. If this data is reliably archived, school leadership can analyze it to identify pockets of student performance that either exceed or lag behind expectations. For example, a school might compare the relative outcomes of two Tier 2 reading groups using the same program to see if there are significant differences across instructors.

This type of advanced RTI/MTSS ‘data mining’ requires that a school first standardize its procedures to ensure that data sources are valid and reliable and that student data is uniformly stored in electronic format for easy retrieval.

4. ‘Recalibrate’ RTI/MTSS Procedures

Every school that follows an RTI/MTSS model has its own procedures to identify students for services, document intervention plans, collect data, move students up and down the tiers of intervention and so on. The end of the school year is the perfect time to review the school’s actual RTI/MTSS practices, identify any gaps in implementation and ‘recalibrate’ to align those day-to-day practices with the expected RTI/MTSS procedures.

Data can help school staff uncover discrepancies in procedures. It is an expectation, for example, that in a ‘typical’ school, 1-5% of students might be referred to the Tier 3 RTI/MTSS Problem-Solving Team in a given school year. If, as summer approaches, fully 10% of a school’s students have been brought to Tier 3 during the current year, the school can follow up by reexamining its criteria for accepting a Tier 3 referral and how those criteria are being enforced by staff.

For a real-world example of how to use this best practice for Tier 1, download the free eBook RTI/MTSS and End of School Year.

5. Recruit Fall Groups Using End-of-Year Screeners

To identify students at academic risk, most schools screen the entire building population 3 times per year (fall/winter/spring). Those data are then used to recruit students whose risk profile indicates they require Tier 2/Tier 3 academic-intervention services. While fall screening data would seem to be the logical data source to recruit fall academic-intervention groups, it presents 2 limitations:

Tier 2/3 interventionists cannot begin work with students until the school has conducted the fall screening and identified groups, resulting in several weeks of dead-time when at-risk learners are not receiving intervention services.
In an effort to speed formation of fall intervention groups, the school may be tempted to screen immediately after the start of school. However, students often experience a ‘summer slide’ — a predictable and temporary drop in reading or math skills over the summer. For most students, the summer-slide effect disappears after 4-5 weeks of school. Therefore, schools that screen within the first 2-3 weeks of school are likely to ‘lock in’ temporary academic deficits and falsely identify at least some students for Tier 2/3 services whose skills would have rebounded on their own.

A solution is to use the end-of-year (spring) academic screening results for 2 purposes: (1) to enter or exit students for current spring Tier 2/3 services and also (2) to identify fall Tier 2/3 intervention groups before the summer break. This approach allows academic-intervention groups to meet immediately when school resumes in the fall and encourages the school to schedule the fall screening when student skills have fully recovered from the summer regression.

6. Update the RTI/MTSS Roll-out Plan

It can take 3 to 5 years to fully implement the RTI/MTSS academic model. Schools in the midst of rolling out RTI/MTSS will find that the final months of the current school year offer a good vantage point from which to firm up plans for the next phase of implementation slated to start in the fall.

While advanced RTI/MTSS planning is always a good idea, some elements of RTI/MTSS require it. Schools seeking to overhaul their system of Tier 2 (supplemental/small-group) interventions, for instance, may need to alter multiple elements, e.g. changing the schedule for those services and training Tier 2 providers to deliver new research-based intervention programs.

For a real-world example of how to update your RTI/MTSS roll-out plan, download the free eBook RTI/MTSS and End of School Year.

7. Prepare RTI/MTSS Professional Development

While schools often do a good job of outlining and implementing a comprehensive RTI/MTSS plan, they sometimes overlook the need to provide ongoing professional development to prepare their staff to understand, accept and work effectively within the plan. As school leaders use the close of the school year to reflect on the quality of RTI/MTSS implementation and proposed next steps, they should also consider what additional training teachers and support staff require to help improve delivery of RTI/MTSS services. This professional development plan should include both the essential RTI/MTSS content to be delivered to teachers and a training calendar extending into the coming school year with opportunities in large- and small-group settings to provide that professional development.

Read examples of RTI/MTSS professional development planning.

A Final Thought on Optimizing RTI/MTSS at End of Year

A key quality for success in implementing an RTI/MTSS model is simply that schools pay attention to the details, verify that records are complete and archived, close gaps between current and best practices and look forward to the next steps in the unfolding RTI/MTSS roll-out plan. The end of the school year is a strategic time for schools to focus their attention — make productive use of this pivotal moment between the recently elapsed and coming school years!

As you reflect on your RTI/MTSS program this spring, consider how Frontline RTI & MTSS Program Management can help you collect and analyze the data you’ll need to make next year even better for staff and students.

How School Districts Can Provide A Great Applicant Experience

Would you want to go through your school district’s hiring process?

A poor applicant experience is a problem even in the best of times. But with the teacher shortage set to worsen as fewer new educators enter the profession, it’s more important than ever that candidates have a delightful experience from the moment they apply to your district.

After all, an exceptional applicant experience means three things for your district:

Fewer barriers to application means a larger applicant pool
Candidates with a good experience are more likely to encourage other educators to apply
New hires are more engaged and prepared to succeed from day one

So, what can you do to ensure that your hiring process is a positive experience?

1. Walk a mile in the job-seeker’s shoes

Look at your hiring process from the perspective of the job-seeker at every stage. Try to find where applicants might hit snags, or what steps might be unnecessarily time-consuming. Remember that great teachers may already be working full-time in another district and may not have hours to spend on your application process.

Are open positions easy to find online, or are they hidden away on your district website? Is it clear to applicants which materials they need to submit, and how to submit them? Do they have to send printed materials through the mail, or can everything required be uploaded electronically? Make sure to look beyond the application itself. Does it take several rounds of phone tag to schedule interviews? Will references be contacted several times by different people from the same district? An applicant tracking system can help streamline the hiring process for both you and applicants, making it a more pleasant experience all around.

2. Communicate

According to Forbes, over 70 percent of online applicants never receive even a generic reply from would-be employers. Look for ways that you can be more communicative with applicants from the beginning, even if you don’t have time to write to each applicant individually. If you’re one of the few school districts that makes a point to acknowledge each and every teacher application — even if it’s an automated form response that isn’t personalized in any way — you’re already ahead of the game. It’s okay to have a template response, especially if you can make it both informative and interesting. It’ll set your school district apart and put the “Human” back in “Human Resources.” Plus, it’s just good etiquette.

Beyond the initial application, the more transparent you can be with job-seekers, the better. You may have a qualified, talented teacher candidate who is a great cultural fit, who never knows they’re one of your top picks. If they don’t hear anything from you for weeks while you work through bureaucratic internal processes, they may assume they won’t be hired and accept another district’s offer — even if working for your school district was their dream job.

3. Set reasonable timeframes

Similarly, keep in mind that teachers need to have a plan. If a great teacher applies for a position in early May, but you aren’t able to make an offer until the end of August, that’s too late! It’s not fair to expect an exceptional candidate to wait months to receive an offer from your district, especially if other, faster districts have already made their hiring decisions. Like anyone else, teachers need to know where they will be working and what they will be doing ahead of time. This is especially true if they would need to relocate from another area in order to work in your school.

Be upfront about how long applicants can expect to wait to hear back. Hiring teachers will always have a degree of uncertainty — current teachers may decide not to return for the next year, or funding may not come through — try to stick to the promised timeline as much as possible. If something does come up, make sure to communicate with candidates so they know where they stand.

4. Seek feedback

You don’t need a “secret shopper” to get the inside scoop on your applicants’ experience with your hiring process. Gather feedback from both new employees and candidates who were not hired on an ongoing basis, so you can continually improve your hiring process. Remember: this isn’t a “once and done” thing: regularly implementing changes based on honest feedback will help your hiring process evolve.

When Cabot Public Schools decided to move to a new applicant tracking system, providing a great applicant experience was one of their top priorities. That’s why they chose Frontline Recruiting & Hiring.

Read the Case Study

Your Section 504 Eligibility Questions Answered

Understanding Section 504’s eligibility criteria is crucial to compliant Section 504 accommodation delivery and implementation. Unlike special education, Section 504 does not expand rights or change the educational experience. Rather, Section 504 protects the general education experience and ensures that students are not discriminated against on account of disability.

Due to the lack of federal and state regulations regarding Section 504, determining eligibility can be a daunting process involving logistical and legal questions. As an education law attorney representing schools, I see these as among the most common questions educators face.

What is the Section 504 eligibility criteria? How should it be used?

The Section 504 eligibility criteria involves two questions:

Does a student present with a physical and/or mental impairment?
If so, does that physical and/or mental impairment substantially limit one or more major life activities?

How is the Criteria Defined?

Let’s define those terms:

“Physical impairment” ― means a diagnosis affecting one or more of the physical systems, such as the neurological, musculoskeletal, special sense organs, respiratory, cardiovascular, etc. Section 504’s broad protections cover all of the body systems. Virtually any diagnosis, affecting any system, constitutes physical impairment.

“Mental impairment” ― means a diagnosis involving virtually any mental disorder listed in the DSM-5, including anxiety, cognitive impairment, brain syndrome, dystonia, oppositional defiant disorder, obsessive compulsive disorder, attentional difficulties or somatoform disorder.

Given the broad nature of these definitions and how many different diagnoses committees may face, the diagnosis itself typically isn’t the tough issue in understanding eligibility. More importantly, committees need to identify data that shows student’s need.

504 criteria requires a showing of impact. What is the need that requires accommodation? https://bit.ly/2FHlH0N

CLICK TO TWEET THIS

What are “Major Life Activities?”

After establishing a diagnosis that affects a body system or the mind, Section 504 committees need to identify the impact of that difficulty on a major life activity. Major life activities are construed broadly also, involving everyday actions like walking, talking, seeing, breathing, hearing, caring for oneself, working, eating, processing, learning – any activity one engages in regularly.

Is Section 504 a “Consolation Prize?”

Applying a real understanding of the eligibility criteria safeguards against the “consolation prize” phenomenon – offering Section 504 accommodations instead of special education. Unlike special education, Section 504 doesn’t provide clear rules or regulations that define its decision makers or the decision-making, eligibility process, itself. In the absence of such rules, getting Section 504 right means understanding and applying the right eligibility criteria, and not simply issuing plans to all former special education students.

Are Any Students “Presumptively Eligible” for 504 Plans?

There is no presumptive eligibility under 504 – simply presenting a diagnosis does not “get you” a plan. In order to affect a compliant process, schools need to consistently implement the right eligibility criteria.

Educators need to recognize that the eligibility process requires consideration of the impact of disability. That means, in order to be eligible for accommodation under a Section 504 plan, students must show symptoms.

When determining Section 504 eligibility in a school, should we only look at activities that impact “learning?”

Section 504 eligibility is broad and involves consideration of all major life activities, not just “learning.” As we continue to discuss, Section 504 protects against disability discrimination so that all students, both disabled and non-disabled, may access the same education. The major life activities that may be considered through the criteria are broad. Learning is certainly involved, but so is walking up the steps to get on a school bus, sitting on that school bus, developing appropriate peer relations so the student may ride that bus successfully to school, climbing off that school bus and being able to walk down the sidewalk to get into the school, and walking through the hallways to the classroom where the student will engage in learning.

To Sum Things Up.

So, where are we? Let’s remember the eligibility criteria for Section 504 is different than the eligibility criteria for IDEA. Remember there are criteria. Remember the eligibility criteria has two primary questions, involving physical and/or mental difficulties. And such physical and/or mental impairment must substantially limit one or more major life activity. Without data that satisfies the eligibility criteria, you should not find students eligible, or provide plans. However, at all times, and with all students, we may never treat any students differently.

Do you have the data you need to make legally sound Section 504 eligibility determinations? Consider how Frontline 504 Program Management can help you efficiently collect, use and securely archive your student data.

Measuring the Impact of Professional Learning

Investing in professional learning for educators comes with the expectation that you’ll be able to evaluate the gains from that investment. But all too often, rolling out a learning plan without an evaluation method means that the second part never happens.

That’s why the Frontline Research & Learning Institute, along with Learning Forward, worked with six districts to examine the best way to measure the impact of professional learning.

Those districts were:

Boston Public Schools, Massachusetts
Greece Central School District, New York
Jenks Public Schools, Oklahoma
Metro Nashville Public Schools, Tennessee
Prior Lake Savage Area Schools, Minnesota
Shaker Heights City Schools, Ohio

Each district brought an existing professional learning program to this small-scale study with the intention of collaboratively determining which changes to the programs would most likely benefit educators at their school. Their five essential findings might surprise you.

1. Plan evaluation as a holistic part of the program

To evaluate effectiveness of professional learning programs on both educators and their students, the program needs to be designed with clear targeted, measurable outcomes and indicators of success. In wrestling with this fact, the districts fell into three buckets of program development:

Existing programs with evaluable outcomes in place.
Existing programs that required reworking for more clarity of outcomes.
In-development programs that didn’t yet have targeted outcomes.

Similar to how educators develop units of learning, planning their final assessments first, these district leaders found it was necessary to do that same. While a “one-off” professional learning event may seem like a good idea, the bigger question is: how does it all fit together?

2. Develop targeted outcomes with existing data sources in mind

In developing their target outcomes, Boston Public School leaders looked to the multiple data sources they already had at their fingertips in the district. They considered which might be useful as indicators of the impact of their new professional learning program. Repurposing data sources in this way can help you more easily and quickly evaluate the program without creating a new data burden (Killion, 2018).

The down side: data sources already in place are approximations of measures of the targeted outcomes of the professional learning program. So, district leaders worked to analyze and interpret the results, then form conclusions about the impact of the program.

Weigh how important exacting data tailor-made for your program is vs. the functional ease of using existing data sources. Consider a combination of the two, if necessary.

3. Consider a systems-approach to better evaluate effectiveness. from its inception.

When professional learning within a district or school lacks coherence — that is, when each event or initiative feels ad hoc or separate — it’s pretty difficult to measure its effects. That’s why Jenks Public Schools took the opportunity to revise their program using a systems-approach. They reworked the planning process to ensure that professional learning met their criteria for quality

Using this planning model, the district and school leaders aligned professional learning with identified needs, provided adequate implementation support, and monitored implementation to increase the likelihood of results.

4. Continually evaluate both new and existing programs

The reasons for evaluating a new program are obvious:

Determine if it’s worth the investment to continue into a second year
See how to refine the program
Incorporate feedback from participants

But what about after the first year or so, when you feel it’s going well?

The districts in the study found it helpful to continue to evaluate existing programs in the following ways:

Run annual data collection from multiple sources about the program to inform continuous upgrades, even after it’s been refined for a year or so.
Go a step further to measure the impact of the program on student achievement, connecting the dots between program outcomes and changes in student learning.

Metro Nashville Schools collected data using the Collaborative Inquiry Process in partnership with REL Appalachia, a system for collecting, analyzing, and using a variety of data to improve programs.

These modes of evaluation helped the districts stay freshly engaged with programs, even if they had been running for more than a year or so. Continuous data collection meant they could go a step further in guiding teachers who were implementing learnings, too.

5. Use reliable and flexible systems for data collection and evaluation

Useful evaluations can take time, resources, and effort. Districts with data systems that allow them to gather, track, analyze and access data quickly are able to monitor the program’s success or needed adjustments more easily.

Data systems that generate analytics using multiple types of educator and student data allow district leaders to see the best way to adjust a program more clearly. This ease shifts the focus from collecting data to analyzing it — a much more effective use of time.

In this small-scale study, the six districts looked at their professional learning programs, however established or nascent, to collaboratively examine methods of evaluating those programs. Rolling evaluation into the holistic design process, beginning with targeted outcomes, taking a systems approach, continually evaluating the program, and using a solid data system all felt like the most important pointers to take away from the study to run an impactful professional learning program for educators.

5 Reasons You Should Be Evaluating Your Professional Development Programs

Wouldn’t it be great if we knew when our professional learning programs were successful? What if we knew more than just the fact that teachers liked the presenter, were comfortable in the room or learned something new? Wouldn’t it be better to know that teachers made meaningful changes in teaching practice that resulted in increased student learning?

We can ascertain all of this and more by conducting program evaluation.

Wouldn’t it be good to know your PD results in meaningful changes in teaching practice and increased student learning? #professionallearning https://bit.ly/2HE4L0h

CLICK TO TWEET THIS

Every day we engage in random acts of evaluation – multiple times per day, in fact. When we get dressed in the morning, we implicitly ask ourselves a set of questions and gather data to answer them.

Will it be warm or cold?
Will the temperature change throughout the day?
Will there be precipitation?
Which clothes do I have that are clean?
What do I have on my schedule? How should I dress for that?

Of course, getting dressed is pretty low stakes. At worst, we might find ourselves too warm or cold, or under- or overdressed for an occasion. Buying a new car, however, is a higher stakes proposition. We could end up with a lemon that costs us a lot of money, or even worse, is unsafe. When we evaluate, we are more or less systematic about it depending on the context. For the car, we may create a spreadsheet and collect data on different models, their price, performance, safety features and gas mileage. Or, at the very least, we would read up on this information and note it in our heads.

But what about our professional learning programs? What does it mean to evaluate a program?

What is Program Evaluation?

Program evaluation is applying systematic methods to collect, analyze, interpret and communicate data about a program to understand its design, implementation, outcomes or impacts. Simply put, program evaluation is gathering data to understand what’s going on with a program, and then using what we learn to make good decisions.

PD program evaluation: applying systematic methods to collect, analyze, interpret, & communicate data about a program to understand its design, implementation, outcomes, or impacts. https://bit.ly/2HE4L0h

CLICK TO TWEET THIS

Program evaluation gives us key insights into important questions we have about our professional learning programs that help inform decisions about them. For example, we may want to know:

How well does the program work? Is it changing teacher practice?
Is the program meeting the needs of the participants?
To what extent has there been progress toward the program’s stated objectives?
Do we have evidence of student learning attributable to the program?
How can the program be improved?

Part of the innate beauty of program evaluation lies in its abundant flexibility. First, there are numerous forms and approaches, and second, evaluation can be conducted both before and during a program, as well as after the program ends.

Systematic Methods

What do we mean by systematic methods? Much like a high-quality lesson or unit plan, program evaluation is the result of good thinking and good planning. It’s knowing what we want our programs to accomplish and what types of assessment will help us determine if we are successful. Being systematic means:

Ensuring that we understand what our programs do and what they are expected to do for both educators and students
Identifying which questions need to be answered
Knowing what data we need to collect to answer those questions
Identifying the primary users of our evaluation results – those who rely on the answers to be able to make good decisions

There are myriad strategies for collecting data. Surveys, interviews or focus group interviews, and observations or walkthroughs are common methods. We can also look at student achievement data, student work samples, lesson plans, teacher journals, logs, video clips, photographs or other artifacts of learning. The data we collect will depend on the questions we ask.

5 Reasons to Evaluate Professional Learning Programs

Learning Forward offers a set of standards, elements essential to educator learning that lead to improved practice and better results for students. The Data Standard in particular calls for professional learning programs to be evaluated using multiple sources of data. While adhering to a set of standards offers justification for action, there are specific advantages of program evaluation that substantiate its need:

Evaluating professional learning programs allows leaders to make data-informed decisions about them.When leaders have evaluation results in hand they can determine the best course of action for program improvement. Will the program be expanded, discontinued, or changed?
Evaluating professional learning programs allows all stakeholders to know how the program is going.How well is it being implemented? Who is participating? Is it meeting participants’ learning needs? How well is the program aligned to the Every Student Succeeds Act (ESSA) definition of professional learning?
Evaluation serves as an early warning system.It allows leaders to peek inside and determine the degree of progress toward expected outcomes. Does it appear that program goals will be achieved? What’s going well? What’s going poorly? Evaluation uncovers problems early on so that they can be corrected before the program ends.
Program evaluation helps us understand not only if the program has been successful (however “success” is defined) but also why the program is or is not successful.It allows us to know what factors influence success of the program.
Program evaluation allows us to demonstrate a program’s success to key stakeholders such as boards of education and community members, or potential grant funders.Evaluation results allow us to document accomplishments and help substantiate the need for current or increased levels of funding.

#Professionallearning program evaluation is an early warning system: https://bit.ly/2HE4L0h

CLICK TO TWEET THIS

All Evaluation is NOT the Same

The word “evaluation” can strike fear into the hearts of teachers and administrators alike. People naturally squirm when they think they are being evaluated. Although personnel or employee evaluation shares some characteristics with program evaluation — such as collecting and analyzing data, using rubrics, assigning a value or score and making recommendations — they serve entirely different purposes.

Program evaluation focuses on program data, not on an individual’s personal performance. The focus of the evaluation is on how the program performs. In education, we take great care not to let program evaluation results influence personnel evaluation. And remember the example about buying a car? That’s product evaluation, and it too shares traits with program evaluation but serves a different purpose.

Are you convinced that program evaluation will help you generate insights that inspire action to improve professional learning in your school or district?

Next up in the series

In Part 2, we’ll take a deeper dive into program evaluation and understand the big picture of how evaluation is conducted, the forms it can take, and how it relates to research.

3 Non-traditional Professional Learning Ideas for Teachers

Tips to Boost Teacher Agency

20 miles east of Portland, Oregon sits Gresham-Barlow School District. With 18 schools, “We’re too small to be big, and too big to be small,” says Sarah Hayden, an instructional coach at the district. Sarah and her colleagues work one-on-one with teachers, but also work at the district level in to provide support where needed.

They wear many hats, and just like many districts, they’re asked to do a lot with limited resources. In response, her team has come up with some creative ways to provide educator-driven, make-an-honest-to-goodness-difference-in-the-classroom professional learning opportunities for teachers.

[Note: this interview has been edited for brevity and clarity.]

Collaboration Walks

FRONTLINE EDUCATION: We’re here today to talk about something you’re doing at Gresham-Barlow called “Collaboration Walks.” Tell me about that — what are they?

SARAH HAYDEN: Collaboration walks are something that we started in our district about three years ago to promote teacher voice, teacher agency and teacher professional growth. On a given day, we get about ten teachers together and explore different classrooms around the elementary schools in our district. Then, teachers sit together to talk and collaborate with each other about what they’ve seen in the classrooms — how they can take what they’ve learned and internalize it.

FRONTLINE: What led to you starting these?

SARAH: We wanted our rubric (Charlotte Danielson’s Framework for Teaching) to be a model for professional growth and not just evaluation. We took all the numbers away from the rubric, and we used it to talk about instruction in a way that was meaningful, safe and promoted growth for teachers. We even rebranded [the walks]. Instead of ‘Learning Walks’, we call them ‘Collaboration Walks.’ We talk about how the collective voice of the teachers in the room is what is needed for everyone to grow — this is not something that’s top-down. It’s a collaborative way to talk about teaching.

Want more details? Listen to our full interview with Sarah Hayden about Collaboration Walks at Gresham-Barlow School District.

FRONTLINE: Describe the process — who’s there? Where do you go? What do you do?

SARAH: There are usually about ten teachers who sign up, usually within a day and a half. We meet at one of the schools in the morning, and we talk about our goals for the day: What do we want to get out of today? How are we going to be reflective? How are we going to move forward collectively?

We focus on two or three of the different standards in our rubric and we ask, “What does best practice look and sound like in the classroom? What does best practice surrounding discussion and question techniques look and sound like in the classroom? What does setting purposeful intentions for students look like and sound like in the classroom?” And in a collaborative way, we come up with, “What does best practice with these indicators, these standards, really mean?”

Then we go into the classrooms with this lens in mind. Teachers bring their cameras, they talk with students, they work alongside teachers. We’re in a classroom for anywhere from 15 to 20 minutes, and we observe. Then the most exciting part of what we do, we sit and we talk, and we talk, and we talk about teaching — what I do in my classroom, what you do in your classroom, what we observed in the teacher’s classroom that we saw.

We keep it really safe and non-evaluative. We use sentence stems that just say, “This is what I observed… This is what I saw… This is what I wonder….” Through this process of collaborative discussion come amazing points about how teachers are going to move their practice forward.

FRONTLINE: What is it about the structure of what you’re doing that makes these effective?

SARAH: It is 100% teacher-driven, teacher-centered, and the entire goal is to elevate teacher voice across our system. The caliber of the teachers in our district is amazing, and if you get some like-minded individuals in a room, we can solve the problems of the world. That’s why I think it’s been so successful, because it’s about meeting the teachers where they’re at, and helping them continue on their personal journey.

Whether they’re a first-year teacher or a veteran of 20+ years in our district, every single person in that room can support their colleagues through collaborative conversations.

Reflective Conversations

FRONTLINE: Collaboration walks aren’t the only thing that you’re doing in professional learning, of course. How else are you working to make professional learning more teacher-centered?

SARAH: We’ve taken the idea of collaboration walks and we’re doing what we call “reflective conversations,” where teachers videotape themselves conducting a lesson in their classroom, and then they come to our professional development session with a trusted peer from their school or their grade level, who has also videotaped themselves.

We had one teacher comment, “In all of my 20 years of teaching, I’ve never had professional development as meaningful as what I experienced with my colleague at this professional learning.” — Sarah Hayden

Then we spend some time talking about what a reflective conversation is, and how to support your colleague in a way that promotes their professional growth. The teachers watch the videos alongside each other and use these reflective conversation skills to discuss their practice. We had one teacher comment, “In all of my 20 years of teaching, I’ve never had professional development as meaningful as what I experienced with my colleague at this professional learning.”

FRONTLINE: What is it about the use of video for these reflective conversations that makes it important?

SARAH: Videotaping yourself as a teacher is absolutely terrifying, which is why we bring that trusted peer in. When you watch yourself teach, you are your own worst critic. And everything that you see, you don’t realize that you do on a day to day basis. Once you get past, “Oh my gosh, I really sound like that?” you see exactly what you’re doing and how your students respond to you. Things that aren’t usually visible in the classroom are very visible when you watch yourself on video.

It’s a chance to go deep into what you’re doing every day, and see how the things you do affect student outcomes. It’s completely and totally career-altering. And the teachers communicated that to us, even after one video.

On using video for self-reflection in teaching: “It’s completely and totally career-altering.” #professionallearning https://bit.ly/2JHUJsQ

CLICK TO TWEET THIS

FRONTLINE: Can you talk more about the training you provide for these reflective conversations?

SARAH: The reflective conversations training is a day long. In the morning, we don’t watch any videos — the trusted peer and the teacher sit with us, and we talk about what a reflective conversation is. “What are ways that you can pose an open question that invites inquiry from your partner?” Because as a teacher, it’s very easy to watch a video alongside a colleague and say, “Well, in my classroom, I…” or “Have you ever tried…?” — which can stifle what that teacher needs to understand about their own practice. So we ask questions in an invitational way, where the trusted peer is asking questions so that the teacher can develop their own understanding.

The trusted peer is never telling, never answering. The trusted peer is facilitating and prompting their peer to think about their teaching in a deeper, different way. The trusted peer is the one who can help you draw out what you need to investigate about your own practice. So it’s way more meaningful when you have someone to support you in that and to ask those questions that you didn’t even know to ask yourself.

You may enjoy this hand-picked content:

White Paper: 10 Strategies to Improve Teaching with Video

FRONTLINE: I would imagine receiving that kind of feedback is both helpful and scary.

SARAH: Exactly. As we’re setting up the day, we talk about what a reflective conversation is, and we say, “It’s rigorous. It’s not mere support group talk.” It’s talking about teaching and giving meaningful feedback. We’ve heard from our teachers that so often in our profession, this is missed. Deep, rich, meaningful feedback is missed by teachers, and they crave it. That’s why they feel so good at the end of the day —they’re finally getting something that’s going to help them, that’s going to take them to the next level.

You’re the trusted peer for your colleague, and then they are the trusted peer for you. So not only are you getting, you’re also giving. Additionally, the teachers have found that being the trusted peer and watching their colleague’s video and asking questions allows them to be reflective of their practice as well.

Inquiry Teams

SARAH: I think one of my favorite things that we’re doing in our district is what we call “inquiry teams.” Inquiry teams are a way for us to allow teachers to experience professional learning completely and totally on their own terms.

In teams, teachers put together a proposal about something that they want to learn about. It could be anything from mindfulness in the classroom to new math strategies for STEM to exploring questions of equity within our school. Then, we put them together with a facilitator and give them time and space to inquire about what they want to learn about in a meaningful way, and then share what they’ve learned with their colleagues across our district.

FRONTLINE: How did they do that? How did they share out those practices?

SARAH: At the end of the inquiry team process, the teams and the facilitators put together a ten-minute presentation about what they learned, and then we have an inquiry celebration where teachers can go and learn from their colleagues in a forum. There’s cake involved, which always is exciting, and then they share out their project and what they’ve learned.

This is our second year of inquiry teams, and last year, some of the presenters said, “You know what? We learned a lot about what doesn’t work through inquiry. We hit some roadblocks, which was completely meaningful for our way of learning. Investigating those things and finding out what doesn’t work was the most beneficial type of learning that we could have done.”

It’s just…it’s amazing. Going over just the inquiry proposals this year was inspiring. What teachers are grappling with, what they want to learn, how they feel that they can move their practice forward, and then seeing how they bring their knowledge through their inquiry team back to their buildings, back to the school district as a whole, is so exciting.

You may enjoy this hand-picked content:

White Paper: The Power of Failure: Encouraging Teachers to Take Risks in a Risk-Free Environment

Explore by ROLE

frontline resources

MEET FRONTLINE

Program Description

Logic Model

Program Theory

Next Up In the Series:

A Loyalty Rate for Loyal Substitutes

Substitute Appreciation Tea

Substitute Teacher Training

So, how do we get it right? Here are a few tips:

Leverage the Talents of Current Staff

1. Streamline meetings and paperwork to increase the time staff can spend with students.

2. Allow staff to play to their strengths; assign roles based on strengths, not titles.

3. Facilitate teamwork with common planning time.

4. Support classroom teachers with in-the-classroom support from staff skilled in behavior management.

Focus on Prevention

5. Focus on prevention by identifying and managing behavioral triggers.

6. Increase access to staff with expertise in behavior management.

7. Align discipline policies to support a commitment to prevention.

8. Stay focused on academic achievement.

Support Local Partnerships

9. Seek local partnerships.

10. Actively support local partnerships.

Working Together to Improve Your School’s Behavioral Climate

Today’s article, focused on the five phases of program evaluation, is Part 2 of a seven-part series on evaluating professional development. In the rest of the series, you’ll learn how you can apply these phases to evaluate your own programs.

What is Program Evaluation?

Program Evaluation Has 5 Key Phases

Phase 1: Engagement and Understanding

Phase 2: Questions

Phase 3: Data Collection

Phase 4: Data Analysis

Phase 5: Reporting and Use

Evaluation as an Iterative Process

Next up in the series

How Franklin Public Schools Used In-District Expertise to Promote Effective Observations

Steps to Making the Most of Spring RTI/MTSS Data

1. Archive RTI/MTSS information

2. Evaluate Effectiveness of Core Instruction

3. Analyze Data to Uncover Performance ‘Pockets’

4. ‘Recalibrate’ RTI/MTSS Procedures

5. Recruit Fall Groups Using End-of-Year Screeners

6. Update the RTI/MTSS Roll-out Plan

7. Prepare RTI/MTSS Professional Development

A Final Thought on Optimizing RTI/MTSS at End of Year

1. Walk a mile in the job-seeker’s shoes

2. Communicate

3. Set reasonable timeframes

4. Seek feedback

What is the Section 504 eligibility criteria? How should it be used?

The Section 504 eligibility criteria involves two questions:

How is the Criteria Defined?

What are “Major Life Activities?”

Is Section 504 a “Consolation Prize?”

Are Any Students “Presumptively Eligible” for 504 Plans?

When determining Section 504 eligibility in a school, should we only look at activities that impact “learning?”

To Sum Things Up.

1. Plan evaluation as a holistic part of the program

2. Develop targeted outcomes with existing data sources in mind

3. Consider a systems-approach to better evaluate effectiveness. from its inception.

4. Continually evaluate both new and existing programs

5. Use reliable and flexible systems for data collection and evaluation

What is Program Evaluation?

Systematic Methods

5 Reasons to Evaluate Professional Learning Programs

All Evaluation is NOT the Same

Next up in the series

Tips to Boost Teacher Agency

Collaboration Walks

Reflective Conversations

You may enjoy this hand-picked content:

Inquiry Teams

You may enjoy this hand-picked content: