Focus Groups: Your Secret Weapon for Enhancing Professional Development Program Evaluation

Focus groups (FGs) — essentially interactive, conversational group interviews — are an underappreciated data collection strategy. When used well, they can provide rich, nuanced and actionable data for your professional development program evaluation. They are fairly easy to add to your repertoire and don’t require extensive planning or resources to carry out. They are often used in other contexts (e.g., market research, usability research) and are flexible enough to be used for different purposes (e.g., to inform the design of a product or service, to collect feedback about a website, to test potential survey questions or dig deeper into survey responses). FGs are best used along with other data collection strategies, such as surveys or interviews.

Possible Scenarios for Using Focus Groups

For professional development program evaluation. I’ve used FGs as part of a plan to evaluate a professional learning program such as Mentoring and New Teacher Induction. In addition to feedback surveys used right after our New Teacher Induction Academy, I used FGs comprised of new teachers and mentors to better understand how our model was working and inform decisions about improving our practices. I have also used FGs to follow up with teachers months after a professional learning course to learn how they were implementing new instructional practices and their perspectives on the impact on student learning.

For professional learning needs assessment. I used focus groups with teachers as part of a larger effort to study a district’s needs and readiness around issues of equity, diversity and inclusion. Along with other data collection strategies, the FGs gave us perspective on what teachers in different buildings understood about these topics and about the district’s effort to address them, where they were struggling, and where they felt they needed support and learning.

Using Focus Groups in Conjunction with Surveys

Using focus groups prior to a survey.

Using open-ended questions and exploring a range of attitudes from FG participants can inform the design of survey questions – for example, questions that may be used on a districtwide professional learning needs assessment. Using FGs in this way means to

…engage in a moderated discussion of the survey topic or construct in hopes of learning more about it as well as eliciting input from participants on issues that should be included in survey questions. Focus group participants, especially if they are either members of the respondent population or closely connected to the topic of study, can provide greater understanding of a topic and lead researchers to explore aspects of a topic that might otherwise be overlooked (Robinson & Leonard, 2019 p. 78).

Using focus groups after a survey.

FGs can facilitate deeper understanding of why and how survey respondents answered certain questions the way they did. Perhaps participants answered a few questions on a feedback survey about what they learned in a professional development course, but now you want to explore more about how they are using that learning, and whether they feel that learning is important to their practice or impacting students.

Common Questions and Concerns

People often ask, “Is this a scientific approach?” and then add, “After all, small groups aren’t representative of all teachers (or all parents, etc.).” Think about the real concerns behind that question. People often dismiss the idea of FGs for data collection because they don’t produce numerical data. What people are really asking is whether or not we can believe what we learn from FGs, whether the data we get constitutes sufficient, reasonable and believable evidence of what we are trying to measure. Most people are not used to making sense of qualitative data. We don’t use statistics for qualitative data. We analyze text and then work to determine whether our findings are indeed trustworthy, credible data that can help answer the program evaluation questions that were posed.

People also ask about anonymity. Using FGs is not an anonymous approach, and therefore may not be the best data collection strategy for the most sensitive topics. That said, the invitation to participate in the group can indicate that the conversation that takes place during the session should be considered confidential, and the moderator can reiterate this request at the outset of the session.

Challenges and Limitations

Typical FG challenges involve group dynamics, personalities, strong opinions and politics. Some of these can be addressed with good planning: selecting participants carefully (i.e., having separate FGs for teachers, parents or administrators to mitigate power differentials) and having the moderator set out clear expectations for participation. The moderator can also have a set of techniques for redirecting conversation, shutting down a dominator and encouraging quieter participants to engage.

FGs cannot measure the full spectrum of perspectives of a population (i.e., all teachers in a district) and are best used with other data collection strategies to obtain a robust picture of what is happening and how people are experiencing your professional learning programs.

Setting up for Success with Focus Groups

Focus group questions or prompts.

You may want to develop questions collaboratively with other stakeholders. Consider your evaluation questions and let them guide what data you need to collect from FGs. Don’t limit yourself to just open-ended questions. Prompts can also be lesson plans, data (achievement, attendance, demographic or discipline), student work samples, other classroom artifacts that participants react to and discuss.

Focus Group roles.

Focus groups generally have a moderator, 1-2 notetakers, and ideally about 6-8 participants. It’s important to think carefully about what will be expected in each role.

Moderator
The moderator facilitates the FG and directs the conversation. Moderators introduce the FG by sharing the purpose and structure of the session with participants. They may ask for permission to record the session. They will pose the questions or prompts participants will discuss during the session, and they will use probes – questions designed to stimulate deeper thinking from participants. Examples of probes are: “Can you say more about that? What else can you share about that? What made you feel that way?”

The moderator will let participants know whether they should wait their turn before speaking, or whether they should jump into the conversation whenever they have something to say. The moderator will let them know the expectations such as whether they should offer just their own opinions or experiences, or whether they should feel free to agree and disagree and build off of what others have said. The moderator will meet with the notetaker(s) to review the focus groups prompts and directions for note taking.

Note-takers
Note-takers are not typically participants in the FG. They are there to capture as much of the conversation as possible, and may also capture observational data about participants and responses. They may note when people smile or frown, when heads nod in agreement or shake in disagreement, when voices are raised, or other non-verbal information that will help the program evaluator make sense of the data. It can be helpful to have two notetakers: one who captures the conversation only, and one who captures the non-conversation data.

One alternative or supplement to note-takers is a conference call transcription service. You can dial into a conference call number, have the conversation recorded (as long as you set a phone close enough to participants), and the transcription sent to you via email. These can be a bit costly, but offer professional transcriptions that ease the pressure on notetakers.

Participants
Participants are the primary conversation-makers. They respond to the FG prompts and to each other as directed by the moderator.

Logistics.

FGs often involve “like groups.” When I conducted an evaluation of an athletics program, I held separate focus groups for coaches, parents and teachers. When I conducted a needs assessment study, I held focus groups at different schools, with teachers in similar grade levels. When evaluating a professional development program, you may want to separate participants by school, subject area, or level (e.g., elementary, middle, high). People are often more forthcoming and comfortable sharing their opinions and experiences with people they know, or at least people with whom they have something in common.

Find a place where people will be comfortable talking. This could be a classroom, a professional development meeting space or even a conference room at a restaurant, hotel or other off-site location. I specifically avoided the Board of Education meeting room at one district because I knew some people didn’t feel comfortable there. It was a more formal, less relaxed space with its large podium, raised semi-circle bench in front of the room and many microphones.

If possible, set up chairs in a circle with no tables. This removes the physical barriers, and encourages people to stay engaged with each other. They can see each other’s faces and body language at all times and can react to that, knowing when to enter the conversation. The notetaker(s) may be seated outside of the circle as not to distract participants or make them uncomfortable.

It’s also a nice gesture to offer light refreshments for participants. Having a few minutes at the beginning of the session for participants to grab a snack and beverage also allows for some small talk that can help loosen people up and get them ready to be actively engaged in conversation around the designated topics.

Are Focus Groups Worth the Effort?

Using FGs as a primary strategy for professional development program evaluation means we have the potential to learn more than we ever could through the use of surveys alone. More importantly, the qualitative and interactive nature of FGs can help us capture data that is different from what any other data collection strategy can yield, and will give us much deeper insight into how people experience professional learning.

For more on using focus groups for professional development program evaluation, see my free guide for conducting focus groups, and my in-depth article: Professional Development Program Evaluation for the Win!

Robinson, S.B., & Leonard, K.F. (2019). Designing Quality Survey Questions. SAGE Publications.