Episode 288: Standardizing Immersive Learning for the Fire Service with Dr. Kate Kapalo

Episode notes

What is the future of fire service training?

Hello Smart Firefighting Community!

Welcome to another episode of covering real world innovations via interviews with fire service and technology industry experts that empower YOU to develop your very own Smart Firefighting strategy!

In this episode:

What must immersive learning focus on for greater impact?

What tools help fire instructors measure performance consistently?

Why there’s a need for stronger collaboration between the fire service, industry, and academia

Hear from Dr. Kate Kapalo - a Course Instructor & Instructional Designer for the U.S. Fire Administration, and a Principal Investigator & Adjunct Professor at the Stevens Institute of Technology.

Kate shares her research and perspective on how immersive technologies like VR and simulation can transform fire service training. She highlights the need for standardized rubrics, national alignment, and stronger collaboration between industry and the fire service to ensure training delivers measurable outcomes.

Kate also emphasizes the importance of situational awareness, communication, and building infrastructure to make immersive learning sustainable. This conversation explores how rigorous research can move from academia into practical, real-world training that prepares firefighters for an all-hazards environment.

So take a listen to hear what she has to share!

Head to www.smartfirefighting.com to discover how SFF accelerates innovation for emergency responders, to find out when our next event is, or review our curated resources!

Facebook | Instagram | Twitter | LinkedIn

Chapters

1. 0:00 Intro from the TRAIN Summit
2. 0:36 Welcome, Dr. Kate Kapalo
3. 1:06 Why standardize immersive learning?
4. 3:07 Building the performance rubric
5. 5:30 Three scenarios: driving, pump ops, highway incidents
6. 6:56 Industry and the fire service need each other
7. 8:06 Vendors can't read our minds
8. 10:03 Two-way communication and co-development
9. 11:05 Academia, prize challenges and KPIs
10. 13:11 DC Fire and EMS: infrastructure and dashboards
11. 15:30 Where immersive learning goes next
12. 18:52 Wrap

Questions answered in this episode

What is the Train Summit and what is its goal?

Dr. Kate Kapalo says the Train Summit focuses on understanding how to better serve the fire service using immersive technologies. The goal is to get everyone on the same page and standardize simulation or immersive learning so learning outcomes can be assessed, addressing the lack of standardization that exists because each state handles training differently.

Hear it at 1:06

Which three scenarios is the performance rubric focused on and why?

Dr. Kate Kapalo explains the rubric focuses on driver training or emergency vehicle operator, pump panel operations, and a highway incident. These were chosen because situation awareness is a huge component in all three and simulation is already being used in these areas, allowing for data collection and evaluation.

Hear it at 3:07

What challenge does the fire service face when working with technology vendors?

Dr. Kate Kapalo says the fire service sometimes expects technology vendors to read their minds. Software engineers don't typically have the domain knowledge needed, and the fire service is not always great at translating what they need. When things are designed iteratively with technology vendors, things tend to go better.

Hear it at 8:06

How is DC Fire EMS using immersive technology?

Dr. Kate Kapalo says DC Fire trains all command staff and incident commanders regularly using immersive technology, and uses immersive technology for EMS based skill practice and assessment. The department is building infrastructure to integrate more immersive learning.

Hear it at 13:46

What does Dr. Kapalo hope happens as a result of the Train Summit work?

She hopes the fire service can come together as a unified voice for training. The goal is to apply an academically rigorous process that actually serves the fire service in practice, potentially incorporating findings into standards later, so state training directors, instructors, and technology vendors all benefit.

Hear it at 16:39

Transcript

Machine transcript of the published audio. Names may be misspelled and some sections are missing.

Ep 288 transcript

Standardizing Immersive Learning for the Fire Service with Dr. Kate Kapalo

Speakers: Dr. Kate Kapalo, Kevin Sofen

Source: Whisper medium.en machine transcription of the published Podbean audio (no original transcript found in Drive), lightly cleaned. Speakers were separated by voice clustering and then checked and assigned by hand at every speaker change. Gaps where the model returned no usable text are marked [inaudible m:ss].

[0:00] Kevin Sofen: Welcome back to the Smart Firefighting Podcast. Today, we're at the Train Summit, where leaders from across the fire service, research, and industry are tackling one big question. How can immersive learning make training better, faster, and safer? From virtual reality and simulators to new performance rubrics, we explore how technology can boost decision-making, generational gaps and bring more sets and reps to every firefighter, whether it's in the big city or a rural volunteer department. It's about turning cool tools into trusted training that delivers real results. Enjoy. Dr. Kate Kapalo, or Kapalo, I guess. Have I been saying it wrong my entire life?

[0:47] Dr. Kate Kapalo: No, it is Kapalo, but that's okay. Everyone says it wrong, so I'm not offended.

[0:50] Kevin Sofen: What is the origin of?

[0:51] Dr. Kate Kapalo: It's Hungarian.

[0:52] Kevin Sofen: Hungarian?

[0:53] Dr. Kate Kapalo: Hungarian, yeah. Everyone thinks it's Italian, but it's spelled with a K. And, yeah, so my dad's side of the family is Hungarian and Italian, in Kapalo.

[1:01] Kevin Sofen: And then here you are, Champaign, Illinois.

[1:03] Dr. Kate Kapalo: Yes.

[1:04] Kevin Sofen: So what brings you here to Champaign?

[1:06] Dr. Kate Kapalo: So we are here for the Train Summit. The Train Summit is focused on understanding how we can better serve the fire service using immersive technologies. So I'm sure you've already covered this, but I think the big thing that we always ask ourselves is we have all these other domains, nursing, military, and they've all done a really good job of standardizing simulation or immersive learning to a point where we can assess learning outcomes. The fire service, because everything is handled in each state, there's a lot of differences and a lot of lack of standardization. And so our goal with this project is really to kind of get everyone on the same page. We have national standards, right? We have NFPA. We have JPRs, job performance requirements. We have all these things that we train to. But I think the key thing here is there's a lot of nuance too that we can't capture. And so the train summit is meant to bring all of the state training directors together, all of our technology stakeholders, all of the people that play a part in this process to make technology work for the fire service and for learning.

[2:02] Kevin Sofen: And I know we talked about on the fire service, or even at USAID, that there's 20,000 fire departments. And then they always say there's 20,000 different ways to do stuff, and they kind of joke about that. But there's some truth behind that. when thinking about how we're going to try and enhance training that's needed. We need more sets and reps. We need better training for the fire service. And we can't just try and close Pandora's box and ignore immersive learning. And so part of this summit is to embrace immersive learning, but doing it in a right way to where it's like, okay, how do we actually create some performance rubrics around it? How do we create some commonality from whether you're in Rhode Island, or Oregon, or Texas, or Wisconsin, wherever? And so I know you've been involved with, I mean, I look at this document and it's the, you know, colors and ABCDE and numbers and symbols and everything. And maybe at first glance, I'm like, wow, I'm not smart enough to know what all this is, but give me some more context and sort of like, why this kind of performance rubric is important and kind of how, what was the methodology that went into creating this performance rubric?

[3:07] Dr. Kate Kapalo: Absolutely, so like the big picture, like why are we here? What are we doing this for? The challenge we have is everything's kind of siloed in the fire service, right? So our standards, we have the job performance requirements. That's what your basic competencies are. So we're training our firefighters to that level. Can you flip on the switch when you need to? The challenge with that is when we actually put you on the rig or when you go on scene, it's not just, can you remember to turn on the switch? It encompasses, am I looking and doing my size up? Am I listening to what's happening on the radio? Am I aware of what's happening in my surroundings? Firegrounds, fire lines, they're all really loud. There's a lot of stuff happening. And so when we look at, you know, how are we training our members? One of the challenges we have is that we don't always factor in, you know, the training environment can be different from the learning environment. And so how do we tie that all together in a bow, right? So if I'm teaching you how to do pump panel operations, we often put you in a situation that's sort of realistic to a certain level. So I'll take you out on the fire ground, we'll connect some lines, you'll pull some levers, push the governor, you do what you're supposed to do. but I don't have firefighters in the background screaming at you for this line. I don't have people screaming on the fire ground in that situation. That time pressure and that stress is not always something that we're able to induce. So the cool thing about this performance rubric and kind of this project is, how do we kind of map that pipeline? We have it for the military. They do that with pilots. We do this with different roles in the military, but it hasn't been fully fleshed out for the fire service. And so that's really where kind of that framework comes from a methodology standpoint, we did a lot of things. It looks pretty simple on paper. And you're like, you know, kind of talking earlier today, somebody mentioned like, oh, you know, it's really easy to poke holes in this because it looks, you know, at a glance like there wasn't a lot of work put into it, but there is a lot of work put into it. So as a scientist, I have to look at what are the needs of my people? What are the needs of people in training? So when we look at our members and our instructors, what are they assessing and what do students need to know? And then kind of from there, we did a survey with the NAFD members, so all 50 state training directors, and we asked them, you know, where are you using immersive learning? Where do you want to be using immersive learning? It was kind of like a temperature check there. And we were able to get feedback and we kind of rolled this out, focused on these different scenarios. So we have three scenarios that we're focusing on. Driver training, so emergency vehicle operator, pump panel operations, and a highway incident. And the reason for that is situation awareness is a huge component in all three of those. And we also have seen a lot of tools in that area. So I can actually start to evaluate some things and collect some data. So that was one of the questions that came up, like why those three scenarios? Well, those three scenarios, we're seeing simulation already being used. And also there's a key component of situation awareness involved. I can go on scene and I can tell whether or not you've observed the right cues and clues based on what's happening in the environment, based on the decisions you're making, based on how you're responding to the incident, based on the tempo, all of that. So we started with a survey and then we moved into the JPR. So I took, JPRs are in a very, I guess, linear format. You could say they're just a list of all the job performance requirements. So we're trying to take all these disparate pieces of the fire service and turn this into something that state training directors can use, that instructors can use. And so we can build out a more mature curriculum. So when we look at, you know, what do fighter pilots do? They start with basic flight familiarization training and then they move into the cockpit. We're designing essentially the equivalent for the fire service. Where does simulation need to be? Where do we need to go? And that's kind of the methodology behind that rubric at a very high level. I can delve more into the academic side, but I don't know if anyone's really interested in that.

[6:56] Kevin Sofen: I am interested. We'll do a slight evolution pivot, but I know Ken talked about this earlier. And so much of this is, again, this kind of age old adage of kind of industry needing the fire service and the fire service needing the industry. and there's kind of this mutual symbiotic relationship, and sometimes it's done really well, and sometimes you have industry driving the boat, but I think what we've seen here is a very healthy dialogue between what are the biggest needs, what are the biggest requirements, and you see some of these immersive learning companies here really chomping at the bit in the best way possible, thinking about, okay, how can we modify our solutions to actually integrate with these fire service operations, and not just like, hey, here's the thing, buy it, use it, Because it all kind of comes down to a fire department or a fire service using taxpayer dollars that need to show some kind of measurable ROI to then going to the question of is the learning actually happening? Is training actually effective? And I'm curious in your thoughts and that kind of continuation of that conversation of the kind of conversation between industry and fire service all leading to like all that the most important ending is like is learning happening more effectively, better, faster, cheaper?

[8:06] Dr. Kate Kapalo: Yeah, no, that's a great question and kind of a great thing to center on. So this is just my personal opinion. I just want to make sure that no one takes this as like an agency's opinion. This is my personal opinion. I think some of the challenge is we expect technology vendors to read our mind in the fire service. So we know what we need and we're not always great at translating that out and back and forth, right? Like software engineers don't typically have the domain, like we have some exceptions. I mean, there's some great people in the fire service that do both things. But software engineers, generally speaking, don't have the domain knowledge that they need to be able to assess, hey, from a learning outcome perspective, this is what I need. Sometimes we focus on the user interface, we make it look pretty, sometimes we focus on the graphics, and all of those are important things when it comes to the user experience per se. But I think the challenge that we have is like, how do I actually give a technology vendor, hey, I need you to design something that does this. And we historically haven't thought a lot about that. And I think you see some of the challenges with that. There's a lot of factors that go into it. Like obviously the space in general in the fire service for sales is kind of small. And that's why you see a lot of adoption from the DOD space. There's also like challenges surrounding every state and department does different things to your point. We talked about that. If you've seen one department, you've seen one department. It doesn't necessarily mean it translates. So I think when we're trying to standardize different things that happen in the fire service and standardized learning, it can be a back and forth kind of conversation that's really challenging because it's, we can't expect technology vendors to read our mind, but if we design things with them iteratively, then things tend to go better. And that's kind of like Chief Boyd's example of the incident command simulator. That's something that was designed iteratively between a fire department and a technology vendor. And I think that's a perfect example of, hey, you know, we can design technology that meets the needs of the fire service more broadly. and in specific nuance. context, which would be like all in Texas fire department.

[10:03] Kevin Sofen: Yeah. Into that kind of reading your mind perspective, it's sort of being a new father and having a son that's two months old. It's sort of funny this dynamic where babies really communicate a lot through crying.

[10:14] Dr. Kate Kapalo: Yeah.

[10:14] Kevin Sofen: I mean, they can't really use their words. And I kind of joke to my son, like, use your words. It's like, you can't. But it all kind of comes down to this, like, this two way street of communication. And like, if one, you have to, we all need to be better communicators in all our relationships.

[10:27] Dr. Kate Kapalo: Yeah.

[10:27] Kevin Sofen: But especially within the fire service to industry perspective of if there isn't, this is one reason why this forum is so important because it's creating the environment. But I think it's a message that we can have with from fire service members to ask more of our industry partners, but also not just grilling them, but how can you be more inviting? How can you create the environment to have co-development, co-conversations? Obviously, sometimes it's easier just to do nothing and just stick with what you're already doing, but wanting to have that mindset of always wanting to be better, but part of that's better communication, finding that kind of two-way street between industry and fire service.

[11:05] Dr. Kate Kapalo: Yeah, and I think there's like different ways to go about it too, right? Like you can set up industry partnerships, but like academia is also an option that's a good option sometimes because we have students that need to learn things. And then we have, you know, practical implementation and it's like kill two birds with one stone. We're able to do both. Our students can learn and we can, you know, something that's useful and factors in what the fire service needs. NIST, a while ago, had the tech to protect challenges. Those were really important. Prize challenges, hackathons, all of that, I think, are things that go into this. But right now, we're in the initial phase where it's like, we're just conceptualizing what this could even look like. And I think Preet brought up some good points today about that, surrounding that. Like, what does that look like for the future? But I guess one of the challenges, too, and this might be a contentious take or a hot take them. But like our KPIs in the fire service aren't necessarily designed around things that always are the best for our learners in the fire service, right? Like we focus a lot on service delivery. Some of these KPIs are vanity metrics. We need to make our department look good to continue to get funding. So there's a give and take there, right? And I think we're able to do both, which I know sounds like, oh my gosh, how is that going to happen? But there are ways to make so that we have a voice in the room and we're also able to show, hey, our department is performing well, we're able to do these things well, we're training people to the standard that we need to be training them. And there's different challenges in different regions and areas like some departments really are struggling with recruitment right now. Other departments, not so much recruitment, but it's like making sure that their staff are competently trained and then able to, they aren't just thrown to the wolves and able to deliver a good service. So I think there's like different nuances and there's different places where different departments are. I mean, obviously metro versus rural, all those kinds of conversations play into it. But yeah, I think technology providers, industry, they can really help us with this KPI problem that we have. Because I mean, yes, you need a pretty printout that shows your department's doing good work, but we also need to show that people are learning and using the technology, and there's a mutually beneficial relationship out of that.

[13:11] Kevin Sofen: Yeah, well said. And Kate, you, like me, wear a couple hats. We have our hands in a few different organizations, and it's a wonderful thing, but I know you've been involved now with DC Fire EMS and worked with, you know, across the department now for five months, six months? Six months, yeah, about six months, crazy template. Yeah, and I know there are some other members from DC Fire here, and I know Chumworth is presenting later, but from a high-level perspective, what's some of the, you know, why are there two of the members of DC Fire here? What are they talking about? And what's, anything you can share about that?

[13:46] Dr. Kate Kapalo: Yeah, absolutely. So Chief Chenworth and Firefighter Jeff Leonard is here. They're both here to discuss a lot of different things. So in DC, we do actually use a lot of immersive technology. We train all of our command staff, our incident commanders, regularly using immersive technology. So they're going to touch on that. We use a lot of immersive technology for EMS-based skill practice and assessment. But I think the thing to take home from this is we have to be able to build the infrastructure for any of this to work, right? And so Chief Chenworth and Jeff are really focused on how can we make, so Chief Chenworth is the chief of IT, just to give some context. And Jeff is a nerd in the finance department who happens to be really tactical, but also really great with technology. So he builds a lot of the dashboards in collaboration with our IT folks, Tim Hutchinson, but it's It's the idea that they're creating the pipeline to be able to build out this infrastructure that we need to integrate more immersive learning. So Chief Chenworth is going to discuss some of those immersive learning things today, but he's also discussing our FOC, our Fire Operations Center, all of the things that went into building that out, what that looks like, how we communicate. And then Jeff will be talking a little bit more about the dashboards we use in operations, which also can be used in a training context as well. Because if we know what's happening in the department, then we have better situation awareness on when it's good to schedule trainings, when things are happening. Like, we have more awareness across the board because it started with, hey, everyone's siloed, operations is doing their thing, trainings doing their thing, and we're butting heads a lot because we can't get schedules straight. So this kind of helps us streamline all of that. So they're gonna be touching on that today.

[15:24] Kevin Sofen: That's awesome. Well, my heart continues to be with the DC Fire Department after everything. I know January was quite the month.

[15:29] Dr. Kate Kapalo: Yes. Thank you.

[15:30] Kevin Sofen: So yeah, to you and everyone, just thank you for your continued service. And I mean, I guess, again, to kind of wrap it up, the final question I have is, I think you were here in 2023, right?

[15:40] Dr. Kate Kapalo: Yeah.

[15:41] Kevin Sofen: So this is kind of, and I talked to Ken Willette where, you know, this for him kind of started in 2020. And, you know, immersive tech has been evolving. And there's some example of the first kind of VR headset stuff, you know, 50 years ago. And of course now it's commercially available in a lot of ways. But obviously now it's May 2025. I'm kind of curious in your thoughts of what do you hope happens from all this? And where do you see immersive learning in VR, AR, MR, all these, XR, all these different terms? Where do you see that being part of the fire service? And I, you know, a little other context, I look at drones now in terms of they maybe they're one of the first new pieces of technology that really started to integrate in it's sort of a less of a nice to have and a must have in many departments in immersive learning. I look at being this next tool that once there's more of these standards and learning rubrics around it will hopefully become more adopted. But what do you hope to happen in the next 5, 10 years? And where do you see this, the whole work we're doing here with the file and train some of its role in it all?

[16:39] Dr. Kate Kapalo: I think that's an excellent question. I think there's a couple different things. So first, I think my hope is that we can kind of come together as a unified voice. You know, we have Dr. Laurie's one voice. We need that for training. We need that. It's critical for training. In order to be able to kind of inform standards and make this ecosystem work for us, we need our state training directors, our instructors, our students to have a voice in that process. And the reason for that is because fire ground needs are changing. Fire departments are all hazard. This has been an evolution coming for a while, but I mean, we expect our firefighters to be wizards and just go out and do whatever call, run they get, they're expected to show up and figure it out. And so I think our training systems need to evolve with that need to be all hazards. If we're going to force people to do that, then we need to look at what's working and what's not. So I'm hoping that this kind of brings us a step forward in terms of we're adding a academically rigorous process to validate this, but we're also implementing it into practice because I can't tell you how many times in research and in academia we do all this work and it never reaches anyone outside of our own echo chamber. We publish a paper and it goes back in a circle where it's like, oh, all these people already agree or disagree and then we don't really implement it into practice. So my hope is that we are taking this methodology, we're taking all these things that we're learning at the train summit and then we're applying it in a way that actually serves the fire service. Ideally, I would love to see some of this incorporated into standards later. I don't know if that's on the horizon or what that will look like right now, but that is kind of an end goal is to say if we're able to help state training directors, if we're able to help instructors with better assessment, if we're able to help technology vendors to, you know, develop what works for the fire service, then we all win. And that's kind of, That's my hope.

[18:28] Kevin Sofen: You might drop. Well, it's okay. Well, I appreciate all the continued work that you do. I mean, you've been involved in a lot of the research and development and working with both the industry and nonprofit and being here. So on the behalf of the fire service and everyone here, thank you for what you're doing. And I'm excited to continue to collaborate moving forward.

[18:46] Dr. Kate Kapalo: Thank you so much. I really appreciate it. You do amazing things for the fire service, so it wouldn't be complete without having you here.

[18:52] Kevin Sofen: Thank you. Appreciate it. Thanks, Kate. Thanks for joining another episode of the Smart Firefighting Podcast. We hope you learned something today, and if so, please consider leaving us a review or a comment so we can keep creating the best content for you. Have a great day, and thanks for all that you do for public safety.