Episode 246: The Role of AR in Real-Time Operations with Yohan Baillot

Episode notes

How can you train and simulate realistic scenarios without any props?

Hello Smart Firefighting Community!

Welcome to another episode of covering real world innovations via interviews with fire service and technology industry experts that empower YOU to develop your very own Smart Firefighting strategy!

In this episode:

What’s the role of augmented reality (AR) in emergency response and training?

How does AR enhance situational awareness during real-time operations?

What potential does AR have in improving emergency response?

Find out from Yohan Baillot - the Founder and CEO of ARCortex Inc .

Yohan talks about the use of augmented reality (AR) in both emergency response and training environments. He explains how AR superimposes graphics onto the real world, allowing users to simulate or annotate information relevant to their specific location. AR technology offers a cost-effective solution for training, enabling realistic scenarios without the need for physical props.

Yohan also discusses the application of AR in real-time operations, where it can be integrated with drones to provide live views and enhance situational awareness. Currently, Yohan is piloting this innovative technology with fire and police departments, and is seeking funding to further develop the project.

So take a listen to hear what he has to share!

Head to www.smartfirefighting.com to discover how SFF accelerates innovation for emergency responders, to find out when our next event is, or review our curated resources!

Facebook | Instagram | Twitter | LinkedIn

Chapters

1. 0:00 Intro to the 5x5 miniseries with FirstNet and NIST
2. 0:38 Yohan Baillot of ARCortex joins
3. 1:03 Why he's at the 5x5: the NIST accelerator
4. 1:33 What augmented reality is
5. 2:50 AR for training without props
6. 4:09 AR in real-time operations with drones
7. 5:52 Pilots, funding and next steps
8. 6:59 What he's excited about: fire propagation, drones and AI
9. 7:56 How the NIST program helped
10. 8:28 How to get in touch
11. 9:13 Wrap

Questions answered in this episode

What is augmented reality and how does it work in firefighting applications?

Yohan Baillot explains that AR is a technology that superimposes graphics on top of the real world. It can annotate the real world with location-specific information or simulate realistic graphics like fire or victims. The technology works through mobile devices or see-through eyewear that allow users to view the real world with added graphics.

Hear it at 1:58

How does AR reduce training costs for fire departments?

Yohan Baillot says AR allows users to use their actual fire station or training building but adds graphics that look like fire or victims instead of physical things. This avoids the problem of not being able to put real fire, reduces resources needed, and lets trainers easily change and reproduce the same scenarios for students.

Hear it at 3:04

How is AR being used with drones for real-time wildfire operations?

Yohan Baillot says the tool used in wildfire shows where personnel on the ground are with respect to the wildfire so they don't get circled by the fire, which is one of the causes of death in those environments. The drone provides a live view and AR displays the location of assets like firefighters in relation to the fire.

Hear it at 4:41

What future AR features is the team working on?

Yohan Baillot mentions they want to work on fire propagation so virtual fire on a building can spread realistically, more work with drones and piloting them, using AI to interpret drone and sensor data, and improving scan to CAD for mapping tools.

Hear it at 7:15

Transcript

Machine transcript of the published audio. Names may be misspelled and some sections are missing.

Ep 246 transcript

The Role of AR in Real-Time Operations with Yohan Baillot

Speakers: Kevin Sofen, Yohan Baillot

Source: Whisper medium.en machine transcription of the published Podbean audio (no original transcript found in Drive), lightly cleaned. Speakers were separated by voice clustering and then checked and assigned by hand at every speaker change. Gaps where the model returned no usable text are marked [inaudible m:ss].

[0:00] Kevin Sofen: Welcome to the Smart Firefighting Podcast. In this miniseries, hosted at the 5x5 Conference with FirstNet and NIST, we're diving into some of the most exciting new technologies supporting first responders. We'll talk about how tools like augmented reality, AI-powered drones, and advanced UAV systems are helping fire departments and emergency teams work smarter, faster, and safer. You're going to hear from people on the front lines of and innovation shaping real world stories about how these tools are making a difference. Stay tuned and enjoy. Alongside Yohan Baillot, did I say it right?

[0:41] Yohan Baillot: That's correct.

[0:42] Kevin Sofen: Yohan, so great to cross paths with you again. Welcome to Chicago.

[0:46] Yohan Baillot: Yeah, thank you. Nice city.

[0:48] Kevin Sofen: Yeah, have you been here before?

[0:49] Yohan Baillot: At the second time, yeah.

[0:50] Kevin Sofen: Okay, you enjoy any Chicago food?

[0:53] Yohan Baillot: A little bit, some dish pizza, yeah.

[0:55] Kevin Sofen: Yeah, the deep dish pizza. I try not to eat it all the time because I usually go into like a food coma for like a couple hours after, but it is delicious.

[1:02] Yohan Baillot: Yeah, it is delicious.

[1:03] Kevin Sofen: So the 5x5, this alignment with NIST and FirstNet, why are you here? What brought you here today?

[1:09] Yohan Baillot: Well, we were part of an accelerator with NIST called the Pulse, and that gave us the ability to work a little bit on our business case, working with drone for situation awareness in wildfire and potentially emergency management. So that's why we're here. They invited us to have a presentation of our pitch, as well as a booth so that we can demo our system.

[1:33] Kevin Sofen: Love it. And so when I've known you, and shout out to the one and only Captain Kirk McKenzie, even though he's retired, he's still captain to me, I've always known you as kind of being tip of the spear of all things, augmented reality, AR. So when I'm looking at this emergency response information systems, ERIS, tell me about kind of what is it, and even kind of love to hear some of the AR overlaying kind of what is AR's role within this initiative?

[1:58] Yohan Baillot: Yes. So AR in general is a technology that superimpose graphics on top of the real world. So you can use the technology to either annotate the real world with information that pertain to the location that you annotate, or simulate graphics that reproduce things that look realistic, like fire or victim or other firefighter. And so this technology is using a different platform, a mobile device in which you look through the mobile device, you see the real world and you overlay those graphics on top of it or inside eyewear that are not like the VR eyewear, but that are see-through so that you can see the real world and you add graphics that annotate or simulate things.

[2:50] Kevin Sofen: and I'm looking at the screen now, and I see ERIS operations and training. Let's start it with training. How is this something that can be used, and what's the kind of user experience, the value for someone to use this kind of technology for training?

[3:04] Yohan Baillot: Yeah, so we started with operation and we continue with that, but training became a very good market fit for this technology, because there is a lot of problem in training with cost, ability to put fire, and other issues. So by using augmented reality, we can let the user actually use the real world. But instead of using physical things like a real fire or a mannequin, we use augmented reality to add graphics that look like those things on top of the real world. So that gives the ability for the user to still use their fire station or the training building, but to create a situation that they can change and adapt to its students very easily and can keep reproducing the same training scenario so that avoids the issue with not being able to put fire and allow to reduce the amount of resource and the cost associated with training, for example.

[4:09] Kevin Sofen: Well, for being such a visual experience, you did a great job explaining that with words. You explained the training stuff. I experience a training dynamic in terms of kind of having this ability to have a tablet and being able to have these different 3D assets and kind of laying out different, whether it's a body, whether it's a fire, whether it's broken glass, whatever it may be, and talking through situation awareness. I really enjoy that. I think there's a lot of value in it. I'm also now here looking at, which I believe is the operation side of it, which you showed me earlier. Now talk me through the use case application in regards to using this for real-time operations.

[4:41] Yohan Baillot: Yeah, so we still see some challenge in being able to address the operational side of a structural fire because of the condition and the precision needed in such a close-by environment. But we see a promise in using augmented reality in operation from drone, because when we look at a drone view, we look usually at large distance and sedent, and we are able to achieve and also accuracy in the tracking the drone and asset on the ground themselves. And so that gives us a live view of the environment, for example, the wildfire or the political demonstration, and being able to understand how things on the ground that we see live in the live video relate to things that we display, such as, for example, the location of a bad guy or the location of fire. So for example, that tool used in wildfire will show where the personnel on the ground is with respect to the wildfire so that they don't get circled by the fire. That's one of the cause of death of those environment.

[5:52] Kevin Sofen: And I mean, I've seen it and I've experienced it in different ways. What do you see as next steps in terms of market commercialization from either the training or operations side? And what are some early adopters or case studies examples that you see in the near future?

[6:06] Yohan Baillot: Yeah, we work with a couple of fire departments on an informal basis right now as a continuation of our grant with the NIST, and we are happy to collaborate with more fire departments. We are also exposed to a lot of SMEs through this accelerator with the NIST, the PERS accelerator. So we're still in the phase of piloting and establishing more product market fit. So we are looking for a research development funding and some investor. And we are starting to get a couple of requests for code to pilot our technology in a couple of fire departments and police departments. So things are starting to go in the right direction, but we can use more help.

[6:50] Kevin Sofen: Hey, man, you work in there for decades and people think you're crazy, then all of a sudden they're like, man, this guy, Yohan, he's got it. He's really got it going on. So what do you see in the next 6, 12, 24 months? What are you excited about? And what are some kind of major milestones? Or, I mean, you just mentioned pilots, but as far as technology development, or what are some dynamics or things that you're looking forward to or excited about?

[7:15] Yohan Baillot: Yeah, well, I'm excited to, at some point, get some funding so that we can continue, actually, to do this development. So we are actually looking for that. But some of the things we want to work on are, for example, the fire propagation, so that when we put a virtual fire somewhere on a building, it can propagate realistically. And once we have also more work to do with drones and in piloting them, so those are kind of the activities we are looking to do. And potentially using a lot of AI in interpreting drone data, sensor data, improving scan to CAD for our mapping tools, So there is a lot of work to be done.

[7:56] Kevin Sofen: Yeah, well, no time like the present, and you're doing a great job on that. So I guess my question would be, how has this NIST program accelerator been beneficial to where you're at now and any sort of comments on the whole NIST and working with the team?

[8:11] Yohan Baillot: The NIST has been a great institution to support us over the last two years. So they have been offering us this accelerator as this last step. We were funded by a TABA grant for a year to pilot our technology. So we're looking to work more with the NIST, and that has been a great support so far.

[8:28] Kevin Sofen: Love it. And people want to get in touch with Yohan. How did they do that?

[8:32] Yohan Baillot: Yeah. So you want to go on their website at www.erisxr.com, erisxr.com. and my email is johann.be like boy a-i-l-l-o-t like tom at a-r-cortex.com that's a-r-c-o-r-t-e-x dot com.

[9:03] Kevin Sofen: Yohan, keep beating the drum, keep working hard and I look forward to continue to cross paths and see where you're at next.

[9:10] Yohan Baillot: Thank you, very much for the opportunity.

[9:13] Kevin Sofen: Thanks for listening to this episode of the Smart Firefighting Podcast. I would love to know what you think. Send me a note, send me a DM, post something on Twitter. Let us know what you think. Thanks again and have a great day.