Episode 248: Automating Tasks for First Responders with Zenext AI with Swarnamouli Majumdar

Episode notes

An assistant for first responders:

Hello Smart Firefighting Community!

Welcome to another episode of covering real world innovations via interviews with fire service and technology industry experts that empower YOU to develop your very own Smart Firefighting strategy!

In this episode:

A voice-based AI assistant designed to automate repetitive tasks for first responders

How does Zenext AI offer a hands-free solution for public safety officers?

How do real-time information and location-based services improve efficiency and safety for first responders?

Hear from Swarnamouli Majumdar - the Founder and CEO of Zenext AI.

Swarnamouli introduces Zenext AI’s voice-based artificial intelligence assistant designed specifically for public safety. The technology offers a hands-free solution for first responders, automating repetitive tasks such as license plate checks, querying databases, and documenting incidents.

Zenext AI is accessible via both mobile and web apps, providing real-time information and location-based services to enhance efficiency and safety. The company is also exploring the integration of drones for fire safety operations. Having validated their product with multiple county police departments, Zenext AI is aiming to start commercialization in 2025.

So take a listen to hear what she has to share!

Head to www.smartfirefighting.com to discover how SFF accelerates innovation for emergency responders, to find out when our next event is, or review our curated resources!

Facebook | Instagram | Twitter | LinkedIn

Chapters

1. 0:00 Intro to the 5x5 miniseries with FirstNet and NIST
2. 0:38 Swarnamouli Majumdar of Zenext AI joins
3. 0:56 Zenext's start in a NIST PSCR contest
4. 1:20 A voice-based AI assistant for public safety
5. 2:19 The user experience: a license plate check case study
6. 3:23 Mobile and web apps, with CAD and RMS integrated
7. 4:06 The NIST program and the move into drones for fire
8. 6:01 Proof of concept and commercialization in 2025
9. 6:46 What excites her most
10. 8:01 Indiana University and mentor Dr. Sunny Kirkley
11. 9:13 Final words: a portal to connect innovators
12. 10:13 Wrap

Questions answered in this episode

What is Zenext AI and what does it do for first responders?

Swarnamouli Majumdar describes Zenext AI as a voice-based artificial intelligence assistant for public safety that uses a heads-up, hands-free approach. It helps first responders with tasks like license plate checks, registration plate checks, voice command reporting and documentation, and dynamic location sharing, reducing dependency on handheld radio devices and dispatch.

Hear it at 1:20

How does an officer use Zenext AI for a license plate check?

An officer can run a license plate check without leaving the car or typing on a laptop. They simply talk to their radio using a trigger word like 'hey, Zenext, can you run a license plate check for me?' and the system handles it hands-free, allowing the officer to keep both hands available for safety.

Hear it at 2:49

How is Zenext AI being applied to fire safety?

The team is developing a dashboard for fire professionals that uses voice-controlled drones. The drones take images and feed them to the dashboard so fire officials can see where help is needed most, such as identifying a disabled person in a basement or a child on the first floor, so they can rescue specific people without searching the entire building.

Hear it at 4:23

When does Zenext AI plan to launch commercially?

Swarnamouli Majumdar says they are aiming to start commercialization in 2025. They have already done proof of concept work with county police departments in Indiana and Illinois and have expressed interest from security companies for their voice AI reporting feature.

Hear it at 6:12

Transcript

Machine transcript of the published audio. Names may be misspelled and some sections are missing.

Ep 248 transcript

Automating Tasks for First Responders with Zenext AI with Swarnamouli Majumdar

Speakers: Kevin Sofen, Swarnamouli Majumdar

Source: Whisper medium.en machine transcription of the published Podbean audio (no original transcript found in Drive), lightly cleaned. Speakers were separated by voice clustering and then checked and assigned by hand at every speaker change. Gaps where the model returned no usable text are marked [inaudible m:ss].

[0:00] Kevin Sofen: Welcome to the Smart Firefighting Podcast. In this miniseries, hosted at the 5x5 Conference with FirstNet and NIST, we're diving into some of the most exciting new technologies supporting first responders. We'll talk about how tools like augmented reality, AI-powered drones, and advanced UAV systems are helping fire departments and emergency teams work smarter, faster, and safer. You're going to hear from people on the front lines of and innovation shaping real world stories about how these tools are making a difference. Stay tuned and enjoy. Here at the 5x5 show floor alongside Swarnamouli Majumdar. How you doing?

[0:43] Swarnamouli Majumdar: I'm doing great. Thank you.

[0:45] Kevin Sofen: Welcome to Chicago. How is Chicago treating you?

[0:47] Swarnamouli Majumdar: Chicago's treating us great. This has been multiple times in the city. And in fact, the journey of Zenext AI started from Chicago.

[0:53] Kevin Sofen: Oh really?

[0:54] Swarnamouli Majumdar: Yeah.

[0:55] Kevin Sofen: How so?

[0:56] Swarnamouli Majumdar: We were part of a regional contest hosted by NIST PSCR in 2019 and 2020 at the Motorola headquarters and we were the first prize winners in that contest.

[1:08] Kevin Sofen: That's amazing. Well, quite the journey from then in 2019 to now 2024. So what's happened since then? And tell us about kind of where Zenext AI is, like what is it and what's the value it brings?

[1:20] Swarnamouli Majumdar: Stepping back a little bit, we founded this company following a series of challenges which we won with the NIST PSCR back in 2020 with the Tech2Protect challenge. We were the first among the nation to innovate a voice-based artificial intelligence assistant for public safety. So it's basically a heads-up, hands-free approach in which we aim to do away with the repetitive tasks, which first responders have, like the handheld radio devices and their laptop computers, in providing a voice command-based AI assistant to be able to help them do license plate checks, registration plate checks, talk to officers, query databases, do voice command reporting and documentation among the few, and also location-based services like dynamic location sharing among officers. So all to alleviate the problems that they have with officer safety, just-in-time help, and the dependency on the radio devices and dispatch for every aspect of theirs in emergency situations.

[2:19] Kevin Sofen: That's amazing. And just to continue to expand upon that, You talked about the sort of using voice hands-free to be able to do something like license plate detection or where's my counterpart? Tell me a little bit about the user experience about how someone experiences Zenext. Is it sort of like their ability to sit in the car driving and have their, to be able to ask a question and it sort of is said back to them or it shows up in the tablet? Like what's the user experience like? Maybe give me a mini case study example if you can.

[2:49] Swarnamouli Majumdar: Yeah, for instance, I'll give you a case study first. So there is a suspicious car in front, and the officer wants to have both his hands on his arms and not to look into his laptop, typing in a registration plate check or a license plate check at the end. So what he can do with Zenext, he can just talk to his radio. By talking, I mean literally having the trigger word like, hey, Zenext, can you run a license plate check for me? And then without needing to step out of his car, He can do it at the comfort of his car. With Zenext, the user experience, as you asked, is in two formats. Right now, we have a mobile app in iOS and Android, and also we have a web app. So that way, if the officer needs to see also what he is checking, that is easily convertible to what he can see. But that is an optional scenario. He can do everything hands-free because we have databases integrated, like the CAD, RMS of that specific county integrated. So he wouldn't need to even lift a finger if he wants to do that. And he is free to use both his hands for safety, for, you know, if there's any much situation in front, he needs to use both his hands.

[4:00] Kevin Sofen: Yeah, I mean, it's important to be able to have your hands and not focus on using your radio or calling if you don't have to.

[4:06] Swarnamouli Majumdar: Yeah, sure.

[4:07] Kevin Sofen: And so this whole program that you've been a part of, you've gone through a few different NIST PSCR initiatives. This particular one, where this is cohort of six teams, six companies, how has the experience been? What's it been like and ultimately getting to hear today? We'd love to hear about some of your experience.

[4:23] Swarnamouli Majumdar: Yeah, sure. So over the years, we've done extensive user research in different county police departments and what the pain points are. We sort of sophisticated our product, made our more compulsive and robust, and now we are more heading into using drones for fire safety with voice control option. So in that case, we have been doing research on our existing system of voice AI, and we have developed a dashboard for fire professionals in which the drones would be able to take images and feed back to the dashboard and the fire officials would be able to know for residentials or commercial areas where the help is needed the most. For instance, if there is a disabled person in the basement, if there's a child in the first floor and if there's a parent, there's no one else in their home, they can possibly only go to those areas, rescue them and come out. In this way, also, they are saving their own lives because they do not need to get to all parts of the house in search for people when there are no people. And there are only people in specific parts. So it's a safety to them. It's a just-in-time help. And also, it's quick. So that's where we are heading. And for this post-accelerator program right now that we are in, it helps us really understand where the drone technology are headed. For instance, we've already started conversations with fellow contestants in this competition, in this accelerator program, and we have learned a lot. And after this, we are more headed into collaborating with them with different partners here to understanding how we can integrate our solution.

[6:01] Kevin Sofen: Yeah, that's amazing. And I kind of, at this point too today, it's June 2024, where are you at with end users or commercialization and actual having first responders deploying this in the field?

[6:12] Swarnamouli Majumdar: So we have validated and done a proof of concept with a few different county police departments in Indiana and Illinois, notably the Hamilton County Police Department. We have got expressed interest in a few security companies in which we would be aiming to deploy our voice AI reporting and documentation feature as a first MVP for them. So for commercialization, we are aiming to start that in 2025 because we still have some nits and pieces to do in user research and sophisticated our product before we sort of launch it to the market.

[6:46] Kevin Sofen: That's awesome. And as far as where you're at now and where you're going, you just talked about commercialization. What excites you most about what you're bringing to market and what should first responders be thinking about in regards to how to work with and experience your technology?

[7:01] Swarnamouli Majumdar: What we aim to bring into the market is Zenext as a product is a software as a service in which the first responders do not, like the police officers, they do not need to do anything with their radios or their laptops. So we are not doing a change management. We are sort of introducing a new hands-free technology which can automate their tasks. So basically they can learn as they go. So it won't be like, if you adopt Zenext, you've got to forget what you're doing. You can continue what you're doing, but you're going to be more safer if you use Zenext AI. That's one thing. And as we move ahead, we are more excited with the fact that we have started this initiative with a PSCR challenge and which is like, we were straight out of school our masters and we were so engaged in this like this has become the number one thing in our lives like this is like as we call our you know first or the second baby that we have so we're pretty excited as this is a very niche market and you know we are very motivated to change the life of responders and how their safety is concerned.

[8:01] Kevin Sofen: I love it and you mentioned too you're are you doing it or you've worked at Indiana University or you went to UIU?

[8:07] Swarnamouli Majumdar: So I did my masters in data science in Indiana University. The team which we founded was full of user experience researchers and designers. We all studied in different cohorts and we met Dr. Sunny Kirkley there who had been our mentor and who had been part of the team back then and now he's our mentor and advisor.

[8:25] Kevin Sofen: Love it. Sunny is like that loving uncle and father figure to all of us and just truly an amazing human. Just a personal thought on him.

[8:35] Swarnamouli Majumdar: So basically I tell my parents that even if I, you know, make a mess, even you can scold me, but you can scold us, you know, me or you can miss me, but then Sunny's such a person, we've pissed him off multiple times. He always maintains his cool and we know that if we mess up, we can go to him. If we do a good job, we can go to him. Like he is the number one mentor we've ever had. And even now when he's not part of the team directly but advising us, if it's 11 p.m. and we need some advice, By 6 a.m., eyes closed, we know we'll get it. Like, he'll be ready to, you know, flex his hours and, you know, be there for us.

[9:09] Kevin Sofen: Yeah. He's an incredibly selfless and wonderful human, so I love hearing that. And so any, as far as, any sort of final thoughts or ideas or comments or words of wisdom that you could close us out with here today?

[9:21] Swarnamouli Majumdar: Yeah, I think that with such a conference, it would be very essential if we, all like-minded professionals, because we're all in public safety and we're all innovating technology for them. If there could be a portal or domain in which we could connect, so we could know if we could collaborate together, like for instance, with any of the companies, startups, ventures out here, it would be really beneficial because most of us are doing stuff which somehow fit into each other's puzzles. So if we could cross each other's paths and you don't have just a one-on-one session of have a portal in which we can choose, okay, here we can talk, or even gain feedback. It would be really beneficial for everyone to grow.

[9:58] Kevin Sofen: and Sunday in Mexico with the first challenge. But, you know, it's called the first challenge, but it's no longer the first challenge. It's the first community, the NIST community. So kudos to Sunny and kudos to the work that you're doing. This is something that has a lot of merit and legs to it. And I'm excited to see whether your team continues to take it.

[10:13] Swarnamouli Majumdar: Yeah. Thank you so much.

[10:14] Kevin Sofen: First, until next time. See you. Thanks for listening to this episode of the Smart Firefighting Podcast. I would love to know what you think. Send me a note, send me a DM, post something on Twitter. Let us know what you think. Thanks again and have a great day.