When someone goes missing, time is of the essence. Those first 24 to 48 hours are critical to search and rescue efforts. The missing could be in grave danger, and finding them often requires a fresh trail with evidence that hasn’t been washed away by rain or disturbed by animals or people. A research team based out of California Polytechnic State University, San Luis Obispo (Cal Poly), used their U.S. National Science Foundation (NSF) ACCESS allocation with the National Center for Supercomputing Applications’ (NCSA) Delta to help shorten the time between when a person goes missing and when they’re found.

What if you could use an AI-trained application, one that could take all kinds of data about a search and rescue attempt in real time and analyze it to help predict where someone would be? Experts could help evaluate and adjust the importance of each clue and with the help of the application, direct searchers into more precise locations. Such an application could become a reality very soon.

Franz Kurfess, a professor in the computer science and software engineering department, leads a team of researchers working on this project. In addition to a rotating group of exceptional student researchers, the team has two experts who’ve worked with search and rescue teams for decades, Gary Bloom and Chris Young.

Through his work with the university, Bloom connected with Kurfess to discuss ways to improve current search and rescue methods, which have been mainly paper-driven until now.

“Essentially, the way we’ve been managing searches is we have a standardized set of forms in Northern California, in the Bay Area,” said Bloom. “We run a lot of things through a copy machine and a command post, and information is flowing verbally between individuals. But, if we started collecting the information electronically, we could then start applying AI technologies to try to help us figure out where we’re likely to locate somebody.”

Before any supercomputers were involved in the process, the research team had to create a database of expert search and rescue knowledge using data from past searches. What started as a digitization project, turning printed, typed or written information into PDFs, eventually evolved into a backend database capable of synchronizing with a mobile device that could access the data through a simple user interface.

An essential part of the project is creating an application that can take all this data and make accurate predictions based on its training – that’s the AI element that requires the use of a supercomputer like Delta. Charles O’Hanlon is one of the student research programmers working on this aspect of the project.

“We’re training a diffusion model to predict the location of lost people,” O’Hanlon said. “The diffusion model is conditioned multimodally, that is to say, on different kinds of data.”

The team of research programmers used more than 1,000 GPU hours on Delta to train their diffusion models. That might not seem like much, but one hour on a GPU could be millions of computations, depending on the complexity of the computation. They’re currently working on training neural networks to create heat maps to try to predict a missing person’s movement.

Kurfess, who has been one of the main faculty advisors on the project, has helped provide students with guidance for the compute research aspects of the work. He envisions the end product as a collection of tools that search and rescue teams can deploy.

“There will be a dashboard at the command post that people use to keep track of what’s going on out in the field – what clues are coming in on the map, displaying where the person is likely to be found, where the clues are located, and then also more specialized tools as they need them,” explained Kurfess. “Then we also have a mobile component that people actually out there in the field or in the woods will use, and they can use that to report what they are finding or get information relayed back from the command post.”

However, AI won’t be able to replace experts’ intuition. But what it can do is help the limited number of experts in the field be more productive in every SAR.

“There will always be someone reviewing the suggestions by the AI,” said Bloom. “The AI will take all that data – weather, terrain, the mental state of the missing – to generate more clarity and more clues and narrow down the scope of where we should be looking. With a tool like that, we’re going to be finding individuals more rapidly.”

Cal Poly is part of the California State University system. Its focus is on undergraduate education with the motto “Learn By Doing.” While the institution has ample expertise to build impressive research teams, it lacks the compute power to get this project to the final stage. The ACCESS program and resources like Delta, have been invaluable, something every member of the team attests to.

“Getting access to Delta, and more broadly the resources in the ACCESS program, was a godsend for us,” said Kurfess, “We’ve been cobbling together computing resources here and there, and none of our cobbled attempts work very well. Charles, especially, spent a lot of time trying to figure out how can we use the best resources. To give you an example, we got a generous $15,000 cloud computing credit from a company. This is a lot of money, but once you translate it into powerful compute resources, it evaporates very quickly. And for a non-research university like ours, getting grants is more complicated. There are a lot of hurdles. On top of that, Cal Poly also does not have the other computing resources and the personnel support that we can get through NCSA.”

Kurfess has been a professor for some time, so he has plenty of experience writing grant proposals. One of the things he appreciated the most about his work with Delta through ACCESS was how easy everything was to set up. 

“This has been one of the easiest processes to get something this useful. Of course, getting cloud computing credits is not too difficult. But it’s not as easy as it used to be, and getting something useful out of it takes more work and time. With ACCESS, the proposal took me maybe an hour or so to write, and then I consulted with a few people and then getting approval took about a day or two, and not much after that, we allocated the resources, and Charles was able to get to work with the resources.”

The quick turnaround gave the team a much-needed boost. Having the resources that fast was a boon, but having the technical support from NCSA experts was the cherry on top. “The support that we’re getting from NCSA personnel is excellent,” said Kurfess. “They respond within a very short time, much shorter than our own ITS people, and they also know much better what we need and how to resolve the issues that we have, which is not surprising because that’s what they do all day long, whereas our ITS people have a lot of other things to take care of.”

The team, particularly those who have dedicated their lives to search and rescue efforts, can’t speak highly enough about programs like ACCESS and resources like Delta – resources that help make it possible for them to have such a huge positive impact on the work of search and rescue. 

“When I can use technology to help aid us in finding someone and perhaps saving a life, I’ll do anything to make that happen,” said Young.“A lot of people in the world don’t know what it’s like to have somebody missing in your family. If it’s your family member that’s missing, the most important thing is finding an individual as rapidly as possible,” said Bloom. “NCSA providing resources to us – if we didn’t have outside organizations helping with some of the resources, the cost of compute capacity, the accessibility of the capacity, would just be out of reach, and we’d be extremely limited in what we’d be able to do. What NCSA provides is an enabler for something that’s going to help find lost people more quickly and will save lives, and that should be celebrated.”

You can find a more in-depth story about this research project here: Using AI to Help Find Missing Persons

---

Resource Provider Institution(s): National Center for Supercomputing Applications (NCSA)
Resources Used: Delta
Affiliations: California Polytechnic State University, San Luis Obispo
Funding Agency: NSF
Grant or Allocation Number(s): CIS240458

The science story featured here was enabled by the U.S. National Science Foundation’s ACCESS program, which is supported by National Science Foundation grants #2138259, #2138286, #2138307, #2137603, and #2138296.