National Guardsmen assist residents affected by flooding caused by Hurricane Harvey in 2017. ... [+]
The National Guard has its work cut out. As first responders to wildfires, floods and other increasingly common catastrophes, Guardsmen must rapidly deploy precisely where needed.
This presents one of the highest-stakes challenges imaginable for infotech as a field: Across the many square miles affected, which school, bridge or neighborhood most needs immediate help? Often, lives are at stake.
Google parent company Alphabet has a moonshot factory called X that aims to solve some of the world’s most critical and vexing issues, and X has developed a breakthrough solution—one that uses predictive AI in much the same way that many companies do. Here’s how X did it—and what all business professionals stand to learn about combating uncertainty and risk with machine learning.
The Problem: Tagging Aerial Photos
Both during and in the aftermath of a serious weather incident, drones and manned aircraft gather thousands of aerial photos of the affected areas. These images potentially reveal which buildings and other infrastructure have been affected—but only after each one has been tagged as to precisely what location it's showing. Unfortunately, the images generally lack this metadata.
Tagging the photos by hand tremendously slows the National Guard's response. After an incident, its team usually needs about 12 hours to complete the task. Unfortunately, this process has until now remained a manual one. It's a challenging task to automate since the photos are taken from varying altitudes and at oblique angles.
But this is exactly the kind of problem for which X was designed: The stakes couldn't be higher, yet it requires a technological breakthrough. X's initiative to take on this and related challenges is called Bellwether, described as "the first prediction engine for the Earth and everything on it."
Sarah Russell, who has been running Bellwether since she founded it in 2020, makes the case. "We took this challenge on because we realized that if we could solve it, we'd slash climate disaster response times and multiply the number of lives saved."
The Solution: Matching Photos With Machine Learning
The breakthrough? Matching real photos with artificial ones. Bellwether has synthesized a database of simulated reference photos to use as exemplars. When a real photo matches one from the database, it's tagged—the system then knows precisely where and what it's a photo of. To synthesize the reference images, X tapped Google's wealth of unique geospatial resources, the underlying basis for products such as Google Earth and Maps.
It works. Only a few years after Bellwether formed and began working on this solution, the National Guard is already deploying it in trials and plans to use it for this summer's wildfire season.
With this solution, National Guard team members can immediately peruse the most affected areas and know what locations they're looking at. They can tell which bridges are out. They can query the surveilled areas, "Show me all hospitals." They can issue informed responses immediately, ridding themselves of the processing delays that have held them back for years.
Machine learning plays a central role in moonshots like this—just as it does in more common enterprise systems. After all, matching photos is exactly the kind of inexact process that ML handles well. No match is a sure thing, since the aerial photos don't match up exactly. They each originate from a unique distance, zoom and angle, they're potentially occluded by weather conditions, and the landscape they capture has often been affected, sometimes disastrously.
ML weeds out most of the uncertainty by assigning a confidence level to each match. With many photos coming in, it turns out that enough of them match with high confidence, so the system can provide visuals to operations personnel that cover almost all affected locations, even after dropping the ones that didn’t find a confident match.
This approach is extensible. "Extending beyond our deployment with the National Guard, our goal is to make this kind of service fundamentally easier for a wider group of disaster responders," says Russell. "It can be applied across rescue and rebuild responses to various weather-related phenomena, including heat waves and tornadoes, for example."
The Universality Of Predictive AI
Whether shooting the moon or shooting for more typical enterprise goals, ML's core capacity to generate confidence levels solves operational challenges universally, across industries. Which customers will probably buy? Marketing targets them. Which transactions are probably fraudulent? Banks block them. Which addresses will probably receive a delivery tomorrow? UPS plans for them.
This well-versed paradigm—driving large-scale operations with ML’s predictions—has a name: predictive AI. It's the practice of systematically filtering out the cases that show less confidence and taking action for the more confident cases that remain.
So, how confident is confident enough? It depends. Each project must determine the best choice of decision threshold based on practical necessity. For example, the National Guard needs photos that have matched with very high confidence. In contrast, marketing and fraud detection can afford to target many cases that don't pan out—an unavoidable part of the numbers games that those kinds of operations inevitably play.
To put it another way, predictive AI reduces uncertainty by quantifying uncertainty. Bellwether is working to extend this prodigious approach in other ways that will also diminish the damage caused by climate disasters, such as by predicting where the most lives could be saved—which affected areas should be the highest priority for evacuation assistance—and by predicting environmental incidents before they happen.
"ML has become the new paradigm for Earth sciences," says Russell. "Until recently, for example, hydrology mostly forecasted floods with site-specific models. Now, with ML, the best models are developed using data taken from across locations—flood behavior on the East Coast of the U.S. can be used to forecast floods on the West Coast."
