Satellite warriors: Today's geospatial imagery provides a high-resolution view of global strife

Arms control and nuclear nonproliferation expert Jeffrey Lewis is pretty sure he was one of the first civilian analysts to realize that Russia’s 2022 invasion of Ukraine had actually started. Poring over radar satellite images from Capella Space in the days before the conflict, he and colleagues at the Middlebury Institute of International Studies at Monterey had already observed military camps in Russia and in Belarus at those nations’ borders with Ukraine, when they noticed armored vehicles lining up near Belgorod, a Russian city close to Ukraine. Soon afterwards, a colleague found a TikTok video of said military vehicles—it was Wednesday afternoon on 23 February 2022 in California, and the researchers were certain war was impending.

“It was this shift of the vehicles from being parked up as though they were going to be there for a long time, to being lined up on the road [that convinced us],” Lewis tells Photonics Focus. “So, we contacted a number of reporters and said, ‘Look, the invasion is going to start in 24 to 48 hours,’ but the reporters thought that the satellite image looked blurry,” and they couldn’t do anything with it.

Lewis and colleagues immediately turned to Google Maps and, using its traffic layer to monitor the road conditions from Belgorod to the Ukraine border, were startled to see a telltale red line of congested traffic exactly where they had located the military convoy from the satellite data and TikTok video.

“If you see a random traffic jam in Russia it doesn’t mean anything, but when you already know there’s a large, armored unit preparing to drive into Ukraine, that’s different,” says Lewis. “It was nearly 3:30 am in Russia—are there often traffic jams at this time on the road from Belgorod? No.”

For the next hour, the Middlebury researchers watched this “traffic jam”—which was actually roadblocks—slowly stretch towards the border; by 5:00 am in Russia on 24 February 2022, President Vladimir Putin had publicly announced his assault on Ukraine. “For me, the satellite image was enough, [to make a call that the war had begun] but I understand the art of interpreting a satellite image is a peculiar skill,” reflects Lewis. “We posted [our findings] on Twitter and the invasion started a couple of hours later.”

Lewis’ use of complementary data sources to discover Putin’s invasion is a remarkable example of “tipping and cueing”—a method used by geospatial intelligence (GEOINT) analysts to build a pattern-of-life understanding of how people or organizations behave. Over the years, he has repeatedly combined satellite data with myriad geographic data sources in this way to track nuclear weapons development in North Korea, rocket-engine testing in Israel, space launches in Iran, and more. “We can also use aircraft and ship tracking data, and the missile or space launch notices given [by authorities] to air crews and mariners to avoid air collisions,” he says.

On 2 August 2023, VIIRS sensors captured a fire caused by a Russian drone attack near Odessa, Ukraine. Photo credit: BlackSky

According to Lewis, while such work relies on satellite data—be it, say, visible, radar, or infrared imagery—and other types of data, the lion’s-share of analysis hinges on inference. Researchers are looking for patterns and deviations in patterns, or anomalies, so they can infer what is taking place. “We don’t ever see a nuclear weapon or missile. We’re lucky if we see a vehicle carrying one of those things,” says Lewis. “Instead, we’re studying infrastructure. If a factory gets bigger, you can work out that perhaps more of something is getting manufactured. Or if a missile base gets rebuilt, you can infer a new kind of missile will soon be received.”

“Look at the Cuban Missile Crisis—President John F. Kennedy nearly ended the world over the presence of missiles in Cuba, yet they didn’t even have a single picture of a missile,” he adds. “It was all pattern-of-life.”

 Scott Herman, geoint expert and chief product officer of geospatial firm Maxar Intelligence, is also no stranger to pattern-of-life. Having spent much of his career developing and deploying remote-sensing and geospatial-analytics systems for national security, he emphasizes the importance of understanding pattern-of-life and identifying anomalies in imagery analysis.

“It’s one of those mantras—‘anomalies make money,’ and they are what people care about,” he says. “Did the tanks leave the garrison? Did we see more fighter planes on the airfield? Have the Russians gotten too close to the border in Ukraine? If we can understand an activity’s pattern-of-life and its operational tempo, then we can look for changes in that tempo.”

The nuances that can be detected in pattern-of-life and operational tempo have never been more subtle. The proliferation in different types of Earth observation satellites and sensors, and their revisit rates over the same point on Earth, mean a range of image resolutions and spectral bands can be blended to track more and more objects over larger areas. Few will forget the true-to-life color, and now iconic, imagery of the 40-mile Russian convoy approaching Ukraine’s capital, Kyiv—and the images of devastation and mass graves at the besieged port city, Mariupol, that followed. This data was all captured by Maxar Intelligence’s WorldView-3 satellite, which has 29 spectral bands, including a panchromatic band that falls within the visible-light range, eight multispectral bands that reach into the near-infrared, and eight short-wave infrared bands.

But satellites with even more spectral bands are launching. While multispectral cameras can intricately map coasts, vegetation, and landforms, hyperspectral satellites from companies like Kuva Space, Open Cosmos, and Pixxel, will record data from hundreds of narrow bands over a continuous spectral range for even more detailed characterization.

“The advantage of having more bands is you can better characterize what is happening on Earth,” explains Lewis. “If I was to look at a mine, then hyperspectral data could help me to characterize the mineral deposits around it so I could say, ‘Oh look, that’s a probably a uranium mine’.”

Beyond hyperspectral imagery, Herman is also keeping a close eye on the latest high-resolution thermal satellites suited to monitoring, say, industrial power and petrochemical plants. As he reflects: “Small satellites with thermal sensors are going up, and this gives us another layer of understanding. For example, with [Maxar] satellites I can see that a facility has eight blast furnaces, and then, with thermal imagery, I could see that three of these are turned on and five are turned off.”

But without a doubt, imagery from synthetic aperture radar (SAR) satellites has proven indispensable time and time again. SAR satellites reflect radio signals off Earth’s surface to track activities through cloud cover, thin canopies, and at night, and they can scan large areas while reaching sub-meter resolutions. In recent months, multiple news outlets have reported how images from SAR satellites indicate Russian ships to be covertly exporting food across the Black Sea from Crimea. And Lewis used SAR images to observe the Ukraine invasion on that cloudy February 2022 morning in Russia.

Speaking generally, Herman says: “If you were to catch the glint on the radar of a particular ship, you could then use that tip to cue one of our exquisite optical satellite sensors to go take a closer look and see what’s happening on the ship itself.

“We can use social media to tip satellite collection and then use that satellite [imagery] to tip and cue investigations using, say, open-source data,” he adds. “This multisource data fusion is a really important trend right now: When we solve problems now, it’s not simply what’s in the picture, right?”

And unquestionably, this is where GEOINT analysis is really changing. The data sources—satellite or otherwise—available to understand pattern-of-life in any defense scenario have now mushroomed to such inhuman scales, analysis needs a helping hand from artificial intelligence. Indeed, since the Russian invasion, Ukraine has been described as a living lab for AI-based warfare.

AI algorithms are being trained to analyze multiple large datasets and provide critical information in minutes, rather than hours or days—think of it as GEOINT on amphetamines. Computer vision can mine satellite and drone imagery, using deep learning to detect and classify objects such as aircraft at a military airfield. Meanwhile, natural language processing will extract information from reams of unstructured text in social media, reports, industry journals, and other intelligence.

Critically, numerous organizations, including Maxar, other commercial satellite firms such as BlackSky and Planet Labs, and government bodies, including the US National Geospatial-Intelligence Agency, are developing AI platforms to analyze all of this different data and then fuse results to create a multilayer picture of what’s going on, on the ground, and even forecast likely events. “Tipping and cueing used to happen with rooms full of people but now takes place very quickly at massive scale and sometimes with no human intervention at all,” says Herman. “We need to quickly validate what’s happening on the ground from, say, social media, with satellite data. But how can you establish veracity against that? At scale, it has to be with artificial intelligence.”

Patrick O’Neil is chief technology officer at BlackSky and, like Herman, one of his key roles is to monitor activity around the world and orchestrate satellite image collection in light of major crises, be they man-made or natural disasters. As he puts it: “What’s interesting about satellite imagery is you can image the best of humanity and the most beautiful locations on the planet as well as the worst of humanity and some of the most devastated places on the planet.”

Maxar WorldView-3 satellite image of the 40-mile convoy approaching Kyiv in the early days of Russia's invasion. The image was captured on 18 February 2022. Photo credit: Maxar

“So, we try to take a measured approach to the data we release,” he adds. “For example, in the early days of Ukraine, we were careful about what we put out, so it didn’t fall into the wrong hands.”

From the outset, BlackSky has focused on high-frequency monitoring, using its visible-light satellites to capture dawn-to-dusk imagery 15 times a day. Many satellites orbit Earth via the North and South poles, collecting imagery across the entire planet once a day. But BlackSky’s constellation follows the equator, with high revisit rates that can provide an hourly monitoring service across thousands of worldwide locations. Unsurprisingly, the volumes of data generated are huge. “We knew we needed a sophisticated AI system to make sense of all of these pixels,” says O’Neil.

Like other firms, BlackSky has automated the re-tasking and repositioning of its satellites to collect multiple shots over a single location ready for AI analysis. This quick-response setup delivered early data on a Chinese naval base in Cambodia back in February 2023—imagery and AI revealed land clearance and rapid construction. However, to track conflict in Ukraine, BlackSky has also been drawing on open-source data from VIIRS—visible infrared imagery radiometer suite—sensors aboard US NASA and US National Oceanic and Atmospheric Administration (NOAA) satellites.

“We can identify hotspots for combat based on a thermal signature, and then automatically task our own satellite constellation to take higher resolution images of that activity,” says O’Neil.

And this is perhaps how more GEOINT analysis is destined to take place. More and more analysts are now tapping into open platforms from the likes of the National Aeronautics and Space Administration, NOAA, and the European Space Agency (ESA), to combine satellite images with other data and AI algorithms, to track change in conflict zones.

Ollie Ballinger, a lecturer in geocomputation at the Centre for Advanced Spatial Analysis, University College London (UCL),  has built numerous open-source AI tools that have been trained on open satellite data. For example, he recently trained neural networks to scan publicly available SAR satellite imagery from ESA’s Sentinel-1 for covert cargo swapping between large ships at sea. Large ships have to be fitted with automatic identification systems and must broadcast their location to prevent collisions. So, Ballinger’s method can identify all ship-to-ship transfers using AI on satellite imagery, and he can then check the GPS signatures of these ships—in some instances, there are more ships visible in the satellite imagery than can be detected via their GPS signals. As he confirms: “This often happens when [ships are] moving illicit cargo such as sanctioned Russian oil or stolen Ukrainian grain.”

Ballinger has also developed damage detection algorithms to map infrastructure damage in both Ukraine and Gaza from Sentinel-1 SAR imagery. Here, he grouped satellite images of locations taken during war-torn months and compared these with a year’s worth of data from before the war. As he points out, rubble has a very different profile to a standing building, so the way in which these objects scatter the SAR satellite signal is different—and his algorithms detect this.

“Satellite imagery works well with machine learning as most satellites have a consistent orbit and look-angle, so they pass over the same point on Earth at the same time of day, observing the same location from the same angle under the same illumination conditions,” he says. “This is really a dream case for object detection and other AI methods.”

Looking to the future, Ballinger also highlights how he and  like-minded analysts are increasingly joining forces to share datasets. “Our work isn’t profit-driven, and damage detection has clear benefits for those trying to respond to emergencies more effectively,” he says. “I really think we’re just beginning to scratch the surface of the insights that we can extract from synthetic aperture radar.”

The same can undoubtedly be said of other satellite technologies and GEOINT analysis in general. At a time when militaries around the world are exploring—and possibly using—AI to select bombing targets, new tech and tools clearly can’t come a moment too soon. But how soon?

“We’re already pretty good at finding things in imagery and combining that with other data to get some understanding of pattern-of-life, but we haven’t reached this holy grail of insight, where you understand the context and know what course of action to take,” says Herman. “Over the next five to 10 years, we’re going to apply artificial intelligence to get a better understanding of what we’re seeing so we can have this insight. This is absolutely the objective of our industry.”

Rebecca Pool is a UK-based freelance writer.