Connect to share and comment

The global market for drones is booming. But what does the coming arms race mean for US national security interests — and the future of warfare? GlobalPost correspondents report from critical locations around the world, from Israel to Iran to Yemen to Brazil — where unmanned aerial vehicles are radically transforming combat and surveillance.

Deadlier drones are coming

How smarter robot warplanes now in development could attack targets on their own.

forces in Europe, told The Los Angeles Times

Greater robot autonomy could herald a major expansion of the drone war. What's less clear is the potential human cost.

The Teal Group, an aerospace research firm, predicted that worldwide military UAV spending could almost double over the next decade from $6.6 billion this year to $11.4 billion in 2022, in constant dollars. A Predator costs around $10 million, not counting the ground stations; the price of a single Reaper is triple that

Today only the US, the UK, Italy and Israel operate armed UAVs — all of them propeller-driven. But scores of nations possess unarmed flying robots. And France, Germany, Sweden, Russia, China and Iran are all developing their own weapons-carrying models, including some with jet power.

In the US, no fewer than four jet-propelled, armed UAVs are in testing and could begin to replace the Predators and Reapers in just a few years. Jet propulsion combined with swept wings translates into higher top speeds and better reaction times in chaotic battle conditions.

So far, only in laboratory simulations at labs like Boeing, Northrop Grumman and MIT have highly autonomous armed drones been cut loose to wreak havoc all on their own.

It's no small task programming a robot to handle every possible problem it might encounter during a combat mission — everything from bad weather to changes in the landscape, unexpected civilians on the battlefield and new types of camouflage and defenses on the part of the enemy.

Robot developers are trying to build massive "what-if" software databases detailing out every possible scenario. Gathering the data is a painstaking effort that's only now getting underway, says Stefanie Tellex, who works alongside Teller and Cummings at MIT. "We're seeing the beginning of efforts to apply large data-sets to robots in order to increase their robustness and level of autonomy."

As drones become capable of handling more of the maneuvering and scanning on their own, human crews should intervene only when the drone encounters a problem it knows it cannot solve. "We don't know a lot about how to tell a machine how to handle surprises," says Randall Davis, another MIT robot developer.

So for the near term developers are focused on devising more, but not totally, autonomous drones requiring less human control.

That could free up the operators to do what people do best: make educated guesses and find creative solutions to unexpected problems — but only when necessary. Robots "aren't going to replace the need for a thinking human being to make decisions that are influenced by experience in a wide range of situational considerations that you just can't program into a machine," Carl Johnson, a Northrop vice president, told GlobalPost last year.

But it's better to keep the drone on constant duty and call in the human beings only rarely.

"Humans contribute the things humans are good at, and robots contribute what robots are good at," is how MIT's Seth Teller describes the dynamic to GlobalPost.

Mike Patzek at the Air Force Research Laboratory says he's working on simplified drone control systems that do away with joysticks, keyboards and multiple computer screens in favor of a single elegant screen combined with two-way voice communications.

A drone operator could simply talk to his robot and the robot would talk back. That technology, which is still in the conceptual phase for UAV operations, is nevertheless in daily use, in an admittedly rudimentary form, in Apple's Siri app for smartphones.

Even as human operators progressively turn over more and more control to the robots, one task in particular will, for now, almost certainly remain in human hands: deciding when a robot should launch a weapon.

"Even though it's possible for a [UAV] to find a target, identify it and give those coordinates electronically to a weapon, it won't do that unless it's told to," Johnson says. "The technology is there, but there is still a need for a human in the loop."

That's not just Johnson's preference: it's government policy that a human operator must approve every weapons release by a drone. In practice, that means a UAV pilot, steering his robot via satellite, literally pulling a trigger to tell the machine to open fire with missiles or bombs.

The reasons for this policy are legal and ethical. "If a UAV is nearly fully autonomous and puts a bomb on a school bus and not a supply truck, which gets held up for the penalty?" asks one Boeing drone developer who spoke on condition of anonymity.

"The absence of human intervention during the weapons release process proves problematic when determining who is to be held accountable following violations of the law of armed conflict," Navy Lt. Cmdr. John Klein explained in a 2003 article

The Air Force is now mapping the policy changes necessary to clear the way for self-directing, armed drones. "Authorizing a machine to make lethal combat decisions is contingent upon political and military leaders resolving legal and ethical questions,"it stated in its 30-year drone plan

Despite the huge obstacles to building fully autonomous killer robots, the military is already thinking over the implications — in essence, clearing the airspace for these more lethal drones to eventually take flight.

Amoral Machines

Highly autonomous robots could pose big problems, and not just legally, Stanford researchers Calo and Lin warn. While remote, there is a chance that a highly sophisticated drone could go rogue in combat. How this could happen has to do with the software that could guide future robots' thinking.

One way to achieve the machine "reasoning" — that's MIT professor Cummings' term — is to program a robot with what Calo calls "genetic algorithms." These sophisticated computer codes refine themselves through trial and error "until they arrive at the best way of doing something," Calo says. "Sometimes the resulting behavior is truly emergent."

"Emergent" is academic-speak for unexpected and amazing.

"Autonomous robots are likely to be learning robots, too," Lin says. "We can't always predict what they will learn and what conclusions they might draw on how to behave."

Genetic algorithms could mutate a smart but obedient robot into something uncontrollable. The worst case scenario is that the Pentagon, CIA, other government agencies and allied armies equip themselves with cutting edge drones that, in teaching themselves to find and kill militants, also learn bad habits. Instead of only attacking men wielding weapons, the robots might decide to kill all men or boys, too.

"If you combine the possibility for emergent behavior with weapons systems, that would be problematic," Calo says.

It's not inconceivable that increasingly sophisticated drones will soon require little more than a few words from a human operator when it comes time to fire a missile or drop a bomb against a target that the robot located itself.

"We're reasonably confident that a human can act ethically, to distinguish right from wrong, but we have no basis yet for this confidence about robots," Lin cautions.

Today roughly a quarter of all the people killed in human-controlled drone strikes are innocent bystanders. In the more optimistic scenario, smarter drones could reduce the percentage of civilians among the dead. But more frequent robotic strikes — a likely consequence of improved technology — could mean an overall increase in the number of innocents killed. 

For all the thousands of people — many of them innocent — that have been killed so far in America's escalating drone war, the real bloodletting could still be in the future. Historians may judge the first decade of lethal, but rudimentary, drone strikes as a prelude to much more sophisticated robot warfare whose efficiency translated directly into more attacks ... and whose autonomy risked unleashing rogue killer robots on an unsuspecting world.