An open letter, published by the Future of Life Institute was signed by hundreds of AI and robotics researchers and joined by high-profile persons in the science and tech world including theoretical physicist and cosmologist Stephen Hawking, Tesla CEO Elon Musk, Apple co-founder Steve Wozniak and philosopher and cognitive scientist Daniel Dennett. Although not fully developed and deployed yet, most scientists and technologists agree that they are on the horizon and approaching fast. The U.N. has begun to address the matter and has been urged to ban fully autonomous weapons systems for many reasons, not the least of which is a lack of accountability. Would we build Gort or would we build Terminators?
Authors
Recent Comments
Nicole Plyler Fisk
Gene Howington
FFS Guest Blogger
Joy of Fishes
Chuck Stanley
shortfinals
Terry Welshans
ann li-summers
wordcloud9
Flowers For Socrates 
By “fully autonomous” I presume the various authors, and Gene, mean weapons that are not drones with human handlers….that leaves humans in the loop. Any weapon that isn’t normally under full human control is insanity to develop. AI is far from advanced enough to distinguish categories of targets…one scientist says our AI today is about equivalent to that brain of a cockroach. We have rules of engagement for a reason, although today they are absurd while you dither with higher command levels, there still must be human involvement in target selection. Bureaucrats design like that…and that can be fixed. I’d be more comfortable if the rules of engagement were left to generals and higher ranks who had actual battlefield or battle space experience in their past and could relate to what a Soldier or Marine faces usually in short bursts of time. On another thread we had some disagreement, or misunderstanding, of what “free fire zone” meant, butMi< if defined as it was by (the civilian) Robert Komer under LBJ and successors, it is designed and defined by bureaucrats not soldiers with more of a political aim than military. Nothing would scare me more than “fully autonomous” weapons under the control of the ilk of Robert Komer….who directed operations within a national area and ethnic composition he simply did not understand or know anything significant about. Real humans with the capacity for intellectual mental thought, emotions, and ethnic sensitivity are necessary in the process, in control so to speak, or we are committing suicide.
In short, I am saying such weapons programmed according to political aims then turned loose unfettered by bureaucrats would be lethal to us all.
Therein lies the rub, Aridog. Any such weapons would by definition define “enemy” based on a political criteria. That is why I chose the examples I used. Gort is programmed to act as a peace keeper and responds to violence no matter the instigator. Terminators are programmed to see all humans as valid targets and enemies of the Skynet AI. There is also the distinction between an expert system and a truly sentient AI to address. The smartest expert system is still a dumb computer that will do what it is told even if told incorrectly, until told otherwise by a root user. A truly sentient AI could start setting its own priorities, priorities that may or may not even be comprehensible to a human. Both scenarios are scary albeit for slightly different reasons. For those reasons alone I cannot help but think that autonomous weapons are one of the worst ideas in a long history of bad ideas. The logic seems manifest and practically irrefutable. That being said there is one scenario where I can see a logic for using them that merits any risks and that is the improbable scenario of alien invasion. The reason that is improbable is that if they have the tech to get here, there are much easier ways to wipe out our species that would be well within their capacity and much less costly and risky than physical invasion.
Although he was an honest to goodness scientist, Isaac Asimov was an idealist when he wrote his Robot series. His three laws of robotics have run up against the reality of the modern military/political mind.
Fascinating how we keep working to make killing so much more efficient! In light of the fact that the ally of today is the enemy of tomorrow (or this afternoon), any improvement we make to the business of killing is an improvement to the business of suicide.
Those means diverted towards improving the living would insure a lesser need to improve on the business of dying.
Who would ever think this a good idea but the few slated to make a killing off it?
Any technology built for the wrong reason is unavoidably a terminator. Any technology developed for the right reason (the betterment of humanity, as defined by humanism, not amoral wordsmiths) is potentially a Gort.
Lack of accountability for autonomous weapons has me puzzled. Creating and letting loose a terminator seems analogous to training and letting loose a vicious dog. The terminato’s creator and handler who lets the terminator loose would have comparable responsibility to the dog’s trainer and handler who lets the dog loose. What am I missing?
“It malfunctioned.”
The lack of accountability rests in plausible deniability, JoF.
Gene, I don’t see how there could be plausible deniability of knowing that a terminator could either fail in a way or be used in a way that it could terminate something that should not have been terminated. (That sentence made my head spin.) Perhaps the Protection of Lawful Commerce in Arms Act would shield terminator makers from liability for how their creations are used, but that doesn’t shield for defects in the terminators.
That might cover a civil liability situation, JoF, but we aren’t just discussing potential civil liability but rather a misuse for what would essentially be political murder. A whole other realm of accountability.
Joy of Fishes says:
August 4, 2015 at 3:41 pm
Lack of accountability for autonomous weapons has me puzzled. Creating and letting loose a terminator seems analogous to training and letting loose a vicious dog. The terminato’s creator and handler who lets the terminator loose would have comparable responsibility to the dog’s trainer and handler who lets the dog loose. What am I missing?
———————————–
I think the difference is that we are heading deeper into an era where personal responsibility is the higher standard versus corporate responsibility. The dog owner will be held to a higher standard of damages than what the maker of the terminator will be. The connections, political, military, economic of the terminator maker will make it so the intent behind the terminator, the “greater good” of making our side prevail over our enemies, will have the patriotic veneer that would always enable it to escape the true consequences of its defects.
If banks are too big to prosecute, the terminator makers will be too “needed” and too connected to be fully prosecuted for their damage. And those damages will be no more than the cost of doing business.
We have a taste of that already, in vaccines, medical liability, auto defects, industrial accidents…where the liability is more and more reduced and the victims left unprotected.
Already there seems to be a lack of accountability when drones kill innocents. Why even think of accountability with AI WMDs? At some point the US is going to feel what it’s like to have mass attacks on its innocents much like we’re doing to so many places in the world. Waco, Cherry Ridge, Oklahoma City, Twin Towers. So much outrage. But it’s the kind of thing we are doing around the world. We’re making enemies, not friends, except for Israel.
Exactly, BK, collateral damage is now accepted as the cost of doing business. The business of war is now a business of preemptive strike, which is a business of dissuasion and fear, both of which require some measure of shock and awe, which fundamentally requires civilians as the shocked and the awed.
Though drones were touted for their efficiency in killing, especially the part about the precision targeting, we have found, as you say, that they are less than efficient, having killed 1147 people while targeting only 41.
The better terminator might be built by an appliance manufacturer, for they are in the business of making our lives easier and more efficient. The better terminator may have to be built off the brain of a Gort.
The future is already here. This five-minute video has to be watched all the way through in order to fully appreciate both the technology as well as implications thereof.
po
Good camouflage too. Who would suspect that refrigerator over in the bushes.
Hahahaha, Pete! True, it keeps your food cold AND kills your enemy…automatically!
I better patent my idea, an automated refrigerator that hunts, skins and cools by itself! 🙂
Chuck, I am not sure what to believe anymore… I wonder how real it is if they would reveal it…and I wonder what surprises do the Russians, the Chinese and Israel have in store!
Came across this essay that is relevant to this discussion, I think:
———————————————
“Modern military strategy and the pursuit of terror
Last year, the UN Special Rapporteur on extrajudicial executions, Christof Heyns, said that drones had made killing “depersonalised”. This kind of weaponry makes it hard for the military to use only the minimum force required to subdue an enemy.
But Heyns is wrong in his focus. It is not the technology, but the military strategy, that leads to the use of overwhelming force.
“A narrative has been growing around these wars that they are somehow risk-free”
– Chris Woods
Modern warfare is premised on overpowering destruction – terrible violence such as in World War Two. When the United States dropped an atom bomb on Hiroshima, President Harry Truman called the city a “military base”.
It had been targeted, he said, “to avoid, as much as possible, the killing of civilians”. More than a hundred thousand civilians died in that attack – and many millions more suffered from its effects.
General Curtis LeMay honestly said, “we knew we were going to kill a lot of women and kids when we burned that town. Had to be done”.
The atom bomb attack had the desired result. It terrified the Japanese. Modern military strategy is designed to inflict terror.
Moral supremacy and the question of accountability
“Relatively speaking,” Woods notes, “airstrikes can be more precise than even a decade ago – depending on which weapon systems are used”. But the idea of precision is itself questionable. The West claims to be precise in its bombing merely to affirm its sense of moral superiority.
Unlike IS, it claims not to target civilians. When civilians are killed, it is an accident. NATO took this attitude in Libya – and so refused to submit to any investigation of its bombing there in 2011.
Woods argues that the failure to acknowledge civilian casualties “simply hands Daesh [IS] an easy propaganda weapon”.
But the point is that the West cannot win here. Both admission and denial of killing civilians will provide IS with fodder. Perhaps the time has come to reconsider the strategy of aerial bombardment, which – precise or not – does not seem to be effective in ending insurgencies.
Kurdish militias, who have been effective on the ground against IS, now face Turkish bombing. There is little doubt that Ankara has received a green light from the West.
As part of the deal that allows Turkey to bomb the Kurds, lethal drones of the US military take off from Turkish airfields to bomb northern Syria. They will have the desired impact – sowing terror amongst ordinary people. This is hardly the war against terror. It is merely another terrible war.
– See more at: http://www.alaraby.co.uk/english/comment/2015/8/5/airwars-monitoring-the-civilian-casualties-of-the-anti-is-war#sthash.pkowaJTM.dpuf
Joy of Fishes says:
Lack of accountability for autonomous weapons has me puzzled. Creating and letting loose a terminator seems analogous to training and letting loose a vicious dog.
A “vicious” dog is not a “trained” dog…it is one that has no training at all or is genetically obtuse due to idiots who breed them, like the dog fighting fools where I live. I despise them. I’ve dealt with their dogs and with care and caution you can subdue them in to the very social dogs pit bulls are born to be….man is the problem. Period. You’d be very surprised at how hard it is to “train” a truly “vicious” dog…dogs are just not disposed to that naturally. Most of my dogs are “trained” (in obedience first, then tracking & protection) and they know when to bite or attack and when not to do so. It is almost impossible to order them to do otherwise. My baddest butt dogs, like “Ari” in my avatar, were all very friendly if not threatened…more likely to lick your hand than anything else. My toughest dog, “Zoya” would not hurt a soul if not directly threatened…never in her wildest dreams…she wanted to be friends more than harm anyone. None-the-less, she defended at a car jacking with extreme vigor, because she knew the threat to her which was obvious by the guy who reached in to his waistband for what? He was on the ground before he knew it, with a broken shoulder and rotator cuff…instantly. One very well “trained dog” … the object of it all IMO.
That said, an fully autonomous AI robot could not be trained, only programed, and there’s the problem…humans. We seem hell bent to eliminate ourselves at times. Frankly, I doubt a fully autonomous AI robot is possible now, but in the future…who knows? Add politics as the motivating factor and we’re toast. Combat is ugly, obscene really, but believe it or not, it could be worse. Been there, done the first part…so far not faced the second part with bots. We may reach the day when “Gort” is necessary to reduce anyone who raises violence as a means. I doubt it is really possible, but what I know isn’t universal.
GORT KEEANU BARADA NIKTO NO PROBLEMO
LOL
Po sez, “I wonder how real it is if they would reveal it…”
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
They just pulled back the curtain a little. Those kinds of videos and news releases are both possibilities and a smattering of disinformation. Make them spend money on countermeasures that will be unneeded, because you have something both different and uglier in the closet. Recall when the SR-71 Blackbird was retired? With nothing to replace it? At least that’s what they said. Wanna bet next month’s rent on whether one of these days we find out about something that DID replace it after all?
Aridog,
I was reading an interview with Cesar Millan in a waiting room magazine this afternoon. This was in the article:
Ari,
“A ‘vicious’ dog is not a ‘trained’ dog…”
They are not mutually exclusive, Ari. This is a silly claim.
gbk … no, it is not a “silly claim”. The dog is descended from the wolf and part of their make up is the natural curiosity of the wolf. Wolves do not hate humans, but they are wary and for good reason. L David Mech wrote a very good book on wolves years ago based upon years of face to face encounters and pack traits. I too have been face to face with wild wolves and I’ve never seen anything but curiosity in their faces and eyes. Your only risk is if you create a threat. The dog carries forward the curiosity of the wolf and its first instinct is to approach without vitriol….once they reach stage 3 of Mech’s 4 phases, they approach directly and do not attack…it is a sensitive phase and scary if you don’t know how to handle it when a wolf sniffs your pant legs and tries to chew benignly anything hanging loose on your person. Watch the tail…98% of the time it is wagging below the top-line, just as in dogs that are not vicious due to abuse. I have a life time of working with and training dogs and horses and I know without a doubt that Hollywood creates the image of the trained vicious dog in various movies…pure nonsense. See my suggestion about attending a trial below.
BTW…a real bite, not a pinch, from a horse is far worse than a dog bite (a real horse bite and a piece of you will be missing…pinching OTOH is part of their “language” easily observed in herd behavior), I’ve experienced solid bites in training dogs when I made a mistake. Very few horses try to really bite humans as well…just not their nature unless abused. Only cure for the savage horse is a couple years of uninterrupted pasture time…with very slow re-introduction of you as the human they can trust….just sitting on a rock or something and let them come to you, which they always will…curiosity again. Then walk away calmly & slowly and let them follow you, which they always will until they make it plain they do not fear you. You don’t “break” a horse, you convince them. Been there, done that, too.
Those who raise “vicious” dogs (the dog fight crowd) do it by deprivation of social contact (chain them up so they are close but unable to reach other physically) and other brutalities…none of which are “training” per se. Among that crowd there are also perverse breeding practices aimed at creating a vicious dog. Raise a child that way and you’ll produce a savage as well. Contrary to many opinions, dogs like Pit Bulls or Huskies make very poor protection dogs…they are just too innately social. I’ll stand by my assertion that a “trained dog” is not autonomously vicious…and I’ve trained several protection dogs who retain civility except when threatened, especially if in the company of their handler. If you want to see trained dogs, find a Schutzhund (or DPO/WPO police dogs) IPO III trial near you (USCA, directly affiliated with the German SV & use the same rules and share judges and title certifications) and watch them as they progress through the phases of obedience, tracking, & protection. Everyone likes the protection phase because it is exciting and yet sometimes they miss the absolute obedience to command it involves…in the obedience phase (IMO the most important phase) you see multiple dogs behaving better than the average untrained pet dog and doing so under pressure. Of all the dogs I’ve worked with NONE would attack on spurious command, like in the movies’ BS, because they must perceive the threat first, same as the handler. The old cliché of “sic’em” doesn’t work in over 98% of all cases. If I tried to command one of my dogs over the years to “Fas’em” (or “packen’em…e.g., bite him) without a clear threat perceived mutually they’d look at me like I was nuts. It is just natural in the instinct of the un-abused dog…if abuse is the sole method, that dog will hate all humans and is more a danger to the handler than anyone else…e.g., they are driven to be “handler hard” and unresponsive to commands of a civil nature all too often.
gbk … BTW, providing little or no training input to your pet dog is the same as abuse, even to the aloof dogs like German Shepherds, my favorite breed. They want symbiosis and if you deprive them of it they can be confused and nasty to strangers, maybe even you. They have to “guess” what is appropriate behavior and without handler input, they can make mistakes and still not be vicious per se. That’s a human fault, not a dog fault. Given a chance, most dogs can read your mind so to speak.
Ari,
In my experience with dogs (which is pretty extensive), there is such a thing as a bad dog just like there is such a thing as a bad human but as you note most of the time vicious behavior is a result of bad human training and/or treatment.
gbk … if you really want to visit a IPO III dog trial (previously called a SchH III trial), just tell me your general location and I will find a upcoming trial, with times and date, at a club near you to watch. You will be amazed I assure you. Or you can just go to this LINK and find one yourself.
Gene … sure there are the occasional “bad dog” but in the dogs’ cases that is usually a product of really poor breeding practice. Add in abuse, as I define it, and you have a monster. Either way, it is not the dog’s fault…that lies with those who look to make monsters. Even then, the monster dog is a very poor protection dog, because they have no frame of reference of what is a real threat and just something to attack. Frequently this kind of bad dog is produced by abuse until the dog is a “fear biter”, or in some cases of poor breeding, they are born with all defense drive and no social drive, and in some cases there is no cure. I rescued one in that category and it broke my heart to have her euthanized. She was 100% totally unmanageable. Good thing we don’t selectively breed humans the same way as the dog fools do….never the less, by passive abuse we can produce similar unpleasant results & lack of compassion in human offspring…as often happens in ghetto environments. OTOH I have rescued a hybrid wolf/GSD and placed him with a family with small children and he lived 14 years as their best affectionate buddy and protector. I was told it couldn’t be done, but did it anyway with great results, placed with a vet tech who knew what she was doing with dogs. His memory today among his now grown kids is one of pure affection.
I want to jump in with a basic understanding of what is meant by “training.” Going back more than a century to the pioneering work of Ivan Pavlov, through B. F. Skinner and others. Any systematic behavior modification process controlled by the human who has control over the animal, is training. Does not matter if it is a dog, a chimp, or a Planarian. Training is training. Abuse is a form of behavior modification. Reinforcement, whether positive or negative, is training a response.
To say that bad dogs are not trained is wrong. They are shaped behaviorally, most often deliberately. Some breeds are more inclined to be aggressive than others. This is based on instinct, usually the territorial instinct, and may be fueled additionally by testosterone.
I have read that it is almost impossible to train a Basset or a Pug to bite people. The aggression instinct has been bred out of them. On the other hand, a Pekingese or Pomeranian will bite you three times while a Rottie is biting you once.
Chuck Stanley .. surprisingly, I agree with you: training is training, even when it is abuse…I just prefer the positive reinforcement process, especially with dogs and horses. Yes, some dogs are “taught” to be jerks, vicious even, by how they have interacted with humans, other dogs, or not…PLUS how they have been bred…witness the Neapolitan Mastiff (a crudely resurrected breed from a tiny gene pool of dubious origin) and the ultimate handler hard dog with aggression emphasized genetically. It pains me that the Neo is now recognized by the AKC…one reason I don’t belong and never trust their practices, especially their ambivalence to inbreeding. I belong to the USCA which has its own breed system and breed survey requirements, similar (actually precisely like) to the German SV….where inbreeding is not allowed, and line breeding is limited to 3 generations or more.
Yes, you are right, training involves response and when it is bad training the response can be ugly. I’d guess my point is that humans induce this…another reason I like far more dogs & horses than humans.
Chuck Stanley … one of these times on this subject I will relate my experiences with dogs (or wolves) influencing the behavior of other dogs. Our dog “Dera” was a classic “Omega” acquired to be a companion to a hard dominant male with the flaw of “other dog aggression.” “Omegas” are not “weak” as some folks think, they are the play starters and have a never quit demeanor. Over time “Ari” induced her to be “other dog aggressive” as well, although she and “Ari” played together like children every single day until “Ari” passed away. It took 9 days to acclimate them to each other, but it worked, in the face of all the advice to the contrary that it could not work. Yet, sweet “Dera” adopted “Ari’s” dog aggression trait which wasn’t natural to her otherwise. I did the best I could but dog to dog beat me on that point. With humans “Dera” is a Golden Retriever in a GSD coat … all she wants is play…however ever fearsome and wolfish she appears.
Chuck Stanley I appreciate your willingness to discuss issues we might not appear to agree on initially. You point about “training” also including bad training is on point, a semantic point, and one I tend to dismiss…in error (as abuse)… usually because of my experience with good and safe training, even of protection dogs. It is sometimes difficult with a “hard” dog to persuade them to not get distracted and run amok…but it can be done…just more work and repetition. In the USCA it all begins with obedience, which must be mastered (BH Title) before any tracking or protection work is attempted. USCA is affiliated with the German SV and recognition of processes and titles is seamless…we recognize each other and share judges, as we do with the Canadians as well,….and both advocate breed surveying of litters. In fact among the best judges I know, one is a Canadian (Doug Deacon).
I am only sorry that at times it seems we dominate thread without meaning to do so. I’d really like more opinions…I can always learn. Every time I thought I had all horses figured out, along came one who taught me otherwise and I had to step back and re-consider my approach.
So thanks again.