terminal velocity michelin’s clever new tires stay just … · playbook canoeing through a ......
TRANSCRIPT
WIRED
GEAR
Autopia
Terminal Velocity
Michelin’s Clever New Tires Stay Just as Grippy as They Wear
100 Years of Automotive Innovation, in One Amazing Display
Gadget Lab
The HTC One Now Comes in a Windows Phone Version
This Android Shield Could Encrypt Apps So Invisibly You Forget It's There
How to Easily Convert a YouTube Clip Into a GIF
Reviews
Finally, a Comfortable Travel Shoe That Doesn't Look Disgusting
The Ups and Downs of an Internet-Connected Garage Door Opener
Doesn't Play Nice
SCIENCE
Science
What's Up With That: How the Weather Guy Knows What It "Feels Like" in Your City
The Strange Blowpipe 19th Century Miners Used to Analyze Ore
All You Can Eat
Science Blogs
How Agriculture's Growth Promoters Might Work: A Mouse Study Sheds Some Light
Earthquakes Still Rocking Iceland's Barðarbunga
Measuring Inbreeding in the Greek Gods
ENTERTAINMENT
Game|Life
Huge Tournament Celebrates End of Oakland's Bizarre 80-Year Pinball Ban
Game|Life Podcast: Sony, Microsoft Trade Exclusives In Germany
Firefall Could Have Been Great, But Poor Design Holds It Back
Playbook
Canoeing Through a No-Escape Gauntlet of Whitecaps and Boulders
How I Survived Biking Through the Mojave Desert for Three Days
On Motorcycles, the Real Fun Begins When the Pavement Ends
Underwire
Behind the Scenes of a Reality Show Where Magicians Fight for Fame
Angry Nerd: Why Body Swapping Is Awesome on TV, But Crappy in Movies
5 Things You Didn't Know About Frank Miller and Sin City
BUSINESS
Business
With New Delivery Service, Uber Declares War on Google and Amazon
The Next Big Thing You Missed: Thanks to Amazon, Tiny Sellers Can Now Reach Across the Globe
The Newsletter That Became a $100M Business Selling Stuff to Dudes
Business Conference
Promotion
In today's business world, disruption is a constant force that never lets up. At the annual WIRED Business Conference: Disruptive by Design, wecelebrate the creative power of bold new ideas and the people that make them happen.
See the event >
Enterprise
Regime Change: Steve Ballmer Has Finally Resigned From Microsoft's Board
Apple's iMessage Is Being Taken Over by Spammers
Find the Perfect Coder Boot Camp for You With This Handy Site
Innovation Insights
Why the Apple-IBM Deal Is a Greater Threat to Android Than You Think
Digital Transformation, Innovation, Optimization, and Disruption: Spin the Dial
Armed and Ready: How Your Data Is Being ‘Weaponized’ Against You
SECURITY
Danger Room
All the Stuff Soldiers Have Carried in Battle, From the 11th Century to Today
The Humble WWI Biplane That Helped Launch Commercial Flight
The Simple Tweaks Mercedes Used to Turn a Sedan Into an Armored Beast
Threat Level
This Android Shield Could Encrypt Apps So Invisibly You Forget It's There
How to Save the Net: A CDC for Cybercrime
How to Save the Net: Keep It Open
DESIGN
Design
Mailbox, the Innovative Email App, Is Now Available for Mac
A Huge, Clay-Filled Robot That Replaces the Potter's Wheel
All the Stuff Soldiers Have Carried in Battle, From the 11th Century to Today
Raw File
The Weird, Totally Charming Hobbies That Unite People
All the Stuff Soldiers Have Carried in Battle, From the 11th Century to Today
Beautiful Portraits of Tsunami Survivors Help a Community Rebuild
OPINION
Opinion
How to Save the Net: A CDC for Cybercrime
How to Save the Net: Keep It Open
How to Save the Net: Build a Backup
MAGAZINE
Current Issue
Subscribe
Terminal Velocity
All You Can Eat
How to Save the Net: A CDC for Cybercrime
Video
Insider
Subscribe
RSS
Search
Renew | Give a Gift | InternationalOpinion
Autonomous Carsethicsrobot cars
Follow WiredTwitterFacebookRSS
Here’s a Terrible Idea: Robot Cars With Adjustable Ethics Settings
By Patrick Lin 08.18.14 | 6:30 am | Permalink
Share on Facebook1.1k
TweetTweet 12
173
U.S. Department of Transportation
Do you remember that day when you lost your mind? You aimed your car at five random people down the road. By the time you realized what you were doing, it wastoo late to brake.
Thankfully, your autonomous car saved their lives by grabbing the wheel from you and swerving to the right. Too bad for the one unlucky person standing on that path,struck and killed by your car.
Did your robot car make the right decision? This scene, of course, is based on the infamous “trolley problem” that many folks are now talking about in AI ethics. It’s aplausible scene, since even cars today have crash-avoidance features: some can brake by themselves to avoid collisions, and others can change lanes too.
ShareShare 169
3
Patrick Lin
Patrick Lin, PhD, is the director of theEthics + Emerging Sciences Group atCalifornia Polytechnic State Universityand affiliate scholar at Stanford LawSchool’s Center for Internet and Society.He is the editor of Robot Ethics. Thestatements expressed here are theauthor’s alone and do not necessarilyreflect the views of the aforementionedorganizations.
The thought-experiment is a moral dilemma, because there’s no clearly right way to go. It’s generally better to harm fewer people than more, to have one person dieinstead of five. But the car manufacturer creates liability for itself in following that rule, sensible as it may be. Swerving the car directly results in that one person’sdeath: this is an act of killing. Had it done nothing, the five people would have died, but you would have killed them, not the car manufacturer which in that case wouldmerely have let them die.
Even if the car didn’t swerve, the car manufacturer could still be blamed for ignoring the plight of those five people, when it held thepower to save them. In other words: damned if you do, and damned if you don’t.
So why not let the user select the car’s “ethics setting”? The way this would work is one customer may set the car (which he paidfor) to jealously value his life over all others; another user may prefer that the car values all lives the same and minimizes harmoverall; yet another may want to minimize legal liability and costs for herself; and other settings are possible.
Plus, with an adjustable ethics dial set by the customer, the manufacturer presumably can’t be blamed for hard judgment calls,especially in no-win scenarios, right? In one survey, 44 percent of the respondents preferred to have a personalized ethics setting,while only 12 percent thought the manufacturer should predetermine the ethical standard. So why not give customers what theywant?
It Doesn’t Solve Liability for the Company
If the goal is to limit liability for the car manufacturer, this tactic fails, as even if the user ultimately determines the weighting ofdifferent values factored into a crash decision the company can still be liable. To draw out that point, let’s make the ethical choicesoutrageous:
Imagine that manufacturers created preference settings that allow us to save hybrid cars over gas-guzzling trucks, or insured carsover uninsured ones, or helmeted motorcyclists over unhelmeted ones. Or more troubling, ethics settings that allow us to savechildren over the elderly, or men over women, or rich people over the poor, or straight people over gay ones, or Christians overMuslims.
In an accident that requires choosing one victim over another, the manufacturer could still be faulted for giving the user any option at all—that is, the option todiscriminate against a particular class of drivers or people. Saving, protecting, or valuing one kind of thing effectively means choosing another kind to target in anunavoidable crash scenario.
Granted, some of these choices seem offensive and inappropriate in the first place. Some are rooted in hate, though not all are. But for many of us, it is also offensiveand inappropriate to assume that your own life matters more than the lives of others—especially more than five, 10, 20, or 100 lives anonymous to you. (Is love ofone’s self much different than hatred or indifference toward others?)
To be that self-centered seems to be a thoughtless or callous mindset that’s at the root of many social problems today. Likewise, many of us would be offended if life-and-death decisions about others were made according to costs (legal or financial) incurred by you. Doing the right thing is often difficult, exactly because it goesagainst our own interests.
Whatever the right value is to put on human life isn’t the issue here, and it’d be controversial any which way. In the same survey above, 36 percent of respondentswould want a robot car to sacrifice their life to avoid crashing into a child, while 64 percent would want the child to die in order to save their own life. This is to saythat we’re nowhere near a consensus on this issue.
The point is this: Even with an ethics setting adjusted by you, an accident victim hit by your robot car could potentially sue the car manufacturer for (1) creating analgorithm that makes her a target and (2) allowing you the option of running that algorithm when someone like her—someone on the losing end of the algorithm—would predictably be a victim under a certain set of circumstances.
Punting Responsibility to Customers
Even if an ethics setting lets the company off the hook, guess what? We, the users, may then be solely responsible for injury or death in an unavoidable accident. Atbest, an ethics setting merely punts responsibility from manufacturer to customer, but it still doesn’t make progress toward that responsibility. The customer would stillneed to undergo soul-searching and philosophical studies to think carefully about which ethical code he or she can live with, and all that it implies.
In an important sense, any injury that results from our ethics setting may be premeditated if it’s foreseen.
And it implies a lot. In an important sense, any injury that results from our ethics setting may be premeditated if it’s foreseen. By valuing our lives over others, weknow that others would be targeted first in a no-win scenario where someone will be struck. We mean for that to happen. This premeditation is the difference betweenmanslaughter and murder, a much more serious offense.
In a non-automated car today, though, we could be excused for making an unfortunate knee-jerk reaction to save ourselves instead of a child or even a crowd ofpeople. Without much time to think about it, we can only make snap decisions, if they’re even true decisions at all, as opposed to merely involuntary reflexes.
Deus in Machina
So, an ethics setting is not a quick workaround to the difficult moral dilemma presented by robotic cars. Other possible solutions to consider include limitingmanufacturer liability by law, similar to legal protections for vaccine makers, since immunizations are essential for a healthy society, too. Or if industry is unwilling orunable to develop ethics standards, regulatory agencies could step in to do the job—but industry should want to try first.
With robot cars, we’re trying to design for random events that previously had no design, and that takes us into surreal territory. Like Alice’s wonderland, we don’tknow which way is up or down, right or wrong. But our technologies are powerful: they give us increasing omniscience and control to bring order to the chaos. Whenwe introduce control to what used to be only instinctive or random—when we put God in the machine—we create new responsibility for ourselves to get it right.
Share on Facebook1.1k
TweetTweet 12
173
RedditDiggStumble UponEmail
ShareShare 169
Tags: Autonomous Cars, ethics, robot cars
137 Comments |Permalink
MOST RECENT WIRED POSTS
Regime Change: Steve Ballmer Has Finally Resigned From Microsoft's Board
What's Up With That: How the Weather Guy Knows What It "Feels Like" in Your City
Huge Tournament Celebrates End of Oakland's Bizarre 80-Year Pinball Ban
Terminal Velocity
The HTC One Now Comes in a Windows Phone Version
How Agriculture's Growth Promoters Might Work: A Mouse Study Sheds Some Light
MOST RECENT WIRED POSTS
WE R E CO M M E N D RECOMMENDED BY
15 Hilariously Bad Designs forEveryday Objects
7 Hand Gestures That MakeYou Look Like a RealIntellectual
The Turbine-Powered, ChevyVolt of Airliners LooksFantastic
Don’t Wrap Your MacBook’sCable Around the Power Brick
The Nissan Altima ST is the bestcruiser, the V6 the sweetestengine- CARADVICE.COM.AU
The My Family sticker angst- NEWS.COM.AU
Regime Change: Steve Ballmer Has Finally Resigned From Microsoft's Board
What's Up With That: How the Weather Guy Knows What It "Feels Like" in Your City
Huge Tournament Celebrates End of Oakland's Bizarre 80-Year Pinball Ban
Terminal Velocity
The HTC One Now Comes in a Windows Phone Version
How Agriculture's Growth Promoters Might Work: A Mouse Study Sheds Some Light
TRENDING NOW ON WIRED
Edward Snowden: The Untold Story
All the Stuff Soldiers Have Carried in Battle, From the 11th Century to Today
I Liked Everything I Saw on Facebook for Two Days. Here’s What It Did to Me
With New Delivery Service, Uber Declares War on Google and Amazon
5 Things You Didn’t Know About Frank Miller and Sin City
TRENDING NOW ON WIRED
Edward Snowden: The Untold Story
All the Stuff Soldiers Have Carried in Battle, From the 11th Century to Today
I Liked Everything I Saw on Facebook for Two Days. Here’s What It Did to Me
With New Delivery Service, Uber Declares War on Google and Amazon
5 Things You Didn’t Know About Frank Miller and Sin City
Wired opinion
EditorEmily Dreyfuss
Send us a tip
Subscribe to Wired Magazine
Enter your e-mail address
Advertisement
Services
Quick Links: Contact Us | Login/Register | Newsletter | RSS Feeds | WIRED Jobs | WIRED Mobile | FAQ | Sitemap
FAQ |Contact Us |WIRED Staff |Advertising |Press Center |Subscription Services |Newsletter |RSS Feeds
Condé Nast Web Sites:Webmonkey |Reddit |ArsTechnica |Details |Golf Digest |GQ |New Yorker
Subscribe to a magazine: Condé Nast web sites: International Sites:
WIRED.com © 2014 Condé Nast. All rights reserved. Use of this Site constitutes acceptance of our User Agreement (effective 01/02/2014) and Privacy Policy(effective 01/02/2014). Your California Privacy Rights.
The material on this site may not be reproduced, distributed, transmitted, cached or otherwise used, except with the prior written permission of Condé Nast.
Ad Choices
Home Loan PreApproval
loans.com.au
3 Month Home Loan Pre Approval.Takes Just 2 Minutes! Apply Now.