E S S AY S

Shouldwebeafraidof
AI?
Machinesseemtobegettingsmarterandsmarterandmuchbetterat
humanjobs,yettrueAIisutterlyimplausible.Why?

byLucianoFloridi<https://aeon.co/users/lucianofloridi>

IllustrationbyMattMurphyviaHandsomeFrank

LucianoFloridi<https://aeon.co/users/lucianofloridi> isprofessorofphilosophyandethicsofinformationattheUniversityof
Oxford,andaDistinguishedResearchFellowattheUehiroCentreforPracticalEthics.HislatestbookisTheFourthRevolution:
HowtheInfosphereIsReshapingHumanReality(2014).

Follow Luciano

3,700words

EditedbyNigelWarburton<https://aeon.co/users/nigelwarburton>
SendtoKindle

Supposeyouenteradarkroominanunknownbuilding.Youmightpanicaboutmonstersthat
couldbelurkinginthedark.Oryoucouldjustturnonthelight,toavoidbumpinginto
furniture.Thedarkroomisthefutureofartificialintelligence(AI).Unfortunately,many
peoplebelievethat,aswestepintotheroom,wemightrunintosomeevil,ultraintelligent
machines.Thisisanoldfear.Itdatestothe1960s,whenIrvingJohnGood,aBritish
mathematicianwhoworkedasacryptologistatBletchleyParkwithAlanTuring,madethe
followingobservation:

Letanultraintelligentmachinebedefinedasamachinethatcanfarsurpassallthe
intellectualactivitiesofanymanhoweverclever.Sincethedesignofmachinesisone
oftheseintellectualactivities,anultraintelligentmachinecoulddesignevenbetter
machinestherewouldthenunquestionablybeanintelligenceexplosion,andthe
intelligenceofmanwouldbeleftfarbehind.Thusthefirstultraintelligentmachineis
thelastinventionthatmanneedevermake,providedthatthemachineisdocileenough
totellushowtokeepitundercontrol.Itiscuriousthatthispointismadesoseldom
outsideofsciencefiction.Itissometimesworthwhiletotakesciencefictionseriously.

Onceultraintelligentmachinesbecomeareality,theymightnotbedocileatallbutbehavelike
Terminator:enslavehumanityasasubspecies,ignoreitsrights,andpursuetheirownends,
regardlessoftheeffectsonhumanlives.

Ifthissoundsincredible,youmightwishtoreconsider.Fastforwardhalfacenturytonow,and
theamazingdevelopmentsinourdigitaltechnologieshaveledmanypeopletobelievethat
Goodsintelligenceexplosionisaseriousrisk,andtheendofourspeciesmightbenear,if
werenotcareful.ThisisStephenHawkingin2014:

Thedevelopmentoffullartificialintelligencecouldspelltheendofthehumanrace.

Lastyear,BillGateswasofthesameview:

Iaminthecampthatisconcernedaboutsuperintelligence.Firstthemachineswilldoa
lotofjobsforusandnotbesuperintelligent.Thatshouldbepositiveifwemanageit
well.Afewdecadesafterthat,though,theintelligenceisstrongenoughtobea
concern.IagreewithElonMuskandsomeothersonthis,anddontunderstandwhy
somepeoplearenotconcerned.

AndwhathadMusk,TeslasCEO,said?
 Weshouldbeverycarefulaboutartificialintelligence.IfIweretoguesswhatour
biggestexistentialthreatis,itsprobablythatIncreasingly,scientiststhinkthere
shouldbesomeregulatoryoversightmaybeatthenationalandinternationallevel,just
tomakesurethatwedontdosomethingveryfoolish.Withartificialintelligence,we
aresummoningthedemon.Inallthosestorieswheretherestheguywiththe
pentagramandtheholywater,itslike,yeah,hessurehecancontrolthedemon.
Didntworkout.

Therealityismoretrivial.ThisMarch,MicrosoftintroducedTayanAIbasedchatrobotto
Twitter.Theyhadtoremoveitonly16hourslater.Itwassupposedtobecomeincreasingly
smarterasitinteractedwithhumans.Instead,itquicklybecameanevilHitlerloving,
Holocaustdenying,incestualsexpromoting,Bushdid9/11proclaimingchatterbox.Why?
Becauseitworkednobetterthankitchenpaper,absorbingandbeingshapedbythenasty
messagessenttoit.Microsoftapologised.

ThisisthestateofAItoday.Aftersomuchtalkingabouttherisksofultraintelligentmachines,
itistimetoturnonthelight,stopworryingaboutscifiscenarios,andstartfocusingonAIs
actualchallenges,inordertoavoidmakingpainfulandcostlymistakesinthedesignanduseof
oursmarttechnologies.

Letmebemorespecific.Philosophydoesntdonuanceswell.Itmightfancyitselfamodelof
precisionandfinelyhoneddistinctions,butwhatitreallylovesarepolarisationsand
dichotomies.Internalismorexternalism,foundationalismorcoherentism,trolleyleftorright,
zombiesornotzombies,observerrelativeorobserverindependent,possibleorimpossible
worlds,groundedorungroundedPhilosophymightpreachtheinclusivevel(girlsorboys
mayplay)buttoooftenindulgesintheexclusiveautaut(eitheryoulikeitoryoudont).

ThecurrentdebateaboutAIisacaseinpoint.Here,thedichotomyisbetweenthosewho
believeintrueAIandthosewhodonot.Yes,therealthing,notSiriinyouriPhone,Roombain
yourlivingroom,orNestinyourkitchen(Iamthehappyownerofallthree).Thinkinsteadof
thefalseMariainMetropolis(1927)Hal9000in2001:ASpaceOdyssey(1968),onwhich
GoodwasoneoftheconsultantsC3POinStarWars(1977)RachaelinBladeRunner(1982)
DatainStarTrek:TheNextGeneration(1987)AgentSmithinTheMatrix(1999)orthe
disembodiedSamanthainHer(2013).Youvegotthepicture.BelieversintrueAIandin
GoodsintelligenceexplosionbelongtotheChurchofSingularitarians.Forlackofabetter
term,IshallrefertothedisbelieversasmembersoftheChurchofAItheists.Letshavealook
atbothfaithsandseewhybotharemistaken.Andmeanwhile,remember:goodphilosophyis
almostalwaysintheboringmiddle.

S ingularitariansbelieveinthreedogmas.First,thatthecreationofsomeformofartificial
ultraintelligenceislikelyintheforeseeablefuture.Thisturningpointisknownasa
technologicalsingularity,hencethename.Boththenatureofsuchasuperintelligenceandthe
exacttimeframeofitsarrivalareleftunspecified,althoughSingularitarianstendtoprefer
futuresthatareconvenientlycloseenoughtoworryaboutbutfarenoughnottobearoundto
beprovedwrong.

Second,humanityrunsamajorriskofbeingdominatedbysuchultraintelligence.Third,a
primaryresponsibilityofthecurrentgenerationistoensurethattheSingularityeitherdoesnot
happenor,ifitdoes,thatitisbenignandwillbenefithumanity.Thishasalltheelementsofa
Manicheanviewoftheworld:GoodfightingEvil,apocalypticovertones,theurgencyofwe
mustdosomethingnoworitwillbetoolate,aneschatologicalperspectiveofhuman
salvation,andanappealtofearsandignorance.

Putallthisinacontextwherepeoplearerightlyworriedabouttheimpactofidioticdigital
technologiesontheirlives,especiallyinthejobmarketandincyberwars,andwheremass
mediadailyreportnewgizmosandunprecedentedcomputerdrivendisasters,andyouhavea
recipeformassdistraction:adigitalopiateforthemasses.

Likeallfaithbasedviews,Singularitarianismisirrefutablebecause,intheend,itis
unconstrainedbyreasonandevidence.Itisalsoimplausible,sincethereisnoreasontobelieve
thatanythingresemblingintelligent(letaloneultraintelligent)machineswillemergefromour
currentandforeseeableunderstandingofcomputerscienceanddigitaltechnologies.Letme
explain.

Sometimes,Singularitarianismispresentedconditionally.Thisisshrewd,becausethethen
doesfollowfromtheif,andnotmerelyinanexfalsoquodlibetsense:ifsomekindof
ultraintelligenceweretoappear,thenwewouldbeindeeptrouble(notmerelycould,as
statedabovebyHawking).Correct.Absolutely.Butthisalsoholdstrueforthefollowing
conditional:iftheFourHorsemenoftheApocalypseweretoappear,thenwewouldbeineven
deepertrouble.

Atothertimes,Singularitarianismreliesonaveryweaksenseofpossibility:someformof
artificialultraintelligencecoulddevelop,couldntit?Yesitcould.Butthiscouldismere
logicalpossibilityasfarasweknow,thereisnocontradictioninassumingthedevelopment
ofartificialultraintelligence.Yetthisisatrick,blurringtheimmensedifferencebetweenI
couldbesicktomorrowwhenIamalreadyfeelingunwell,andIcouldbeabutterflythat
dreamsitsahumanbeing.

HowsomenastyultraintelligentAIwilleverevolveautonomously
fromthecomputationalskillsrequiredtoparkinatightspotremains
unclear
Thereisnocontradictioninassumingthatadeadrelativeyouveneverheardofhasleftyou
$10million.Thatcouldhappen.So?Contradictions,likehappilymarriedbachelors,arent
possiblestatesofaffairs,butnoncontradictions,likeextraterrestrialagentslivingamongusso
wellhiddenthatweneverdiscoveredthem,canstillbedismissedasutterlycrazy.Inother
words,thecouldisnotthecouldhappenofanearthquake,buttheitisnttruethatit
couldnthappenofthinkingthatyouarethefirstimmortalhuman.Correct,butnotareasonto
startactingasifyouwillliveforever.Unless,thatis,someoneprovidesevidencetothe
contrary,andshowsthatthereissomethinginourcurrentandforeseeableunderstandingof
computersciencethatshouldleadustosuspectthattheemergenceofartificialultraintelligence
istrulyplausible.

HereSingularitariansmixfaithandfacts,oftenmoved,Ibelieve,byasinceresenseof
apocalypticurgency.Theystarttalkingaboutjoblosses,digitalsystemsatrisk,unmanned
dronesgoneawryandotherrealandworrisomeissuesaboutcomputationaltechnologiesthat
arecomingtodominatehumanlife,fromeducationtoemployment,fromentertainmentto
conflicts.Fromthis,theyjumptobeingseriouslyworriedabouttheirinabilitytocontroltheir
nextHondaCivicbecauseitwillhaveamindofitsown.HowsomenastyultraintelligentAI
willeverevolveautonomouslyfromthecomputationalskillsrequiredtoparkinatightspot
remainsunclear.Thetruthisthatclimbingontopofatreeisnotasmallsteptowardsthe
Moonitistheendofthejourney.Whatwearegoingtoseeareincreasinglysmartmachines
abletoperformmoretasksthatwecurrentlyperformourselves.

Ifallotherargumentsfail,Singularitariansarefondofthrowinginsomemaths.Afavourite
referenceisMooresLaw.Thisistheempiricalclaimthat,inthedevelopmentofdigital
computers,thenumberoftransistorsonintegratedcircuitsdoublesapproximatelyeverytwo
years.Theoutcomehassofarbeenmorecomputationalpowerforless.Butthingsare
changing.Technicaldifficultiesinnanotechnologypresentseriousmanufacturingchallenges.
Thereis,afterall,alimittohowsmallthingscangetbeforetheysimplymelt.MooresLawno
longerholds.Justbecausesomethinggrowsexponentiallyforsometime,doesnotmeanthatit
willcontinuetodosoforever,asTheEconomistputitin2014:

Throughoutrecordedhistory,humanshavereignedunchallengedasEarthsdominant
species.Mightthatsoonchange?Turkeys,heretoforeharmlesscreatures,havebeen
explodinginsize,swellingfromanaverage13.2lb(6kg)in1929toover30lbtoday.
Ontherocksolidscientificassumptionthatpresenttrendswillpersist,TheEconomist
calculatesthatturkeyswillbeasbigashumansinjust150years.Within6,000years,
turkeyswilldwarftheentireplanet.Scientistsclaimthattherapidgrowthofturkeysis
theresultofinnovationsinpoultryfarming,suchasselectivebreedingandartificial
insemination.Theartificialnatureoftheirgrowth,andthefactthatmosthavelostthe
abilitytofly,suggestthatnotallislost.Still,withnearly250mturkeysgobblingand
paradinginAmericaalone,thereiscauseforconcern.ThisThanksgiving,thereisbut
 oneprudentcourseofaction:eatthembeforetheyeatyou.

FromTurkzillatoAIzilla,thestepissmall,ifitwerentforthefactthatagrowthcurvecan
easilybesigmoid,withaninitialstageofgrowththatisapproximatelyexponential,followed
bysaturation,slowergrowth,maturityand,finally,nofurthergrowth.ButIsuspectthatthe
representationofsigmoidcurvesmightbeblasphemousforSingularitarianists.

Singularitarianismisirresponsiblydistracting.Itisarichworldpreoccupation,likelytoworry
peopleinleisuredsocieties,whoseemtoforgetaboutrealevilsoppressinghumanityandour
planet.Oneexamplewillsuffice:almost700millionpeoplehavenoaccesstosafewater
<http://www.unicef.org/publications/index_82419.html>.Thisisamajorthreattohumanity.
Oh,andjustincaseyouthoughtpredictionsbyexpertswereareliableguide,thinktwice.
Therearemanystaggeringlywrongtechnologicalpredictionsbyexperts(seesomehilarious
onesfromDavidPogue<http://www.scientificamerican.com/article/poguealltimeworsttech
predictions/>andonCracked.com<http://www.cracked.com/photoplasty_777_26hilariously
inaccuratepredictionsaboutfuture/>).In2004Gatesstated:Twoyearsfromnow,spamwill
besolved.Andin2011Hawkingdeclaredthatphilosophyisdead(sowhatsthisyouare
reading?).

ThepredictionofwhichIammostfondisbyRobertMetcalfe,coinventorofEthernetand
founderofthedigitalelectronicsmanufacturer3Com.In1995hepromisedtoeathiswordsif
provedwrongthattheinternetwillsoongosupernovaandin1996willcatastrophically
collapse.Amanofhisword,in1997hepubliclyliquefiedhisarticleinafoodprocessorand
drankit.IwishSingularitarianswereasboldandcoherentashim.

D eeplyirritatedbythosewhoworshipthewrongdigitalgods,andbytheirunfulfilled
Singularitarianprophecies,disbelieversAItheistsmakeittheirmissiontoproveonceand
forallthatanykindoffaithintrueAIistotallywrong.AIisjustcomputers,computersarejust
TuringMachines,TuringMachinesaremerelysyntacticengines,andsyntacticenginescannot
think,cannotknow,cannotbeconscious.Endofstory.

Thisiswhythereissomuchthatcomputers(still)cannotdo,looselythetitleofseveral
publicationsIraWilson(1970)<http://www.amazon.com/WhatcomputerscannotGaulbert
Wilson/dp/0877690316>HubertDreyfus(19721979)
<https://archive.org/stream/whatcomputerscan017504mbp/whatcomputerscan017504mbp_djvu.txt>
Dreyfus(1992)<https://mitpress.mit.edu/books/whatcomputersstillcantdo>DavidHarel
(2000)<http://www.academia.edu/2839622/Computers_Ltd_what_they_really_cant_do>
JohnSearle(2014)<http://www.nybooks.com/articles/2014/10/09/whatyourcomputercant
know/>thoughwhatpreciselytheycantdoisaconvenientlymovabletarget.Itisalsowhy
theyareunabletoprocesssemantics(ofanylanguage,Chineseincluded,nomatterwhat
Googletranslationachieves).Thisprovesthatthereisabsolutelynothingtodiscuss,letalone
worryabout.ThereisnogenuineAI,soafortioritherearenoproblemscausedbyit.Relax
andenjoyallthesewonderfulelectricgadgets.

AItheistsfaithisasmisplacedastheSingularitarians.BothChurcheshaveplentyof
followersinCalifornia,whereHollywoodscififilms,wonderfulresearchuniversitiessuchas
Berkeley,andsomeoftheworldsmostimportantdigitalcompaniesflourishsidebyside.This
mightnotbeaccidental.Whenthereisbigmoneyinvolved,peopleeasilygetconfused.For
example,GooglehasbeenbuyingAItechcompaniesasiftherewerenotomorrow(disclaimer:
IamamemberofGooglesAdvisoryCouncilontherighttobeforgotten),sosurelyGoogle
mustknowsomethingabouttherealchancesofdevelopingacomputerthatcanthink,thatwe,
outsideTheCircle,aremissing?EricSchmidt,Googlesexecutivechairman,fuelledthis
view,whenhetoldtheAspenInstitutein2013:ManypeopleinAIbelievethatwerecloseto
[acomputerpassingtheTuringTest]withinthenextfiveyears.

TheTuringtestisawaytocheckwhetherAIisgettinganycloser.Youaskquestionsoftwo
agentsinanotherroomoneishuman,theotherartificialifyoucannottellthedifference
betweenthetwofromtheiranswers,thentherobotpassesthetest.Itisacrudetest.Thinkof
thedrivingtest:ifAlicedoesnotpassit,sheisnotasafedriverbutevenifshedoes,she
mightstillbeanunsafedriver.TheTuringtestprovidesanecessarybutinsufficientcondition
foraformofintelligence.Thisisareallylowbar.Andyet,noAIhasevergotoverit.More
importantly,allprogramskeepfailinginthesameway,usingtricksdevelopedinthe1960s.
Letmeofferabet.Ihateaubergine(eggplant),butIshalleataplateofitifasoftwareprogram
passestheTuringTestandwinstheLoebnerPrizegoldmedalbefore16July2018.Itisasafe
bet<http://www.philosophyofinformation.net/wp
content/uploads/sites/67/2014/05/tigsaicfamasj.pdf>.

BothSingularitariansandAItheistsaremistaken.AsTuringclearlystatedinthe1950article
<http://www.loebner.net/Prizef/TuringArticle.html>thatintroducedhistest,thequestionCan
amachinethink?istoomeaninglesstodeservediscussion.(Ironically,orperhaps
presciently,thatquestionisengravedontheLoebnerPrizemedal.)Thisholdstrue,nomatter
whichofthetwoChurchesyoubelongto.YetbothChurchescontinuethispointlessdebate,
suffocatinganydissentingvoiceofreason.

TrueAIisnotlogicallyimpossible,butitisutterlyimplausible.Wehavenoideahowwe
mightbegintoengineerit,notleastbecausewehaveverylittleunderstandingofhowourown
brainsandintelligencework.Thismeansthatweshouldnotlosesleepoverthepossible
appearanceofsomeultraintelligence.Whatreallymattersisthattheincreasingpresenceof
eversmartertechnologiesishavinghugeeffectsonhowweconceiveofourselves,theworld,
andourinteractions.Thepointisnotthatourmachinesareconscious,orintelligent,orableto
knowsomethingaswedo.Theyarenot.Thereareplentyofwellknownresultsthatindicate
thelimitsofcomputation,socalledundecidableproblemsforwhichitcanbeprovedthatitis
impossibletoconstructanalgorithmthatalwaysleadstoacorrectyesornoanswer.

Weknow,forexample,thatourcomputationalmachinessatisfytheCurryHoward
correspondence,whichindicatesthatproofsystemsinlogicontheonehandandthemodelsof
computationontheother,areinfactstructurallythesamekindofobjects,andsoanylogical
limitappliestocomputersaswell.Plentyofmachinescandoamazingthings,including
playingcheckers,chessandGoandthequizshowJeopardybetterthanus.Andyettheyareall
versionsofaTuringMachine,anabstractmodelthatsetsthelimitsofwhatcanbedonebya
computerthroughitsmathematicallogic.

Quantumcomputersareconstrainedbythesamelimits,thelimitsofwhatcanbecomputed
(socalledcomputablefunctions).Noconscious,intelligententityisgoingtoemergefroma
TuringMachine.Thepointisthatoursmarttechnologiesalsothankstotheenormous
amountofavailabledataandsomeverysophisticatedprogrammingareincreasinglyableto
dealwithmoretasksbetterthanwedo,includingpredictingourbehaviours.Sowearenotthe
onlyagentsabletoperformtaskssuccessfully.

ThisiswhatIhavedefinedastheFourthRevolution
<http://onlinelibrary.wiley.com/doi/10.1111/j.14679973.2008.00573.x/abstract>inourself
understanding.WearenotatthecentreoftheUniverse(Copernicus),ofthebiological
kingdom(CharlesDarwin),orofrationality(SigmundFreud).AndafterTuring,weareno
longeratthecentreoftheinfosphere<http://www.indiana.edu/~tisj/readers/fulltext/25
3.PDF>,theworldofinformationprocessingandsmartagency,either.Wesharethe
infospherewithdigitaltechnologies.Theseareordinaryartefactsthatoutperformusinever
moretasks,despitebeingnoclevererthanatoaster.Theirabilitiesarehumblingandmakeus
reevaluatehumanexceptionalityandourspecialroleintheUniverse,whichremainsunique.
Wethoughtweweresmartbecausewecouldplaychess.Nowaphoneplaysbetterthana
Grandmaster.Wethoughtwewerefreebecausewecouldbuywhateverwewished.Nowour
spendingpatternsarepredictedbydevicesasthickasaplank.

Whatsthedifference?Thesameasbetweenyouandthedishwasher
whenwashingthedishes.Whatstheconsequence?Thatany
apocalypticvisionofAIcanbedisregarded

Thesuccessofourtechnologiesdependslargelyonthefactthat,whilewewerespeculating
aboutthepossibilityofultraintelligence,weincreasinglyenvelopedtheworldinsomany
devices,sensors,applicationsanddatathatitbecameanITfriendlyenvironment,where
technologiescanreplaceuswithouthavinganyunderstanding,mentalstates,intentions,
interpretations,emotionalstates,semanticskills,consciousness,selfawarenessorflexible
intelligence.Memory(asinalgorithmsandimmensedatasets)outperformsintelligencewhen
landinganaircraft,findingthefastestroutefromhometotheoffice,ordiscoveringthebest
priceforyournextfridge.

Digitaltechnologiescandomoreandmorethingsbetterthanus,byprocessingincreasing
amountsofdataandimprovingtheirperformancebyanalysingtheirownoutputasinputfor
thenextoperations.AlphaGo,thecomputerprogramdevelopedbyGoogleDeepMind,wonthe
boardgameGoagainsttheworldsbestplayerbecauseitcoulduseadatabaseofaround30
millionmovesandplaythousandsofgamesagainstitself,learninghowtoimproveits
performance.Itislikeatwoknifesystemthatcansharpenitself.Whatsthedifference?The
sameasbetweenyouandthedishwasherwhenwashingthedishes.Whatstheconsequence?
ThatanyapocalypticvisionofAIcanbedisregarded.Weareandshallremain,forany
foreseeablefuture,theproblem,notourtechnology.Soweshouldconcentrateonthereal
challenges.Bywayofconclusion,letmelistfiveofthem,allequallyimportant.

WeshouldmakeAIenvironmentfriendly.Weneedthesmartesttechnologieswecanbuildto
tackletheconcreteevilsoppressinghumanityandourplanet,fromenvironmentaldisastersto
financialcrises,fromcrime,terrorismandwar,tofamine,poverty,ignorance,inequalityand
appallinglivingstandards.

WeshouldmakeAIhumanfriendly.Itshouldbeusedtotreatpeoplealwaysasends,neveras
meremeans,toparaphraseImmanuelKant.

WeshouldmakeAIsstupidityworkforhumanintelligence.Millionsofjobswillbedisrupted,
eliminatedandcreatedthebenefitsofthisshouldbesharedbyall,andthecostsborneby
society.

WeshouldmakeAIspredictivepowerworkforfreedomandautonomy.Marketingproducts,
influencingbehaviours,nudgingpeopleorfightingcrimeandterrorismshouldnever
underminehumandignity.

Andfinally,weshouldmakeAImakeusmorehuman.Theseriousriskisthatwemight
misuseoursmarttechnologies,tothedetrimentofmostofhumanityandthewholeplanet.
WinstonChurchillsaidthatweshapeourbuildingsandafterwardsourbuildingsshapeus.
Thisappliestotheinfosphereanditssmarttechnologiesaswell.

SingularitariansandAItheistswillcontinuetheirdiatribesaboutthepossibilityorimpossibility
oftrueAI.Weneedtobetolerant.Butwedonothavetoengage.AsVirgilsuggestsinDantes
Inferno:Speaknotofthem,butlook,andpassthemby.Fortheworldneedssomegood
philosophy,andweneedtotakecareofmorepressingproblems.

FutureofTechnology PhilosophyofMind Computing&ArtificialIntelligence Alltopics