Professional Documents
Culture Documents
ACM
CACM.ACM.ORG OF THE 10/2018 VOL.61 NO.10
Human-Level Intelligence
or Animal-Like Abilities?
32 Viewpoint
Are CS Conferences (Too)
IMAGE © LUCASF ILM LTD. & T M. ALL RIGH TS RESERVED.
Closed Communities?
Assessing whether newcomers
have a more difficult time
achieving paper acceptance
at established conferences.
By Jordi Cabot,
Javier Luis Cánovas Izquierdo,
and Valerio Cosentino
Association for Computing Machinery
Advancing Computing as a Science & Profession
44 56 86
Communications of the ACM is the leading monthly print and online magazine for the computing and information technology fields.
Communications is recognized as the most trusted and knowledgeable source of industry information for today’s computing professional.
Communications brings its readership in-depth coverage of emerging areas of computer science, new trends in information technology,
and practical applications. Industry leaders use Communications as a platform to present and debate various technology implications,
public policies, engineering challenges, and market trends. The prestige and unmatched reputation that Communications of the ACM
enjoys today is built upon a 50-year commitment to high-quality editorial content and a steadfast dedication to advancing the arts,
sciences, and applications of information technology.
ACM, the world’s largest educational STA F F EDITORIAL BOARD ACM Copyright Notice
and scientific computing society, delivers DIRECTOR OF PU BL ICATIONS E DITOR- IN- C HIE F Copyright © 2018 by Association for
resources that advance computing as a Scott E. Delman Andrew A. Chien Computing Machinery, Inc. (ACM).
science and profession. ACM provides the cacm-publisher@cacm.acm.org eic@cacm.acm.org Permission to make digital or hard copies
computing field’s premier Digital Library Deputy to the Editor-in-Chief of part or all of this work for personal
and serves its members and the computing Executive Editor Lihan Chen or classroom use is granted without
profession with leading-edge publications, Diane Crawford cacm.deputy.to.eic@gmail.com fee provided that copies are not made
conferences, and career resources. Managing Editor S E NIOR E DITOR or distributed for profit or commercial
Thomas E. Lambert Moshe Y. Vardi advantage and that copies bear this
Executive Director and CEO Senior Editor notice and full citation on the first
Vicki L. Hanson Andrew Rosenbloom page. Copyright for components of this
NE W S
Deputy Executive Director and COO Senior Editor/News work owned by others than ACM must
Patricia Ryan Co-Chairs be honored. Abstracting with credit is
Lawrence M. Fisher
Director, Office of Information Systems William Pulleyblank and Marc Snir permitted. To copy otherwise, to republish,
Web Editor
Wayne Graves Board Members to post on servers, or to redistribute to
David Roman
Director, Office of Financial Services Monica Divitini; Mei Kobayashi; lists, requires prior specific permission
Rights and Permissions
Darren Ramdin Michael Mitzenmacher; Rajeev Rastogi; and/or fee. Request permission to publish
Barbara Ryan
Director, Office of SIG Services François Sillion from permissions@hq.acm.org or fax
Editorial Assistant
Donna Cappo Jade Morris (212) 869-0481.
Director, Office of Publications VIE W P OINTS
Scott E. Delman Art Director Co-Chairs For other copying of articles that carry a
Andrij Borys Tim Finin; Susanne E. Hambrusch; code at the bottom of the first or last page
Associate Art Director John Leslie King; Paul Rosenbloom or screen display, copying is permitted
ACM CO U N C I L
Margaret Gray Board Members provided that the per-copy fee indicated
President
Assistant Art Director Stefan Bechtold; Michael L. Best; Judith Bishop; in the code is paid through the Copyright
Cherri M. Pancake
Mia Angelica Balaquiot Andrew W. Cross; Mark Guzdial; Haym B. Hirsch; Clearance Center; www.copyright.com.
Vice-President
Production Manager Richard Ladner; Carl Landwehr; Beng Chin Ooi;
Elizabeth Churchill
Bernadette Shade Francesca Rossi; Loren Terveen; Subscriptions
Secretary/Treasurer
Advertising Sales Account Manager Marshall Van Alstyne; Jeannette Wing; An annual subscription cost is included
Yannis Ioannidis
Ilia Rodriguez Susan J. Winter in ACM member dues of $99 ($40 of
Past President
Alexander L. Wolf which is allocated to a subscription to
Chair, SGB Board Columnists Communications); for students, cost
Jeff Jortner David Anderson; Michael Cusumano; P R AC TIC E is included in $42 dues ($20 of which
Co-Chairs, Publications Board Peter J. Denning; Mark Guzdial; Co-Chairs is allocated to a Communications
Jack Davidson and Joseph Konstan Thomas Haigh; Leah Hoffmann; Mari Sako; Stephen Bourne and Theo Schlossnagle subscription). A nonmember annual
Members-at-Large Pamela Samuelson; Marshall Van Alstyne Board Members subscription is $269.
Gabriele Anderst-Kotis; Susan Dumais; Eric Allman; Samy Bahra; Peter Bailis;
Renée McCauley; Claudia Bauzer Mederios; C O N TAC T P O IN TS Terry Coatta; Stuart Feldman; Nicole Forsgren; ACM Media Advertising Policy
Elizabeth D. Mynatt; Pamela Samuelson; Copyright permission Camille Fournier; Jessie Frazelle; Communications of the ACM and other
Theo Schlossnagle; Eugene H. Spafford permissions@hq.acm.org Benjamin Fried; Tom Killalea; Tom Limoncelli; ACM Media publications accept advertising
SGB Council Representatives Calendar items Kate Matsudaira; Marshall Kirk McKusick; in both print and electronic formats. All
Sarita Adve; Jeanna Neefe Matthews calendar@cacm.acm.org Erik Meijer; George Neville-Neil; advertising in ACM Media publications is
Change of address Jim Waldo; Meredith Whittaker at the discretion of ACM and is intended
BOARD C HA I R S acmhelp@acm.org to provide financial support for the various
Letters to the Editor activities and services for ACM members.
Education Board C ONTR IB U TE D A RTIC LES
letters@cacm.acm.org Current advertising rates can be found
Mehran Sahami and Jane Chu Prey Co-Chairs
by visiting http://www.acm-media.org or
Practitioners Board James Larus and Gail Murphy
W E B S IT E by contacting ACM Media Sales at
Terry Coatta and Stephen Ibaraki Board Members
http://cacm.acm.org (212) 626-0686.
William Aiello; Robert Austin; Kim Bruce;
REGIONA L C O U N C I L C HA I R S Alan Bundy; Peter Buneman; Carl Gutwin;
WEB BOARD Single Copies
ACM Europe Council Yannis Ioannidis; Gal A. Kaminka;
Chair Single copies of Communications of the
Chris Hankin Ashish Kapoor; Kristin Lauter; Igor Markov;
James Landay ACM are available for purchase. Please
ACM India Council Bernhard Nebel; Lionel M. Ni; Adrian Perrig;
Board Members contact acmhelp@acm.org.
Abhiram Ranade Marie-Christine Rousset; Krishan Sabnani;
Marti Hearst; Jason I. Hong;
ACM China Council m.c. schraefel; Ron Shamir; Alex Smola;
Jeff Johnson; Wendy E. MacKay COMMUN ICATION S OF THE ACM
Wenguang Chen Josep Torrellas; Sebastian Uchitel;
(ISSN 0001-0782) is published monthly
AU T H O R G U ID E L IN ES Hannes Werthner; Reinhard Wilhelm
by ACM Media, 2 Penn Plaza, Suite 701,
PUB LICATI O N S BOA R D http://cacm.acm.org/about- New York, NY 10121-0701. Periodicals
Co-Chairs communications/author-center RES E A R C H HIGHLIGHTS
postage paid at New York, NY 10001,
Jack Davidson; Joseph Konstan Co-Chairs
and other mailing offices.
Board Members Azer Bestavros and Shriram Krishnamurthi
ACM ADVERTISIN G DEPARTM E NT
Phoebe Ayers; Edward A. Fox; Board Members
2 Penn Plaza, Suite 701, New York, NY POSTMASTER
Chris Hankin; Xiang-Yang Li; Martin Abadi; Amr El Abbadi; Sanjeev Arora;
10121-0701 Please send address changes to
Sue Moon; Michael L. Nelson; Michael Backes; Maria-Florina Balcan;
T (212) 626-0686 Communications of the ACM
Sharon Oviatt; Eugene H. Spafford; David Brooks; Stuart K. Card; Jon Crowcroft;
F (212) 869-0481 2 Penn Plaza, Suite 701
Stephen N. Spencer; Divesh Srivastava; Alexei Efros; Bryan Ford; Alon Halevy;
New York, NY 10121-0701 USA
Robert Walker; Julie R. Williamson Gernot Heiser; Takeo Igarashi; Sven Koenig;
Advertising Sales Account Manager
Greg Morrisett; Tim Roughgarden;
Ilia Rodriguez
ACM U.S. Public Policy Office Guy Steele, Jr.; Robert Williamson; Printed in the USA.
ilia.rodriguez@hq.acm.org
Adam Eisgrau, Margaret H. Wright; Nicholai Zeldovich;
Director of Global Policy and Public Affairs Andreas Zeller
Media Kit acmmediasales@acm.org
1701 Pennsylvania Ave NW, Suite 300,
Washington, DC 20006 USA
T (202) 659-9711; F (202) 667-1066 Association for Computing Machinery S P EC IA L S EC TIONS
(ACM) Co-Chair
Computer Science Teachers Association 2 Penn Plaza, Suite 701 Sriram Rajamani A
SE
REC
Y
CL
PL
Executive Director T (212) 869-7440; F (212) 869-0481 Tao Xie; Kenjiro Taura; David Padua
NE
TH
S
I
Z
I
M AGA
DOI:10.1145/3273019
A
Organizing Committee:
Y. Amit, University of Chicago A.M.
WARDING ACM’S 2017 Certainty Is Unobtainable” (Feb. 2018)
R. Basri, Weizmann Institute Turing Award to John included a number of misleading state-
A. Berg, University of NC Hennessy and David Pat- ments, the most important that: “Mean-
T. Berg, University of NC terson was richly deserved while, Gödel’s results were based on
P. Felzenszwalb, Brown Univ. and long overdue, as de- first-order logic, but every moderately
B. Fux Svaiter, IMPA
scribed by Neil Savage in his news sto- powerful first-order theory is inconsis-
S. Geman, Brown University
B. Gidas, Brown University ry “Rewarded for RISC” (June 2018). tent. Consequently, computer science
D. Jacobs, University of MD RISC was a big step forward. In their is changing to use higher-order logic.”
O. Veksler, Univ of W. Ontario acceptance speech, Patterson also Computer science is based on logic,
graciously acknowledged the contem- mostly first-order logic, and program-
Program Description: porary and independent invention mers make their coding decisions us-
Computer vision is an of the RISC concepts by John Cocke, ing logic every day. The most important
inter-disciplinary topic
another Turing laureate, at IBM, as results of logic (such as Kurt Gödel’s
crossing boundaries
between computer science,
described by Radin.1 Unfortunately, Incompleteness Theorems) are taught
statistics, mathematics, Cocke, who was the principal inven- in theory courses and are the funda-
engineering, and cognitive tor but rarely published, was not in- mentals on which computer science
science. Research in cluded as an author, and it would and software engineering are based. No
computer vision involves have been good if Savage had men- inconsistencies have ever been found in
the development and tioned his contribution. any of the standard first-order theories
evaluation of computa-
It is noteworthy that RISC archi- used in logic, ranging from moderately
tional methods for image
analysis.
tectures depend on and emerged powerful to very powerful, and none are
from optimizing compilers. So far as believed to be inconsistent.
The focus of the program I can tell, all the RISC inventors had Harvey Friedman, Columbus, OH, USA,
will be on problems that
strong backgrounds in both architec- and Victor Marek, Lexington, KY, USA
involve modeling, machine
learning and optimization. ture and compilers.
The program will also
Reference
bridge a gap between 1. Radin, G. The 801 minicomputer. IBM Journal of
Author Responds:
theoretical approaches and Research & Development (1983), 237–246. Powerful first-order theories of intelligent
practical algorithms, Fred Brooks, Chapel Hill, NC, USA information systems are inconsistent
involving researchers with because these systems are not compact,
a variety of backgrounds.
thus violating a fundamental principle
Associated Workshops: No Inconsistencies in Fundamental of first-order theories. Meanwhile, the
First-Order Theories in Logic properties of self-proof of inferential
• Theory and Practice
in Machine Learning Referring to Martin E. Hellman’s Tur- completeness and formal consistency in
and Computer Vision ing Lecture article “Cybersecurity, Nu- higher-order mathematical theories are
(February 18 - 22, 2019) clear Security, Alan Turing, and Illogical the opposite of incompleteness and the
• Image Description for Logic” (Dec. 2017), Carl Hewitt’s letter self-unprovability of consistency Gödel
Consumer and to the editor “Final Knowledge with showed for first-order theories. Differing
Overhead Imagery properties between higher-order and
(February 25 - 26, 2019) first-order theories are reconciled by
• Computational Imaging It is noteworthy that Gödel’s “I’mUnprovable” proposition’s
(March 18 - 22, 2019) nonexistence in higher-order theories.
• Optimization Methods in
RISC architectures First-order theories are not foundational
Computer Vision and depend on and to computer science, which indeed relies
Image Processing on the opposite of Gödel’s results.
(April 29 - May 3, 2019) emerged from Carl Hewitt, Palo Alto, CA, USA
DOI:10.1145/3264623 http://cacm.acm.org/blogs/blog-cacm
Can We Use AI
for Global Good?
Amir Banifatemi observes how the AI for Good Summit
“allowed us to start a dialogue, find a common frame of reference,
and decide how our steps would be smart and structured.”
q Join ACM-W: ACM-W supports, celebrates, and advocates internationally for the full engagement of women in
computing. Membership in ACM-W is open to all ACM members and is free of charge.
Priority Code: CAPP
Payment Information
Payment must accompany application. If paying by check
or money order, make payable to ACM, Inc., in U.S. dollars
Name or equivalent in foreign currency.
Credit Card #
City/State/Province
Exp. Date
ZIP/Postal Code/Country
q Please do not release my postal address to third parties
Signature
Email
q Yes, please send me ACM Announcements via email Return completed application to:
q No, please do not send me ACM Announcements via email ACM General Post Office
P.O. Box 30777
Purposes of ACM New York, NY 10087-0777
ACM is dedicated to:
Prices include surface delivery charge. Expedited Air
1) Advancing the art, science, engineering, and Service, which is a partial air freight delivery service,
application of information technology is available outside North America. Contact ACM for
2) Fostering the open interchange of information more information.
to serve both professionals and the public Satisfaction Guaranteed!
3) Promoting the highest professional and ethics standards
1-800-342-6626 (US & Canada) Hours: 8:30AM - 4:30PM (US EST) acmhelp@acm.org
1-212-626-0500 (Global) Fax: 212-944-1318 acm.org/join/CAPP
N
news
F
EW M OVI E S CE N E Shave had Even The Walt Disney Company, which points to an anticipated implementa-
such an effect on display- bought Lucasfilm and the distribution tion of having the 3D image seem to be
technology research and rights for the movie franchise in 2012, standing on an illuminated pedestal,
development as the droid is among those with engineers working similar to the game table on the Mil-
R2D2 projecting a three- on the idea. lennium Falcon that appears in a scene
dimensional (3D) image of Princess Two years ago, Daniel Joseph and later in Star Wars.
Leia pleading for help in 1977’s block- colleagues in entertainment giant Dis- The Disney system suffers from a
buster film Star Wars. Numerous en- ney’s Burbank, CA-based research and problem that is shared with similar
gineers have wondered just how they development operation filed for a pat- systems: the image is formed from an
might achieve that effect, of an image ent on a projector intended to display array of light sources fed through beam
you can see from any angle, in real life. floating 3D images. The U.S. patent splitters and mirrors some distance
IMAGE © LUCASF ILM LTD. & T M. ALL RIGH TS RESERVED.
O C TO B E R 2 0 1 8 | VO L. 6 1 | N O. 1 0 | C OM M U N IC AT ION S OF T HE ACM 11
news
behind the pedestal, which limits the his move away from holographic tech-
viewing angle to those looking toward nologies. In the first installment of the
the projection optics, and so cannot All volumetric movie series, protagonist Tony Stark
emulate the movies. displays to date share uses a 3D projector not just to visualize
Daniel Smalley, an assistant profes- the elements of his powered suit, but
sor of electrical and computer engineer- the same problem, also to create a virtual gauntlet around
ing at Brigham Young University, says, Smalley says. his hand.
“Like many in the holography field, I Smalley’s team overcame the need
felt that holograms would provide the “You don’t have to encapsulate their display by trap-
3D images of the future, but the annoy- the self-occlusion ping and moving a single dust-sized
ing issue is you have to be looking in the particle. The prototype uses an ul-
direction of the screen that generates to make objects traviolet laser taken from a Blu-ray
them. It’s counter to what you expect 3D that look realistic.” player to capture and move the piece
displays to do in the future.” of dust. A visible-light source tracks
Builders of volumetric displays that and illuminates it. Physicists have yet
can be viewed from any angle face their to develop a theory that fully explains
own challenge. “Fundamentally, you the process of such photophoretic
have the problem that photons will into the air and illuminate them. A team trapping, but it appears to rely on lo-
just keep traveling until they bounce led by John Howell, a professor of phys- cal heating from being struck by pho-
off something,” says V. Michael Bove, ics and optics based at the University of tons. Gas molecules hitting the hotter
principal research scientist and head Rochester, used cesium vapor to create surface acquire more kinetic energy
of the object-based media group at the the voxels in their experimental volumet- as they bounce off, pushing the par-
Massachusetts Institute of Technology. ric display; the cesium atoms glow where ticle away.
Systems such as the VX1 built by the light from two steerable lasers cross. Says Smalley, “On average it doesn’t
Australian company Voxon Photonics Yet in these displays, moving parts and work very well at all, but in the [statis-
use a fast-moving sheet to provide a re- poisonous particles need to be encapsu- tical] tails you see incredible behavior.
flective surface for photons. At a high- lated in a transparent dome or sphere. The particle just stays there. You can
enough speed, the sheet will seem to “What’s of increased interest is even blow on it gently. We had one par-
disappear, but bright lights bounced not have a display in the table but to ticle trapped in there for 15 hours. It
off it will persist to the viewer; the re- interact with it in a meaningful way. could have stayed for longer: we had to
sult is the illusion of a slightly translu- Volumetric displays do have this talk- switch the machine off.”
cent 3D object floating in space. Bove ing-head-in-a-jar character that works The particle’s composition seems
says the need to move the sheet at high against that. You have the sense that to be crucial. Smalley’s team settled
speed makes this an intrinsically noisy this imagery is bottled up,” Bove says. on black liquor—a by-product of the
option, and one likely to suffer from Smalley also sees interaction as key, paper-making process—after trying
mechanical wear. citing another Disney movie franchise, numerous candidates. “I do not be-
Another option is to disperse particles Iron Man, as additional inspiration for lieve we can say this is definitively the
ACM News
best material. It seems unlikely that it ments are complex movement and dy-
is,” he says. namics, not super-high resolution,”
It is possible to produce freestand- “The general public Blundell argues.
ing volumetric images without inject- has for 40 years Smalley envisages applications
ing particles into the air. More than a where the user needs to inspect the
decade ago, Hidei Kimura, founder been seeing shape closely and move around it. The
and CEO of Japanese company Bur- cinematic depictions ability to produce mid-air streamers in
ton Inc., and Taro Uchiyama of Keio fluid-dynamics simulations and models
University found that when focused of physically of organs to help with planning medi-
on specific points, microsecond bursts impossible things, cal operations seem good examples.
of high-intensity infrared light could “A lot of 3D technologies can’t give you
cause air molecules to become glow- and when they do a strong spatial sense when you get up
ing plasma. Kimura envisaged the see what’s possible, close. With ours, you can,” he says.
technology being used to create levi- Bove says by looking closely at re-
tating signs above head height for use they’re disappointed.” quirements for target applications
in emergencies; the bursts would be and working with user-interface de-
intense enough to burn the hand of a signers, the developers of volumetric
user foolish enough to try to touch the displays can move from experiment to
glowing voxels. market more easily. “Can it be behind
Much shorter pulses could yield a age to a volume the size of a ping-pong a transparent barrier? Is it important
safer system. Yoichi Ochiai of the Uni- ball, and the results demonstrated so that it be viewable from any angle or
versity of Tsukuba and Kota Kumagai far are based on long-exposure images is 90 degrees OK? Is it acceptable for it
of the University of Utsunomiya in Ja- that took up to a minute to generate. to have moving parts?” he suggests as
pan showed at the ACM SIGGRAPH Says Barry Blundell, senior lecturer questions to be asked.
conference in 2015 the results of a pro- in computing at the University of Derby Developing volumetric technolo-
totype based on lasers that fire bursts in the U.K. and a researcher into volu- gies for specific applications may lead
no more than 100 femtoseconds long. metric displays since the late 1980s, to the problem of no individual market
According to Ochiai, users would “With the optical-trap display, I would being large enough to support research
simply get a tingling sensation from have to see images generated a lot and development, but such displays
touching the plasma voxels, though faster. The only way to do that is paral- look more technologically feasible,
users would need to be careful to not lelism; you’ve got to have more lasers Bove says. “The problem with the Leia
let their eyes get too close to the im- surrounding the display, and more par- display is that it needs all of the boxes
ages, as retinal damage is a distinct ticles. The problem could be that you to be ticked.”
possibility. Robert Stone, professor of need to have so much physical appara-
interactive multimedia systems at the tus that you lose the viewing freedom.”
Further Reading
University of Birmingham in the U.K., Smalley claims the technology ex-
says he has concerns over the eye form- ists to drive and illuminate a collection Smalley, D.E. et al
A Photophoretic-Trap Volumetric Display,
ing strong afterimages because of the of particles in the shape of the spatial
Nature, 553, pp486–490 (25 January 2018),
brightness of the plasma. light modulator, the same kind of de- doi:10.1038/nature25176
The plasma projector has the ad- vice as that used to research holograph-
Ochiai, Y., Kumagai, K., Hoshi, T., Rekimoto, J.,
vantage of being far more resistant to ic displays and optical computers. Bove Hasegawa, S., and Hayasaki, Y.
disturbance by moving hands than argues the laser and light-modulator Fairy Lights in Femtoseconds: Aerial and
the particle-based option. However, components needed for scaled-up dis- Volumetric Graphics Rendered by Focused
all volumetric displays to date have a plays are now relatively cheap. Femtosecond Laser Combined with
Computational Holographic Fields, ACM
common problem, Smalley says: “It Still, expectations may be set too high.
Transactions on Graphics, Volume 35, Issue
is like taking a bunch of fireflies and “The general public has for 40 years 2, (May 2016), doi:10.1145/2850414
organizing them into patterns. Every- been seeing cinematic depictions
Blundell, B.
thing looks like a ghost. You don’t have of physically impossible things, and On the Uncertain Future of the Volumetric
the self-occlusion to make objects that when they do see what’s possible, they 3D Display Paradigm, 3D Research, 8 (2)
look realistic. are disappointed,” says Bove. p11, doi:10.1007/s13319-017-0122-2
“We want to be able to take a point Smalley concedes, “At this stage, you Joseph, D.M., Smoot, L.S.,
and have it shine light in only one di- don’t have to be an expert to realize that Smithwick, Q.Y., and Ilardi, M.J.
rection. That would mean it begins to this isn’t the Princess Leia display you Retroreflector Display System for
look solid.” are looking for. But, if given the oppor- Generating Floating Image Effects, U.S.
Patent Application 2018/0024373 A1 (25
The lack of self-occlusion in the op- tunity to be developed further, I don’t January 2018)
tical-trap display is, for the moment, a think you would be disappointed.”
secondary issue. It is difficult to move Researchers may be trying too hard Chris Edwards is a Surrey, U.K.-based writer who reports
the single particle that flies around the to make fact out of fiction. “What some on electronics, IT, and synthetic biology
Brigham Young display any faster than of the people working on volumetrics
is possible today; that limits its cover- haven’t realized is that the key ele- © 2018 ACM 0001-0782/18/10 $15.00
O C TO B E R 2 0 1 8 | VO L. 6 1 | N O. 1 0 | C OM M U N IC AT ION S OF T HE ACM 13
news
Transient Electronics
Take Shape
Advances in materials science and chemistry are leading
to self-destructing circuits and transient electronics,
which could impact many fields.
O
N E O F T H E intriguing as- Mission: Possible
pects of the popular 1960s There are myriad possible uses for
television show “Mission self-destructing electronic circuits. For
Impossible” was the open- example, the technology would allow
ing sequence of every epi- farmers to place monitoring devices
sode, which featured a secret agent in a field and not have to worry about
listening to a recorded message about removing them later. Using different
an upcoming mission. At the end of the materials or combinations of materials
recording each week, the tape would that avoid toxic residue ranging from
sizzle, crackle, and disintegrate into a Tungsten to formulated polymers, the
heap of smoke and debris, ensuring no circuits would simply disintegrate at a
one else could access the top-secret in- certain point. The remaining material
formation it contained. would have little or no impact on the
Until recently, self-destructing elec- environment.
tronic systems remained within the The same technology also would let
realm of science fiction, but advances in doctors insert biomedical devices into
chemistry, engineering, and materials the human body to dispense medicine
science are finally allowing research- in a controlled way; in some cases, such
ers to construct circuits that break as with chemotherapy, such micro-
down on their own timetable. This targeting of cells could dramatically
includes systems that rely on conven- reduce side effects, and there would be
tional complementary oxide semicon- no need to surgically remove the device
ductor (CMOS) technology. The self-destructing audio tape of the at the end of treatment.
“Mission Impossible” television show
“The goal is to develop functional anticipated by decades the advent of Transient electronics could allow the
circuits that can operate for a period self-destructing electronics. military to deploy drones, robots, and
of time and then vaporize,” says Amit other electronic devices into the field
Lal, Robert M. Scharf 1977 Professor destructing circuits that could be used without the worry adversaries could re-
of Engineering in the Electrical and in smartphones, drones, and even in- cover them and benefit in any way.
Computer Engineering Department at side the human body. The environmental benefits of self-
Cornell University in Ithaca, NY, and While the technology is still in the destructing circuits are also obvious,
director of the university’s SonicMEMs early stages of development, it could considering tens of millions of tons of e-
lab. “It’s the Biblical ashes-to-ashes have a commercial impact within a few waste are generated every year, and toxic
concept applied to electronics.” years. For now, the biggest obstacles re- substances including mercury, lead,
The technology could reshape nu- volve around perfecting transient elec- cadmium and arsenic are not always
merous fields, including medicine, tronics and self-destructing circuits recycled, or completely destroyed dur-
agriculture, and the military. It could and scaling them for mass use. There’s ing incineration. In some cases, e-waste
also reduce environmental damage also a need to gain a deeper under- winds up in landfills, particularly in de-
caused by materials in semiconduc- standing of polymers and composite veloping nations. The resulting toxins
tors and electronics, which require re- materials, and to ensure these systems that leach into the soil, air, and water
IMAGE F RO M CSCOTT RO LLINS.BLOGSPOT.COM
cycling and too often wind up in land- fully vaporize without leaving traces of create health hazards that can result in
fills and water supplies. Already, Lal toxic chemicals. As Lal explains, “It’s neurological damage, reproductive dis-
and a team at Cornell have obtained not easy to design a circuit that works orders, and cancers.
a patent for water-soluble circuits perfectly and delivers a high level of New types of designs and encapsula-
that biodegrade without leaving toxic performance for a period of time, and tion layers will allow electronic systems
materials behind. Other researchers then make it vaporize in the desired formed with specialized materials to
at Northwestern University and the situation or at a precise moment, and operate in a stable, high-performance
University of Houston have built self- within a relatively short period of time.” manner for a prescribed period and
O C TO B E R 2 0 1 8 | VO L. 6 1 | N O. 1 0 | C OM M U N IC AT ION S OF T HE ACM 15
news
A
S K P OVERT Y AT TORN E Y Jo-
Applications of Robotic
anna Green Brown for an Caseworker Client
Process Automation (RPA) • Automating application INTAKE • Scheduling appointments for
example of a client who and cognitive technologies screening human services programs
fell through the cracks and across the life cycle of a • Automating verification • Addressing queries
1
• Predicting high-risk cases • Auto-filling application forms
lost social services benefits human services case • Automating eligibility
determinations
they may have been eligible for because
AI Technologies:
of a program driven by artificial intelli- Chatbot
gence (AI), and you will get an earful. RPA
There was the “highly educated and Machine learning
the murky waters of applying for gov- show a person isn’t necessarily getting report “AI-augmented human services:
ernment assistance programs like So- all benefits they need, but it doesn’t Using cognitive technologies to trans-
cial Security and Medicaid. necessarily mean it’s correct informa- form program delivery.”
There are so many factors that go tion, and it’s not always indicative of “AI can augment the work of case-
into an application or appeals process eligibility of benefits.” workers by automating paperwork,
for social services that many people There are well-documented ex- while machine learning can help case-
just give up, Green Brown says. They amples of bias in automated systems workers know which cases need urgent
can also lose benefits when a line of used to provide guidelines in sentenc- attention. But ultimately, humans are
questioning ends in the system, but ing criminals, predicting the likeli- the users of AI systems, and these sys-
O C TO B E R 2 0 1 8 | VO L. 6 1 | N O. 1 0 | C OM M U N IC AT ION S OF T HE ACM 17
news
tems should be designed with human Machines fall short in that they have
needs in mind,’’ the report states. That no “common sense,” so if a data error
means they first need to determine the is input, it will continue to apply that “Humans are better
biggest pain points for caseworkers, error, Etzioni says. Likewise, if there is than computers
and the individuals and families they a pattern in the data that is objection-
serve. Issues to factor in are what are able because the data is from the past at exploring those
the most complex processes; can they but is being used to create predictive grey areas around
be simplified; what activities take the models for the future, the machine will
most time and whether they can be not override it. the edges of problems.
streamlined, the report suggests. “It won’t say, ‘this behavior is racist or Computers are
Use of these systems is in the early sexist and we want to change that’; on the
stages, but we can expect to see a grow- contrary, the behavior of the algorithm is better at the
ing number of government agencies to amplify behaviors found in the data,’’ black-and-white
implementing AI systems that can au- he says. “Data codifies past biases.”
tomate social services to reduce costs Because machine learning systems decisions in
and speed up delivery of services, says seek a signal or pattern in the data, “we the middle.”
James Hendler, director of the Rensse- need to be very careful in the applica-
laer Institute for Data Exploration and tion of these systems,” Etzioni says. “If
Applications and one of the originators we are careful, there’s a great potential
of the Semantic Web. benefit as well.”
“There’s definitely a drive, as more To make AI and machine learning systems, since a lot of these models
people need social services, to bring systems work appropriately, many cog- make assumptions that don’t always
in any kind of computing automation nitive technologies need to be trained hold in practice. They felt it was im-
and obviously, AI and machine learn- and retrained, according to the De- portant to consider the possibility that
ing are offering some new opportuni- loitte report. “They improve via deep an algorithm could respond “I don’t
ties in that space,’’ Hendler says. learning methods as they interact with know” or “pass,” which led them to
One of the ways an AI system can be users. To make the most of their invest- think about the relationship between
beneficial is in instances in which some- ments in AI, agencies should adopt an a model and its surrounding system.
one seeking benefits needs to access agile approach [with software systems], “There is often an assumption in ma-
cross-agency information. For example, if continuously testing and training their chine learning that the data is a repre-
someone is trying to determine wheth- cognitive technologies.” sentative sample, or that we know exact-
er they can get their parents into a gov- David Madras, a Ph.D. student and ly what objective we want to optimize.”
ernment-funded senior living facility, machine learning researcher at the That has proven not to be the case in
there are myriad questions to answer. University of Toronto (U of T), believes many decision problems, he says.
“The potential of AI and machine learn- if an algorithm is not certain of some- Madras acknowledges the difficulty
ing is figuring out how to get people to the thing, rather than reach a conclusion, of knowing how to add fairness to (or
right places to answer their questions, it should have the option to indicate subtract unfairness from) an algo-
and it may require going to many places uncertainty and defer to a human. rithm. “Firstly, unfairness can creep
and piecing together information. AI can Madras and colleagues at U of T in at many points in the process, from
help you pull it together as one activity.” developed an algorithmic model that problem definition, to data collection,
One of the main, persistent prob- includes fairness. The definition of to optimization, to user interaction.”
lems these systems have, however, is fairness they used for their model is Also, he adds, “Nobody has a great
inherent bias, because data is input by based on “equalized odds,” which they single definition of ‘fairness.’ It’s a very
biased humans, experts say. found in a 2016 paper, “Equality of Op- complex, context-specific idea [that]
Just like “Murphy’s Law,” which portunity in Supervised Learning,” by doesn’t lend itself easily to one-size-
states that “anything that could go computer scientists from Google, the fits-all solutions.”
wrong, will,” Oren Etzioni, chief ex- University of Chicago, and the Univer- The definition they chose for their
ecutive officer of the Allen Institute sity of Texas, Austin. According to that model could just as easily be replaced
for Artificial Intelligence, says there’s paper, Madras explains, “the model’s by another, he notes.
a Murphy’s Law for AI: “It’s a law of false positive and false negative rates In terms of whether social services
unintended consequences, because should be equal for different groups systems can be unbiased when the al-
a system looks at a vast range of pos- (for example, divided by race). Intui- gorithm running them may have built-
sibilities and will find a very counter- tively, this means the types of mistakes in biases, Madras says that when mod-
intuitive solution to a problem.” should be the same for different types els learn from historical data, they will
“People struggle with their own bi- of people (there are mistakes that can pick up any natural biases, which will
ases, whether racist or sexist—or be- advantage someone, and mistakes that be a factor in their decision-making.
cause they’re just plain hungry,’’ he can disadvantage someone).” “It’s also very difficult to make an
says. “Research has shown that there The U of T researchers wanted to ex- algorithm unbiased when it is operat-
are [judicial] sentencing differences amine the unintended side effects of ing in a highly biased environment;
based on the time of day.” machine learning in decision-making especially when a model is learned
from historical data, the tendency researchers stated. “However, what “The real danger is people over-
is to repeat those patterns in some we should be focusing on is design- trusting these ‘unbiased’ AI systems,”
sense,’’ Madras says. ing AI that delivers results that are in he says. “What I’m afraid of is most
Etzioni believes an AI system can line with peoples’ well-being. By ob- people don’t understand these issues
be bias-free even when bias is input, serving human reactions to various … and just will trust the system the way
although that is not an easy thing to outcomes, AI could learn through they trust other computer systems. If
achieve. An original algorithm tries to a technique called ‘cooperative in- they don’t know these systems have
maximize consistency with data, he verse reinforcement learning’ what these limitations, they won’t be look-
says, but that past data may not be the our preferences are, and then work ing for the alternatives that humans
only criteria. towards producing results consistent are good at.”
“If we can define a criterion and with those preferences.”
mathematically describe what it AI systems need to be held account-
Further Reading
means to be free of bias, we can give able, says Alexandra Chouldechova,
that to the machine,’’ he says. “The an assistant professor of statistics and Madras, D., Creager, E., Pitassi, T., and Zemel, R.
Learning Adversarially Fair and
challenge becomes describing formal- public policy at Carnegie Mellon Uni-
Transferable Representations, 17 Feb. 2018,
ly or mathematically what bias means, versity’s Heinz College of Information Cornell University Library, https://arxiv.org/
and secondly, you have to have some Systems and Public Policy. abs/1802.06309
adherence to the data. So there’s really “Systems fail to achieve their pur- Buolamwini, J. and Gebru, T.
a tension between consistency with the ported goals all the time,’’ Choul- Gender Shades: Intersectional Accuracy
data, which is clearly desirable, and be- dechova notes. “The questions are: Disparities in Commercial Gender
ing bias-free.” Why? Can it be fixed? Could it have Classification, Proceedings of Machine
Learning Research, 2018, Conference on
People are working so both consis- been prevented in the first place?
Fairness, Accountability and Transparency.
tency and being bias-free can be sup- “By being clear about a system’s in- http://proceedings.mlr.press/v81/
ported, he adds. tended purpose at the outset, transpar- buolamwini18a/buolamwini18a.pdf
For AI to augment the work of gov- ent about its development and deploy- Dovey Fishman, T., Eggers, W.D., and Kishnani, P.
ernment case workers and make social ment, and proactive in anticipating its AI-augmented human services: Using
programs more efficient is to couple impact, we can hopefully reach a place cognitive technologies to transform
the technical progress being made where there will be fewer adverse unin- program delivery, Deloitte Insights, 2017,
with educating people on how to use tended consequences.” https://www2.deloitte.com/insights/
us/en/industry/public-sector/artificial-
these programs, Etzioni says. For the foreseeable future, Hen- intelligence-technologies-human-services-
“Part of the problem is when a hu- dler believes humans and computers programs.html
man just blindly adheres to the rec- working together will outperform ei- Zhao, J., Wang, T., Yatskar, M.,
ommendations of the system without ther one separately. For the partner- Ordonez, V., and Chang, K..
trying to make sense of them, and the ship to work, a human must be able Men Also Like Shopping: Reducing
system says, ‘It must be true,’ but if to understand the decision-making of Gender Bias Amplification using Corpus-
the machine’s analysis is one output the AI system, he says. level Constraints, University of Virginia.
Proceedings of the 2017 Conference on
and a sophisticated person analyzes “We currently teach people to take Empirical Methods in Natural Language
it, we find ourselves in the best of the data and feed it into AI systems to Processing, pages 2979–2989 Copenhagen,
both worlds.” get an ‘unbiased answer.’ That unbi- Denmark, Sept. 7–11, 2017. https://pdfs.
AI, he says, really should stand for ased answer is used to make predic- semanticscholar.org/566f/34fd344607693e
490a636cdbf3b92f74f976.pdf?_
“augmented intelligence,” where tech- tions and help people find services,’’
ga=2.37177120.1400811332.1523294823-
nology plays a supporting role, he says. Hendler says. “The problem is, the 1569884054.1523294823
“Humans are better than com- data coming in has been chosen in
Tan, S., Caruana, R., Hooker, G., and Lou, Y.
puters at exploring those grey ar- various ways, and we don’t educate Auditing Black-Box Models Using
eas around the edges of problems,’’ computer or data scientists how to Transparent Model Distillation With
agrees Hendler. “Computers are bet- know the data in your database will Side Information, 17 Oct. 2017, Cornell
ter at the black-and-white decisions model the real world.” University Library,
https://arxiv.org/abs/1710.06169
in the middle.” This is certainly not a new prob-
The issue of transparency of algo- lem. Hendler recalls the famous case O’Neil, C.
rithms and bias was discussed at a of Stanislov Petrov, a Soviet lieuten- Weapons of Math Destruction. 2016.
Crown Random House.
November 2017 conference held by ant-colonel whose job was to monitor
the Paris-based Organization for Eco- his country’s satellite system. In 1983, Hardt, M., Price, E., and Srebro, N.
Equality of Opportunity
nomic Cooperation and Development the computers sounded an alarm in-
in Supervised Learning
(OECD). Although several beneficial so- dicating the U.S. had launched nu- October 11, 2016
cietal use-cases of AI were mentioned, clear missiles. Instead of launching a https://arxiv.org/pdf/1610.02413.pdf
researchers said the solution lies in ad- counterattack, Petrov felt something
dressing system bias from a policy per- was wrong and refused; it turned out Esther Shein is a freelance technology and business
writer based in the area of Boston, MA, USA.
spective as well as a design perspective. to be a computer malfunction. AI sci-
“Right now, AI is designed so as entists, says Hendler, should learn
to optimize a given objective,” the from Petrov. © 2018 ACM 0001-0782/18/10 $15.00
O C TO B E R 2 0 1 8 | VO L. 6 1 | N O. 1 0 | C OM M U N IC AT ION S OF T HE ACM 19
V
viewpoints
Technology Strategy
and Management
The Business of
Quantum Computing
Considering the similarities of quantum computing development
to the early years of conventional computing.
I
Laureate Richard
N 1 9 8 1 , N O BE L represent information equal to the es- computer needs multiple qubits to do
Feynman challenged the com- timated number of particles in the calculations, and at least 50 qubits to
puting community to build a known universe.21 To perform calcula- do anything useful.14 We might need
quantum computer. We have tions, qubits exploit superposition 4,000 to 8,000 entangled qubits to sur-
come a long way. In 2015, and “entanglement.” This refers to pass current encryption technology us-
McKinsey estimated there were 7,000 when two quantum systems (such as ing very large integers.3 Programming
researchers working on quantum com- an electron or a nucleus), once they in- the devices also requires specialized
puting, with a combined budget of $1.5 teract, become connected and retain a hardware design skills, not conven-
billion.20 In 2018, dozens of universi- specific correlation in their spin or en- tional software programing skills.3
ties, approximately 30 major compa- ergy states (which represent combina- Entangled qubits are difficult to
nies, and more than a dozen startups tions of 0 and 1), even if physically sep- use and scale because of another phe-
had notable R&D efforts.a Now seems arate. Entanglement makes it possible nomenon called “decoherence.” The
like a good time to review the business. for quantum bits to work together and specific correlations between quan-
How do quantum computers work? represent multiple combinations of tum states can dissipate over time,
Quantum computers are built around values simultaneously, rather than thus destroying the ability of qubits to
circuits called quantum bits or qubits. represent one combination at a time. explore multiple solutions simultane-
One qubit can represent not just 0 or 1 Once a calculation is finished, you ob- ously. A useful analogy is to think of
as in traditional digital computers, but serve the qubits directly as 0 or 1 values qubit outputs like smoke rings blown
0 or 1 or both simultaneously—a phe- to determine the solution, as with a from a cigar.14 The rings can represent
nomenon called “superposition.” A classical computer. information but disintegrate (lose
pair of qubits can represent four states, What are the technical hurdles? Qu- their “coherence”) quickly. Since en-
three qubits eight states, and so on. bits resemble hardwired logic gates tangled qubits have a small probabili-
N qubits can represent 2n bits of in- usually made of atomic particles and su- ty of taking on different values due to
formation, and even 300 qubits can perconductor materials chilled to near- external interactions, the computa-
absolute zero. A one-qubit system is not tions require another process to de-
a https://bit.ly/2OXEA5n so difficult to build, but a quantum tect and correct errors.
The D-Wave 2000Q chip, designed to run quantum computing problems, increases from 1,000 qubits to 2,000 qubits, allowing larger
problems to be run—increasing the number of qubits yields an exponential increase in the size of the feasible search space.
How many different ways are there to time. The answer that requires the low- more than 400 per year in 2016–2017.
build quantum computers? There are est energy represents the optimal solu- The U.S. leads with approximately 800
several competing technologies. D- tion.10 However, some critics note that total patents, three to four times the
Wave was founded in 1999 to accumu- D-Wave qubits do not all seem to work numbers from Japan and China. The
late patent rights in exchange for re- together or exhibit quantum entangle- company with the largest portfolio is
search grants.17 It has been funded ment, and may not operate faster than D-Wave, followed by IBM (which
mainly by venture capital, corporate conventional computers.4 started research in 1990) and then
investors such as Goldman Sachs, and Google and IBM, as well as startups Microsoft. IBM leads in annual pat-
more recently, Jeff Bezos and the CIA.13 such as Quantum Circuits and Righet- ent filings. At universities, the lead-
The company has focused on “adiabat- ti Computing, deploy a different logic- ers in patent applications are MIT,
ic quantum computing,” also known gate approach, using entangled elec- Harvard, Zhejiang (China), Yale, and
as “quantum annealing.” D-Wave used trons or nuclei.19 Xanadu, a Toronto Tsinghua (China).2
this approach to build a 28-qubit de- startup, uses photons.b Microsoft’s de- What are some applications where
vice in 2007 and has been marketing a sign relies on quasi-particles called quantum computers should excel?
2,000-qubit device since 2017. Each anyons. Arranged into “topological qu- Experts list mathematical problems that
D-Wave qubit is a separate lattice con- bits,” these resemble braided knots on require massive parallel computations
tained within a magnetic field of Jo- a string, with (theoretically) high levels such as in optimization and simula-
sephson Junctions (logic circuits made of stability and coherence. Microsoft tion, cryptography and secure commu-
of superconductor materials that ex- plans to build a device within five years nications, pattern matching and big-
ploit quantum tunneling effects) and and make it commercially available via data analysis, and artificial intelligence
couplers (which link the circuits and the cloud.1,16 and machine learning.
IMAGE COURTESY OF D -WAVE
pass information). You program the Who leads in the patent race? Pat- D-Wave computers seem to generate
device by loading mathematical equa- ent-related publications have in- “good enough” solutions to complex
tions into the lattices. The processor creased from a handful in the 1990s to combinatorial optimization problems
then explores all possible solutions si- with many potential solutions. For ex-
multaneously, rather than one at a b https://bit.ly/2B04tP1 ample, in 2012, Harvard researchers
O C TO B E R 2 0 1 8 | VO L. 6 1 | N O. 1 0 | C OM M U N IC AT ION S OF T HE ACM 21
viewpoints
R
“ E A L ” C Y B E R S E C U R I T Y TO D AY
devotes enormous effort to
non-code vulnerabilities and
responses. The Cybersecu-
rity Workforce Frameworka
of the National Initiative for Cyberse-
curity Education lists 33 specialty ar-
eas for cybersecurity jobs. Ten of the
specialty areas primarily involve cod-
ing, but more than half primarily in-
volve non-code work (15 areas, in my
estimate) or are mixed (eight areas,
per my assessment).
This column proposes a Pedagogic
Cybersecurity Framework (PCF) for
categorizing and teaching the jumble
of non-code yet vital cybersecurity
topics. From my experience teach-
ing cybersecurity to computer sci-
ence and other majors at Georgia
Tech, the PCF clarifies how the var-
ied pieces in a multidisciplinary cy-
bersecurity course fit together. The
framework organizes the subjects
that have not been included in tra-
ditional cybersecurity courses, but
instead address cybersecurity man-
agement, policy, law, and interna-
tional affairs.
The PCF adds layers beyond the the “user layer.”b The framework pro- framework would benefit cybersecuri-
traditional seven layers in the Open posed here adds three layers—layer ty students, instructors, researchers,
Systems Interconnection model 8 is organizations, layer 9 is govern- and practitioners. Layers 8–10 clas-
(“OSI model” or “OSI stack”). Previ- ments, and layer 10 is international. sify vulnerabilities and mitigations
IMAGE BY UNCO NVENTIONA L
ous writers have acknowledged the This column explains how the new that are frequently studied by non-
possibility of a layer or layers beyond computer scientists, but are also
seven, most commonly calling layer 8 b Varying previous definitions of higher
critical for a holistic understand-
layers of the OSI Model are available at ing of the cybersecurity ecosystem by
a https://bit.ly/2McPRB3 https://en.wikipedia.org/wiki/Layer_8. computing professionals.
O C TO B E R 2 0 1 8 | VO L. 6 1 | N O. 1 0 | C OM M U N IC AT ION S OF T HE ACM 23
viewpoints
Table 1. Vulnerabilities at each layer of the expanded OSI stack. The Abstraction Layers
of the OSI Model
As discussed in the column, for layers 8–10, “A” refers to vulnerabilities The PCF builds on the Open Systems
and risk mitigation arising within the organization or nation; “B” refers Interconnection model (OSI) stack fa-
to vulnerability and risk mitigation in relation with other actors at that miliar to most computer scientists. It
level; and “C” refers to other limits created by actors at that level.
treats the stack primarily as a concep-
Layer Vulnerability tual framework for organizing how we
1. Physical Cut the wire; stress equipment; wiretap understand computing systems, par-
ticularly in the security domain. The
2. Data link Add noise or delay (threatens availability)
OSI model describes abstraction layers
3. Network DNS and BGP attacks; false certificates that enable the student or practitioner
4. Transport Man in the middle to focus on where a problem may ex-
ist, such as the physical, network, or
5. Session Session splicing (Firesheep); MS SMB
application layer. While retaining the
6. Presentation Attacks on encryption; ASN-1 parser attack abstraction layers from the OSI model,
7. Application Malware; manual exploitation of vulnerabilities; SQL injection; buffer overflow the PCF does not emphasize the role
8. Organization A: Insider attacks; poor training or policies
of the OSI model as a standardizing
B: Sub-contractors with weak cybersecurity; lack of information sharing model. Instead, it broadens students’
C: Weak technical or organizational standards understanding by focusing attention
9. Government A: Laws prohibiting effective cybersecurity (for example, limits on encryption); on the critical domains that introduce
weak laws for IoT or other security well-documented and well-understood
B: Badly drafted cybercrime laws (for example, prohibiting security research)
C: Excessive government surveillance
risks from management, government,
and international affairs. I provide
10. International A: Nation-state cyberattacks
B: Lack of workable international agreements to limit cyberattacks supplemental materials online that
C: Supranational legal rules that weaken cybersecurity further discuss the relationship of the
(for example, some International Telecommunications Union proposals) PCF to the OSI model and expand other
points made in this column.c
As a conceptual framework for un-
derstanding computer systems, the
Table 2. The pedagogic cybersecurity framework. seven traditional layers apply intuitive-
ly to cybersecurity risks, as discussed
A: Risk Mitigation
by Glenn Surman in his 2002 article
Layer of the Within an
Expanded OSI Organization or B: Relations with C: Other Limits Protocol “Understanding Security Using the OSI
Stack Nation Other Actors from This Level Data Unit Model.”2 Surman concluded: “The most
8: Organization 8A: Internal 8B: Vulnerability 8C: Standards and Contracts critical thing you should take from this
policies or plans management in limits originating paper is that for every layer there are at-
of action to reduce contracts with from the private tacks being created, or attacks awaiting
risk within an other entities, sector (for
organization (for like vendors (for example, PCI DSS
activation as a result of poor defence.”
example, incident example, cyber- standard, led by Bob Blakley from Citicorp assisted with
response plans). insurance). the PCI Cyber these illustrations of vulnerabilities
Security Standards that exist at each of the seven layers,
Council).
and I have added vulnerabilities exist-
9: Government 9A: Laws that 9B: Laws that 9C: Government Laws ing at layers 8, 9, and 10.
govern what an govern how limits on its
individual or organizations own actions (for
As a way to introduce layers 8 through
organization can and individuals example, Fourth 10, each horizontal layer highlights im-
or must do (for interact (for Amendment, portant types of cybersecurity vulner-
example, HIPAA example, limits on illegal
abilities. At layer 8, organizations face
Security Rule). Computer Fraud searches).
and Abuse Act). a wide range of cyber-risks, and take
many actions to mitigate such risks. At
10: International 10A: Unilateral 10B: Formal 10C: Limits on Diplomacy
actions by one and informal nations that layer 9, governments enact and enforce
government relationship come from laws—good laws can reduce cyberse-
directed at one management with other nations curity risks, while bad laws can make
or more other other nations (for example, the
nations (for (for example, United Nations and
them worse. At layer 10, the interna-
example, U.S. the Budapest international law). tional realm, no one nation can impose
Cyber Command Convention’s its laws, but treaties or discussions with
launching a provisions about Russia and China, for instance, may im-
cyberattack on a cybercrime and
hostile nation). Mutual Legal prove cybersecurity. As shown in Table
Assistance).
c Supplementary materials on the framework
are available at https://bit.ly/2MJCrZq
O C TO B E R 2 0 1 8 | VO L. 6 1 | N O. 1 0 | C OM M U N IC AT ION S OF T HE ACM 25
viewpoints
Kode Vicious
The Obscene Coupling
Known as Spaghetti Code
Teach your junior programmers how to read code.
Dear KV,
Forgive me, for my ACM member-
ship has lapsed, and for my sins I
have been saddled with mentoring a
spaghetti coder.
I am working on a piece of new soft-
ware—greenfield for once—but with
stiff reliability requirements. My help-
er, a young, self-proclaimed “devop,”
aims to improve as a programmer, and,
unfortunately, this person got stuck
with me.
No matter how hard I constrain
the work I dole out, I just cannot stop
this helper from the obscene coupling
known as spaghetti code, all masquer-
ading under obsessive, perfect syntax.
We cannot even get into the hard reli-
ability aspects of the software, because
tangled messes that lint perfectly and
break opaquely just keep piling up.
After many approaches, each one
narrower in scope than the last, I mentality in implementation and Once upon a time, spaghetti code
have come down to doling out work open this person’s mind to engage was defined by the fact that it jumped
units that are constrained to writ- the actual problem at hand—what the all over the place without any rhyme
ing single, well-defined functions software does! or reason, but, as you say, you have
in a Python library, but even then I I do not want to botch this and pro- someone, who even when given a con-
am failing to keep this person from duce the next Darth Vader! strained contract such as single func-
needlessly chaining functions, si- Mr. Function Defines Form tions, is still able to make a plate of
lently mixing and transparently pasta of it.
passing data through multiple lay- Perhaps it is time to introduce the
ers of interfaces, and, most pain- Dear Function, idea of narrative to your Padawan.
fully, burying important error out- Well, at least you didn’t mention goto, Code, as I have pointed out countless
put in ways we all know too well as the root of much of the spaghetti code times, is a form of communication
IMAGE BY KONGSAK
spaghetti code. of my well-spent youth. Yes, KV was between the people who write and
Assuming this apprentice is will- once young, but because of program- maintain it and is only incidentally
ing and eager, how can one go about mers such as your ward, he has never executable on a machine, which we
breaking this fundamental coupling looked young or beautiful. call a computer. I cannot seem to say
O C TO B E R 2 0 1 8 | VO L. 6 1 | N O. 1 0 | C OM M U N IC AT ION S OF T HE ACM 27
viewpoints
Viewpoint
Building the Universal
Archive of Source Code
A global collaborative project for the benefit of all.
S
OFTWARE IS BECOMING the
fabric that binds our personal
and social lives, embodying a
vast part of the technologi-
cal knowledge that powers
our industry and fuels innovation. Soft-
ware is a pillar of most scientific research
activities in all fields, from mathematics
to physics, from chemistry to biology,
from finance to social sciences. Soft-
ware is also an essential mediator for ac-
cessing any digital information.
In short, a rapidly increasing part of
our collective knowledge is embodied
in, or dependent on, software artifacts.
Our ability to design, use, understand,
adapt, and evolve systems and devices
on which our lives have come to depend
relies on our ability to understand,
adapt, and evolve the source code of
the software that controls them.
Software source code is a precious, Source code is spread around a variety attention to software safety, security,
unique form of knowledge. It can be of platforms and infrastructures that we reliability, and traceability. But un-
readily translated into a form execut- use to develop and/or distribute it, and like other scientific fields, we lack
able by a machine, and yet it is human software projects often migrate from large-scale research instruments for
readable: Harold Abelson wrote “Pro- one to another: there is no universal enabling massive analysis of all the
grams must be written for humans to catalog that tracks it all. available software source code.
read,”1 and source code is the preferred Software can be deleted, corrupted, As computer scientists and profes-
form for modification of software arti- or misplaced. What’s even more worry- sionals, it is our duty, responsibility,
facts by developers.3 Quite differently ing, in recent years we have seen major and privilege to build a shared infra-
from other forms of knowledge, we code forges shut down, endangering structure that answers these needs.
have grown accustomed to use version- hundreds of thousands of publicly Not just for our community, not just
control systems that trace source code available software projects at once.6 for the technical and scientific com-
development, and provide precious in- We clearly need a universal archive munity, but for society as a whole.
sight into its evolution. As Len Shustek of software source code. Software Heritagea is an initiative
puts it, “Source code provides a view The deep penetration of software launched at Inria—the French Institute
IMAGE BY AND REY VP
into the mind of the designer.”4 in all aspects of our world brings for Research in Computer Science and
And yet, we have not been taking along failures and risks whose po- Automation—precisely to take up this
good care of this precious form of tential impact is growing. Users now
knowledge. understand the need for an organized a See https://www.softwareheritage.org
O C TO B E R 2 0 1 8 | VO L. 6 1 | N O. 1 0 | C OM M U N IC AT ION S OF T HE ACM 29
viewpoints
mission. While a full article detailing and enables full deduplication (mas-
our approach is available online,2 we sively reducing storage costs), integrity
focus here on the challenges raised by We are at a unique checking, and tracking of reuse across
the three main goals: collecting, pre- turning point in all software projects at the file level.
serving, and sharing the source code But it also poses novel challenges
of all the software ever written. the history when it comes to efficiently indexing
of computer science and querying its contents.
Collection
There are various kinds of source code. and technology. Sharing
Some is current, actively developed, The raw material that Software Heritage
and technically easy to make available; collects must be properly organized
some other is legacy source code that to ease its fruition. On top of the infor-
must be painfully retrieved from offline mation captured by version-control
media. Some is open, and free for all to ness need to keep software closed fades systems, we need metadata describing
read and reuse; some is closed behind away, a focused search (that requires a the software and means to classify the
proprietary doors. Software Heritage’s costly and dedicated effort) can succeed millions of harvested projects, written
ambition is to collect it all. in recovering and liberating its source in one of the thousands of known pro-
For current, open source code, we code, growing our software commons. gramming languages.e We need to ex-
need an automated process to harvest all Finally, by providing a means to tract and reconcile existing information
software projects, with all the available safely keep closed source software un- from many different sources, encoded
development history, from the many der embargo, much like what happens in one of the many different software
places where development and distri- already with software escrow, we may ontologies, and complete it using either
bution take place, like forges and pack- succeed in collecting current and future automatic tools or crowdsourcing.
age repositories. Yes, we really mean closed source, and be ready to liberate it We must also support the many use
harvesting everything available, with no when time comes, dispensing altogeth- cases that it enables. Programmers
a priori filtering. Because the value of er with costly technical recovery efforts. may want to search for specific project
an active software project will only be versions or code snippets to reuse, and
known in the future, and because stor- Preservation then browse them online or download
ing all present and future source code In the extensive literature on digital history-full source code bundles. Com-
can be done at a reasonable cost. preservation, it is now well established panies may want to access an API to
The technical challenge is to build that long-term preservation requires build applications that use the archive.
crawlers for each code-hosting plat- full access to the source code of the Researchers may want to access the
form, as there is no common protocol tools used for the task. Software Heri- whole corpus to perform big data opera-
available, and to develop adapters for tage uses and develops exclusively free tions or train machine learning models.
all version-control systems and package and open source software tools for We must carefully assess which
formats. It is a significant undertaking, building its archive. functionalities are generic enough to
but once a standard platform is avail- Also, replication and diversifica- be incorporated in the archive, and
able each of these crawlers and adapters tion are best practices to mitigate the which are so specific that they are best
can be developed in parallel. threats—from technical failures to implemented externally by third par-
For legacy, open source code, we legal and economic decisions—that ties. And there are of course legal and
need a crowdsourcing platform to endanger any long-term preservation ethical issues to be dealt with when
empower the volunteers that are will- initiative. Hence, we want to foster a redistributing parts —or all—of the
ing to help recover their preferred geographically distributed network of contents of the archive.
software artifacts. Guidelines must be mirrors, implemented using a variety
offered to help properly reconstruct of storage technologies, in different ad- Current Status
from the raw material the interesting ministrative domains, controlled by a Software Heritage is an active project
history that lies behind it, like in the plurality of institutions, and located in that has already assembled the largest
beautiful work that has been done for different jurisdictions. existing collection of software source
the history of Unix.5 Finally, preserving software source code. At the time of writing the Software
Closed software contains precious code also requires preserving the de- Heritage Archive contains more than
knowledge that is more difficult to re- velopment history of source code, four billion unique source code files and
cover. For example, the Computer His- which carries precious insights into one billion individual commits, gath-
tory Museumb and Living Computersc the structure of programs and also ered from more than 80 million pub-
have shown, in the case of the mythi- tracks inter-project relationships. licly available source code repositories
cal Alto system,d that once the busi- Software Heritage’s unique approach (including a full and up-to-date mirror
is to store all available source code of GitHub) and packages (including a
b See http://www.computerhistory.org/ and its revisions into a single Merkle full and up-to-date mirror of Debian).
c See http://www.livingcomputers.org/
d See http://xeroxalto.computerhistory.org and
DAG (Directed Acyclic Graph), shared Three copies are currently maintained,
http://www.livingcomputers.org/Discover/ among all software projects. This
News/ContrAlto-A-Xerox-Alto-Emulator.aspx data structure facilitates distribution e See http://hopl.info/
O C TO B E R 2 0 1 8 | VO L. 6 1 | N O. 1 0 | C OM M U N IC AT ION S OF T HE ACM 31
V
viewpoints
DOI:10.1145/3209580 Jordi Cabot, Javier Luis Cánovas Izquierdo, and Valerio Cosentino
Viewpoint
Are CS Conferences (Too)
Closed Communities?
Assessing whether newcomers have a more difficult time
achieving paper acceptance at established conferences.
P
UBLICATION IN TOP confer-
ences is a key factor, albeit
controversial,3,4 in the dis-
semination of ideas and ca-
reer promotion in many ar-
eas of computer science. Therefore, it
is a major goal for every CS researcher.
However, many researchers believe
publishing in a top conference is
something reserved for the established
members of the conference commu-
nity. For newcomers, this is a tough
nut to crack. Indeed, when talking with
fellow researchers the assumed unspo-
ken truth is always the same: If you are
not one of “them,” you have no chance
to get “in” on your own.
If this were true, it would imply that
senior researchers wishing to change
fields during their research career may
have a difficult time doing so. And the
impact would be even more dramatic change it)? Our goal in this Viewpoint Computer Software category, for which
for junior researchers: they could only is to shed some light on these issues. we were able to find available data in
access top venues by going together with the DBLP dataset, the well-known on-
their supervisor, limiting their options Looking at the Data line reference for computer science
to make a name for themselves—exactly To assess whether it is actually true bibliographic information. The choice
the opposite of what evaluation commit- that newcomers have a difficult time of CORE as ranking system is based on
tees typically require from candidates. getting their papers accepted, we have its widespread use.
Indeed, candidates are supposed to evaluated the number of newcomer We have analyzed the conferences us-
show their ability to propose and de- papers (research papers where all au- ing a seven-year window (that is, an au-
velop valid research lines independently thors are new to the conference, that is, thor is considered new to a conference if
of their supervisor, even better if it is in none of the authors has ever published he or she has not published in that confer-
a slightly different research field and a paper of any kind in that same con- ence in the last seven years). We only count
IMAGE F RO M SH UTT ERSTOCK.CO M
hence in a different community. ference) in 65 conferences. The list of full papers in the main research track
But is it true that conferences are selected conferences corresponds to (since getting short papers, posters,
closed communities? Or is it just a the list of international CS conferences demos, and so forth is typically easier
myth spread by those that tried and in the CORE ranking,a 2015 edition, but it barely counts toward promotion).
failed? And if so, how do we change Results show that newcomers’ pa-
this situation (and do we really need to a https://bit.ly/2MnAncz pers are indeed scarce. Most confer-
ences (88%) show a percentage of new- research lines in a new field, senior
comer papers under 40%. This value is researchers moving to a new research
significantly lower in top conferences, Satellite events interest, industrial researchers trying
with a median value of 14%. As specific play a positive role to disseminate their results …) able to
examples, well-regarded conferences bring a completely fresh perspective
show the following values: ICSE (5%), in the community. to the community.
OOPSLA (13%), ICFP (11%), RE (6%). The main challenge in opening up
We may be tempted to quickly dis- conferences comes from the fact that
miss these numbers by attributing the we do not really know the reasons why
low percentage of newcomers papers these numbers are so low. Do some
to a lack of newcomer submissions. potential newcomers refrain from sub-
While it is true that CS communities ties. Most likely, some readers believe mitting in the first place? Do they get
are shrinking (at least based on ACM this is exactly how things should be rejected more often than established
tables for SIG memberships), which and that newcomers must first learn authors? If the latter, are they being
could imply that the “newcomers pool” the community’s particular “culture” fairly rejected because their papers do
is smaller, our analysis suggests that (in the widest sense of the word, in- not follow the right structure, process,
newcomer paper submissions rep- cluding its topics of interest, pre- or evaluation standards? Or is there
resent at least one-third of the total ferred research methods, social be- a positive (unconscious) bias toward
number of submissions.b havior, vocabulary, and even writing known community members during
Additionally, for each conference, style) either by simply attending the the review phase?
we have also calculated the number of conference or warming-up publish- Narrowing down a root cause—or
semi-newcomer papers. A semi-new- ing in satellite events, before being causes—requires much more confer-
comer is a researcher that has never able to get their papers accepted in ence data to be publicly disclosed for
published in the main track but that the main research track. analysis. We hope this is a direction
has published before in other tracks We dare to disagree and argue that we will follow as a community. In the
(for example, a demo or a poster). Data the situation is getting to a point in meantime, we would like to suggest a
indicates publishing a paper as a semi- which is worth discussing how to few ideas we think are worth pursuing
newcomer is also difficult but slightly change course. The overall presence and that, most likely, should be com-
easier than doing so as a complete of newcomers decreases over time.2 bined in order to tackle this multifac-
newcomer. If you want to be part of a Besides, increasing travel and eco- eted challenge:
given community, it seems to pay off nomical restrictions make it difficult ˲˲ Open the review process. More
to first participate in that community to follow the (so far) “easier” path to and more conferences are adopting
via lesser competitive tracks or collo- enter the community, for example, a double-blind review model to avoid
cated satellite events. And the good many outsider researchers will not get bias. Its usefulness to avoid author
news is that, unsurprisingly, newcom- funded to attend a satellite event, pre- identification seems to be confirmed6
ers have reasonable chances of suc- venting them from learning the ropes but it is probably still fairly easy to
cess to get papers accepted in those of that particular community. spot whether the authors are at least
satellite events. Our data indicates the While closed communities have members of the community so bias
percentage of newcomer papers in sat- indeed some positive aspects (for ex- is not completely out of the question.
ellite events is over 30% in most confer- ample, a particular focus, a heritage We could go even further and aim for
ences and it frequently goes up to 50% to build upon, sense of security, and triple-blind reviews or, alternatively,
and over. Clearly, satellite events play a so forth) we believe they are now be- open reviews (where reviewers sign
positive role in the growth of the com- coming too closed. In our opinion, the reviews and/or reviews are later
munity. The full data is available, in- a healthier number for conferences released publicly).
cluding all conferences values and the would be having at least 25% of new- ˲˲ Identify and promote research
corresponding boxplot distributions comer papers in each edition. This topics with a lower entry barrier for
based on the conference rankings.c would ensure a continuous influx of newcomers either because they are new
fresh ideas and new members in the topics, and therefore not many people
Opening Up Conferences community among other benefits in the community work on them, or
We believe the data confirms CS con- of open communities such as better because they require less advanced
ferencesd behave as closed communi- diversity and inclusiveness. While skills/infrastructure.
junior researchers co-authoring a ˲˲ Increasing acceptance rates to
O C TO B E R 2 0 1 8 | VO L. 6 1 | N O. 1 0 | C OM M U N IC AT ION S OF T HE ACM 33
viewpoints
SIGIR, SIGCOMM, SIGCSE, mentoring may have a limited success Analysis of co-authorship graphs of CORE-ranked
software conferences. Scientometrics 109, 3 (Dec.
in getting the newcomers’ papers in 2016), 1665–1693.
SIGMOD/PODS, immediately, it could have a positive 3. Franceschet, M. The role of conference publications in
CS. Commun. ACM 53, 12 (Dec. 2010), 129.
and many more. long-lasting effect in speeding up the 4. Freyne, J. et al. Relative status of journal and
conference publications in computer science.
newcomer learning. Commun. ACM 53, 11 (Nov. 2010), 124.
˲˲ Draw ideas from other domains 5. Gebert, D. and Boerner, S. The open and the closed
For available titles and where they may face similar problems. corporation as conflicting forms of organization. J.
Appl. Behav. Sci. 35, 3 (Sept. 1999), 341–359.
ordering info, visit: For instance, in the open source com- 6. Le Goues, C. et al. Effectiveness of anonymization in
double-blind review. Commun. ACM 61, 6 (June 2018),
munity, many projects struggle to at-
librarians.acm.org/pod tract new contributors and have come
30–33.
7. Steinmacher, I. et al. A systematic literature review
on the barriers faced by newcomers to open source
up with proposals to attract more peo- software projects. Inf. Softw. Technol. 59 (2015), 67–85.
ple.7 Examples (adapted to our field) 8. Vardi, M.Y. Divination by program committee.
Commun. ACM 60, 9 (Aug. 2017), 7.
would be to have a dedicated portal for 9. Vasilescu, B. et al. How healthy are software
newcomers clearly explaining how pa- engineering conferences? Sci. Comput. Program. 89,
PART C (2014), 251–272.
pers in the conference are evaluated,
showing examples of good papers (in Jordi Cabot (jordi.cabot@icrea.cat) is an ICREA
terms of style and structure), listing typ- Research Professor at the Universitat Oberta de
Catalunya (UOC), an Internet-centered open university
ical mistakes first submitters do based based in Barcelona, Spain.
on the experience of PC members, and Javier Luis Cánovas Izquierdo (jcanovasi@uoc.edu) is
so forth. And, importantly, encourag- a Postdoctoral Research Fellow at the Universitat Oberta
de Catalunya.
ing them to keep trying if they are not
Valerio Cosentino (vcosentino@uoc.edu) was a
initially successful—they may not be Postdoctoral Research Fellow at the Universitat Oberta
aware senior researchers also get many de Catalunya. Since September 2017, he is a software
developer at Bitergia, an open source company devoted to
papers rejected. offer software development analytics, part of the CHAOSS
Despite the number of works ana- project of the Linux Foundation.
lyzing co-authorship graphs, newcom-
ers metrics have been mostly ignored Copyright held by authors
The Mythos
nopoly on agency in society. If you ap-
plied for a job, loan, or bail, a human
decided your fate. If you went to the
hospital, a human would attempt to
of Model
categorize your malady and recom-
mend treatment. For consequential
decisions such as these, you might de-
Interpretability
mand an explanation from the deci-
sion-making agent.
If your loan application is denied,
for example, you might want to under-
stand the agent’s reasoning in a bid to
strengthen your next application. If
the decision was based on a flawed
premise, you might contest this prem-
ise in the hope of overturning the de-
cision. In the hospital, a doctor’s ex-
planation might educate you about
models boast
S U P E R V I S E D M AC H I N E - L E A R N I N G your condition.
remarkable predictive capabilities. But can you trust In societal contexts, the reasons for a
your model? Will it work in deployment? What else decision often matter. For example, in-
tentionally causing death (murder) vs.
can it tell you about the world? Models should be not unintentionally (manslaughter) are
only good, but also interpretable, yet the task of distinct crimes. Similarly, a hiring de-
cision being based (directly or indirect-
interpretation appears underspecified. The ly) on a protected characteristic such as
academic literature has provided diverse and race has a bearing on its legality. How-
sometimes non-overlapping motivations for ever, today’s predictive models are not
capable of reasoning at all.
interpretability and has offered myriad techniques Over the past 20 years, rapid prog-
for rendering interpretable models. Despite this ress in machine learning (ML) has led
to the deployment of automatic deci-
ambiguity, many authors proclaim their models to be sion processes. Most ML-based deci-
interpretable axiomatically, absent further argument. sion making in practical use works in
Problematically, it is not clear what common properties the following way: the ML algorithm
is trained to take some input and pre-
unite these techniques. dict the corresponding output. For ex-
This article seeks to refine the discourse on ample, given a set of attributes char-
acterizing a financial transaction, an
interpretability. First it examines the objectives of ML algorithm can predict the long-
previous papers addressing interpretability, finding term return on investment. Given im-
them to be diverse and occasionally discordant. ages from a CT scan, the algorithm
can assign a probability that the scan
Then, it explores model properties and techniques depicts a cancerous tumor. The ML al-
thought to confer interpretability, identifying gorithm takes in a large corpus of (in-
some label, only that certain inputs are ther: the definition of interpretability is sider these refining questions: What is
correlated with that label. For exam- universally agreed upon, but no one has interpretability? Why is it important?
ple, shown a dataset in which the only bothered to set it in writing; or the term Let’s address the second question
orange objects are basketballs, an im- interpretability is ill-defined, and, thus, first. Many authors have proposed in-
age classifier might learn to classify all claims regarding interpretability of var- terpretability as a means to engender
orange objects as basketballs. This ious models exhibit a quasi-scientific trust.9,24 This leads to a similarly vexing
model would achieve high accuracy character. An investigation of the litera- epistemological question: What is
even on held out images, despite fail- ture suggests the latter. Both the objec- trust? Does it refer to faith that a model
ing to grasp the difference that actually tives and methods put forth in the liter- will perform well? Does trust require a
makes a difference. ature investigating interpretability are low-level mechanistic understanding
O C TO B E R 2 0 1 8 | VO L. 6 1 | N O. 1 0 | C OM M U N IC AT ION S OF T HE ACM 37
practice
of models? Or perhaps trust is a subjec- you could consider the model’s com-
tive concept? plexity: Is it simple enough to be ex-
Other authors suggest that an inter- amined all at once by a human?
pretable model is desirable because it Other work has investigated so-
might help uncover causal structure in
observational data.1 The legal notion of What is trust? called post hoc interpretations. These
interpretations might explain predic-
a right to explanation offers yet another
lens on interpretability. Finally, some-
Is it simply tions without elucidating the mecha-
nisms by which models work. Exam-
times the goal of interpretability might confidence ples include the verbal explanations
simply be to get more useful informa-
tion from the model.
that a model will produced by people or the saliency
maps used to analyze deep neural net-
While the discussed desiderata, or perform well? works. Thus, human decisions might
objectives of interpretability, are di- admit post hoc interpretability despite
verse, they typically speak to situations the black-box nature of human brains,
where standard ML problem formula- revealing a contradiction between two
tions, for example, maximizing accu- popular notions of interpretability.
racy on a set of hold-out data for which
the training data is perfectly represen- Desiderata of
tative, are imperfectly matched to the Interpretability Research
complex real-life tasks they are meant This section spells out the various
to solve. Consider medical research desiderata of interpretability research.
with longitudinal data. The real goal The demand for interpretability arises
may be to discover potentially causal when a mismatch occurs between the
associations that can guide interven- formal objectives of supervised learn-
tions, as with smoking and cancer.29 ing (test-set predictive performance)
The optimization objective for most and the real-world costs in a deploy-
supervised learning models, however, ment setting.
is simply to minimize error, a feat that Typically, evaluation metrics re-
might be achieved in a purely correla- quire only predictions and ground-
tive fashion. truth labels. When stakeholders ad-
Another example of such a mis- ditionally demand interpretability,
match is that available training data you might infer the existence of ob-
imperfectly represents the likely de- jectives that cannot be captured in
ployment environment. Real environ- this fashion. In other words, because
ments often have changing dynamics. most common evaluation metrics for
Imagine training a product recom- supervised learning require only pre-
mender for an online store, where new dictions, together with ground truth, to
products are periodically introduced, produce a score, the very desire for an
and customer preferences can change interpretation suggests that some-
over time. In more extreme cases, ac- times predictions alone and metrics
tions from an ML-based system may calculated on them do not suffice to
alter the environment, invalidating fu- characterize the model. You should
ture predictions. then ask, what are these other objec-
After addressing the desiderata of tives and under what circumstances
interpretability, this article considers are they sought?
which properties of models might Often, real-world objectives are dif-
render them interpretable. Some pa- ficult to encode as simple mathemati-
pers equate interpretability with un- cal functions. Otherwise, they might
derstandability or intelligibility,16 just be incorporated into the objective
(that is, you can grasp how the models function and the problem would be
work). In these papers, understand- considered solved. For example, an al-
able models are sometimes called gorithm for making hiring decisions
transparent, while incomprehensible should simultaneously optimize pro-
models are called black boxes. But ductivity, ethics, and legality. But how
what constitutes transparency? You would you go about writing a func-
might look to the algorithm itself: tion that measures ethics or legality?
Will it converge? Does it produce a The problem can also arise when you
unique solution? Or you might look to desire robustness to changes in the
its parameters: Do you understand dynamics between the training and
what each represents? Alternatively, deployment environments.
Trust. Some authors suggest inter- variables. You might hope, however, never encountered during training.
pretability is a prerequisite for trust.9,23 that by interpreting supervised learn- However, these are mistakes a human
Again, what is trust? Is it simply confi- ing models, you could generate hy- would not make, and it would be pref-
dence that a model will perform well? potheses that scientists could then erable that models not make these
If so, a sufficiently accurate model test. For example, Liu et al.14 empha- mistakes, either. Already, supervised
should be demonstrably trustworthy, size regression trees and Bayesian learning models are regularly subject
and interpretability would serve no neural networks, suggesting these to such adversarial manipulation. Con-
purpose. Trust might also be defined models are interpretable and thus bet- sider the models used to generate cred-
subjectively. For example, a person ter able to provide clues about the it ratings; higher scores should signify
might feel more at ease with a well-un- causal relationships between physio- a higher probability that an individual
derstood model, even if this under- logic signals and affective states. The repays a loan. According to its own
standing serves no obvious purpose. task of inferring causal relationships technical report, FICO trains credit
Alternatively, when the training and from observational data has been ex- models using logistic regression,6 spe-
deployment objectives diverge, trust tensively studied.22 Causal inference cifically citing interpretability as a mo-
might denote confidence that the mod- methods, however, tend to rely on tivation for the choice of model. Fea-
el will perform well with respect to the strong assumptions and are not widely tures include dummy variables
real objectives and scenarios. used by practitioners, especially on representing binned values for average
For example, consider the growing large, complex datasets. age of accounts, debt ratio, the number
use of ML models to forecast crime Transferability. Typically, training of late payments, and the number of
rates for purposes of allocating police and test data are chosen by randomly accounts in good standing.
officers. The model may be trusted to partitioning examples from the same Several of these factors can be ma-
make accurate predictions but not to distribution. A model’s generalization nipulated at will by credit-seekers. For
account for racial biases in the training error is then judged by the gap between example, one’s debt ratio can be im-
data or for the model’s own effect in its performance on training and test proved simply by requesting periodic
perpetuating a cycle of incarceration data. Humans exhibit a far richer capac- increases to credit lines while keeping
by over-policing some neighborhoods. ity to generalize, however, transferring spending patterns constant.
Another sense in which an end user learned skills to unfamiliar situations. Similarly, simply applying for new
might be said to trust an ML model ML algorithms are already used in accounts when the probability of ac-
might be if they are comfortable with these situations, such as when the en- ceptance is reasonably high can in-
relinquishing control to it. Through vironment is nonstationary. Models crease the total number of accounts.
this lens, you might care not only about are also deployed in settings where Indeed, FICO and Experian both ac-
how often a model is right, but also for their use might alter the environment, knowledge that credit ratings can be
which examples it is right. If the model invalidating their future predictions. manipulated, even suggesting guides
tends to make mistakes on only those Along these lines, Caruana et al.3 de- for improving one’s credit rating.
kinds of inputs where humans also scribe a model trained to predict prob- These rating-improvement strategies
make mistakes, and thus is typically ac- ability of death from pneumonia that may fundamentally change one’s un-
curate whenever humans are accurate, assigned less risk to patients if they also derlying ability to pay a debt. The fact
then you might trust the model owing had asthma. Presumably, asthma was that individuals actively and success-
to the absence of any expected cost of predictive of a lower risk of death be- fully game the rating system may inval-
relinquishing control. If a model tends cause of the more aggressive treatment idate its predictive power.
to make mistakes for inputs that hu- these patients received. If the model Informativeness. Sometimes, deci-
mans classify accurately, however, were deployed to aid in triage, these pa- sion theory is applied to the outputs of
then there may always be an advantage tients might then receive less aggres- supervised models to take actions in
to maintaining human supervision of sive treatment, invalidating the model. the real world. In another common use
the algorithms. Even worse, there are situations, paradigm, however, the supervised
Causality. Although supervised such as machine learning for security, model is used instead to provide infor-
learning models are only optimized where the environment might be ac- mation to human decision-makers, a
directly to make associations, re- tively adversarial. Consider the recent- setting considered by Kim et al.11 and
searchers often use them in the hope ly discovered susceptibility of convo- Huysmans et al.8 While the machine-
of inferring properties of the natural lutional neural networks (CNNs). The learning objective might be to reduce
world. For example, a simple regres- CNNs were made to misclassify images error, the real-world purpose is to pro-
sion model might reveal a strong as- that were imperceptibly (to a human) vide useful information. The most ob-
sociation between thalidomide use perturbed.26 Of course, this is not over- vious way that a model conveys infor-
and birth defects, or between smoking fitting in the classical sense. The mod- mation is via its outputs. However, we
and lung cancer.29 els both achieve strong results on train- might hope that by probing the pat-
The associations learned by super- ing data and generalize well when used terns that the model has extracted, we
vised learning algorithms are not guar- to classify held out test data. The cru- can convey additional information to a
anteed to reflect causal relationships. cial distinction is that these images human decision maker.
There could always be unobserved have been altered in ways that, while An interpretation may prove infor-
causes responsible for both associated subtle to human observers, the models mative even without shedding light on
O C TO B E R 2 0 1 8 | VO L. 6 1 | N O. 1 0 | C OM M U N IC AT ION S OF T HE ACM 39
practice
a model’s inner workings. For exam- in reasonable time step through every fragile with respect to feature selection
ple, a diagnosis model might provide calculation required to produce a pre- and preprocessing. For example, the
intuition to a human decision maker diction. This accords with the common coefficient corresponding to the asso-
by pointing to similar cases in support claim that sparse linear models, as ciation between flu risk and vaccina-
of a diagnostic decision. In some cas- produced by lasso regression,27 are tion might be positive or negative, de-
es, a supervised learning model is more interpretable than dense linear pending on whether the feature set
trained when the real task more close- models learned on the same inputs. includes indicators of old age, infancy,
ly resembles unsupervised learning. Ribeiro et al.23 also adopt this notion or immunodeficiency.
The real goal might be to explore the of interpretability, suggesting that an Algorithmic transparency. A final no-
underlying structure of the data, and interpretable model is one that “can tion of transparency might apply at the
the labeling objective serves only as be readily presented to the user with level of the learning algorithm itself. In
weak supervision. visual or textual artifacts.” the case of linear models, you may un-
Fair and ethical decision making. At The trade-offs between model size derstand the shape of the error surface.
present, politicians, journalists, and and computation to apply a single pre- You can prove that training will con-
researchers have expressed concern diction varies across models. For exam- verge to a unique solution, even for pre-
that interpretations must be produced ple, in some models, such as decision viously unseen datasets. This might
for assessing whether decisions pro- trees, the size of the model (total num- provide some confidence that the mod-
duced automatically by algorithms ber of nodes) may grow quite large el will behave in an online setting re-
conform to ethical standards.7 Recidi- compared to the time required to per- quiring programmatic retraining on
vism predictions are already used to form inference (length of pass from previously unseen data. On the other
determine whom to release and whom root to leaf). This suggests simulatabil- hand, modern deep learning methods
to detain, raising ethical concerns. ity may admit two subtypes: one based lack this sort of algorithmic transpar-
How can you be sure predictions do not on the size of the model and another ency. While the heuristic optimization
discriminate on the basis of race? Con- based on the computation required to procedures for neural networks are de-
ventional evaluation metrics such as perform inference. monstrably powerful, we do not under-
accuracy or AUC (area under the curve) Fixing a notion of simulatability, the stand how they work, and at present
offer little assurance that ML-based de- quantity denoted by reasonable is sub- cannot guarantee a priori they will
cisions will behave acceptably. Thus, jective. Clearly, however, given the lim- work on new problems. Note, however,
demands for fairness often lead to de- ited capacity of human cognition, this that humans exhibit none of these
mands for interpretable models. ambiguity might span only several or- forms of transparency.
ders of magnitude. In this light, nei- Post hoc interpretability represents a
The Transparency ther linear models, rule-based systems, distinct approach to extracting infor-
Notion of Interpretability nor decision trees are intrinsically in- mation from learned models. While
Let’s now consider the techniques and terpretable. Sufficiently high-dimen- post hoc interpretations often do not
model properties that are proposed sional models, unwieldy rule lists, and elucidate precisely how a model works,
to confer interpretability. These fall deep decision trees could all be consid- they may nonetheless confer useful in-
broadly into two categories. The first ered less transparent than compara- formation for practitioners and end us-
relates to transparency (that is, how tively compact neural networks. ers of machine learning. Some com-
does the model work?). The second Decomposability. A second notion of mon approaches to post hoc
consists of post hoc explanations (that transparency might be that each part interpretations include natural lan-
is, what else can the model tell me?) of the model—input, parameter, and guage explanations, visualizations of
Informally, transparency is the oppo- calculation—admits an intuitive expla- learned representations or models,
site of opacity or “black-boxness.” It con- nation. This accords with the property of and explanations by example (for ex-
notes some sense of understanding the intelligibility as described by Lou ample, a particular tumor is classified
mechanism by which the model works. et al.15 For example, each node in a as malignant because to the model it
Transparency is considered here at the decision tree might correspond to a looks a lot like certain other tumors).
level of the entire model (simulatabili- plain text description (for example, all To the extent that we might consider
ty), at the level of individual compo- patients with diastolic blood pressure humans to be interpretable, this is the
nents such as parameters (decompos- over 150). Similarly, the parameters of sort of interpretability that applies. For
ability), and at the level of the training a linear model could be described as all we know, the processes by which hu-
algorithm (algorithmic transparency). representing strengths of association mans make decisions and those by
Simulatability. In the strictest sense, between each feature and the label. which they explain them may be dis-
a model might be called transparent if Note this notion of interpretability tinct. One advantage of this concept of
a person can contemplate the entire requires that inputs themselves be in- interpretability is that opaque models
model at once. This definition suggests dividually interpretable, disqualifying can be interpreted after the fact, with-
an interpretable model is a simple some models with highly engineered out sacrificing predictive performance.
model. For example, for a model to be or anonymous features. While this no- Text explanations. Humans often
fully understood, a human should be tion is popular, it should not be accept- justify decisions verbally. Similarly,
able to take the input data together ed blindly. The weights of a linear mod- one model might be trained to gener-
with the parameters of the model and el might seem intuitive, but they can be ate predictions, and a separate model,
O C TO B E R 2 0 1 8 | VO L. 6 1 | N O. 1 0 | C OM M U N IC AT ION S OF T HE ACM 41
practice
you may get a very different saliency related work in Bayesian methods,
map. This contrasts with linear mod- investigating case-based reasoning
els, which model global relationships approaches for interpreting genera-
between inputs and outputs. tive models.
Another attempt at local explana-
tions is made by Ribeiro et al.23 In this An inspection Discussion
work, the authors explain the decisions
of any model in a local region near a
of the perturbed The concept of interpretability ap-
pears simultaneously important and
particular point by learning a separate inputs can give slippery. Earlier, this article analyzed
sparse linear model to explain the deci-
sions of the first. Strangely, although
clues to what both the motivations for interpretabil-
ity and some attempts by the research
the method’s appeal over saliency the model community to confer it. Now let’s con-
maps owes to its ability to provide ex-
planations for non-differentiable mod-
has learned. sider the implications of this analysis
and offer several takeaways.
els, it is more often used when the ˲˲ Linear models are not strictly more
model subject to interpretation is in interpretable than deep neural networks.
fact differentiable. In this case, what is Despite this claim’s enduring popular-
provided, besides a noisy estimate of ity, its truth value depends on which
the gradient, remains unclear. In this notion of interpretability is employed.
paper, the explanation is offered in With respect to algorithmic transpar-
terms of a set of superpixels. Whether ency, this claim seems uncontrover-
or not this is more informative than a sial, but given high-dimensional or
plain gradient may depend strongly on heavily engineered features, linear
how one chooses the superpixels. models lose simulatability or decom-
Moreover, absent a rigorously defined posability, respectively.
objective, who is to say which hyper- When choosing between linear and
parameters are correct? deep models, you must often make a
Explanation by example. One post tradeoff between algorithmic transpar-
hoc mechanism for explaining the de- ency and decomposability. This is be-
cisions of a model might be to report cause deep neural networks tend to op-
(in addition to predictions) which erate on raw or lightly processed
other examples are most similar with features. So, if nothing else, the features
respect to the model, a method sug- are intuitively meaningful, and post hoc
gested by Caruana et al.2 Training a reasoning is sensible. To get compara-
deep neural network or latent variable ble performance, however, linear mod-
model for a discriminative task pro- els often must operate on heavily hand-
vides access to not only predictions engineered features. Lipton et al.13
but also the learned representations. demonstrate such a case where linear
Then, for any example, in addition to models can approach the performance
generating a prediction, you can use of recurrent neural networks (RNNs)
the activations of the hidden layers to only at the cost of decomposability.
identify the k-nearest neighbors based For some kinds of post hoc interpre-
on the proximity in the space learned tation, deep neural networks exhibit a
by the model. This sort of explanation clear advantage. They learn rich repre-
by example has precedent in how hu- sentations that can be visualized, ver-
mans sometimes justify actions by balized, or used for clustering. Consid-
analogy. For example, doctors often ering the desiderata for interpretability,
refer to case studies to support a linear models appear to have a better
planned treatment protocol. track record for studying the natural
In the neural network literature, world, but there seems to be no theo-
Mikolov et al.19 use such an approach to retical reason why this must be so.
examine the learned representations Conceivably, post hoc interpretations
of words after training the word2vec could prove useful in similar scenarios.
model. Their model is trained for dis- ˲˲ Claims about interpretability must
criminative skip-gram prediction, to be qualified. As demonstrated here,
examine which relationships the mod- the term interpretability does not ref-
el has learned they enumerate nearest erence a monolithic concept. To be
neighbors of words based on distanc- meaningful, any assertion regarding
es calculated in the latent space. Kim interpretability should fix a specific
et al.10 and Doshi-Velez et al.5 have done definition. If the model satisfies a form
of transparency, this can be shown models and environments. This capa- via intuitive interaction. Massachusetts Institute of
Technology, Cambridge, MA, 2015.
directly. For post hoc interpretabil- bility, however, may come at the cost of 12. Krening, S., Harrison, B., Feigh, K., Isbell, C., Riedl,
ity, work in this field should fix a clear allowing models to experiment in the M. and Thomaz, A. Learning from explanations using
sentiment and advice in RL. IEEE Trans. Cognitive and
objective and demonstrate evidence world, incurring real consequences. Developmental Systems 9, 1 (2017), 41–55.
that the offered form of interpretation Notably, reinforcement learners 13. Lipton, Z.C., Kale, D.C. and Wetzel, R. Modeling missing
data in clinical time series with RNNs. In Proceedings
achieves it. are able to learn causal relationships of Machine Learning for Healthcare, 2016.
˲˲ In some cases, transparency may be between their actions and real-world 14. Liu, C., Rani, P. and Sarkar, N. 2006. An empirical study
of machine-learning techniques for affect recognition
at odds with the broader objectives of impacts. Like supervised learning, in human-robot interaction. Pattern Analysis and
AI (artificial intelligence). Some argu- however, reinforcement learning re- Applications 9, 1 (2006), 58–69.
15. Lou, Y., Caruana, R. and Gehrke, J. Intelligible models
ments against black-box algorithms lies on a well-defined scalar objective. for classification and regression. In Proceedings of the
18th ACM SIGKDD Intern. Conf. Knowledge Discovery
appear to preclude any model that For problems such as fairness, where and Data Mining, 2012, 150–158.
could match or surpass human abili- we struggle to verbalize precise defi- 16. Lou, Y., Caruana, R., Gehrke, J. and Hooker, G. Accurate
intelligible models with pairwise interactions. In
ties on complex tasks. As a concrete nitions of success, a shift of the ML Proceedings of the 19th ACM SIGKDD Intern. Conf.
example, the short-term goal of build- paradigm is unlikely to eliminate the Knowledge Discovery and Data Mining, 2013, 623–631.
17. Mahendran, A. and Vedaldi, A. Understanding
ing trust with doctors by developing problems we face. deep image representations by inverting them. In
transparent models might clash with Proceedings of the IEEE Conf. Computer Vision and
Pattern Recognition, 2015, 1–9.
the longer-term goal of improving 18. McAuley, J. and Leskovec, J. Hidden factors and
Related articles
health care. Be careful when giving on queue.acm.org
hidden topics: Understanding rating dimensions with
review text. In Proceedings of the 7th ACM Conf.
up predictive power that the desire Recommender Systems, 2013, 165–172.
Accountability in
for transparency is justified and not 19. Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S. and
Algorithmic Decision Making Dean, J. Distributed representations of words and
simply a concession to institutional Nicholas Diakopoulos phrases and their compositionality. In Proceedings of
biases against new methods. the 26th Intern. Conf. Neural Information Processing
https://queue.acm.org/detail.cfm?id=2886105 Systems 2, 2013, 3111–3119.
˲˲ Post hoc interpretations can poten- 20. Mordvintsev, A., Olah, C. and Tyka, M. Inceptionism:
Black Box Debugging
tially mislead. Beware of blindly em- Going deeper into neural networks. Google AI Blog;
James A. Whittaker and Herbert H. Thompson https://ai.googleblog.com/2015/06/inceptionism-
bracing post hoc notions of interpret- https://queue.acm.org/detail.cfm?id=966807 going- deeper-into-neural.html.
21. Mounk, Y. Is Harvard unfair to Asian-Americans?
ability, especially when optimized to Hazy: Making It Easier to Build New York Times (Nov. 24, 2014); http://www.nytimes.
placate subjective demands. In such and Maintain Big-Data Analytics com/2014/11/25/opinion/is-harvard-unfair-to-asian-
Arun Kumar, Feng Niu, and Christopher Ré americans.html.
cases, one might—deliberately or 22. Pearl, J. Causality. Cambridge University Press,
not—optimize an algorithm to pres- https://queue.acm.org/detail.cfm?id=2431055 Cambridge, MA, 2009.
23. Ribeiro, M.T., Singh, S. and Guestrin, C. ‘Why should
ent misleading but plausible expla- I trust you?’ Explaining the predictions of any
References
nations. As humans, we are known to 1. Athey, S. and Imbens, G.W. Machine-learning
classifier. In Proceedings of the 22nd SIGKDD Intern.
Conf. Knowledge Discovery and Data Mining, 2016,
engage in this behavior, as evidenced methodsm 2015; https://arxiv.org/abs/1504.01132v1.
1135–1144.
2. Caruana, R., Kangarloo, H., Dionisio, J. D, Sinha, U. and
in hiring practices and college admis- Johnson, D. Case-based explanation of non-case-
24. Ridgeway, G., Madigan, D., Richardson, T. and O’Kane,
J. Interpretable boosted naïve Bayes classification.
sions. Several journalists and social based learning methods. In Proceedings of the Amer.
In Proceedings of the 4th Intern. Conf. Knowledge
Med. Info. Assoc. Symp., 1999, 12–215.
scientists have demonstrated that 3. Caruana, R., Lou, Y., Gehrke, J., Koch, P., Sturm, M.
Discovery and Data Mining, 1998, 101–104.
25. Simonyan, K., Vedaldi, A., Zisserman, A. Deep
acceptance decisions attributed to and Elhadad, N. Intelligible models for healthcare:
inside convolutional networks: Visualising image
Predicting pneumonia risk and hospital 30-day
virtues such as leadership or origi- classification models and saliency maps, 2013; https://
readmission. In Proceedings of the 21st SIGKDD
arxiv. org/abs/1312.6034.
nality often disguise racial or gender Intern. Conf. Knowledge Discovery and Data Mining,
26. Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J.,
2017, 1721–1730.
discrimination.21 In the rush to gain Erhan, D., Goodfellow, I. and Fergus, R. Intriguing
4. Chang, J., Gerrish, S., Wang, C., Boyd-Graber, J.L., Blei,
properties of neural networks, 2013; https://arxiv.org/
acceptance for machine learning and D.M. 2009. Reading tea leaves: how humans interpret
abs/1312.6199.
topic models. In Proceedings of the 22nd Intern.
27. Tibshirani, R. 1996. Regression shrinkage and selection
to emulate human intelligence, we Conf. Neural Information Processing Systems, 2009,
via the lasso. J. Royal Statistical Society: Series B:
288–296.
should all be careful not to reproduce 5. Doshi-Velez, F., Wallace, B. and Adams, R. Graph-
Statistical Methodology 58, 1 (1996), 267–288.
28. Van der Maaten, L. and Hinton, G. Visualizing data
pathological behavior at scale. sparse lDA: A topic model with structured sparsity.
using t-SNE. J. Machine Learning Research 9 (2008),
In Proceedings of the 29th Assoc. Advance. Artificial
2579–2605.
Intelligence Conf., 2015, 2575–2581.
29. Wang, H.-X., Fratiglioni, L., Frisoni, G. B., Viitanen,
Future Work 6. Fair Isaac Corporation (FICO). Introduction to model
M. and Winblad, B. Smoking and the occurrence
builder scorecard, 2011; http://www.fico.com/en/
There are several promising directions latest- thinking/white-papers/introduction-to-model-
of Alzheimer’s disease: Cross-sectional and
longitudinal data in a population-based study. Amer. J.
for future work. First, for some prob- builder- scorecard.
Epidemiology 149, 7 (1999), 640–644.
7. Goodman, B. and Flaxman, S. European Union
lems, the discrepancy between real-life regulations on algorithmic decision-making and
30. Wang, Z., Freitas, N. and Lanctot, M. Dueling network
architectures for deep reinforcement learning. In
and machine-learning objectives could a ‘right to explanation,’ 2016; https://arxiv.org/
Proceedings of the 33rd Intern. Conf. Machine Learning
abs/1606.08813v3.
be mitigated by developing richer loss 8. Huysmans, J., Dejaeger, K., Mues, C., Vanthienen,
48, 2016, 1995–2003.
O C TO B E R 2 0 1 8 | VO L. 6 1 | N O. 1 0 | C OM M U N IC AT ION S OF T HE ACM 43
practice
DOI:10.1145/ 3233239
Article development led by
queue.acm.org
The Secret
Formula for
Choosing
the Right
Next Role and the one after that, and the one
after that.
When you are looking at the options
for your next role, there are smarter
choices that you can make. Here are
the most important factors to consider
when picking your next opportunity.
O C TO B E R 2 0 1 8 | VO L. 6 1 | N O. 1 0 | C OM M U N IC AT ION S OF T HE ACM 45
practice
˲˲ What is this job setting me up for? the future), but if it’s not, then you extra invested in that new hire doing
˲˲ What
will I have gained from this are just stuck. well once he or she joined the team?
role in two years, and are those gains For example, I have a friend who re- Even a minimal investment will have
valuable to me? ally wanted to work on machine learn- a psychological impact on your po-
ing, so he joined a team doing that type tential coworkers. If they meet you or
Pick People, Not Projects of work. For the 18 months he didn’t interview you, they will have already
Another easy trap to fall into when get to do anything related to machine invested some amount of time in you
picking your next job is to focus too learning, and instead was stuck writ- and will be more inclined to want to
much on the projects you think you ing deployment scripts and updates to see that investment rewarded.
will get to work on. data loaders—work that was much less ˲˲ You will not be “brand new” on
Of course, we all want to work on interesting to him than the project he your first day. As humans, we are
things that are interesting and excit- was on previously. naturally resistant to change and to
ing or that could make us rich and Projects are never guaranteed, so new people whom we know nothing
famous. The truth is projects get can- ensure you understand the specifics about. If you show up on your first
celed all the time. They change and and exactly what work you will get the day having met no one yet, you are
become less exciting. The roles within chance to do. Also, instead of think- a stranger; your coworkers are more
them change, and you could end up ing just about the work, I recommend likely to see you as an “outsider” tak-
doing legwork that is not actually very thinking also about whom you will be ing up space. Even a short meeting in
interesting or exciting to you. working with. advance will prime them to see you as
In college, I got a job working in Basing your decision on the people familiar the next time you see them.
a lab. I was so happy because I was you will be working with is one of the Plus, you will have some baseline
envisioning myself working on excit- best ways to pick a job. If you must knowledge about the team that can
ing experiments and getting my work choose between an exciting project help you fit in more quickly, as op-
published in major journals. While or a great team, always go for the posed to starting to learn about the
those exciting projects did happen great team. team culture after you have joined.
in this lab, I never got to do them. I Some 99% of my happiness in a job
ended up running the same experi- has to do with who my manager and Be Smart When You
ment day after day, collecting the coworkers are. I bet it is the same for Choose Your Next Role
same data over and over again. This you. You spend so much time at work; When you are searching for the next
is often what research is—you need if you work full time, you probably step in your career, don’t just think
to make sure any results are statisti- spend as much (or more) time with about the surface-level benefits. Drill
cally significant, so you do the same your coworkers than you do with your down on your biggest goals and do a
thing repeatedly. friends or family. little thinking about whether or not
The projects the lab was working In some organizations, it is com- each job will help you get closer to
on were exciting, but my life in the mon to interview with the boss and at those goals.
lab was not. least one other member of the team, The best careers are not defined
It is so important to consider what though this does not always happen. by titles or résumé bullet points. The
your day-to-day life will be like in a You should always ask for the opportu- smarter you are about what you choose
role. What will you actually spend nity to meet more of the people you will next, the closer you will get to the
your time doing? Will it add value be working with. things you truly want from your life
to your career? What will you get the This has a few benefits: and your work.
chance to learn? ˲˲ You can meet with the people you
Remember, when you are new to a will work with every day. Not only will
Related articles
team, you have no career capital built you get a feel for what it will be like on queue.acm.org
up with this organization. Career capi- working with them, you can also ask
10 Ways to Be a Better Interviewer
tal is your currency at work; when you them for insight into other aspects of
Kate Matsudaira
provide a lot of concrete, visible value the role. Do they like working there? https://queue.acm.org/detail.cfm?id=3125635
to the team or the organization, you How much turnover is there on the
Avoiding Obsolescence
have more leverage to do the things team? How does collaboration work? Kode Vicious
you want, such as work on the most ex- Does leadership listen to input on deci- https://queue.acm.org/detail.cfm?id=1781175
citing projects or get more flexibility in sions? What are the things they would A Generation Lost in the Bazaar
your schedule. want to change about the team/com- Poul-Henning Kamp
When you are new, you have not pany/culture? Why do they work there https://queue.acm.org/detail.cfm?id=2349257
earned this leverage. That means if vs. anywhere else?
you are assigned to a boring role on ˲˲ Your coworkers will feel invested Kate Matsudaira (katemats.com) is an experienced
technology leader. She has worked at Microsoft and
an exciting project, you pretty much in your success if they are part of Amazon and successful startups before starting her own
just have to do it. Sometimes that the process of hiring you. Think company, Popforms, which was acquired by Safari Books.
can be OK (maybe you actually want- about it—if you met with a candi-
ed to learn this boring skill because date you liked and fought for him Copyright © 2018 held by owner/author.
it will help you get a job you want in or her to be hired, wouldn’t you be Publication rights licensed to ACM. $15.00
Article development led by
queue.acm.org
Mind Your
State for Your
State of Mind
O C TO B E R 2 0 1 8 | VO L. 6 1 | N O. 1 0 | C OM M U N IC AT ION S OF T HE ACM 47
practice
fulfillment of the business needs and rally leads to applications and app code There have been and continue to
clarity of expectations. encapsulating the data so the distrusted be significant changes to the style of
This article provides a partial taxon- outsider cannot just modify the data computation, to storage, and to how
omy of diverse storage solutions avail- with abandon. these application patterns are used to
able over a distributed cluster. Part of As the industry started running stuff access storage.
this is an exploration of the interac- at huge scale, it learned that busting a This is only a partial list of storage
tions among different features of a service into smaller microservices has a and compute models. It is not meant to
store. The article then considers how couple of big advantages: be complete.
distinct application patterns have ˲˲ Better engineering. Breaking your Challenges in modern microservice-
grown over time to leverage these services (that is, trust boundaries) into based applications. These days, mi-
stores and the business requirements smaller pieces allows better engineer- croservices power many scalable apps.
they meet. This may have surprising ing flexibility as small teams make Microservices are pools of identical or
implications. quicker changes. equivalent services running over a col-
˲˲ Better operability. Making these lection of servers. Incoming requests
The Evolution of State, Storage, smaller pieces stateless and restartable are load balanced across the pool.
And Computing … At Least So Far allows for more resilient operations as
This section starts by examining some of failures, rolling upgrades of versions,
the profound changes that have occurred and adjustments for varying demand When a request waits for a microser-
in both storage and computation. The are dynamically handled. vice, any one from the same pool will
focus then turns to a discussion of both Microservices became an essential do the job. Sometimes, systems imple-
durable state and session state and how part of the software engineering and op- ment affinitization, where a subsequent
they have evolved over time. Finally, there erations landscape. request is likely to go to the same spe-
is a brief reminder of how data is treated cific microservice. Still, the outcome
differently inside a classic database and must be correct if you land on any of the
outside as it moves across trust and trans- Careful Replacement Variations microservices.
actional boundaries. ˲˲ A write may trash the previous
Trends in storage and computing. value … write somewhere else first.
Changes in storage and computing ˲ ˲ A client crash may interrupt a Microservices help scalable systems
have put demands on how storage is sequence of writes … plan carefully. in two broad ways:
accessed and the expected behavior in ˲˲ Improved software engineering.
doing so. This is especially interesting Building systems consisting of small
as work is smeared over pools of small Computing’s use of storage has and independent microservices results
computation known as microservices. evolved. It has been quite a wild ride of in agility. Teams owning the microser-
Storage has evolved. It used to be application changes as their use of stor- vices must be accountable and have
that storage was only directly attached age has evolved: independence and ownership. When
to your computer. Then came shared ˲˲ Direct file I/O used careful replace- something needs changing, change it.
appliances such as storage area net- ment for recoverability. Careful replace- When something is broken, the owning
works (SANs). These are big, expensive ment is a technique that is at least as team is responsible.
devices with a lot of sophisticated soft- old as the 1960s. It involves thoughtful ˲˲ Improved operations. Health-medi-
ware and hardware to provide highly ordering of changes to durable storage ated deployment allows for slow rollout
available storage to a bunch of servers such that failures can be tolerated. of new versions into the running system.
attached to them. This led to storage ˲˲ Transactional changes were sup- By watching the system’s health, new
clusters of commodity servers con- ported for application developers, pro- versions can be rolled back. These roll-
tained in a network. viding a huge improvement. It meant ing upgrades to the microservices can
Computing has evolved. A few de- the app developer did not need to be be sensitive to fault zones so an inde-
cades ago, it was only a single pro- so careful when dealing with storage. pendent failure during a flaky upgrade
cess on a single server. Years went by It also allowed a grouping of changes is not too damaging. Simply having a lot
before people started worrying about to records so a bunch of records were of separate and equivalent microser-
communicating across multiple pro- atomically updated. This was a lot eas- vices means a failure of one or more of
cesses on a single server. Then the ier. SANs implemented the required them is automatically repaired.
world moved on with great excitement careful replacement for the hardware Durable state is not usually kept
to RPCs (remote procedure calls) storage, allowing bigger and better da- in microservices. Instead, it is kept in
across a tiny cluster of servers. At the tabases. Databases evolved to support back-end databases, key-value stores,
time, we didn’t think about trust since two-tier and N-tier applications using caches, or other things. The remainder
everyone was in the same trust zone. transactional updates. of this article looks at some of these.
We were all in the family! ˲˲ Key-value stores offered more scale Microservices cannot easily update
In the 2000s, the concept of services but less declarative functionality for the state across all of the microser-
or SOA began to emerge, sometimes un- processing the application’s data. Mul- vices in the pool. This is especially true
der different names.6 The basic aspect tirecord transactions were lost as scale when they are coming and going willy-
of a service is trust isolation. This natu- was gained. nilly. It is common to keep the latest
state out of reach of the microservices Outside data always has some form of Then, an interesting development
and provide older versions of the state a unique identifier such as a URI (uni- in storage occurred. Some stores are
that are accessible in a scalable cache. form resource identifier) or a key. The fast but sometimes return stale values.
Sometimes, this leads to read-through identifier may be implicit within a ses- Others always return the latest value
requests by the scalable cache to du- sion or an environment. Outside data but occasionally stall when one of the
rable state that is not directly address- typically is manifest as a message, file, servers is slow. This section shows how
able to the calling microservice. or key-value pair. predictable answers result in unpre-
This is now becoming a tried and dictable latencies.10 Finally, it exam-
true pattern. Figure 1 is taken from a The Evolution of Durable ines the role immutable data can play
2007 paper by DeCandia et al. on Ama- State Semantics in supporting very large systems with
zon’s Dynamo.2 While the nomencla- Storage systems and databases have predictable answers and response
ture is slightly different, it shows three evolved through the decades and so times for some business functions.
tiers of microservices accessing a back- have the semantics of updating their Careful replacement of disk
end tier of different stores. state. This section begins in the bad old blocks. It used to be, back in the 1970s
Durable state and session state. days when I first started building sys- and 1980s, that a disk write might
Durable state is stuff that gets remem- tems. Back in the 1970s and 1980s, disk leave data unreadable. The write went
bered across requests and persists storage had to be carefully updated to through a number of state changes
across failures. This may be captured avoid trashing disk blocks. From there, from the old V1 version, to unreadable
as database data, file-system files, we move forward to atomic record up- garbage, to the new V2 version. When
key values, and more. Durable state dates and the challenges that arose be- the disk head was writing a block, the
is updated in a number of different fore transactions. When transactions magnetic representation of the bits in
ways, largely dependent on the kind came along a lot of things got a lot the block would be turned to mush on
of store holding it. It may be changed easier—if you were making a change at the way to being updated to the new
by single updates to a key value or one place and one time. Adding cross- version. A power failure would cause
file, or it may be changed by a trans- database and cross-time behavior led you to lose the old value (see Figure 2).
action or distributed transaction to the same challenges you had with When implementing a reliable ap-
implemented by a database or other more primitive storage systems. This plication, it’s essential that you do not
store. was helped by using messaging subsys- lose the old value of the data. For exam-
Session state is the stuff that gets re- tems to glue stuff together. ple, if you’re implementing the trans-
membered across requests in a session
but not across failures. Session state Figure 1. Example of Amazon’s Dynamo microservice architecture.
exists within the endpoints associated
with the session. Multioperation trans-
actions use a form of session state.7 Client Requests
Session state is hard to do when the
session is smeared across service in- Page
stances. If different microservices in ... Rendering
the pool process subsequent messages Components
in the transaction, session state is chal-
lenging to implement. It’s difficult to re- Request Routing
tain session state at the instance when
the next message to the pool may land
Aggregator
at a different service instance. Services
Data on the outside versus data on
the inside. The 2005 paper “Data on
the Outside Versus Data on the Inside”5 Request Routing
speaks about the fundamental differ-
ences between data kept in a locked
transactional store (for example, a rela- Services
tional database) and data kept in other
representations.
Data on the inside refers to locked
transactionally updated data. It lives
in one place (for example, a database)
Amazon
and at one time, the transactional S3
point in time.
Data on the outside is unlocked
and immutable, although it may be Dynamo Instances Other Datastores
versioned with a sequence of versions
that are in their own right immutable.
O C TO B E R 2 0 1 8 | VO L. 6 1 | N O. 1 0 | C OM M U N IC AT ION S OF T HE ACM 49
practice
Figure 2. V1 is trashed before V2 is written. be updated. Typically, you would write action by transaction. This leads to
to record X, wait to know it’s perma- messaging semantics.
file 1 V1 nent, and then write to record Y. Messaging semantics. In transac-
So, could you untangle the mess if a tional messaging a transaction makes
crash happened? a bunch of changes to its data and then
Frequently, there was an application- expresses a desire to send a message.
file 1
dependent pattern that provided in- This desire is atomically recorded with
sight into the order you needed to write. the transaction. A transaction may
After a crash and restart: atomically consume an incoming mes-
file 1 V2 ˲˲ If record A was updated but record sage. That means the work of the trans-
B was not written, the application can action, including changes to the appli-
clean up the mess. cation data, occurs if, and only if, the
Figure 3. “Ping-Pong” technique delays ˲˲ If record B was updated but record incoming message is consumed.
overwrite of V1.
A was not written, the application could It is possible to support the seman-
not cope and could not recover. tics of exactly-once delivery. The desire
file 1 V1 An example of careful replacement to send is atomically committed with
for records is message queuing. If the the sending transaction. A committed
application writes and confirms the desire to send a message causes one
file 1 V1 V2 presence of a message in a queue (call or more transmissions. The system
it record A), and the work to process retries until the destination acknowl-
that message is idempotent, then the edges it has received the message in
V2
application can cope with crashes its queue. The message must be pro-
file 1
based on careful replacement for re- cessed at the receiver at most once.
cords. Idempotent means it is correct This means it must be idempotently
if restarted.4,7 processed (see Figure 4).
file 1 V2 V2 Transactions and careful replace- There are challenges with at-
ment. Transactions bundle and solve most-once processing at the destina-
careful record replacement. Multiple ap- tion. To accomplish this, you need
action system for a database, it’s really plication records may be updated in a to remember the messages you have
bad to lose the most recently commit- single transaction, and they are all-or- processed so you don’t process them
ted transactions because the partially nothing. The database system ensures twice. But how do you remember the
full last block of your transaction log is the record updates are atomic. messages? You have to detect dupli-
being rewritten. One trick to avoid this ˲˲ Databases automatically handle cates. How long do you remember?
is to take turns writing to mirrored logs any challenges with careful storage re- Does the destination split? Does
on different disks. Only after know- placement. Users are not aware of the the destination move? If you mess
ing for sure that mirror A has the new funky failure behaviors that may oc- this up, will the application process
block do you write it to mirror B. After cur when systems crash or power fails. the message more than once? What
a crash, you rewrite the last block of the If present, databases also support if the message is being delivered to
log onto both mirrors to ensure a con- distributed transactions over a small a microservice-based application?
sistent answer. number of intimate database servers. Where is the knowledge of the set of
Another well-known technique, es- ˲˲ Work across time (that is, work- processed messages kept?
pecially for the tail of the log, is called flow) needs careful transactional re- Read your writes? Yes? No? It used
ping-pong.4 In this approach, the last placement. While the set of records in to be, back in the day, if you wrote
(and incomplete) block of the log is a transaction is atomically updated something, you could read it. Now,
left where it lies at the end of the log. with the help of the database, long- it’s not always that simple. Consider
The next version of that block, contain- running workflows3,4 are essential to the following:
ing the previous contents and more, is accomplish correct work over time. Linearizable stores offer read-
written to a later block. Only after the Failures, restarts, and new work can your-write behavior. In a linearizable
extended contents are durable on the advance the state of the application store each update creates a new ver-
later block will the new version over- transaction by transaction. Work sion of the value, and the store never
write the earlier version. In this fashion, across time leverages message pro- returns an old value or a different
there are no windows in which a power cessing. value. It always returns the latest in a
failure will lose the contents of the log ˲˲ Work across space (that is, across linear series of values.
(see Figure 3). boundaries) also needs careful transac-
Careful replacement for record tional replacement. Different systems,
writes. Updates to records in pre-data- applications, departments, and/or Linearizable stores will sometimes de-
base days didn’t have transactions. As- companies have separate trust bound- lay for a looooong time.
suming each record write was atomic, aries and typically do not do transac- To ensure they always give the cor-
you still couldn’t update two records tions across them. Work across space rect value, they will always update
and get any guarantees they would both necessitates work across time, trans- every replica.
If a server is slow or dead and con- store a brand-new value for an identi- Slip-Slidin’ Away …
tains one of the replicas, it may take fier and, later on, delete it. Many ap- This section looks at a number of guar-
tens of seconds to decide what to do … plication patterns are based on immu- antees that are slipping away. Everyone
Meanwhile, the user waits. table items. wishes they had a computational mod-
Imagine a system where you are sim- el such as a von Neumann machine,12
ply recording stuff you have seen. Ev- which provides computation, storage,
Nonlinearizable stores do not offer erything you know is based on observa- and predictable linear behavior. Once
to read your writes. A nonlinearizeable tions. The past is never changed—sort distribution kicks in, however, that’s in-
store means there’s no guarantee that a of like an accountant’s ledger where deed only a wish.
write will update all the replicas. Some- nothing is updated. You can put a Single-process computation as John
times, a read may find an old value. unique ID on each artifact and look at it von Neumann conceived has evolved
Reading and writing a nonlinearizable later but never change it. This is an ex- to multiprocess- and multiserver-using
store has a very consistent response tremely common pattern. sessions and session state. These state-
time with much higher probability. When keeping immutable objects ful sessions supported composable
A read or write can skip over a sick or or values in a key/value store, you transactions that spanned multiple
dead server. Occasionally, this results never get a stale answer. There’s only records and multiple servers working
in an older value coming back from the one immutable value for the unique together. As the work started decom-
skipped server. But, hey, it’s fast—and key. That means a nonlinearizable posing into microservices, however, it
predictably so. store offers the one and only correct became hard to use transactions the
Imagine a key/value store where key- answer. All the store types give the way they had been used.
K has value V1 and the store keeps it correct answer, just with different To cope with scalable environments,
on servers S1, S2, and S3. You decide to characteristics for read and write la- data had to be busted up into key val-
update the value to V2. The store tries tencies (see Figure 6). Storing immu- ues. Scalable stores worked well for up-
to change the values on its three serv- table data means you never get a stale dating a single key at a time but not for
ers, but S2 does not answer because it version because there is not one. atomic transactions across keys. Most
is down. Therefore, the store decides
to write V2 onto S1, S3, and S4 so that Figure 4. Transaction messaging.
the new value is always written to three
servers. Later, when S2 comes up, a At Least
read might find the old value V1. This Transaction T1 Transaction T2
Once
has the following trade-offs: Delivery
Writes To Data Writes To Data
˲˲ The write of three stores always
happens quickly. W X Y Z A B C D
˲˲ The store is not linearizable and
sometimes returns an old value.
This very useful technique underlies At Most
a number of scalable storage systems Once Processing
such as Dynamo2 and Cassandra.11
Cached data offers scalable read
throughput with great response time. Figure 5. Different types of storage offer different guarantees.
Key-value pairs live in many servers
and are updated by propagating new
Fast Predictable Fast Predictable Read
versions. Each read hits one of the Reads? Writes? Your Writes?
servers and returns one of the versions
Linearizable Store No No Yes
(see Figure 5).
Non-Linearizable Store Yes Yes No
Scalable Cache Yes w/Scale No No
Different Stores for Different Uses
OK to stall on reads?
OK to stall on writes?
OK to return stale versions? Figure 6. Immutable data allows “read-your-write-behavior.”
You can’t have everything!
Fast Predictable Fast Predictable Read
Reads? Writes? Your Writes?
Immutability: A solid rock to stand Linearizable Store No No Immutable
on. When you store immutable data,
Non-Linearizable Store Yes Yes Immutable
each lookup always returns the same
result.8 Immutable stores do not ever Scalable Cache Yes w/Scale No Immutable
O C TO B E R 2 0 1 8 | VO L. 6 1 | N O. 1 0 | C OM M U N IC AT ION S OF T HE ACM 51
practice
of these scalable key-value stores en- tion. In many circumstances, rich and sion-1]), you can view the version as im-
sured linearizable, strongly consistent complex transactions could occur mutable data. Each version becomes an
updates to their single keys. Unfortu- over N-tier environments, even across immutable thing to be kept. Using the
nately, these linearizable stores would multiple back-end databases using extended [Key, Version], you can refer-
occasionally cause delays seen by users. distributed transactions. ence immutable data in the store.
This led to the construction of nonlin- Transactions, sessions, and microser- Version history may be linear, mean-
earizable stores with the big advantage vices. Microservices leave much to be de- ing one version supersedes the previous
that they have excellent response times sired when it comes to session state. Re- one. This is achieved by using a lineariz-
for reads and writes. In exchange, they quests are load balanced through a router, able store. Version history may be a di-
sometimes give a reader an old value. and one of many microservice instances is rected acyclic graph (DAG). This happens
Finally, this section points out that selected. Usually, later traffic is sent to the when writing to a nonlinearizable store.
some uses of data find the correct an- same instance but not always. You cannot Imagine you have a notepad on
swer important enough to use care- count on getting back to where you were. which to scribble stuff. But you really
ful replacement of the stored values. Without session state, you can- have multiple notepads. You scribble
These uses are not the best for nonlin- not easily create transactions cross- stuff on whichever notepad is closest
earizable stores. ing requests. Typically, microservice to you at the time. When you want to
Honestly, it ain’t like it used to be. environments support a transaction read the information, you look at the
Same process evolves to different within a single request but not across closest notepad even if it’s not the one
process. Applications and the database multiple requests. you wrote on most recently. Sometimes,
used to run in the same process. A li- Furthermore, if a microservice ac- you get two notepads next to each other,
brary call to the database code allowed cesses a scalable key-value store as it look at both, and write something in
access to the data. Sometimes, multiple processes a single request, the scalable both to consolidate the scribbles. This
applications were loaded together. key-value store will usually support only is the kind of behavior that comes from
Later, the database and applica- atomic updates to a single key. While it a nonlinearizable store. Updates do not
tions were split into different processes won’t break the data by failing in the mid- march forward in linear order.
connected by a session. The session dle of updating a key as older file systems Careful replacement and read your
described the session state and had in- did, programmers are on their own when writes. In careful replacement you need
formation about the user, transaction changing values tied to multiple keys. to be careful about the ordering of what
in flight, the application being run, and Keys, versions, and nonlinear his- you update. This is essential to handle
the cursor state and return values. tory. Each key is represented by some some failures, as discussed earlier. Pre-
Later still, the application and da- number, string, key, or URI. That key dictable behavior across trust boundar-
tabase moved to different servers. The can reference something that’s im- ies is needed when working with other
session state made that possible. mutable. For example, “The New York companies. It’s also essential when do-
Stateful sessions and transactions. Times, June 1, 2018, San Francisco Bay ing long-running workflows.
Stateful sessions were a natural out- Area edition” is immutable across space Careful replacement is predicated
come of shared processes. You knew and time. A key may also reference some- on read-your-writes behavior, which
who you were talking to and you could thing that changes over time—for exam- depends on a linearizable store. Lin-
remember stuff about the other guy. ple, “today’s New York Times.” earizable stores almost always have the
Stateful sessions worked well for When a key references something property of occasionally stalling when
classic SOA. When talking to a ser- that changes, it can be understood as waiting for a bum server.
vice, you expected a long session with referencing a sequence of versions, each
state on each side. Stateful sessions of which is immutable. By first binding Some Example Application Patterns
meant the application could do mul- the changing value of the key to a unique Let’s look at some application patterns
tiple interactions within a transac- version of the key (for example, [Key, Ver- and how they impact the management
of durable state (see Figure 7).
Figure 7. Applications patterns. Workflow over key-value with care-
ful replacement. This pattern demon-
strates how applications perform work-
workflow over key-value A traditional workflow application over a scalable collection
of key-value data. flow when the durable state is too large
transactional blobs-by-ref A centralized and transactional system managing
to fit in a single database.
very large collections of immutable blobs. An object is uniquely identified by its
e-commerce—shopping cart The familiar but still surprising world of e-commerce key. Work arrives from the outside via
shopping carts. human interaction or messaging. Work-
e-commerce—product Consider a very large ecommerce product catalog with flow can be captured in the values. New
catalog enormous numbers of product descriptions and huge traffic values replace old ones. The messages
reading the catalog.
are contained as data within the object.9
search Track a ginormous number of document (for example, the entire Scalable workflow applications can
Web) and organize searchable indices to locate documents by
words and phrases. Must scale to ever increasing read workload. be built over key-value stores. You must
have single-item linearizability (read
your writes, see Figure 8.) With a linear
version history, one new version always exist in the version history DAG. Rela- the partition for the product identifier
supersedes the earlier one. A nonlinear tively simple shopping-cart semantics and then load-balanced across replicas.
history has a DAG version history. In this facilitate combining different versions Back-end processing of the feeds
case, the linearizable behavior of the of a single user’s shopping cart.2 and crawls, as well as the pub-sub dis-
store also implies that a stall within one E-commerce—Product catalog. tribution of updates to the caches, are
of the store servers will stall the write Product catalogs for large e-commerce throughput sensitive, not latency-sensi-
to the store. This is the “must be right” sites are processed offline and stuffed tive. Different replicas may be updated
even if it’s not “right now” case. into large scalable caches. Feeds from
The workflow implemented by careful partners and crawls of the Web are Figure 8. Linear vs. nonlinear histories.
replacement will be a mess if you can’t crunched to produce a sanitized
read the last value written. Hence, this and hopefully consistent collection of
usage pattern will stall and not be stale. product-catalog entries.
Transactional blobs-by-ref. This is a Each product in the catalog has a
pretty common application pattern. The unique identifier. Typically, the identifi-
application runs using transactions and er takes you to a partition of the catalog.
a relational database. It also stores big The partition has a bunch of replicas,
blobs such as documents, photos, PDFs, each containing many product descrip-
videos, music, and more. The blobs can tions (see Figure 9). One typical imple-
be large and numerous. Hence, these are mentation of a scalable product cache
a challenge to implement directly in the has partitions with replicas. In this
relational database. depiction, the columns are partitions
Each of these blobs is an immutable and the rows depict replicas. The back-
set of bits. To modify a blob (for exam- end processing produces new product Linear Directed Acyclic Graph
Version History Version History
ple, editing a photo), you always create a descriptions that are distributed with
new blob to replace the old one. The im- pub-sub. Incoming requests are sent to
mutable blobs typically have a univer-
sally unique identifier (UUID) as their Figure 9. Partitions with replicas.
key in a scalable key-value store.
Storing immutable blobs in a non-
linearizable database does not have any
problems with returning a stale version.
Since there’s only one immutable ver- Incoming Read Requests
O C TO B E R 2 0 1 8 | VO L. 6 1 | N O. 1 0 | C OM M U N IC AT ION S OF T HE ACM 53
practice
General Co-Chairs
Klara Nahrstedt, University of Illinois at Urbana-Champaign, USA
Olaf Landsiedel, Chalmers University, Sweden
Program Co-Chairs
Gian Pietro Picco, University of Trento, Italy
Prashant Shenoy, University of Massachussetts, Amherst, USA
Important Dates
Abstract Registration: Oct. 10, 2018
Paper Submission: Oct. 17, 2018
Author Rebuttal: Dec. 10, 2018
Notification Date: Jan. 15, 2019
http://conferences.computer.org/iotdi/2019/
CPS-IoT Week
contributed articles
DOI:10.1145/ 3271625
The triumph of these achievements
What just happened in artificial intelligence has led some to describe the automa-
tion of these tasks as having reached
and how it is being misunderstood. human-level intelligence. This percep-
tion, originally hinted at in academic
BY ADNAN DARWICHE circles, has gained momentum more
broadly and is leading to some impli-
Human-Level
cations. For example, some coverage
of AI in public arenas, particularly
comments made by several notable fig-
ures, has led to mixing this excitement
Intelligence
with fear of what AI might bring us all
in the future (doomsday scenarios).b
Moreover, a trend is emerging in which
machine learning research is being
or Animal-Like
streamlined into neural network re-
search, under its newly acquired label
“deep learning.” This perception has
also caused some to question the wis-
Abilities?
dom of continuing to invest in other
machine learning approaches or even
other mainstream areas of AI (such as
knowledge representation, symbolic
reasoning, and planning).
This turn of events in the history of
AI has created a dilemma for research-
ers in the broader AI community. On
the one hand, one cannot but be im-
pressed with, and enjoy, what we have
been able to accomplish with neural
“The vision systems of the eagle and the snake networks. On the other hand, main-
stream scientific intuition stands in
outperform everything that we can make in the way of accepting that a method
the laboratory, but snakes and eagles cannot b Stephen Hawking said: “The development of
build an eyeglass or a telescope or a microscope.” full artificial intelligence could spell the end
of the human race;” and Elon Musk said AI is:
— Judea Pearla “ … potentially more dangerous than nukes.”
key insights
of neural networks in
T H E REC ENT SUC C E S S E S ˽˽ The recent successes of deep learning
have revealed something very interesting
applications like speech recognition, vision, and about the structure of our world, yet this
autonomous navigation has led to great excitement by seems to be the least pursued and talked
about topic today.
members of the artificial intelligence (AI) community, ˽˽ In AI, the key question today is not
as well as by the general public. Over a relatively short whether we should use model-based or
function-based approaches but how to
time, by the science clock, we managed to automate integrate and fuse them so we can realize
ILLUSTRATION BY H UGH SYME
that does not require explicit model- reasoning is required to compute the
ing or sophisticated reasoning is suf- function outputs from its inputs. The
ficient for reproducing human-level main tool of this approach is the neural
intelligence. This dilemma is further network. Many college students have
amplified by the observation that re-
cent developments did not culminate In my own quest exercised a version of it in a physics
or chemistry lab, where they fit simple
in a clearly characterized and profound
scientific discovery (such as a new
to fully appreciate functions to data collected from vari-
ous experiments, as in Figure 2. The
theory of the mind) that would nor- the progress main difference here is we are now em-
mally mandate massive updates to the
AI curricula. Scholars from outside AI
enabled by deep ploying functions with multiple inputs
and outputs; the structure of these
and computer science often sense this learning, I came functions can be quite complex; and
dilemma, as they complain they are
not receiving an intellectually satisfy-
to the conclusion the problems being tackled are ones
we tend to associate with perception or
ing answer to the question: “What just that recent cognition, as opposed to, say, estimat-
happened in AI?”
The answer lies in a careful assess- developments tell ing the relationship between volume
and pressure in a sealed container.d
ment of what we managed to achieve us more about the The main observation in AI recently
with deep learning and in identifying
and appreciating the key scientific out- problems tackled is that the function-based approach
can be quite effective at certain AI
comes of recent developments in this
area of research. This has unfortunate-
and the structure tasks, more so than the model-based
approach or at least earlier attempts at
ly been lacking to a great extent. My of our world than using this approach. This has surprised
aim here is to trigger such a discussion,
encouraged by the positive and curious
about neural not only mainstream AI researchers,
who mainly practice the model-based
feedback I have been receiving on the networks per se. approach, but also machine learning
thoughts expressed in this article. researchers who practice various ap-
proaches, of which the function-based
Background approach is but one.e This has had
To lay the ground for the discussion, I many implications, some positive and
first mark two distinct approaches for some giving grounds for concern.
tackling problems that have been of On the positive side is the increas-
interest to AI. I call the first one “mod- ing number of tasks and applications
el-based” and the second “function- now within reach, using a tool that can
based.” Consider the object-recogni- be very familiar to someone with only
tion and -localization task in Figure 1. a broad engineering background, par-
To solve it, the model-based approach ticularly one accustomed to estimat-
requires one to represent knowledge ing functions and using them to make
about dogs and hats, among other predictions. What is of concern, how-
things, and involves reasoning with ever, is the current imbalance between
such knowledge. The main tools of exploiting, enjoying, and cheering
the approach today are logic and prob- this tool on the one hand and thinking
ability (mathematical modeling more about it on the other. This thinking is
generally) and can be thought of as not only important for realizing the full
the “represent-and-reason”c approach potential of the tool but also for scien-
originally envisioned by the founders tifically characterizing its potential
of AI. It is also the approach normally
expected, at some level, by informed
d This is also called the “curve-fitting” ap-
members of the scientific community. proach. While the term “curve” highlights the
The function-based approach, on the efficient evaluation of a function and captures
other hand, formulates this task as a the spirit of the function-based approach, it
function-fitting problem, with func- underplays the complex and rich structure of
functions encoded by today’s (deep) neural
tion inputs coming directly from the networks, which can have millions if not bil-
image pixels and outputs correspond- lions of parameters.
ing to the high-level recognitions we e Machine learning includes the function-based
seek. The function must have a form approach but has a wide enough span that it
that can be evaluated efficiently so no overlaps with the model-based approach; for
example, one can learn the parameters and
structure of a model but may still need non-
c This term might be likened to what has been trivial reasoning to obtain answers from the
called “good old-fashioned AI.” learned model.
reach. The lack of such characteriza- of labeled data; the increased com- a class of practical applications that
tion is a culprit of current misconcep- putational power we now have at correspond to functions that, we now
tions about AI progress and where it our hands; and the increasingly so- know, are simple enough to allow
may lead us in the future. phisticated statistical and optimiza- compact representations that can be
tion techniques for fitting functions evaluated efficiently (again, without
What Just Happened in AI? (including new activation functions the need for reasoning), and whose
In my own quest to fully appreciate the and new/deeper network structures). estimation is within reach of current
progress enabled by deep learning, The second is that we have identified thresholds for gathering data, com-
I came to the conclusion that recent
developments tell us more about the Figure 1. Object recognition and localization in an image (ImageNet).
problems tackled and the structure of
our world than about neural networks
per se. These networks are param-
eterized functions that are expressive
enough to capture any relationship
between inputs and outputs and have
a form that can be evaluated efficiently.
This has been known for decades and
described at length in textbooks. What
caused the current turn of events?
To shed some light on this question,
let me state again what we have discov-
ered recently. That is, some seemingly
complex abilities that are typically as-
sociated with perception or cognition
can be captured and reproduced to
a reasonable extent by simply fitting
functions to data, without having to ex-
plicitly model the environment or sym-
bolically reason about it. While this
is a remarkable finding, it highlights
problems and thresholds more than it
highlights technology, a point I explain
next.
Every behavior, intelligent or not,
can be captured by a function that
maps inputs (environmental sensing) Figure 2. Fitting a simple function to data.
to outputs (thoughts or actions). How-
ever, the size of this function can be
quite large for certain tasks, assuming
the function can be evaluated efficient- 10
O C TO B E R 2 0 1 8 | VO L. 6 1 | N O. 1 0 | C OM M U N IC AT ION S OF T HE ACM 59
contributed articles
putational speed, and estimation fort, if not general agreement, with the speed up automation.h To address these
techniques. This includes recogniz- remarks I made. I did get a few “I beg to concerns, the focus should be shifted
ing and localizing objects in some differ” responses though, all centering toward policy and regulatory consider-
classes of images and certain tasks on recent advancements relating to op- ations for dealing with the new level of
that pertain to natural language and timizing functions, which are key to the automation our society is embarking
speech. The third development, successful training of neural networks on, instead of fearing AI.
which goes largely unnoticed, is (such as results on stochastic gradient
that we gradually changed our ob- descent, dropouts, and new activation On Objectives and Success
jectives and measures for success functions). The objections stemmed Let me now address the third reason for
in ways that reduced the technical from not having named them as break- the current turn of events, which relates
challenges considerably, at least as throughs (in AI). My answer: They all to the change in objectives and how we
entertained by early AI researchers, fall under the enabler I outlined earlier: measure success as a broad AI com-
while maintaining our ability to cap- “increasingly sophisticated statistical munity. This reason is quite substantial
italize on the obtained results com- and optimization techniques for fitting yet goes largely unnoticed, especially by
mercially, a point I discuss further functions.” Follow up question: Does younger researchers. I am referring here
later in the section on objectives and it matter that they are statistical and to the gradual but sustained shift over AI
success. optimization techniques, as opposed history from trying to develop technolo-
Interestingly, none of these develop- to classical AI techniques? Answer: It gies that were meant to be intelligent and
ments amounts to a major technical does not matter as far as acknowledg- part of integrated AI systems to develop-
breakthrough in AI per se (such as the ing and appreciating scientific inquiry ing technologies that perform well and
establishment of probability as a foun- and progress, but it does matter as far are integrated with consumer products;
dation of commonsense reasoning in as explaining what just happened and, this distinction can be likened to what
the late 1980s and the introduction of more important, forecasting what may has been called “Strong AI” vs. “Weak AI.”
neural networks more than 50 years happen next. This shift was paralleled by a sharp-
ago).f Yet the combination of these fac- Consider an educated individual sit- ening of performance metrics and by
tors created a milestone in AI history, as ting next to you, the AI researcher, on progress against these metrics, partic-
it had a profound impact on real-world a plane; I get that a lot. They figure out ularly by deep learning, leading to an
applications and the successful deploy- you do AI research and ask: What are the increased deployment of AI systems.
ment of various AI techniques that have developments that enabled the current However, these metrics and corre-
been in the works for a very long time, progress in AI? You recount the func- sponding progress did not necessarily
particularly neural networks.g tion-based story and lay out the three en- align with improving intelligence, or
ablers. They will likely be impressed and furthering our understanding of intelli-
‘I Beg to Differ’ also intellectually satisfied. However, if gence as sought by early AI researchers.i
I shared these remarks in various con- the answer is, “We just discovered a new One must thus be careful not to draw
texts during the course of preparing this theory of the mind,” you will likely not certain conclusions based on current
article. The audiences ranged from AI be surprised if they also end up worry- progress, which would be justified only
and computer science to law and pub- ing about a Skynet coming soon to mess if one were to make progress against
lic-policy researchers with an interest up our lives. Public perceptions about AI earlier objectives. This caution particu-
in AI. What I found striking is the great progress and its future are very impor- larly refers to current perceptions that
interest in this discussion and the com- tant. The current misperceptions and as- we may have made considerable prog-
sociated fears are being nurtured by the ress toward achieving “full AI.”
f Research on neural networks has gone through absence of scientific, precise, and bold Consider machine translation, which
many turns since their early traces in the 1940s.
perspectives on what just happened, received significant attention in the early
Nils Nilsson of Stanford University told me he
does not think the pessimistic predictions of leaving much to the imagination. days of AI. The represent-and-reason ap-
the 1969 book Perceptrons: An Introduction to This is not to suggest that only a proach aimed to comprehend text before
Computational Geometry by Marvin Minsky and new theory of the mind or an advance translating it and is considered to have
Seymour Papert was the real reason for the de- of such scale would justify some of the failed on this task, with function-based ap-
cline in neural network research back then, as
legitimate concerns surrounding AI. In proaches being the state of the art today.
is widely believed. Instead, it was the inability
to train multiple layers of weights that Nilsson fact, even limited AI technologies can In the early days of AI, success was mea-
also wrestled with at SRI during that time “but lead to autonomous systems that may sured by how far a system’s accuracy was
couldn’t get anywhere,” as he explained to me. pose all kinds of risks. However, these
g A perspective relayed to me by an anonymous re- concerns are not new to our industrial-
viewer is that science advances because instru- h See also the first report of the One Hundred
ments improve and that recent developments
ized society; recall safety concerns when Year Study on Artificial Intelligence (AI100) for
in neural networks could be viewed as improve- the autopilot was introduced into the a complementary perspective; https://ai100.
ments to our machine learning instruments. aerospace industry and job-loss con- stanford.edu/
The analogy given here was to genomics and the cerns when ATMs were introduced into i An anonymous reviewer said that throughout
development of high-throughput sequencing, the banking industry. The headline here AI there are metrics for evaluating task per-
which was not the result of a scientific break- formance but not for evaluating the fit among
through but rather of intense engineering ef-
should therefore be “automation” more an agent, its goals, and its environment. Such
forts, yet such efforts have indeed revealed a vast than “AI,” as the latter is just a tech- global metrics may be needed to assess and
amount about the human genome. nology that happened to improve and improve the intelligence of AI systems.
from 100% compared to humans, and Perhaps one of the broadest applica-
successful translation was predicated tions of these systems today is in user
on the ability to comprehend text. Gov- interfaces (such as automated tech-
ernment intelligence was a main driv- nical support and the commanding
ing application; a failure to translate
correctly can potentially lead to a politi- Some seemingly of software systems, as in phone and
navigation systems in vehicles). These
cal crisis. Today, the main application
of machine translation is to webpages
complex abilities systems fail often; try to say something
that is not very prototypical or not to
and social-media content, leading to a that are typically hide your accent if you have one. But
new mode of operation and a different
measure of success. In the new context,
associated with when these systems fail, they send
the user back to a human operator or
there is no explicit need for a transla- perception or force the user to command the soft-
tion system to comprehend text, only
to perform well based on the adopted
cognition can ware through classical means; some
users even adjust their speech to get
metrics. From a consumer’s viewpoint, be captured and the systems to work. Again, while the
success is effectively measured in terms
of how far a system’s accuracy is from reproduced to a performance of these systems has im-
proved, according to the adopted met-
0%. If I am looking at a page written in reasonable extent rics, they are today embedded in new
French, a language I do not speak, I am
happy with any translation that gives me by simply fitting contexts and governed by new modes
of operation that can tolerate lack of
a sense of what the page is saying. In fact,
the machine-translation community
functions to data. robustness or intelligence. Moreover,
as in text, improving their performance
rightfully calls this “gist translation.” It against current metrics is not neces-
can work impressively well on prototypi- sarily directed toward, nor requires
cal sentences that appear often in the addressing, the challenge of compre-
data (such as in social media) but can hending speech.l
fail badly on novel text (such as poetry). Moving to vision applications, it
It is still very valuable yet corresponds to has been noted that some object-rec-
a task that is significantly different from ognition systems, based on neural net-
what was tackled by early AI researchers. works, surpass human performance in
We did indeed make significant progress recognizing certain objects in images.
recently with function-based translation, But reports also indicate how making
thanks to deep learning. But this prog- simple changes to images may some-
ress has not been directed toward the times hinder the ability of neural net-
classical challenge of comprehending works to recognize objects correctly.
text, which aimed to acquire knowledge Some transformations or deformations
from text to enable reasoning about its to objects in images, which preserve
content,j instead of just translating it.k the human ability to recognize them,
Similar observations can be made can also hinder the ability of networks
about speech-recognition systems. to recognize them. While this does not
measure up to the expectations of early
j There are other views as to what “comprehen- AI researchers or even contemporary vi-
sion” might mean, as in, say, what might be
sion researchers, as far as robustness
revealed about language from the internal en-
codings of learned translation functions. and intelligence is concerned, we still
k With regard to the observation that the repre- manage to benefit from these technolo-
sent-and-reason approach is considered to have gies in a number of applications. This
failed on machine translation, Stuart Russell of includes recognizing faces during au-
the University of California, Berkeley, pointed
tofocus in smart cameras (people do
out to me that this is probably a correct descrip-
tion of an incorrect diagnosis, as not enough ef- not normally deform their faces but if
fort was directed toward pursuing an adequate they do, bad luck, an unfocused image);
represent-and-reason approach, particularly looking up images that contain cats in
one that is trainable, since language has too online search (it is ok if you end up get-
many quirks to be captured by hand. This ob-
servation is part of a broader perspective I sub-
ting a dog instead); and localizing sur-
scribe to calling for revisiting represent-and-rea- rounding vehicles in an image taken by
son approaches while augmenting them with
advances in machine learning. This task would, l An anonymous reviewer suggested that tran-
however, require a new generation of research- scription is perhaps the main application of
ers well versed in both approaches; see the sec- speech systems today, with substantial prog-
tion in this article on the power of success for ress made toward the preferred metric of
hints as to what might stand in the way of having “word error rate.” The same observation ap-
this breed of researchers. plies to this class of applications.
O C TO B E R 2 0 1 8 | VO L. 6 1 | N O. 1 0 | C OM M U N IC AT ION S OF T HE ACM 61
contributed articles
the camera of a self-driving car (the vul- and regulatory levels.m The second
nerability of these systems to mistakes is that while function-based systems
remains controversial in both its scope have been an enabling and positive
and how to deal with it at the policy and development, we do need to be acute-
regulatory levels).
The significance of these observa- We succeeded in ly aware of the reasons behind their
success to better understand the im-
tions stems from their bearing on our
ability to forecast the future and deci-
these applications plications. A key finding here is that
some tasks in perception and cogni-
sions as to what research to invest in. by having tion can be emulated to a reasonable
In particular, does the success in ad-
dressing these selected tasks, which
circumvented extent without having to understand
or formalize these tasks as originally
are driven by circumscribed com- certain technical believed and sought, as in some text,
mercial applications, justify the worry
about doomsday scenarios? Does it
challenges instead speech, and vision applications. That
is, we succeeded in these applica-
justify claims that AI-based systems of having solved tions by having circumvented certain
can now comprehend language or
speech or do vision at the levels that them directly. technical challenges instead of hav-
ing solved them directly.n This ob-
humans do? Does it justify this cur- servation is not meant to discount
rent imbalance of attitudes toward current success but to highlight its
various machine learning and AI ap- nature and lay the grounds for this
proaches? If you work for a company question: How far can we go with this
that has an interest in such an appli- direction? I revisit this issue later in
cation, then the answer is perhaps, the article.
and justifiably, yes. But, if you are con-
cerned with scientific inquiry and un- Human-Level or Animal-Level?
derstanding intelligence more broad- Let me now get to the thoughts that
ly, then the answer is hopefully no. triggered the title of this article in
In summary, what has just hap- the first place. I believe human-level
pened in AI is nothing close to a break- intelligence is not required for the
through that justifies worrying about tasks currently conquered by neural
doomsday scenarios. What just hap- networks, as such tasks barely rise
pened is the successful employment to the level of abilities possessed by
of AI technology in some widespread many animals. Judea Pearl cited ea-
applications, aided greatly by devel- gles and snakes as having vision sys-
opments in related fields, and by new tems that surpass what we can build
modes of operation that can tolerate today. Cats have navigation abilities
lack of robustness or intelligence. that are far superior to any of those
Put another way—and in response to in existing automatous-navigation
headlines I see today, like “AI Has Ar- systems, including self-driving cars.
rived” and “I Didn’t See AI Coming”— Dogs can recognize and react to hu-
AI has not yet arrived according to the
early objective of capturing intelli-
m Eric Horvitz of Microsoft Research brought
gent behavior. What really has arrived
up the idea of subjecting certain AI systems to
are numerous applications that can trials as is done to approve drugs. The proper
benefit from improved AI techniques labeling of certain AI systems should also be
that still fall short of AI ambitions but considered, also as is done with drugs. For
are good enough to be capitalized on example, it has been suggested that the term
“self-driving car” is perhaps responsible for
commercially. This by itself is posi-
the misuse of this AI-based technology by
tive, until we confuse it with some- some drivers who expect more from the tech-
thing else. nology than is currently warranted.
Let me close this section by n For example, one can now use learned func-
stressing two points: The first is tions to recognize cats in images without
having to describe or model what a cat is, as
to reemphasize an earlier observa- originally thought and sought, by simply fitting
tion that while current AI technol- a function based on labeled data of the form:
ogy is still quite limited, the impact (image, cat), (image, not cat). While this ap-
it may have on automation, and proach works better than modeling a cat (for
hence society, may be substantial now), it does not entail success in “learning”
what a cat is, to the point where one can rec-
(such as in jobs and safety). This ognize, say, deformed images of cats or in-
in turn calls for profound treat- fer aspects of cats that are not relayed in the
ments at the technological, policy, training dataset.
man speech, and African grey parrots functions, even though we may not ity into this consequential question:
can generate sounds that mimic hu- be there today, given current thresh- How effective will function-based
man speech to impressive levels. Yet olds. But it begs the question: If it approaches be when applied to new
none of these animals has the cogni- is a matter of thresholds, and given and broader applications than those
tive abilities and intelligence typically current successes, why not focus all already targeted, particularly those
attributed to humans. our attention on moving thresholds that mandate more stringent mea-
One of the reactions I received to further? While there is merit to this sures of success? The question has
such remarks was: “I don’t know of any proposal, which seems to have been two parts: The first concerns the class
animal that can play Go!” This was in adopted by key industries, it does of cognitive tasks whose correspond-
reference to the AlphaGo system, which face challenges that stem from both ing functions are simple enough to al-
set a milestone in 2016 by beating the academic and policy considerations. low compact representations that can
world champion in the game. Indeed, I address academic considerations be evaluated efficiently (as in neural
we do not know of animals that can play next while leaving policy consider- networks) and whose estimation is
a game as complex as Go. But first recall ations to a later section. within reach of current thresholds—
the difference between performance From an academic viewpoint, the or thresholds we expect to attain in,
and intelligence: A calculator outper- history of AI tells us to be quite cau- say, 10 to 20 years. The second al-
forms humans at arithmetic without tious, as we have seen similar phe- ludes to the fact that these functions
possessing human or even animal cog- nomena before. Those of us who have are only approximations of cognitive
nitive abilities. Moreover, contrary to been around long enough can recall tasks; that is, they do not always get it
what seems to be widely believed, Al- the era of expert systems in the 1980s. right. How suitable or acceptable will
phaGo is not a neural network since At that time, we discovered ways to such approximations be when tar-
its architecture is based on a collection build functions using rules that were geting cognitive tasks that mandate
of AI techniques that have been in the devised through “knowledge engi- measures of success that are tighter
works for at least 50 years.o This includes neering” sessions, as they were then than those required by the currently
the minimax technique for two-player called. The functions created through targeted applications?
games, stochastic search, learning from this process, called “expert systems”
self-play, use of evaluation functions and “knowledge-based systems,” were The Power of Success
to cut off minimax search trees, and claimed to achieve performance that Before I comment on policy consid-
reinforcement learning, in addition to surpassed human experts in some erations, let me highlight a relevant
two neural networks. While a Go player cases, particularly in medical diagno- phenomenon that recurs in the his-
can be viewed as a function that maps a sis.q The term “knowledge is power” tory of science, with AI no exception.
board configuration (input) to an action was used and symbolized a jubilant I call it the “bullied-by-success” phe-
(output), the AlphaGo player was not state of affairs, resembling what “deep nomenon, in reference to the sub-
built by learning a single function from learning” has come to symbolize to- duing of a research community into
input-output pairs; only some of its day.r The period following this era mainly pursing what is currently suc-
components were built that way.p The came to be known as the “AI Winter,” cessful, at the expense of pursuing
issue here is not only about assigning as we could finally delimit the class of enough what may be more successful
credit but about whether a competitive applications that yielded to such sys- or needed in the future.
Go function can be small enough to be tems, and that class fell well short of Going back to AI history, some of
represented and estimated under cur- AI ambitions. the perspectives promoted during
rent data-gathering, storage, and com- While the current derivative for the expert-systems era can be safely
putational thresholds. It would be progress on neural networks has been characterized today as having been
quite interesting if this was the case, impressive, it has not been sustained scientifically absurd. Yet, due to the
but we do not yet know the answer. I long enough to allow sufficient visibil- perceived success of expert systems
should also note that AlphaGo is a then, these perspectives had a domi-
great example of what one can achieve q One academic outcome of the expert system nating effect on the course of scientific
today by integrating model-based and era was the introduction of a dedicated mas- dialogue and direction, leading to a
ter’s degree at Stanford University called the
function-based approaches. bullied-by-success community.s I saw a
“Master’s in AI” that was separate from the
master’s in computer science and had sig- similar phenomenon during the tran-
Pushing Thresholds nificantly looser course requirements. It sition from logic-based approaches
One cannot of course preclude the was a two-year program, with the second to probability-based approaches for
possibility of constructing a competi- year dedicated to building an expert system. commonsense reasoning in the late
I was a member of the very last class that
tive Go function or similarly complex graduated from the program before it was
1980s. Popular arguments then, like
terminated and recall that one of its justifi- “People don’t reason probabilistically,”
o Oren Etzioni of the Allen Institute for Artificial cations was that classical computer science
Intelligence laid out this argument during a techniques can be harmful to the “heuris- s A colleague could not but joke that the broad
talk at UCLA in March 2016 called Myths and tic” thinking needed to effectively build ex- machine learning community is being bullied
Facts about the Future of AI. pert systems. today by the success of its deep learning sub-
p AlphaZero, the successor to AlphaGo, used one r The phrase “knowledge is power” is appar- community, just as the broader AI community
neural network instead of two and data generat- ently due to English philosopher Sir Francis has been bullied by the success of its machine
ed through self-play, setting another milestone. Bacon (1561–1626). learning sub-community.
O C TO B E R 2 0 1 8 | VO L. 6 1 | N O. 1 0 | C OM M U N IC AT ION S OF T HE ACM 63
contributed articles
which I believe carries merit, were genuine academic interests instead formation that goes beyond what can
completely silenced when proba- of just yielding to current fashions.t be extracted from data. To elaborate
bilistic approaches started solving on these points, I first need to explain
commonsense reasoning problems Policy Considerations why a function may not qualify as a
that had defied logical approaches Let me now address some policy con- model, a question I received during a
for more than a decade. The bullied- cerns with regard to focusing all our discussion on the subject.
by-success community then made attention on functions instead of Consider an engineered system
even more far-reaching choices in also on models. A major concern that allows us to blow air into a bal-
this case, as symbolic logic almost here relates to interpretability and loon that then raises a lever that is
disappeared from the AI curricula. explainability. If a medical-diagnosis positioned on top of the balloon.
Departments that were viewed as system recommends surgery, we The input to this system is the
world centers for representing and would need to know why. If a self- amount of air we blow (X), while the
reasoning with symbolic logic bare- driving car kills someone, we would output is the position of the lever
ly offered any logic courses as a re- also need to know why. If a voice (Y). We can learn a function that
sult. Now we are paying the price. command unintentionally shuts captures the behavior of the system
As one example: Not realizing that down a power-generation system, it by collecting X-Y pairs and then esti-
probabilistic reasoning attributes would need to be explained as well. mating the function Y = f (X). While
numbers to Boolean propositions in Answering “Why?” questions is cen- this function may be all we need for
the first place, and that logic was at tral to assigning blame and respon- certain applications, it would not
the heart of probabilistic reasoning sibility and lies at the heart of legal qualify as a model, as it does not
except in its simplest form, we have systems. It is also now recognized capture the system mechanism.
now come to the conclusion that we that opacity, or lack of explainabili- Modeling that mechanism is essen-
need to attribute probabilities to ty, is “one of the biggest obstacles tial for certain explanations (Why is
more complex Boolean propositions to widespread adoption of artificial the change in the lever position not
and even to first-order sentences. The intelligence.”u a linear function of the amount of
resulting frameworks are referred to Models are more interpretable air blown?) and for causal reasoning
as “first-order probabilistic models” than functions.v Moreover, models more generally (What if the balloon
or “relational probabilistic models,” offer a wider class of explanations is pinched?). One may try to address
and there is a great need for skill in than functions, including explana- these issues by adding more inputs
symbolic logic to advance these for- tions of novel situations and expla- to the function but may also blow up
malisms. The only problem is that nations that can form a basis for the function size, among other dif-
this skill has almost vanished from “understanding” and “control.” This ficulties; more on this next.
within the AI community. is due to models having access to in- In his The Book of Why: The New Sci-
The blame for this phenomenon ence of Cause and Effect, Judea Pearl
cannot be assigned to any particular t I made these remarks over a dinner table that explained further the differences be-
party. It is natural for the successful included a young machine learning researcher, tween a (causal) model and a function,
to be overjoyed and sometimes also whose reaction was: “I feel much better now.” He even though he did not use the term
was apparently subjected to this phenomenon “function” explicitly. In Chapter 1, he
inflate that success. It is expected that
by support-vector-machine (SVM) researchers
industry will exploit such success in during his Ph.D. work when SVMs were at their
wrote: “There is only one way a thinking
ways that may redefine the employ- peak and considered “it” at the time. Another entity (computer or human) can work
ment market and influence the aca- young vision researcher, pressed on whether out what would happen in multiple
demic interests of graduate students. deep learning is able to address the ambitions of scenarios, including some that it has
vision research, said, “The reality is that you can-
It is also understandable that the rest never experienced before. It must pos-
not publish a vision paper today in a top confer-
of the academic community may play ence if it does not contain a deep learning com- sess, consult, and manipulate a mental
along for the sake of its survival: win a ponent, which is kind of depressing.” causal model of that reality.” He then
grant, get a paper in, attract a student. u See Castellanos, S. and Norton, S. Inside gave an example of a navigation system
While each of these behaviors seems Darpa’s push to make artificial intelligence based on either reasoning with a map
explain itself. The Wall Street Journal (Aug.
rational locally, their combination (model) or consulting a GPS system that
10, 2017); http://on.wsj.com/2vmZKlM; DAR-
can be harmful to scientific inquiry PA’s program on “explainable artificial intel- gives only a list of left-right turns for ar-
and hence irrational globally. Beyond ligence”; https://www.darpa.mil/program/ riving at a destination (function). The
raising awareness about this recur- explainable-artificial-intelligence; and the rest of the discussion focused on what
ring phenomenon, decision makers E.U. general data protection regulation on “ex- can be done with the model but not the
plainability”; https://www.privacy-regulation.
at the governmental and academic eu/en/r71.htm
function. Pearl’s argument particularly
levels bear a particular responsibility v I am referring here to learned and large func- focused on how a model can handle
for mitigating its negative effects. Se- tions of the kind that stand behind some of the novel scenarios (such as encountering
nior members of the academic com- current successes (such as neural networks roadblocks that invalidate the function
with thousands or millions of parameters). recommendations) while pointing to
munity also bear the responsibility
This excludes simple or well-understood
of putting current developments in learned functions and functions synthesized
the combinatorial impossibility of en-
historical perspective, to empower from models, as they can be interpretable or coding such contingencies in the func-
junior researchers in pursuing their explainable by design. tion, as it must have a bounded size.
O C TO B E R 2 0 1 8 | VO L. 6 1 | N O. 1 0 | C OM M U N IC AT ION S OF T HE ACM 65
contributed articles
mostly in instinct-based perception profound scientific contributions.aa tinuing to share its contents verbally
(such as computer vision and language On the other hand, I am reminded how in various contexts and revising ac-
processing). I agree with this obser- times of achievements can potentially cordingly. The decision to eventually
vation, except nothing at this stage slow scientific progress by shifting aca- release a first draft in July 2017 was
prohibits functions from providing demic interests, resources, and brain triggered by two events: a discussion
reasonable approximations to more power too significantly toward exploit- of these thoughts at a workshop orga-
high-level cognitive tasks. In fact, Go ing what was just discovered, at the nized by the UCLA School of Law and
functions have been constructed us- expense of understanding the discov- other discussions with colleagues
ing neural networks, even though they eries and preparing for the moment outside of AI, including architecture,
are not yet competitive with hybrid when their practical applications have programming languages, networks,
systems (such as AlphaGo). Admit- been delimited or exhausted. and theory. These discussions re-
tedly, it is also possible that we might There are many dimensions to vealed a substantial interest in the
later realize that functions (of practical such preparation. For the deep learn- subject and led me to conclude that
size) cannot provide reasonable ap- ing community, perhaps the most sig- the most important objective I should
proximations to a wide enough class nificant is a transition from the “look be seeking is “starting a discussion.”
of cognitive functions despite prog- what else we can do” mode to a “look I may have erred in certain parts, I
ress on pushing computational and what else you can do” mode. This is may have failed to give due credit, and
data thresholds. The association with not only an invitation to reach out to I may have missed parts of the evolv-
perception would then be more estab- and empower the broader AI commu- ing scene. I just hope the thoughts I
lished in that case. Time will tell. nity; it is also a challenge since such a share here will start that discussion,
transition is not only a function of at- and the collective wisdom of the com-
Conclusion titude but also an ability to character- munity will correct what I may have
This article was motivated by concerns ize progress in ways that enable people gotten wrong.
I and others have had on how current from outside the community to under-
progress in AI is being framed and stand and capitalize on it. The broader Acknowledgments
perceived. Without a scholarly discus- AI community is also both invited and I benefited greatly from the feedback
sion of the causes and effects of recent challenged to identify fundamental I received from anonymous review-
achievements, and without a proper ways in which functions can be turned ers and from colleagues who are
perspective on the obtained results, into a boon for building and learning too many to enumerate but whose
one stands to hinder further progress models. Given where we stand today, input and discussions were critical
by perhaps misguiding the young gen- the question is not whether it is func- to shaping the thoughts expressed
eration of researchers or misallocating tions or models but how to profoundly here. However, I must specifically
resources at the academic, industrial, integrate and fuse functions with mod- acknowledge Judea Pearl for inspir-
and governmental levels. One also els.ab This aim requires genuine cross- ing the article and for helping with
stands to misinform a public that has fertilization and the training of a new various arguments; Stuart Russell
developed a keen interest in AI and its generation of researchers who are well- for providing very thoughtful and
implications. The current negative dis- versed in and appreciative of various AI constructive feedback; Guy Van den
cussions by the general public on the methods, and who are better informed Broeck for keeping me interested in
AI singularity, also called “super intel- about the history of AI. the project every time I almost gave
ligence,” is partly due to the lack of ac- I conclude with this reflection: up; and Arthur Choi for being a gen-
curate framings and characterizations I wrote the first draft of this article erous and honest companion to the
of recent progress. With almost every- in November 2016. A number of col- thinking that went into it. Finally, I
one being either overexcited or over- leagues provided positive feedback wish to thank Nils Nilsson for telling
whelmed by the new developments, then, with one warning about a nega- me that he wished he had written the
substantial scholarly discussions and tive tone. I put the draft on hold for article and for kindly inviting me to
reflections have gone missing. some months as a result while con- share his feedback with others. This
I had the privilege of starting my is an ultimate reward.
research career in AI around the mid-
aa Judea Pearl’s seminal work on probabilistic
to-late 1980s during one of the major Adnan Darwiche (darwiche@cs.ucla.edu) is a professor
approaches to commonsense reasoning is one in and chairman of the Computer Science Department at
crises in the field, a period marked example outcome of the crisis. the University of California, Los Angeles, CA, USA.
by inability instead of ability. I was ab An anonymous reviewer brought to my atten-
dismayed then, as I sat in classes at tion works on the analyses of human cogni- Copyright held by author.
tion, particularly Daniel Kahneman’s book
Stanford University, witnessing how Thinking Fast and Slow. The reviewer said
AI researchers were being significant- “fast” naturally maps onto function-based
ly challenged by some of the simpler and “slow” onto model-based, and there is a
tasks performed routinely by humans. strong argument in the literature on cogni-
I now realize how such crises can be tive science that people must at least com- Watch the author discuss
bine them both. The reviewer further pointed his work in this exclusive
enabling for scientific discovery, as out that there are a variety of cognitive ar- Communications video.
https://cacm.acm.org/videos/
they fuel academic thinking, empower chitectures that embody specific hypotheses human-level-intelligence-or-
researchers, and create grounds for about such hybrids. animal-like-abilities
O C TO B E R 2 0 1 8 | VO L. 6 1 | N O. 1 0 | C OM M U N IC AT ION S OF T HE ACM 67
contributed articles
DOI:10.1145/ 3230627
One might think surviving such
Verified software secures the Unmanned an attack is not a big deal, certainly
that military aircraft would be robust
Little Bird autonomous helicopter against against cyber attacks. In reality, a
mid-flight cyber attacks. “red team” of professional penetra-
tion testers hired by the Defense Ad-
BY GERWIN KLEIN, JUNE ANDRONICK, MATTHEW FERNANDEZ, vanced Research Projects Agency
IHOR KUZ, TOBY MURRAY, AND GERNOT HEISER (DARPA) under its High-Assurance
Cyber Military Systems (HACMS)
Formally
program had in 2013 compromised
the baseline version of the ULB, de-
signed for safety rather than secu-
rity, to the point where it could have
Verified
crashed it or diverted to any location
of its choice. In this light, risking
an in-flight attack with a human on
board indicates that something had
Software in
changed dramatically.
This article explains that change
and the technology that enabled it.
Specifically, it is about technology de-
compromised some subsystems but could not affect ˽˽ High assurance can be retrofitted to
suitable existing systems with only
the safe operation of the aircraft. moderate redesign and refactoring.
was done by Boeing engineers, not by tecture, enforce the desired security Formal Verification
formal verification researchers. property, and our verified component Mathematical correctness proofs of pro-
By far, not all the software on the framework, CAmkES. The CAmkES grams go back to at least the 1960s,14
HACMS vehicles was built on the basis framework integrates with architec- but for a long time, their real-world
of mathematical models and reason- ture analysis tools from Rockwell Col- benefit to software development was
ing; the field of formal verification is lins and the University of Minnesota, limited in scale and depth. However, a
not yet ready for such scale. However, along with trusted high-assurance number of impressive breakthroughs
HACMS demonstrated that significant software components using domain- have been seen in recent years in the
improvement is feasible by applying specific languages from Galois Inc. formal code-level verification of real-
formal techniques strategically to the The HACMS achievements are life systems, from the verified C com-
most critical parts of the overall sys- based on the software engineer’s trusty piler CompCert28 to the verified seL4
tem. The HACMS approach works for old friend—modularization. What is microkernel,22,23,33 verified conference
systems in which the desired secu- new is that formal methods provide system CoCon,21 verified ML compiler
IMAGE: COPYRIGHT © BOEING. ALL RIGH TS RESERVED.
rity property can be achieved through proof that interfaces are observed and CakeML,25 verified interactive theorem
purely architecture-level enforcement. module internals are encapsulated. provers Milawa,9 and Candle,24 veri-
Its foundation is our verified microker- This guaranteed enforcement of mod- fied crash-resistant file system FSCQ,5
nel, seL4, discussed later, which guar- ularization allows engineers, like those verified distributed system IronFleet,19
antees isolation between subsystems at Boeing, who are not formal-method and verified concurrent kernel frame-
except for well-defined communica- experts, to construct new or even ret- work CertiKOS,17 as well as significant
tion channels that are subject to the rofit existing systems, as discussed mathematical theorems, including the
system’s security policy. This isolation later, and achieve high resilience, even Four Colour Theorem,15 mechanized
is leveraged by system-level compo- though the tools do not yet provide an proof of the Kepler Conjecture,18 and
nent architectures that, through archi- overall proof of system security. Odd Order Theorem.16 None of these
O C TO B E R 2 0 1 8 | VO L. 6 1 | N O. 1 0 | C OM M U N IC AT ION S OF T HE ACM 69
contributed articles
Figure 1. Isolation and controlled communi- hold—through other means of verifi- other publicly available software in hu-
cation with seL4. cation like testing. Moreover, in many man history in terms not only of lines
cases systems consist of a combina- of proof but strength of properties
Untrusted VM Untrusted Trusted tion of verified and non-verified code, proved. At the heart of this verification
and in them, formal verification acts story sits the proof of “functional cor-
Guest apps Native apps Native apps
as a lens, focusing review, testing, and rectness” of the kernel’s C implemen-
Guest OS debugging on the system’s critical non- tation,23 guaranteeing every behavior
verified code. of the kernel is predicted by its formal
seL4
abstract specification; see the online
Hardware seL4 appendix (dl.acm.org/citation.cfm?
We begin with the foundation for build- doid=3230627&picked=formats) for an
ing provably trustworthy systems—the idea of how these proofs look. Following
are toy systems. For instance, Comp- operating system (OS) kernel, the sys- this guarantee, we added further proofs
Cert is a commercial product, the seL4 tem’s most critical part and enabler we explain after first introducing the
microkernel is used in aerospace, au- of cost-effective trustworthiness of the main kernel mechanisms.
tonomous aviation, and as an Internet entire system. seL4 API. The seL4 kernel provides a
of Things platform, and the CoCon The seL4 microkernel provides a minimal set of mechanisms for imple-
system has been used in multiple full- formally verified minimal set of mecha- menting secure systems: threads, ca-
scale scientific conferences. nisms for implementing secure sys- pability management, virtual address
These verification projects required tems. Unlike standard separation ker- spaces, inter-process communication
significant effort, and for verification nels31 they are purposefully general and (IPC), signaling, and interrupt delivery.
to be practical for widespread use, the so can be combined for implementing a The kernel maintains its state in
effort needs to decrease. Here, we dem- range of security policies for a range of “kernel objects.” For example, for each
onstrate how strategically combining system requirements. thread in a system there is a “thread
formal and informal techniques, par- One of the main design goals of object” that stores information about
tially automating the formal ones, and seL4 (see the sidebar “Proof Effort”) scheduling, execution, and access con-
carefully architecting the software to is to enforce strong isolation between trol. User-space programs can refer to
maximize the benefits of isolated com- mutually distrusting components that kernel objects only indirectly through
ponents, allowed us to dramatically in- may run on top of it. The mechanisms “capabilities”10 that combine a refer-
crease the assurance of systems whose support its use as a hypervisor to, say, ence to an object with a set of access
overall size and complexity is orders- host entire Linux operating systems rights to this object. For example, a
of-magnitude greater than that of the while keeping them isolated from se- thread cannot start or stop another
systems mentioned earlier. curity-critical components that might thread unless it has a capability to the
Note we primarily use formal veri- run alongside, as outlined in Figure 1. corresponding thread object.
fication to provide proofs about cor- In particular, this functionality allows Threads communicate and syn-
rectness of code that a system’s safety system designers to deploy legacy chronize by sending messages through
or security relies on. But it has other components that may have latent vul- IPC “endpoint” objects. One thread
benefits as well. For example, code nerabilities alongside highly trustwor- with a send capability to an appropri-
correctness proofs make assumptions thy components. ate endpoint can message another
about the context in which the code is The seL4 kernel is unique among thread that has a receive capability to
run (such as behavior of hardware and general-purpose microkernels. Not that endpoint. “Notification” objects
configuration of software). Since for- only does it deliver the best perfor- provide synchronization through sets
mal verification makes these assump- mance in its class, 20 its 10,000 lines of binary semaphores. Virtual address
tions explicit, developer effort can of C code have been subjected to translation is managed by kernel ob-
focus on ensuring the assumptions more formal verification than any jects that represent page directories,
Figure 2. Kernel objects for an example seL4-based system with two threads communicating via an endpoint.
A B
CSpace CSpace
Thread ObjectA Thread Object
VSpace EP B
CNodeA1 CNodeB1
PDA
PTA1
Receive
Send
CONTEXT
CNodeA2
CONTEXT
FRAME
...
...
...
FRAME
...
...
... VSpace
O C TO B E R 2 0 1 8 | VO L. 6 1 | N O. 1 0 | C OM M U N IC AT ION S OF T HE ACM 71
contributed articles
representative of the ULB, and is, at hicle’s flight behavior. The operating behavior of that component, keys can-
this level of abstraction, the same as assumption is that the camera is un- not be leaked, as no other component
the ULB architecture. trusted and potentially compromised, has access to them; the link between
The figure includes two main com- or malicious, that its drivers and the Linux and the crypto component in Fig-
puters: a mission computer that com- legacy payload software are poten- ure 4 is for message passing only and
municates with the ground-control tially compromised, and any outside does not give access to memory. Only
station and manages mission-payload communication is likewise potentially authenticated messages can reach the
software (such as for controlling a compromised. For the purpose of this CAN bus, as the crypto component is
camera); and a flight computer with example, we assume a correct and the only connection to the driver. Un-
the task of flying the vehicle, reading strong cryptography implementation, trusted payload software and WiFi are,
sensor data, and controlling motors. or the key cannot be guessed, and that as part of the Linux VM, encapsulated
The computers communicate via an basic radio jamming and denial-of-ser- by component isolation and can com-
internal network, a controller area net- vice by overwhelming the ground sta- municate to the rest of the system only
work, or CAN bus, on the quadcopter, tion radio link are out of scope. via the trusted crypto component.
a dedicated Ethernet on the ULB. On Figure 4 outlines how we design It is easy to imagine that this kind
the quadcopter, the mission computer the quadcopter architecture to achieve of architecture analysis could be auto-
also has an insecure WiFi link, giving these properties. We use a virtual ma- mated to a high degree through model
us the opportunity to demonstrate fur- chine (VM) running Linux as a contain- checking and higher-level mechanized
ther security techniques. ment vessel for legacy payload soft- reasoning tools. As observed in MILS
The subsystem under consider- ware, camera drivers, and WiFi link. systems,1 component boundaries in
ation in this example is the mission We isolate the cryptography control an architecture are not just a conve-
computer. Four main properties must module in its own component, with nient decomposition tool for modu-
be enforced: only correctly authenti- connections to the CAN bus compo- larity and code management but,
cated commands from the ground sta- nent, to the ground station link, and with enforced isolation, provide ef-
tion are sent to the flight computer; to the Linux VM for sending image- fective boundaries for formal reason-
cryptographic keys are not leaked; no recognition data back to the ground ing about the behavior of the system.
additional messages are sent to the station. The purpose of the crypto However, the entire argument hinges
flight computer; and untrusted pay- component is to forward (only) autho- on the fact that component boundar-
load software cannot influence the ve- rized messages to the flight computer ies in the architecture are correctly en-
via the CAN interface stack and send forced at runtime in the final, binary
Figure 3. Autonomous-air-vehicle architecture. back diagnostic data to the ground sta- implementation of the system.
tion. The radio-link component sends The mechanisms of the seL4 kernel
WiFi
and receives raw messages that are en- discussed earlier can achieve this en-
Ground Mission
Station Link Computer crypted, decrypted, and authenticated, forcement, but the level of abstraction
Camera respectively, by the crypto component. of the mechanisms is in stark contrast
Establishing the desired system to the boxes and arrows of an architec-
NET
properties is now reduced purely to the ture diagram; even the more abstract
isolation properties and information- access-control policy still contains
Flight
Sensors Motors flow behavior of the architecture, and far more detail than the architecture
Computer
to the behavior of the single trusted diagram. A running system of this size
crypto component. Assuming correct contains tens of thousands of kernel
objects and capabilities that are cre-
Figure 4. Simplified quadcopter mission-computer architecture. ated programmatically, and errors in
configuration could lead to security
violations. We next discuss how we not
only automate the configuration and
Radio Data
Link
Linux VM construction of such code but also how
Driver
we can automatically prove that archi-
WiFi tecture boundaries are enforced.
and
Camera
Crypto Verified Componentization
The same way reasoning about secu-
Non-critical
rity becomes easier with the formal ab-
CAN bus
CAN
Critical
Untrusted stractions of security policies, abstrac-
Driver Trusted tion also helps in building systems.
Contained
The CAmkES component platform,27
which runs on seL4 abstracts over the
seL4 low-level kernel mechanisms, provides
communication primitives, as well as
support for decomposing a system into
CONTEXT
CONTEXT
Receive
Send
connectors use endpoint objects, and
...
...
CAmkES generates glue code to mar- VSpace VSpace
shal and unmarshal messages and
send them over IPC endpoints. Like-
wise, a dataport connector is imple-
mented through shared memory,
initialized system + proof
shared frame objects present in the ad-
dress spaces of two components, and
optionally restricting the direction of
communication. Finally, an event con- Figure 6. RPC-generated code.
nector is implemented using seL4’s
notification mechanism. A B
CAmkES also generates, in the cap- g() {
DL language,26 a low-level specification f(); ...
of the system’s initial configuration of handwritten }
kernel objects and capabilities. This
capDL specification is the input for the f() { g_stub() {
generic seL4 initializer that runs as the //glue: //glue:
// marshalling // seL4_Recv(ep,...)
first task after boot and performs the // unmarshalling
generated // seL4_Send(ep,...) seL4
necessary seL4 operations to instanti- // seL4_Recv(ep,...) // g_invoke()
ate and initialize the system.4 // unmarshalling // marshalling
} // seL4_Send(ep,...)
In summary, a component platform }
provides free code. The component ar-
chitecture describes a set of boxes and
arrows, and the implementation task is
reduced to simply filling in the boxes; tomate large parts of system construc- lent to calling g. The lemma the system
the platform generates the rest while tion without expanding the trusted generates ensures the invocation of the
enforcing the architecture. computing base. generated RPC glue code f behaves as
With a traditional component plat- Developers rarely look at the output a direct invocation of g, as if it were co-
form, the enforcement process would of code generators, focusing instead on located with the caller.
mean the generated code increases the the functionality and business logic of To be useful, the proofs the system
trusted computing base of the system, their systems. In the same way, we in- generates must be composable with
as it has the ability to influence the tend the glue code proofs to be artifacts (almost) arbitrary user-provided proofs,
functionality of the components. How- that do not need to be examined, mean- both of the function g and of the contexts
ever, CAmkES also generates proofs. ing developers can focus on proving the where g and f are used. To enable this
Automated proofs. While generat- correctness of their handwritten code. composability, the specification of the
ing glue code, CAmkES produces for- Mirroring the way a header generated by connectors is parameterized through
mal proofs in Isabelle/HOL, following CAmkES gives the developer an API for user-provided specifications of remote
a translation-validation approach,30 the generated code, the top-level gener- functions. In this way, proof engineers
demonstrating that the generated glue ated lemma statements produce a proof can reason about their architecture,
code obeys a high-level specification API. The lemmas describe the expected providing specifications and proofs for
and the generated capDL specification behavior of the connectors. In the ex- their components, and rely on specifica-
is a correct refinement of the CAmkES ample of RPC glue code outlined in Fig- tions for the generated code.
description.12 We have also proved that ure 6, the generated function f provides To date, we have demonstrated this
the generic seL4 initializer correctly a way to invoke a remote function g in process end-to-end using a specific
sets up the system in the desired ini- another component. To preserve the CAmkES RPC connector.12,13 Extending
tial configuration. In doing so, we au- abstraction, calling f must be equiva- the proof generator to support other
O C TO B E R 2 0 1 8 | VO L. 6 1 | N O. 1 0 | C OM M U N IC AT ION S OF T HE ACM 73
contributed articles
connectors, allowing construction of izer satisfies the one described in the mission-computer functionality. The
more diverse verified systems, should given specification.4 This proof holds system was built and re-engineered by
be simpler to achieve, because other for a precise model of the initializer Boeing engineers, using the methods,
connector patterns (data ports and but not yet at the implementation lev- tools, and components provided by the
events) are significantly less complex el. Compared to the depth of the rest HACMS partners.
than RPC. of the proof chain, this limitation may Step 1. Virtualization. The first step
Next to communication code, appear weak, but it is already more for- was to take the system as is and run it
CAmkES produces the initial access mal proof than would be required for in a VM on top of a secure hypervisor
control configuration that is designed the highest level (EAL7) of a Common (see Figure 7). In the seismic-retrofit
to enforce architecture boundaries. To Criteria security evaluation. metaphor, doing so corresponds to
prove the two system descriptions— situating the system on a more flex-
capDL and CAmkES—correspond, we Seismic Security Retrofit ible foundation. A VM on top of seL4
consider the CAmkES description as In practice, there are few opportuni- in this system consists of one CAmkES
an abstraction of the capDL descrip- ties to engineer a system from scratch component that includes a virtual ma-
tion. We use the established frame- for security, so the ability to retrofit chine monitor (VMM) and the guest
work36 mentioned earlier to infer for security is crucial for engineer- operating system, in this case Linux.
authority of one object over another ing secure systems. Our seL4-based The kernel provides abstractions of
object from a capDL description to lift framework supports an iterative pro- the virtualization hardware, while the
reasoning to a policy level. Addition- cess we call “seismic security retrofit,” VMM manages these abstractions for
ally, we have defined rules for inferring as a regular structural architect might the VM. The seL4 kernel constrains not
authority between components in a retrofit an existing building for great- only the guest but also the VMM, so the
CAmkES description. The produced er resilience against earthquakes. VMM implementation does not need
proof ensures the capDL objects, when We illustrate the process by walking to be trusted to enforce isolation. Fail-
represented as an authority graph through an example that incremental- ure of the VMM will lead to failure of
with objects grouped per component, ly adapts the existing software archi- the guest but not to failure of the com-
have the same intergroup edges as the tecture of an autonomous air vehicle, plete system.
equivalent graph between CAmkES moving it from a traditional testing Depending on system configura-
components.12 Intuitively, this corre- approach to a high-assurance sys- tion, the VM may have access to hard-
spondence between the edges means tem with theorems backed by formal ware devices through para-virtualized
an architecture analysis of the policy methods. While this example is based drivers, pass-through drivers, or both.
inferred by the CAmkES description on work done for a real vehicle—the In the case of pass-through drivers,
will hold for the policy inferred by the ULB—it is simplified for presentation developers can make use of a system
generated capDL description, which in and does not include all details. MMU or IOMMU to prevent hardware
turn is proved to satisfy authority con- The original vehicle architecture devices and drivers in the guest from
finement, integrity, and confidential- is the same as the architecture out- breaching isolation boundaries. Note
ity, as mentioned earlier. lined in Figure 3. Its functionality is that simply running a system in a VM
Finally, to prove correct initializa- split over two separate computers: a adds no additional security or reliabil-
tion, CAmkES leverages the generic flight computer that controls the ac- ity benefits. Instead, the reason for this
initializer that will run as the first user tual flying and the mission computer first step is to enable step 2.
task following boot time. In seL4, this that performs high-level tasks (such as Step 2. Multiple VMs. The second
first (and unique) user task has access ground-station communication and step in a seismic retrofit strengthens
to all available memory, using it to cre- camera-based navigation). The origi- existing walls. In software, the devel-
ate objects and capabilities accord- nal version of the mission computer oper can improve security and reli-
ing to the detailed capDL description was a monolithic software application ability by splitting the original system
it takes as input. We proved that the running on Linux. The rest of the ex- into multiple subsystem partitions,
state following execution of the initial- ample concentrates on a retrofit of this each consisting of a VM running the
Figure 7. All functionality in a single VM. Figure 8. Functionality split into multiple VMs. Figure 9. Functionality in native components.
O C TO B E R 2 0 1 8 | VO L. 6 1 | N O. 1 0 | C OM M U N IC AT ION S OF T HE ACM 75
contributed articles
Fisher for having the vision to start the International Conference (Noordwijk, the Netherlands, Proceedings of the First ACM Asia-Pacific Workshop
May 12–14). Curran, Red Hook, NY, 2008. on Systems (New Delhi, India, Aug. 30–Sept. 3). ACM
program. John Launchbury coined 9. Davis, J. and Myreen, M.O. The reflective Milawa Press, New York, 2010, 31–35.
the term “seismic security retrofit.” theorem prover is sound (down to the machine code 27. Kuz, I., Liu, Y., Gorton, I., and Heiser, G. CAmkES:
that runs it). Journal of Automated Reasoning 55, 2 A component model for secure microkernel-based
We thank Lee Pike for feedback on an (Aug. 2015), 117–183. embedded systems. Journal of Systems and Software
earlier draft. We would also like to ac- 10. Dennis, J.B. and Van Horn, E.C. Programming (Special Edition on Component-Based Software
semantics for multi-programmed computations. Engineering of Trustworthy Embedded Systems) 80, 5
knowledge our HACMS project part- Commun. ACM 9, 3 (Mar. 1966), 143–155. (May 2007), 687–699.
ners from Rockwell Collins, the Univer- 11. Elliott, T., Pike, L., Winwood, S., Hickey, P., Bielman, 28. Leroy, X. Formal verification of a realistic compiler.
J., Sharp, J., Seidel, E., and Launchbury, J. Guilt-free Commun. ACM 52, 7 (July 2009), 107–115.
sity of Minnesota, Galois, and Boeing. Ivory. In Proceedings of the ACM SIGPLAN Haskell 29. Murray, T., Matichuk, D., Brassil, M., Gammie, P.,
While we concentrated on the oper- Symposium (Vancouver, Canada, Sept. 3–4). ACM Bourke, T., Seefried, S., Lewis, C., Gao, X., and Klein, G.
Press, New York, 189–200. seL4: From general-purpose to a proof of information
ating system aspects of the HACMS 12. Fernandez, M. Formal Verification of a Component flow enforcement. In Proceedings of the 2013 IEEE
Platform. Ph.D. thesis. School of Computer Science & Symposium on Security and Privacy (San Francisco,
project here, the rapid construction CA, May 19–22). IEEE Press, Los Alamitos, CA, 2013,
Engineering, University of New South Wales, Sydney,
of high-assurance systems includes Australia, July 2016. 415–429.
13. Fernandez, M., Andronick, J., Klein, G., and Kuz, 30. Pnueli, A., Siegel, M., and Singerman, E. Translation
many further components, including I. Automated verification of RPC stub code. In validation. In Proceedings of the Fourth International
a trusted build, as well as architecture Proceedings of the 20th International Symposium on Conference on Tools and Algorithms for Construction
Formal Methods (Oslo, Norway, June 22–26). Springer, and Analysis of Systems (Lisbon, Portugal, Mar. 28–
and security-analysis tools. This mate- Heidelberg, Germany, 2015, 273–290. Apr. 4). Springer, Berlin, Germany, 1998, 151–166.
rial is based on research sponsored by 14. Floyd, R.W. Assigning meanings to programs. 31. Rushby, J. Design and verification of secure systems.
Mathematical Aspects of Computer Science 19, In Proceedings of the Eighth Symposium on Operating
the U.S. Air Force Research Laboratory (1967), 19–32. System Principles (Pacific Grove, CA, Dec. 14–16).
and the Defense Advanced Research 15. Gonthier, G. A Computer-Checked Proof of the Four- ACM Press, New York, 1981, 12–21.
Colour Theorem. Microsoft Research, Cambridge, U.K, 32. Ryzhyk, L., Chubb, P., Kuz, I., Le Sueur, E., and Heiser,
Projects Agency under agreement 2005; https://www.microsoft.com/en-us/research/wp- G. Automatic device driver synthesis with Termite. In
number FA8750-12-9-0179. The U.S. content/uploads/2016/02/gonthier-4colproof.pdf Proceedings of the 22nd ACM Symposium on Operating
16. Gonthier, G., Asperti, A., Avigad, J., Bertot, Y., Cohen, Systems Principles (Big Sky, MT, Oct. 11–14). ACM
government is authorized to repro- C., Garillot, F., Le Roux, S., Mahboubi, A., O’Connor, Press, New York, 2009, 73–86.
duce and distribute reprints for gov- R., Biha S.O., Pasca, I., Rideau, L., Solovyev, A., Tassi, 33. seL4 microkernel code and proofs; https://github.
E., and Théry, L. A machine-checked proof of the com/seL4/
ernmental purposes notwithstanding Odd Order Theorem. In Proceedings of the Fourth 34. Sewell, T., Kam, F., and Heiser, G. Complete, high-
any copyright notation thereon. The International Conference on Interactive Theorem assurance determination of loop bounds and infeasible
Proving, Volume 7998 of LNCS (Rennes, France, July paths for WCET analysis. In Proceedings of the 22nd
views and conclusions contained here- 22–26). Springer, Heidelberg, Germany, 2013, 163–179. IEEE Real Time and Embedded Technology and
Applications Symposium (Vienna, Austria, Apr. 11–14).
in are those of the authors and should 17. Gu, R., Shao, Z., Chen, H., Wu, X.(N.)., Kim, J., Sjöberg, V.,
IEEE Press, 2016.
and Costanzo, C. CertiKOS: An extensible architecture
not be interpreted as necessarily repre- for building certified concurrent OS kernels. In 35. Sewell, T., Myreen, M., and Klein, G. Translation
validation for a verified OS kernel. In Proceedings
senting the official policies or endorse- Proceedings of the 12th USENIX Symposium on
of the 34th Annual ACM SIGPLAN Conference on
Operating Systems Design and Implementation
ments, either expressed or implied, (Savannah, GA, Nov. 2–4). ACM Press, New York, 2016. Programming Language Design and Implementation
18. Hales, T.C., Adams, M., Bauer, G., Dang, D.T., Harrison, (Seattle, WA, June 16–22). ACM Press, New York,
of the Air Force Research Laboratory, 2013, 471–481.
J., Le Hoang, T., Kaliszyk, C., Magron, V., McLaughlin, S.,
Defense Advanced Research Projects Nguyen, T.T., Nguyen, T.Q., Nipkow, T., Obua, S., Pleso, 36. Sewell, T., Winwood, S., Gammie, P., Murray, T.,
J., Rute, J., Solovyev, A., Ta, A.H.T., Tran, T.N., Trieu, T.T., Andronick, J., and Klein, G. seL4 enforces integrity.
Agency, or U.S. government. In Proceedings of the International Conference
Urban, J., Vu, K.K., and Zumkeller, R. A formal proof
of the Kepler Conjecture. Forum of Mathematics, Pi, on Interactive Theorem Proving (Nijmegen, the
Volume 5. Cambridge University Press, 2017. Netherlands, Aug. 22–25). Springer, Heidelberg,
References
19. Hawblitzel, C., Howell, J., Kapritsos, M., Lorch, J.R., Germany, 2011, 325–340.
1. Alves-Foss, J., Oman, P.W., Taylor, C., and Harrison, S.
The MILS architecture for high-assurance embedded Parno, B., Roberts, M.L., Setty, S.T.V., and Zill, B.
systems. International Journal of Embedded Systems IronFleet: Proving practical distributed systems
Gerwin Klein (gerwin.klein@data61.csiro.au) is a Chief
2, 3-4 (2006), 239–247. correct. In Proceedings of the 25th ACM Symposium on
Research Scientist at Data61, CSIRO, and Conjoint
2. Blackham, B., Shi, Y., Chattopadhyay, S., Operating Systems Principles (Monterey, CA, Oct. 5–7).
Professor at UNSW, Sydney, Australia.
Roychoudhury, A., and Heiser, G. Timing analysis of a ACM Press, New York, 2015, 1–17.
protected operating system kernel. In Proceedings of 20. Heiser, G. and Elphinstone, K. L4 microkernels: The June Andronick (june.andronick@data61.csiro.au) is a
the 32nd IEEE Real-Time Systems Symposium (Vienna, lessons from 20 years of research and deployment. Principal Research Scientist at Data61, CSIRO, Conjoint
Austria, Nov. 29–Dec. 2). IEEE Computer Society ACM Transactions on Computer Systems 34, 1 (Apr. Associate Professor at UNSW, Sydney, Australia, and
Press, 2011, 339–348. 2016), 1:1–1:29. the leader of the Trustworthy Systems group at Data61,
3. Boeing. Unmanned Little Bird H-6U; http://www. 21. Kanav, S., Lammich, P., and Popescu, A. A known for the formal verification of the seL4 operating
boeing.com/defense/unmanned-little-bird-h-6u/ conference management system with verified system microkernel.
4. Boyton, A., Andronick, J., Bannister, C., Fernandez, document confidentiality. In Proceedings of the
M., Gao, X., Greenaway, D., Klein, G., Lewis, C., and 26th International Conference on Computer Aided Matthew Fernandez (matthew.fernandez@gmail.com)
Sewell, T. Formally verified system initialisation. In Verification (Vienna, Austria, July 18–22). ACM Press, participated in this project while he was a Ph.D. student
Proceedings of the 15th International Conference New York, 2014, 167–183. at UNSW, Sydney, Australia, and is today a researcher at
on Formal Engineering Methods (Queenstown, New 22. Klein, G., Andronick, J., Elphinstone, K., Murray, T., Intel Labs, USA.
Zealand, Oct. 29–Nov. 1). Springer, Heidelberg, Sewell, T., Kolanski, R., and Heiser, G. Comprehensive
Ihor Kuz (ihor.kuz@data61.csiro.au) is a Principal
Germany, 2013 70–85. formal verification of an OS microkernel. ACM Research Engineer at Data61, CSIRO , and also a Conjoint
5. Chen, H., Ziegler, D., Chajed, T., Chlipala, A., Frans Transactions on Computer Systems 32, 1 (Feb. 2014), Associate Professor at UNSW, Sydney, Australia.
Kaashoek, M., and Zeldovich, N. Using Crash 2:1–2:70.
Hoare logic for certifying the FSCQ file system. In 23. Klein, G., Elphinstone, K., Heiser, G., Andronick, J., Toby Murray (toby.murray@unimelb.edu.au) is a lecturer
Proceedings of the 25th ACM Symposium on Operating Cock, D., Derrin, P., Elkaduwe, D., Engelhardt, K., at the University of Melbourne, Australia, and a Senior
Systems Principles (Monterey, CA, Oct. 5–7). ACM Kolanski, R., Norrish, M., Sewell, T., Tuch, H., and Research Scientist at Data61, CSIRO.
Press, New York, 2015, 18–37. Winwood, S. seL4: Formal verification of an OS kernel.
6. Cock, D., Ge, Q., Murray, T., and Heiser, G. The last mile: In Proceedings of the 22nd ACM Symposium on Gernot Heiser (gernot@unsw.edu.au) is a Scientia
An empirical study of some timing channels on seL4. Operating Systems Principles (Big Sky, MT, Oct. 11–14). Professor and John Lions Chair of Computer Science at
In Proceedings of the ACM SIGSAC Conference on ACM Press, New York, 2009, 207–220. UNSW, Sydney, Australia, a Chief Research Scientist at
Computer and Communications Security (Scottsdale, 24. Kumar, R., Arthan, R., Myreen, M.O., and Owens, S. Data61, CSIRO, and a fellow of the ACM, the IEEE, and
AZ, Nov. 3–7). ACM Press, New York, 2014, 570–581. Self-formalisation of higher-order logic: Semantics, the Australian Academy of Technology and Engineering.
7. Cock, D., Klein, G., and Sewell, T. Secure microkernels, soundness, and a verified implementation. Journal of
state monads and scalable refinement. In Automated Reasoning 56, 3 (Apr. 2016), 221–259.
Proceedings of the 21st International Conference on 25. Kumar, R., Myreen, M., Norrish, M., and Owens,
Theorem Proving in Higher Order Logics (Montreal, S. CakeML: A verified implementation of ML. In
Canada, Aug. 18–21). Springer, Heidelberg, Germany, Proceedings of the 41st ACM SIGPLAN-SIGACT
2008, 167–182. Symposium on Principles of Programming Languages
8. Colbert, E. and Boehm, B. Cost estimation for (San Diego, CA, Jan. 22–24). ACM Press, New York,
secure software & systems. In Proceedings of 2014, 179–191.
the International Society of Parametric Analysts / 26. Kuz, I., Klein, G., Lewis, C., and Walker, A. capDL: A Copyright held by authors.
Society of Cost Estimating and Analysis 2008 Joint language for describing capability-based systems. In Publication rights licensed to ACM. $15.00
O C TO B E R 2 0 1 8 | VO L. 6 1 | N O. 1 0 | C OM M U N IC AT ION S OF T HE ACM 77
contributed articles
DOI:10.1145/ 3183583
outlined bold goals for HIT adoption
New York State healthcare providers increased as a key facet of each of their health-
care reform efforts, promising sig-
their use of the technology but delivered nificant benefits for healthcare pro-
only mixed results for their patients. viders and patients alike.20 Clinical
HIT systems, including electronic
BY QUANG “NEO” BUI, SEAN HANSEN, health records (EHRs), health infor-
MANLU LIU, AND QIANG (JOHN) TU mation exchanges (HIEs), comput-
erized provider order entry (CPOE),
The Productivity
and telemedicine technologies, are
seen as critical remedies to the com-
plexity and inefficiency that have long
plagued the U.S. healthcare industry.a
Paradox in
In 2009, the U.S. allocated more
than $30 billion, aiming to reduce
healthcare costs and increase quality
Health
of care through adoption and use of
HIT systems.1 In that same year, the
Office of the National Coordinator for
Information
Health Information Technology (ONC)
was established as part of the Health
Information Technology for Economic
Technology
and Clinical Health (HITECH) Act of
2009 to drive HIT adoption and co-
ordinate development of critical HIT
infrastructure. The ONC oversees a
range of programs (such as regional
extension centers, HIEs, privacy and
security policies, workforce develop-
ment, and curriculum development).
The HITECH Act introduced the prin-
ciple of “meaningful use” of HIT, a set
“HEALTH INFORMATION TECHNOLOGY connects doctors of guidelines for the substantive adop-
tion and application of HIT, including
and patients to more complete and accurate health
records … This technology is critical to improving a HIT reflects a range of technologies that can be
applied to the delivery and administration of
patient care, enabling coordination between providers healthcare service. In the present study, we fo-
and patients, reducing the risk of dangerous drug cus primarily on clinical HIT systems, empha-
sizing EHR and HIE systems, as they have been
interactions, and helping patients access prevention the leading areas of emphasis in the ongoing
wave of HIT adoption in the U.S.
and disease management services.”
— President Barack Obama, Presidential Proclamation key insights
on National Health Information Technology Week,
˽˽ No conclusive evidence has shown HIT
September 12, 2011 contribution to health outcomes among
New York State healthcare providers.
governmental support, evidence of gests the observable effects are limited evidence from the State of New York.
HIT’s contribution to health outcomes or even negative, marked by the risk of As the country’s fourth most popu-
remains mixed.7 A 2014 report from disrupted workflows, degradation of lous state and a national leader in
the U.S. Government Accountability physician-patient relationships, and HIT investment and adoption, New
Office (GAO) suggested that meaning- reduced clinical insight.25 In light of York offers a valuable context for as-
ful use requirements have had a mod- these findings, many researchers and sessing the effect of growing use of
est effect, and a comprehensive strate- public-policy observers have called for clinical HIT. Since 2007, New York
gy is needed to achieve better quality of additional studies to provide credible has invested more than $840 millionb
care through HIT.14 In addition, while evidence of improved health outcomes
several studies highlight perceived through expanded use of HIT.26 b https://www.health.ny.gov/technology/
O C TO B E R 2 0 1 8 | VO L. 6 1 | N O. 1 0 | C OM M U N IC AT ION S OF T HE ACM 79
contributed articles
Figure 1. Adoption of EHR functionalities by hospitals in New York State. 2015 for more than 180 hospitals across
the state. We tested a structural model
in which higher HIT investments
would lead to increased adoption and
36 use of EHR systems and HIEs that
Number of EHR Functionalities
HIT Investment
120
systems, implementation of clinical
25 decision-support functionality, and
100 significant participation in HIEs.
20 Specifically, New York healthcare
80 providers implemented most EHR
15
functionalities classified as “basic”
60
(see Figure 1). On average, New York
10
40 hospitals implemented 5.48 out of
six basic EHR functions (such as
5 20 electronic document viewing, results
viewing, CPOE, and decision sup-
0 0
2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 port); and hospitals differ only by the
Year degree of implementation around
other advanced EHR functionalities
(such as barcode identification, tele-
in health information infrastructure. dination among healthcare providers health, mobile device connections).
In that time, a variety of initiatives across boundaries.c Additionally, the number of new hos-
within the state have sought to fos- To understand HIT effects among pitals joining local HIEs corresponds
ter information exchange, improve New York healthcare providers, we to the surge in the state’s public
quality and outcomes of care, reduce conducted a mixed-methods study us- funding for HIT investment in 2008,
healthcare costs, and engage con- ing both quantitative and qualitative significantly augmented in 2015 and
stituents in their care.22 Specifically, approaches. Our quantitative analy- 2016 (see Figure 2).e As of 2018, over
the state has focused on establishing ses used publicly available data from 80% of New York healthcare-provider
governance and policies that increase New York HIEs, New York State web-
participation in regional HIEs and sites, and databases made available d Details of our research methodology is
encourage EHR system adoption by by the not-for-profit American Hospi- provided in the online appendix “Re-
hospitals and individual providers. tal Association and the U.S. Centers search Methodology”; dl.acm.org/citation.
These efforts align with federal HIT for Medicare and Medicaid Services. cfm?doid=3183583&picked=formats
e These local HIEs received public grants from
meaningful-use initiatives aimed The dataset covered the period 2014– New York State to increase information shar-
at creating better management of ing among hospitals; https://www.health.
medical records and seamless coor- c https://www.healthit.gov/ ny.gov/technology/financial_investment.htm
organizations—162 out of 197—had rather than enhance the quality of are generated have an awful lot of
joined HIEs and regularly exchange care providers render. Prominent words but communicate very little.”
medical records data electronically. concerns include the perception — Physician, Family Practice
While the majority of New York hos- that HIT adoption results in extra “The highlighted efficiency from
pitals have implemented and used EHR workload, ineffective communica- reducing duplicate lab tests and cut-
and HIEs in their practice, the evidence tion, poor information quality, and ting costs is just not there yet. I am not
is inconclusive with respect to how ineffectiveness addressing opera- really sure that an EHR will provide
these initiatives have affected quality of tional needs. The following illustra- the savings that are talked about.”
care and broad health outcomes across tive statements highlight the con- — Physician, Internal Medicine
the state. We found no evidence of a re- cerns shared by our respondents: “I have charting at home. I ended up
lationship between HIT use and such “This whole business about elec- having to get a laptop through my work
critical health outcomes as improved tronic medical records helping with budget to bring home so that I wasn’t
interpersonal care, customer satisfac- communication I think is a total falla- sitting at the office until ... I would
tion, customer loyalty, patient mortal- cy. I think it really hinders communica- see my last person around 4:20, and I
ity, and reduced ER waiting times (see tion, unless you freehand-type or you would be there until 6:30 doing chart-
Figure 3). These results are in line with dictate, which defeats the main pur- ing because of being slow with the
previous studies suggesting unclear pose of electronic medical records.” system and be more attentive to the
evidence of HIT effects.15 — Physician, Pediatrics patient than I was to the computer.”
While HIE participation and EHR “I hear complaints from patients say- — Nurse Practitioner
use levels reveal no significant rela- ing, ‘They’re looking at the computer In summary, our mixed-methods
tionships with most outcome mea- and not at me.’” — Physician, Pediatrics analyses suggest strong evidence of
sures, we were surprised to find EHR “This is my issue with all electron- increased adoption and use of EHR
use also does have a significant ad- ic medical records: The notes that and HIE among New York healthcare
verse relationship with patient re-
admission rates and complication Figure 3. Effects of HIT investment on hospital performance.
rates. To further explore this coun-
terintuitive result, we looked at the Interpersonal
social-capital index in each county Care
where the hospitals operate. The so-
cial-capital index27 reflects the socio- Overall
Rating
economic growth of a community.f
The post-analyses suggest areas with EMR Exchange –0.227*** HIE Participation
0.017* Capabilities Level
low social capital often see higher Loyalty
readmission rates and complication HIT
Investments
rates. This low score is due to such
factors as rural market, low social ER Waiting
0.172*** Time
support, and low educational rate. EHR 0.269*** 0.145*
Functionalities EHR Use
One possible explanation for our
Readmission
counterintuitive finding is that hos- Rates
pitals in areas with low social capital * p-value < 0.05; ** p-value < 0.01; *** p-value < 0.001
encounter inherent difficulties that 0.036**
Statistically significant relationship Complication
in turn increase patient readmission Rates
Statistically insignificant relationship
and complication rates regardless of
their use of HIT. We encourage future
research into this relationship.
Augmenting our quantitative Explaining the IT productivity paradox in HIT contexts.
analysis, our conversations with
healthcare providers suggest mixed
Causes Description
feelings and skepticism toward the
HIT mismeasurement Most HIT measures focus on efficiency rather than effective-
expected values of HIT. In particu- ness. Recent efforts like “meaningful use” level 2 are useful
lar, many clinicians were concerned but far from satisfactory.
that HIT initiatives were too often Delay delivering HIT benefits HITs are complex systems that require an average of two
not motivated by patient-oriented to four years to deliver significant benefits to healthcare
objectives and might undermine providers.
Redistribution of HIT benefits HIT gains are offset by unintended consequences in health-
care processes and procedures, including extra work and
f The social capital index was developed by
lack of human-doctor interaction.
the Northeast Regional Center for Rural
Development (http://aese.psu.edu/nercrd) Mismanagement of HIT systems Healthcare managers are not adequately trained to deal with
and uses an array of individual and commu- the complexity of HIT systems.
nity factors to measure the socioeconomic
growth of a community.
O C TO B E R 2 0 1 8 | VO L. 6 1 | N O. 1 0 | C OM M U N IC AT ION S OF T HE ACM 81
contributed articles
providers but cast doubt on the Brynjolfsson5 said the productivity most exclusively on measures of input,
claim of substantial HIT effects on paradox could derive from the fact that or use of a certified system for records
health outcomes. “IT really is not productive at the firm capture, reporting, and data exchange.
level” or that managers have not been Despite incentivizing inputs, the ulti-
Assessing HIT able to apply IT resources effectively. mate objective of the meaningful-use
The challenge of finding evidence of “One of the health plans locally guidelines is substantive improvement
practical benefits accruing from IT in- made an attempt at doing reporting in health outcomes (such as quality of
vestment is not unique to healthcare. [on provider efficiency]. They based it care and fewer medical errors). This
Indeed, the IT productivity paradox,5 totally on cost. So [when they looked at disconnect suggests we may need bet-
an apparent disconnect between in- the report] one of the physicians that ter measures to capture the contribu-
vestment in IT resources and discern- was in the top had died six months tion of HIT investment to those ulti-
ible impact on organizational per- before. He looked very efficient mate objectives.29
formance, has been widely observed from a cost perspective. He hadn’t With respect to the question of a
with earlier waves of IT adoption in generated any cost to the system.” temporal lag, a number of studies have
manufacturing and other industrial — Director, Medical Society suggested this is a critical issue in the
sectors. In a seminal disposition on the In the years since the initial explo- healthcare context. For example, Me-
phenomenon, Erik Brynjolfsson5 sum- rations of the productivity paradox, non et al.21 found it takes, on aver-
marized a number of concerns that the apparent disconnect between IT age, from two to four years for HIT
emerged in the 1980s and early 1990s investment and organizational out- systems to improve health outcomes
around a lack of productivity gains comes has been largely resolved; that in a given healthcare-provider orga-
corresponding to rapid adoption of IT is, researchers have concluded that nization. Many providers lack the
resources. Several analysts had noted the first two explanations—mismea- necessary IT skills to quickly get
significant growth in technological surement and lagged effects—were acquainted with new HIT tools and
investment and innovation across the primary drivers of the paradoxical procedures, making implementation
developed economies had coincided observations6 and that IT investment more challenging. Given the fact that
with disappointing gains—or even de- is indeed correlated with significant the uptick of HIT investment com-
clines—in productivity.11,23 It appears improvement in various measures of menced only in 2009, it may take many
that just as in the manufacturing sec- value at firm, industry, and country more years for HIT influence to ripple
tor, HIT is struggling to produce cred- levels, but such gains might take years across healthcare providers.
ible improvements in key measures to materialize.12,13,28 However, the idio- The possibility of redistributive ef-
of performance. The IT productivity syncratic characteristics of the health- fects also warrants consideration in the
paradox has once again surfaced in the care sector (such as institutional het- HIT context. As the comments of our
healthcare industry. erogeneity, combination of public and study respondents underscore, many
In his exploration of the phenome- private influences, and comparatively healthcare providers fear the efficiency
non, Brynjolfsson5 suggested four pos- late adoption of IT innovations) un- in reporting and data analysis HIT en-
sible explanations: mismeasurement, derscore important differences with genders for insurance firms and regula-
temporal lags, redistribution, and the sectors explored previously. Con- tors comes at the expense of decreased
mismanagement. Mismeasurement re- sequently, a thorough consideration efficiency for clinicians who actually
fers to the idea that we lack appropriate of diverse possible factors is war- deliver clinical care. Indeed, this shift-
measures for productivity in a service- ranted.17,19 Indeed, the four proposed ing of efficiencies and burdens can be
based economy, with most traditional, explanations associated with the IT seen in one of the most common orga-
manufacturing-oriented measures of productivity paradox suggest critical nizational responses to HIT adoption:
productivity failing to account for in- clues for considering the inconclusive dedicated “scribes” to capture data dur-
direct benefits (such as quality and effects of contemporary HIT invest- ing a clinical encounter. The question
customer satisfaction). The issue of a ment (see the table here). of whether efficiency gains in one facet
temporal lag centers on the possibility As our analysis highlights, the idio- of the healthcare system are partially
that gains from IT investment could syncratic nature of the healthcare do- outweighed by efficiency or process
take years to develop as organizations main introduces a range of relatively losses elsewhere in the system thus re-
change their ways of working and the novel outcome measures for HIT in- quires additional analysis.
skills of their personnel. Less opti- vestment, including quality of care, Finally, the mismanagement of IT
mistically, redistribution suggests readmission rates, complication rates, resources may well play a role in the
the dearth of productivity improve- and diagnostic accuracy. While these mixed results of HIT adoption. Con-
ments could be the result of new IT are well-established measures of ef- cerns expressed to us by healthcare
resources merely shifting productiv- fectiveness for health services, their providers regarding the usefulness of
ity gains (or losses) from some mar- appropriateness for evaluation of the HIT resources suggest the possibility
ket participants to others. That is, IT efficiency and effectiveness of HIT re- of missteps in the design, implementa-
may indeed create productivity gains mains to be seen. Interestingly, the tion, and/or ongoing use of these sys-
for some players, but such gains are concept of “meaningful use” that has tems. These concerns lead to negative
counterbalanced by losses for other driven adoption of much HIT since the perceptions of HIT that likely result in
individuals or organizations. Finally, passage of the HITECH Act focuses al- misuse and jeopardize overall perfor-
mance. Yet such concerns from multi- Efforts by the healthcare commu-
ple stakeholders are hardly captured in nity. Resolution of the apparent HIT
HIT development, and IT staff is inex- productivity paradox will require more
perienced in helping and adjusting the than the isolated efforts of healthcare
new systems to local needs. In our in-
terviews, several healthcare providers We found no providers, calling for a community ef-
fort. To this end, we suggest a stronger
expressed their struggles in managing
new systems due to their limited time
evidence of a leadership role for HIE-facilitating en-
tities, including regional health infor-
and personal technology anxiety. relationship mation organizations (RHIOs). As the
While each of the proposed mech-
anisms for paradoxical outcomes has
between HIT use ONC acknowledges, RHIOs are central
to data exchange across healthcare
some applicability in the healthcare and such critical institutions.30 Given the challenges in
context, the rich vein of research that
grew out of the productivity paradox
health outcomes the healthcare industry, we propose
that RHIOs should be more than mere
also offers some critical caveats for as improved data clearinghouses but formalized
assessing the practical effect of IT
investment and use.12,13,28 First, sig- interpersonal institutions that significantly improve
HIT use, especially in two major roles:
nificant variation exists across firms care, customer Encourage learning and adaptation
and industries with respect to the ef-
fect of IT investment on organization satisfaction, mechanisms in HIT practices. As with
many enterprise IT systems, HIT plat-
performance.9 Second, this varia-
tion and the existence of temporal
customer loyalty, forms are frequently complex and rig-
id, requiring significant resources and
lags are tied to the fact that perfor- patient mortality, enterprise-level effort to implement
mance gains are often associated not
merely with the adoption of new IT
and reduced ER effectively. For such complex projects
to yield tangible results, it takes time
resources but with the concomitant waiting times. for users to adapt to new routines and
redesign of business processes and practices, patients to get accustomed
investment in complementary assets to new processes and functionality,
and skills.6,28 Finally, the healthcare and in-house IT staff to discern what
literature reveals that measures of system modifications would make
productivity or business value remain the new system better fit with local
ambiguous and highly contingent on needs. RHIOs can serve as a platform
firm or industry conditions. Apply- through which different parties can
ing these lessons in the context of share resources, help others learn,
HIT, the evidence points to the need and contribute back to the broader
for more research to understand the community. In addition to creating a
complex nature of the healthcare in- mechanism for the development and
dustry and its business processes, exchange of a shared knowledgebase,
along with interdependence among these organizations represent a bridge
healthcare stakeholders in HIT devel- between different types of hospitals:
opment, adoption, and use. large/small, public/private, urban/ru-
ral. Managers can consider practices
Beyond the Paradox proposed in RHIO-based discourses to
Based on our analyses of the effects foster learning and adaptation in HIT
of clinical HIT adoption, we find that adoption (such as using collaborative
a number of viable mechanisms are teams to explore HIT functionalities,
available for achieving enhanced rewards to enforce positive behaviors,
health outcomes as a result of expand- and centers of excellence around HIT
ed HIT use, moving from meaningful best practices).
use to meaningful results. The U.S. Put users at the center of the HIT ex-
healthcare sector is an interdepen- perience. Commonly found in our in-
dent system. Leveraging and extend- terviews and in the HIT literature is
ing past insights from research on the the concern that HIT policies have
productivity paradox and IT business pushed healthcare providers toward a
value in general, we find it would ben- techno-centric perspective in which
efit from a collective approach that HIT is pursued “for IT’s sake” and HIT
brings together such diverse entities systems are designed without sub-
as hospitals, insurance companies, stantive input from prospective us-
regulators, and HIT vendors to seek ers.10 It is critical not to lose sight of the
systemic improvements. most important HIT stakeholders—
O C TO B E R 2 0 1 8 | VO L. 6 1 | N O. 1 0 | C OM M U N IC AT ION S OF T HE ACM 83
contributed articles
fective, enhanced, or idiosyncratic attention since 2007. Yet concrete Managed Care 17, 12 (Dec. 2011), 117–124.
17. Jones, S.S., Heaton, P.S., Rudin, R.S., and Schneider,
use of IT resources.2,8 Insight from and credible evidence that HIT im- E.C. Unraveling the IT Productivity Paradox: Lessons
the research could inform the afore- proves health outcomes remains in- for Health Care. The New England Journal of Medicine
366, 24 (June 14, 2012), 2243–2245.
mentioned efforts among healthcare conclusive. Our investigation of New 18. Jones, S.S., Rudin, R.S., Perry, T., and Shekelle, P.G.
system participants to identify and York State healthcare providers fur- Health information technology: An updated systematic
review with a focus on meaningful use. Annals of
disseminate best practices and foster ther indicates the healthcare industry Internal Medicine 160, 1 (Jan. 2014), 48–54.
more productive use patterns. may be experiencing an ongoing HIT 19. Lapointe, L. The IT productivity paradox in health: A
stakeholder’s perspective. International Journal of
Efforts by policymakers. Policymak- productivity paradox, mirroring earli- Medical Informatics 80, 2 (Feb. 2011), 102–115.
ers play a significant role in each of er patterns in manufacturing and oth- 20. Leidner, D.E., Preston, D., and Chen, D. An
examination of the antecedents and consequences
the measures we have proposed, as in er industrial sectors. While potential of organizational IT innovation in hospitals. Journal
of Strategic Information Systems 19, 3 (Sept. 2010),
community building through RHIOs HIT contribution to health outcomes 154–170.
and advancing outcome-oriented mea- remains an open question, we suggest 21. Menon, N.M., Yaylacicegi, U., and Cezar, A. Differential
effects of the two types of information systems:
sures of HIT use. While they should a collective approach is needed to ad- A hospital-based study. Journal of Management
work with academic researchers and dress the many issues raised by the Information Systems 26, 1 (July 2009), 297–316.
22. New York eHealth Collaborative. State HIE
the industry to identify more relevant HIT productivity paradox and hope Cooperative Agreement Program Strategic Plan. New
metrics for healthcare providers, it our research invites further inquiry York, 2009; https://www.healthit.gov/topic/onc-hitech-
programs/state-health-information-exchange
is equally important they maintain a into this important issue. 23. Panko, R.R. Is office productivity stagnant? MIS
holistic view of the healthcare value Quarterly 15, 2 (June 1991), 191–203.
24. Payne, P.R.O., Lussier, Y., Foraker, R.E., and Embi, P.J.
chain. Instead of focusing on policies References
Rethinking the role and impact of health information
1. Adler-Milstein, J., Bates, D.W., and Jha, A.K. A survey
that incentivize only EHR adoption of health information exchange organizations in the
technology: Informatics as an interventional
discipline. BMC Medical Informatics and Decision
or HIE participation, policymakers United States: Implications for meaningful use. Annals
Making 16, 40 (Mar. 29, 2016), 1–7.
of Internal Medicine 154, 10 (May 2011), 666–671.
should also consider how to promote 25. Rosenbaum, L. Transitional chaos or enduring harm?
2. Bagayogo, F.F., Lapointe, L., and Bassellier, G.
The EHR and the disruption of medicine. The New
experimentation both within and Enhanced use of IT: A new perspective on post-
England Journal of Medicine 373, 17 (Oct. 22, 2015),
adoption. Journal of the Association for Information
across geographic boundaries. This 1585–1588.
Systems 15, 7 (July 2014). 361–387.
26. Rudin, R.S., Motala, A., Goldzweig, C.L., and Shekelle,
might include more flexible use-style 3. Blumenthal, D. and Tavenner, M. The ‘meaningful use’
P.G. Usage and effect of health information exchange:
regulation for electronic health records. The New
A systematic review. Annals of Internal Medicine 161,
incentive programs that reward not England Journal of Medicine 363, 6 (Aug. 5, 2010),
11 (Dec. 2014), 803–812.
501–504.
only hospital-by-hospital efforts but 4. Boyatzis, R.E. Transforming Qualitative Information:
27. Rupasingha, A., Goetz, S.J., and Freshwater, D.
The production of social capital in U.S. counties.
also cross-hospital, cross-state, and Thematic Analysis and Code Development. Sage
The Journal of Socio-Economics 35, 1
Publications, Thousand Oaks, CA, 1998.
cross-boundary initiatives. It is dif- 5. Brynjolfsson, E. The productivity paradox of
(Feb. 2006), 83–101.
28. Schryen, G. Revisiting IS business value research:
ficult today to promote technologies information technology. Commun. ACM 36, 12 (Dec.
What we already know, what we still need to know,
1993), 66–77.
that provide value across geographi- 6. Brynjolfsson, E. and Hitt, L.M. Beyond the productivity
and how we can get there. European Journal of
Information Systems 22, 2 (Mar. 2013), 139–169.
cal locations (such as telemedicine) or paradox. Commun. ACM 41, 8 (Aug. 1998), 49–55.
29. Sharma, L., Chandrasekaran, A., Boyer, K.K., and
7. Buntin, M.B., Burke, M.F., Hoaglin, M.C., and
across institutional boundaries (such Blumenthal, D. The benefits of health information
McDermott, C.M. The impact of health information
technology bundles on hospital performance:
as healthcare supply-chain systems). technology: A review of the recent literature shows
An econometric study. Journal of Operations
predominantly positive results. Health Affairs 30, 3
In order to promote innovation and (2011), 464–471.
Management 41 (Jan. 2016), 25–41.
30. Vest, J.R. and Gamm, L.D. Health information
collaboration, policymakers might 8. Burton-Jones, A. and Grange, C. From use to effective
exchange: Persistent challenges and new strategies.
use: A representation theory perspective. Information
thus want to consider measures that Journal of the American Medical Informatics
Systems Research 24, 3 (Mar. 2012), 632–658.
Association 17, 3 (May 2010), 288–294.
target multiple parties in a health- 9. Chari, M.D., Devaraj, S., and David, P. The impact of
information technology investments and diversification
care value chain rather than a limited strategies on firm performance. Management Science
Quang “Neo” Bui (qnbui@saunders.rit.edu) is an assistant
number of dominant players. This 54, 1 (Jan. 2008), 224–234.
professor of management information systems in the
10. Cho, K.W., Bae, S.-K., Ryu, J.-H., Kim, K.N., An, C.-H.,
Saunders College of Business of the Rochester Institute
would include support for public-pri- and Chae, Y.M. Performance evaluation of public
of Technology, Rochester, NY, USA.
hospital information systems by the information
vate partnerships that bring together system success model. Healthcare Informatics Sean Hansen (shansen@saunders.rit.edu) is an associate
healthcare providers, payer organiza- Research 21, 1 (Jan. 2015), 43–48. professor of management information systems in the
11. David, P.A. The dynamo and the computer: An Saunders College of Business of the Rochester Institute
tions, and HIT providers or initiatives historical perspective on the modern productivity of Technology, Rochester, NY, USA.
that include large-scale participation paradox. The American Economic Review 80, 2 (May
1990), 355–361. Manlu Liu (manluliu@saunders.rit.edu) is an associate
groups (such as the Precision Medi- 12. Dedrick, J., Gurbaxani, V., and Kraemer, K.L. professor of management information systems and
cine Initiative). Such efforts could le- Information technology and economic performance: accounting in the Saunders College of Business of the
A critical review of the empirical evidence. ACM Rochester Institute of Technology, Rochester, NY, USA.
verage emergent technologies (such Computing Surveys 35, 1 (Mar. 2003), 1–28.
Qiang (John) Tu (jtu@saunders.rit.edu) is a professor
as big data analytics platforms, mo- 13. Devaraj, S. and Kohli, R. Information technology
of management information systems and the Senior
payoff in the healthcare industry: A longitudinal study.
bile health apps, and social media) to Journal of Management Information Systems 16, 4
Associate Dean in the Saunders College of Business of the
Rochester Institute of Technology, Rochester, NY, USA.
quickly assess the efficacy of a diverse (Apr. 2000), 41–67.
14. Government Accountability Office. Electronic Health
set of HIT projects and channel re- Record Programs: Participation Has Increased,
sources toward the ones that show the but Action Is Needed to Achieve Goals, Including
Improved Quality of Care. Washington, D.C., 2014;
greatest promise for bridging the gap https://www.gao.gov/assets/670/661399.pdf
between HIT use and health outcomes 15. Harrison, M.I., Koppel, R., and Bar-Lev, S. Unintended
consequences of information technologies in health
across populations. care: An interactive sociotechnical analysis. Journal
of the American Medical Informatics Association 14, 5
(Sept. 2007), 542–549.
Conclusion 16. Jha, A.K., Burke, M.F., DesRoches, C., Joshi, M.S.,
Kralovec, P.D., Campbell, E.G., and Buntin, M.B.
IT use in the healthcare industry has Progress toward meaningful use: Hospitals’ adoption
experienced tremendous growth and of electronic health records. American Journal of © 2018 ACM 0001-0782/18/10 $15.00
O C TO B E R 2 0 1 8 | VO L. 6 1 | N O. 1 0 | C OM M U N IC AT ION S OF T HE ACM 85
review articles
DOI:10.1145/ 3183582
or the amount of pollution an ecosys-
The future of computing research tem can bear, limits are less obvious in
computing. Many believe the only limit
relies on addressing an array worth considering is human ingenu-
of limitations on a planetary scale. ity, and that we can surpass any and all
other limits if we, as a global communi-
BY BONNIE NARDI, BILL TOMLINSON, ty, pool our creative resources. But we
DONALD J. PATTERSON, JAY CHEN, DANIEL PARGMAN, collectively face new global conditions
BARATH RAGHAVAN, AND BIRGIT PENZENSTADLER that warrant our attention.
In this article we explore the relation-
Computing
ship between these potential futures
and computing research. What hidden
assumptions about the future are em-
bedded in most computing research?
within Limits
What possible or even probable futures
are we ignoring? What work should we
be doing to respond to fundamental
planetary limits, and to the ecological
and energy constraints that global soci-
ety faces over the coming years and de-
cades? Confronting such limits is likely
to present challenges that we—human-
ity—have never before faced.
Given that computing underlies vir-
tually all the infrastructure of global so-
COMPU TING R ESE ARC H E RS AND practitioners are often ciety—in commerce, communication,
transportation, agriculture, manufac-
seen as inventing the future. As such, we are implicitly turing, education, science, healthcare,
also in the business of predicting the future. We plot and governance—computing has an
trajectories for the future in the problems we select, enormous role to play in responding to
global limits and in shaping a society
the assumptions we make about technology and that meaningfully adapts to them. We
societal trends, and the ways we evaluate research. contend that the root of much of com-
puting research has been driven pre-
However, a great deal of computing research focuses dominantly by growth-oriented visions
on one particular type of future, one very much like
the present, only more so. This vision of the future key insights
assumes that current trajectories of ever-increasing ˽˽ Most computing work is premised on
industrial civilization’s default worldview
production and consumption will continue. This focus in which ongoing economic growth is
both achievable and desirable.
is perhaps not surprising, since computing machinery ˽˽ This growth-focused worldview, however,
as we know it has existed for only 80 years, in a period is at odds with findings from many other
scientific fields, which see growth as
of remarkable industrial and technological expansion. deeply problematic for ecological and
social reasons.
But humanity is rapidly approaching, or has already
˽˽ We proposed that the computing field
exceeded, a variety of planet-scale limits related to the transition toward “computing within
limits,” exploring ways that new forms
global climate system, fossil fuels, raw materials, and of computing supported well-being while
biocapacity.28,32,38 enabling human civilizations to live within
global ecological and material limits.
It is understandable that in computing we would not ˽˽ Computing underlies virtually all the
focus on limits. While planetary limits are obvious in infrastructure of global society, and will
therefore be critical in shaping a society
areas such as extractive capacity in mining or fishing, that meaningfully adapts to global limits.
O C TO B E R 2 0 1 8 | VO L. 6 1 | N O. 1 0 | C OM M U N IC AT ION S OF T HE ACM 87
review articles
Background
to global limits and tain directions. If we use ‘growth’ to
mean quantitative change, and ‘devel-
Since the beginning of computing, all in shaping a society opment’ to refer to qualitative change,
research and development has taken
place against a backdrop of exponen-
that meaningfully then we may say that a steady-state
economy develops but does not grow,
tial growth of, for example, transistors adapts to them. just as the planet Earth, of which the
per integrated circuit (Moore’s Law), human economy is a subsystem, devel-
disk storage density (Kryder’s Law), ops but does not grow.” Daly suggests
bandwidth capacity (Nielsen’s Law), that a single-minded focus on grow-
and fiber-optic capacity (Keck’s Law). ing the economy comes at the eventual
These developments have led to the cost of decreasing human well-being
establishment of a “cornucopian para- and quality of life. Such growth results
digm”23 where the design of new ser- in, for example, charging for things
vices stimulates demand, which drives that used to be free, the health conse-
growth of increased infrastructure quences of polluting the environment,
capacity, which then cycles back to en- and decreasing long-term possibilities
able the design of new services in a self- to produce food or earn a livelihood.
perpetuating cycle. The idea that expo- Looking at societal trends through
nential growth of computing capacity the lens of human history, archae-
and an ever-expanding infrastructure ologist Joseph Tainter’s book The
for computing will continue into the Collapse of Complex Societies argues
future is usually taken for granted. We that civilizations eventually collapse,
draw from research in ecological eco- declining over a period of decades or
nomics and the historical record in ar- centuries.33 Analyzing extensive his-
cheology to question this assumption. torical and archaeological materials,
This research suggests that other Tainter presented collapse as a pro-
futures are not just possible but prob- cess that arises from increasing so-
able. While most economists sidestep cietal complexity, which, over time,
questions of finite resources,6 econo- creates burdens for systems that they
mists in the subfield of ecological eventually cannot sustain.
economics have grappled with these Decline will result in less mate-
questions for decades. How can we rial abundance as we push the limits
maintain or increase well-being while of the Earth’s resources necessary for
staying within ecological limits? How economic activity. But it is not neces-
can we promote well-being and not ex- sary for our society to end in abject
ceed the assimilative and regenerative collapse. The societies that Tainter
capacities of the Earth’s biochemical studied—the Maya, the Mesopota-
life-support systems? We have already mians, the Minoans, the Inca, the Ro-
exceeded many such limits through, for mans, the Egyptians, and others—did
example, overfishing, deforestation, not possess the resources of science,
soil depletion, falling water tables, ris- history, and technology that we have
ing temperatures, and emitting CO2 amassed in the last 500 years. These
and other greenhouse gases at rates resources have the potential to be use-
that dangerously increase their con- fully deployed to fashion a transition
centrations in the atmosphere.28,32,38 from the current, unsustainable sys-
Ecological economist Herman Daly tem to a new system based on today’s
has proposed that we abandon the realities. We optimistically assume
idea of striving for economic growth that with advances in science and prog-
in favor of a steady-state economy (in ress in philosophies of human rights,
line with classical economist Adam we have a good chance of transforma-
Smith’s idea that the economy would tive change to a system more like the
steady-state economy Herman Daly of scarcity.” This work helped lay the constitutes an important subfield of
envisions. The implication of the work groundwork, along with papers from human-computer interaction.19 There
in ecological economics and archaeol- other subfields of computing24,37 for are some key differences between cri-
ogy is that we should endeavor to build LIMITS research. sis informatics and LIMITS, although
computer systems that aim at increas- LIMITS has drawn heavily from col- we think that in the future the two may
ing well-being and quality of life while lapse informatics but shifts emphasis increasingly mutually inform one an-
contributing to staying within ecologi- to planetary limits rather than societal other. At present, crisis informatics
cal limits. Foregrounding human well- decline. LIMITS focuses on exposing research generally assumes an external
being is supported by the ACM Code of basic processes of resource use and entity that enacts a rescue when a disas-
Ethics and Professional Conduct, the waste management in complex human ter, such as a flood or earthquake, oc-
first imperative of which states: “As an systems. The metrics used to assess sus- curs. Events are conceived as localized,
ACM member I will contribute to so- tainability must shift correspondingly. describing a space into which the sur-
ciety and human well-being.” (https:// As examples, Pargman and Raghavan’s rounding society can pour resources
www.acm.org/aboutacm/acm-code-of- “Rethinking Sustainability in Comput- to alleviate the resulting disorder and
ethics-and-professional-conduct) ing: From Buzzword to Non-negotiable disruption. These scenarios accurately
We turn now to a review of comput- Limits”20 and Raghavan and Pargman’s describe an important subset of possi-
ing literature that has been founda- “Means and Ends in Human-Comput- ble issues confronting human civiliza-
tional for the development of comput- er Interaction: Sustainability through tions. LIMITS, however, assumes long
ing within LIMITS perspectives. Disintermediation,”25 offer major con- time frames and a global spatial scale.
tributions, arguing that “sustainabil- There is no external entity to provide
SCHI: Sustainable ity” must be grounded in rigorous met- relief. LIMITS emphasizes phenomena
Human-Computer Interaction rics arising from planetary limits, and such as climate change, soil erosion,
The Sustainable Human-Computer that the complexity of societal systems water pollution, civic instability, mass
Interaction community is about a might be reduced, easing resource use migration, reduced infrastructure, and
decade old, and a number of LIMITS and waste production. The forthcom- an economy that requires continuous
researchers have roots in this area. Eli ing edited collection Digital Technology growth.4,5,14,20,21,24,30,36
Blevis’s “Sustainable Interaction De- and Sustainability: Engaging the Para- Potentially there is a strong link be-
sign”3 is a primary source, offering a ru- dox10 incorporates influences from tween LIMITS and crisis informatics.
bric to identify how interaction designs LIMITS research. Several of the papers Some crisis informatics researchers
lead to material effects, as well as sev- mentioned here as well as Preist et al.23 are beginning to examine long-term
eral principles for engaging in sustain- have won best paper awards, signaling processes underlying crises, suggest-
able interaction design. Early papers interest in the issues. ing that when looked at more broadly,
that sparked interest among LIMITS “crises” are often more than acute
researchers were Jeff Wong’s “Prepare Crisis Informatics events of short duration, with roots in
for Descent: Interaction Design in Our We are often asked if computing within underlying processes that may have
New Future”40 and Silberman and Tom- LIMITS is the same as crisis informat- been developing over decades.1 This
linson’s “Precarious Infrastructure and ics. Crisis informatics is concerned understanding provides a bridge for
Postapocalyptic Computing.”31 Several with technology-based studies of di- future development and crossfertiliza-
high-profile CHI papers drew attention saster planning and response, and tion between the two subdisciplines.
to the challenges of sustainability and
the shortcomings of SHCI work in fail-
ing to address questions of physical,
material, and energy limits. DiSalvo et
al.’s “Mapping the Landscape of Sus-
tainable HCI”8 sought to provide struc-
ture to the array of papers in SHCI,
and identified gaps in the areas being
studied, such as the need to focus on
collectives and broader contexts, not
just individuals, the importance of en-
gaging with policy issues, and stronger
connections to sustainability work in
fields outside of computing.
From this context, Tomlinson et
al.’s “Collapse Informatics”35 was the
PHOTO BY VALENTIN VALKOV
O C TO B E R 2 0 1 8 | VO L. 6 1 | N O. 1 0 | C OM M U N IC AT ION S OF T HE ACM 89
review articles
Computing Within
Limits Workshops
LIMITS ideas have been developed
through three workshops (2015–2017)
convened by the LIMITS community
(the latter two in cooperation with
ACM). The first two were held at the
University of California, Irvine, and
the third at Westmont College in Santa
Barbara, with funding from the two
universities as well as from Facebook
and Google. Participants came from
institutions in Abu Dhabi, Canada,
Hong Kong, Pakistan, Spain, Sweden,
Switzerland, the U.K., and the U.S.,
consistent with the global nature of
LIMITS concerns and research. The
ICTD: Information and equal distribution of wealth and the 2018 workshop was held in Toronto,
Communication Technology consequent problem of poverty in the co-located with the Fifth International
for Development Global South, the Global North must Conference on Information and Com-
ICTD is a relatively young field that shrink its resource footprint enough munication Technology for Sustain-
has explored the potential of comput- that countries in the Global South are ability (ICT4S). Sparked by discussions
ing for improving the socioeconomic afforded some space for necessary eco- at the workshops, LIMITS participants
situation of the poor. While comput- nomic growth. However, everyone— have co-authored several papers pub-
ing within LIMITS typically focuses North and South—must operate within lished in mainstream conferences and
on the future, Tomlinson et al.35 note some absolute global limits. The ethi- a research grant. The LIMITS work-
that our imagined “future” LIMITS cal argument for improving the quality shop papers are available at comput-
scenarios may already exist today in of life of the poor is easy to make, but ingwithinlimits.org
the conditions in which poor commu- reducing the Global North’s consump-
nities live around the world. However, tive (and exploitative) practices to af- Three Key Principles
few studies within the ICTD literature ford the Global South opportunities to We propose three principles that can
consider global ecological, material, grow, especially in the face of mount- help frame computing research and
and energy limits. Most research is ing resource and climate pressures, practice in a way that is consistent with
situated in resource-constrained con- remains an enormous challenge, and the ideas described in this paper and
texts and assumes the constraints will one computing should be cognizant of. the literature we have surveyed.
be relaxed in the future after sufficient Despite differing perspectives, LIM- Question growth. The industrial-
economic growth has occurred.12,15 ITS and ICTD have much in common ized world’s current economic system,
The only paper so far that explicitly and potential for integration and col- capitalism, is predicated on growth.
makes the link between LIMITS and laboration.4 For example, LIMITS work Economic growth has brought more
ICTD in an ICTD venue is Tomlinson has studied the use of digital technol- than an order of magnitude rise in per
et al.’s DEV paper, “Toward alternative ogy to design habitations in refugee capita income from $3 a day in 1800 to
decentralized infrastructures.”36 The camps,29 problems of networking in $100 in the early 2000s for most of Eu-
vacuum regarding the implications of rural populations in Zambia and Gua- rope and North America.16 However,
phenomena such as climate change in temala30 and infrastructure in condi- despite such unprecedented prosper-
the ICTD literature could be filled by a tions of scarcity in Haiti.21 While these ity, global income inequality is increas-
LIMITS perspective. are classic ICTD topics, the authors in ing. Wealth is accumulating in the
There is, however, a tension be- each case considered ecological, mate- hands of fewer and fewer astoundingly
tween economic development in poor rial, and energy limits in their analyses, rich persons.22 Poverty is widespread.
countries—the focus of ICTD—and unlike typical ICTD studies. The papers Such social dysfunction, along with
sustainability. As Herman Daly points engage models of scarcity, examining the burdens on ecosystems produced
out, the total resource footprint of the the cases as possible future global LIM- by economic activity,28,32,38 suggest we
PHOTO BY GENCHO PETKOV
Global North and the Global South ITS scenarios. Drought, flooding, envi- must rethink the growth paradigm.
combined together must stay within ronmental disasters, infrastructure The ubiquity and power of computing
the boundaries of a global steady state disruption, mass migration, and per- make it well positioned to act as an
economy that is sustainable in the long manent settlement in refugee camps agent of change to influence proposals
run. To ameliorate the problem of un- in low-resource environments are seen for transformative economic systems
and methods of governance. While dis- become more numerous and more
cussion of specific proposals is beyond powerful as outcomes of global envi-
the scope of this article, we point to the ronmental changes. Our track record
work of, for example, Daniel O’Neill,18 of being prepared for dealing with un-
Peter Frase,9 and Tim Jackson13 as
thoughtful responses to current prob- We encourage predictable catastrophic events is not
encouraging. We would benefit from
lems that might inform the ways we
practice computing.
those working seriously considering LIMITS-related
scenarios rather than blithely denying
Daly’s notion of promoting devel- in computing to their possibility or treating their fore-
opment rather than economic growth
suggests a sound mechanism for mov-
build systems and shocks as isolated incidents. Engaging
with these difficult scenarios before
ing civilization forward, deploying our envision worlds that they occur, rather than only in their af-
creativity and capacity for innovation
in LIMITS-compliant ways. An econo-
are neither reliant termath, will help us evaluate our level
of preparedness and perhaps prevent
my that demands endless growth en- on nor contributing certain undesirable future scenarios
tails a cycle of consequences that must
be interrupted if we are to address mas- to runaway growth. from happening.21
To speak of LIMITS-scenarios only
sive problems such as climate change in the future tense, however, is mis-
and resource depletion.20 Exploring leading. These events are here now, as
relations between computing and the several climate-related catastrophes in
economy will be an important direc- the U.S. and Europe have shown, even
tion for future development of the during the writing of this article. Sci-
computing community and a consider- ence fiction author William Gibson
able challenge. famously said, “The future is already
Currently, the implicit organizing here—it’s just not evenly distributed.”
framework for a great deal of comput- We see this future currently on display
ing work puts a focus on increasing the in places such as Flint, Michigan where
proximate financial value of compa- toxic wastes have poisoned the water
nies. Even when particular products, supply. It is thus possible to frame LIM-
from a narrow perspective, are seeking ITS scenarios (including, for example,
to make people’s lives better through heat waves, drought, rising sea levels,
new technology, these products are and floods) not in terms of random ir-
typically embedded in a rapid churn regularities or threats that might afflict
of objects and services that foster run- us in the future, but in terms of an in-
away consumption.23,27 By shifting the creasing incidence of phenomena aris-
explicit focus, first and foremost, to ing from intensive economic activity.
the pursuit of long-term well-being, A concrete research strategy is to de-
we may finally escape the growth para- velop case studies of current changes
digm and build systems that more ef- that may model futures of relative scar-
fectively lead to sustainable improve- city. For example, a study of the con-
ments in the quality of life for humans tinuing impact of the 2010 earthquake
and other species. in Haiti found that the regrowth of in-
To make this principle actionable, frastructures was occurring in a more
we encourage researchers and practi- distributed fashion than would be typi-
tioners to consider whether their work cal for countries with more resources.21
is a) reliant on growth, b) seeking to Distribution networks for clean water,
make growth happen, c) contributing electricity, Internet, and gasoline were
to growth. We encourage those work- severely damaged in the earthquake.
ing in computing to build systems Corporate and government responses
and envision worlds that are neither were hampered by political and finan-
reliant on nor contributing to runaway cial obstacles. In many cases, survivors
growth. A number of existing LIMITS themselves began to rebuild the infra-
relevant papers have addressed this structures in a bottom up manner. For
principle.24,31,35 example, large private water tanks were
Consider models of scarcity. Clever installed on local properties. Wealthier
technological fixes may help us de- residences allowed adjacent poorer
fer catastrophes for some time, but households to tap into power lines via
not indefinitely, and especially not if jerry-rigged extension cords without
events such as wildfires, hundred-year paying for the service—a generous if
storms, and Category 5 hurricanes somewhat precarious arrangement.
O C TO B E R 2 0 1 8 | VO L. 6 1 | N O. 1 0 | C OM M U N IC AT ION S OF T HE ACM 91
review articles
O C TO B E R 2 0 1 8 | VO L. 6 1 | N O. 1 0 | C OM M U N IC AT ION S OF T HE ACM 93
research highlights
P. 95 P. 96
Technical Fundamental Concepts
Perspective
A Control Theorist’s of Reactive Control for
View on Reactive Autonomous Drones
Control for By Luca Mottola and Kamin Whitehouse
Autonomous Drones
By John Baillieul
P. 105 P. 106
Technical Enabling Highly Scalable
Perspective Remote Memory Access
The Future of MPI
By Marc Snir
Programming with
MPI-3 One Sided
By Robert Gerstenberger, Maciej Besta, and Torsten Hoefler
Technical Perspective
To view the accompanying paper,
visit doi.acm.org/10.1145/3264417 rh
O C TO B E R 2 0 1 8 | VO L. 6 1 | N O. 1 0 | C OM M U N IC AT ION S OF T HE ACM 95
research highlights
DOI:10.1145/ 32 6 441 7
Fundamental Concepts
of Reactive Control for
Autonomous Drones
By Luca Mottola and Kamin Whitehouse
Time
O C TO B E R 2 0 1 8 | VO L. 6 1 | N O. 1 0 | C OM M U N IC AT ION S OF T HE ACM 97
research highlights
for example, in human tracking applications for function- aspects by employing a form of auto-tuning of the
ality such as fall detection.15 conditions leading to running the control logic.
2) An indication for running the control logic may
2.2. Intuition originate from different sensors, at different rates,
Through our continuous work with drones as mobile com- and asynchronously with respect to each other.
puting platforms,16, 19 we eventually noticed that the auto- A problem is thus how to handle the possible inter-
pilots’ PID controllers are mostly tuned so that it is the leavings. Moreover, not running the control loop for
Proportional component to dictate the actual controller too long may negatively affect the drone’s stability,
operation. The Derivative component can be kept to a mini- possibly preventing to reclaim the correct behavior.
mum though a careful distribution of weights,6, 11 whereas We tackle these issues by only changing the execu-
precise sensor calibration may spare the Integral compo- tion of the control logic over time, rather than the
nent almost completely.6, 11, 22 logic itself.
As a result of this observation, we concluded that a sim- 3) Reactive control must run on resource-constrained
ple relation exists between current inputs from the navi- embedded hardware. When implementing reactive
gation sensors and the corresponding actuator settings. control, however, the code quickly turns into a “call-
With little impact from the time-dependent Derivative and back hell”10 as the operation becomes inherently
Integral components, and with the Proportional compo- event-driven. We experimentally find that, using
nent dominating, small variations in the current sensor standard languages and compilers, this negatively
inputs likely correspond to small variations in the actuator affects the execution speed, thus limiting the gains.7
settings. As an extreme case, as long as the sensor inputs We design and implement a custom realization of
do not change, the actuator settings should remain almost Reactive Programming (RP) techniques3 to tackle this
unaltered. In such a case, at least in principle, one may not problem.
run the control logic and simply retain the previous actua-
tor settings. The context where we are to address these issues shapes
Reactive control builds upon this intuition. We constantly the challenge in unseen ways. For example, aerial drone
monitor the navigation sensors to understand when the con- demonstrations exist showing motion control in tasks such
trol logic does need to run as a function of the instantaneous as throwing and catching balls,21 flying in formation,23 and
environment conditions. These manifest as changes in the carrying large payloads.14 In these settings, the low-level con-
inputs of navigation sensors. If these are sufficiently signifi- trol does not operate aboard the drone. At 100Hz or more, a
cant to warrant a change in the physical drone behavior to be powerful computer receives accurate localization data from
compensated, reactive control executes the control logic to high-end motion capture systems, runs sophisticated con-
compute new actuator settings. Otherwise, reactive control trol algorithms based on drone-specific mechanical models
retains the existing configuration. expressed through differential equations, and sends actua-
As we explain next, reactive control abstracts the problem tor commands to the drones. Differently, we aim at improv-
of recognizing such significant changes in a way that makes ing the performance of mainstream low-level control on
it computationally tractable with little processing resources. embedded hardware, targeting mobile sensing applications
Moreover, because of the aforementioned characteristics of that operate in the wild.
sensor hardware on autopilot boards, monitoring the sensor On the surface, reactive control may also resemble the
readings at the maximum possible rate usually bears very notion of event-based control.1 Here, however, the control
little energy overhead. Reactive control, nonetheless, makes logic is often expressly redesigned for settings different
it possible to rely on the low-power interrupt-driven modes than ours; for example, in distributed control systems to
if available. cope with limited communication bandwidth or unpredict-
As a result, when sensor inputs change often, reactive con- able latency. This requires a different theoretical frame-
trol makes control run repeatedly, possibly at rates higher work.1 In contrast, we aim at re-using existing control logic,
than the static settings of a time-triggered implementation. whose properties are well understood, and at doing so with
When sensor inputs exhibit small or no variations, the rate of little or no knowledge of its corresponding implementation
control execution reduces, freeing up processing resources and its parameter tuning. Different than event-based con-
that may be needed at different times. trol, in addition, reactive control is mainly applicable only
to PID-like controllers where the Proportional component
2.3. Challenge dominates.
Realizing reactive control is, however, non-trivial. Three
issues are to be solved, as we illustrate in Section 3: 3. REACTIVE CONTROL
The key issues we discussed require dedicated solutions, as
1) What is a “significant” change in the sensor input we explain here.
depends on several factors, including the accuracy of
sensor hardware, the physical characteristics of the 3.1. Conditions for reacting
drone, the control logic, and the granularity of actua- Problem. It may seem intuitive that the more “significant” is
tor output. We opt for a probabilistic approach to a change in a sensor reading, the more likely is the necessity
tackle this problem, which abstracts from all these to run the control loop. Such a condition would indicate that
O C TO B E R 2 0 1 8 | VO L. 6 1 | N O. 1 0 | C OM M U N IC AT ION S OF T HE ACM 99
research highlights
dependability. To address this issue, we run the control c requires an explicit assignment following the changes
loop anyways at very low frequency, typically in the range in a or b. It becomes an issue to determine where to place
of a few Hz. If such executions compute new actuator set- such an assignment without knowing when a or b might
tings, the drone most likely applies some significant cor- change.
rection to the flight operation that causes reactive control Using RP, one declaratively describes the data dependen-
to be triggered immediately after. If logistic regression cies between variables a, b, and c. As variables a and b
originally indicated that the current changes in sensor change, the value of c is constantly kept up-to-date. Then,
readings did not demand to run the control logic, the cur- variable c may be input to the computation of further state
rent iteration is considered a false negative and feed back variables. The data dependencies thus take the form of an
to the data set used for tuning the regression parameters. (acylic) graph, where the nodes represent individual values,
The next time the least square estimation executes, as and edges represent input/output relations.
explained above, these false negatives are also taken into The RP run-time support traverses the data depen-
account. dency graph every time a data change occurs, stopping
Note that the techniques hitherto described do not whenever a variable does not change its value as a result
require one to alter the control logic itself; they solely of changes in its inputs. Any further processing would be
drive its execution differently over time. The single itera- unnecessary because the other values in the graph would
tion remains essentially the same as in a traditional time- remain the same. This is precisely what we need to effi-
triggered implementation. This means reactive control ciently implement reactive control; however, RP is rarely
does not require to conceive a new control logic; the exist- employed in embedded computing because of resource
ing ones can be re-used provided an efficient implemen- constraints.
tation of such asynchronous processing is possible, as we RP-Embedded. We rely on a few key characteristics
discuss next. of reactive control to realize a highly efficient RP imple-
mentation. First, the data dependency graph encodes the
3.3. Implementation control logic; therefore, its layout is known at compile-
Problem. The control logic is implemented as multiple pro- time. Second, the sensors we wish to use as initial inputs
cessing steps arranged in a complex multi-branch pipeline. are only a handful. Finally, the highest frequency of data
Moreover, each such processing step may—in addition to changes is known; for each sensors, we are aware or can
producing an output immediately useful to take control deci- safely approximate the highest sampling frequency.
sions—update global state used at a different iteration else- Based on these, we design and implement RP-Embed-
where in the control pipeline. ded: a C++ library to support RP on embedded resource-con-
Using reactive control, depending on what sensor indi- strained hardware. RP-Embedded trades generality for
cates the need to execute the control loop, different slices efficiency, both in terms of memory consumption and pro-
of the code may need to run while other parts may not. The cessing speed, which are limited on our target platforms.
parts of the control pipeline that do not run at a given itera- We achieve this by relying heavily on statically-allocated
tion, however, may need to run later because of new updates compact data structures to encode the data dependency
to global state. Thus, any arbitrary processing step—not graph. These reduce memory occupation compared with
just those directly connected to the sensors’ inputs—might container classes of the STD library used in many existing
potentially need to execute upon recognizing a significant C++ RP implementations, and improve processing speed
change in given sensor inputs. by sparing pointer dereferences and indirection opera-
Employing standard programming techniques in these tion during the traversal. This comes at the cost of
circumstances quickly turns implementations into a “call- reduced flexibility: at run-time, the data dependency graph
back hell”.10 This fragments the program’s control flow can only change within strict bounds determined at
across numerous syntactically-independent fragments of compile-time.
code, hampering compile-time optimizations. We experi- In addition, RP-Embedded provides custom time
mentally found that this causes an overhead that limits the semantics to handle the issues described in Section 3.2.
benefits of reactive control.7 The traditional RP semantics would trigger a traversal of
Approach. We tackle this issue using RP.3 RP is increas- the data dependency graph for any change of the inputs.
ingly employed in applications where it is generally impos- With reactive control, however, the traversal caused by
sible to predict when interesting events arrive.3 It provides changes in a high-frequency sensor may be immedi-
abstractions to automatically manage data dependencies in ately superseded by the traversal caused by changes in
programs where updates to variables happen unpredictably. another sensor within the same hyperperiod. The out-
Consider for example: put that matters, however, is only the one produced by
the second traversal.
a = 2; To avoid unnecessary processing, RP-Embedded allows
b = 3; one to characterize the inputs to the data dependency
c = a + b; graph with their maximum rate of change. This informa-
tion is used to compute the system’s hyperperiod. Every
In sequential programming, variable c retains the value 5 time a value is updated in the data dependency graph,
regardless of any future update to variable a or b. Updating RP-Embedded waits for the completion of the current
safety, most GCS implementations instruct the drone given environment. This cannot be attributed to its struc-
to return to the launch point upon reaching this thresh- tural robustness; the Y6 is definitely the least “sturdy” of
old. In general, the lifetime of aerial drones is currently the three. We conjecture that the different control logic of
extremely limited. State of the art technology usually pro- the Y6 offers additional opportunities to reactive control.
vides at most half an hour of operation. This aspect is A similar reasoning applies to Cleanflight, as shown in
thus widely perceived as a major hampering factor. Figure 7(a). Being the youngest of the autopilot we test,
In the following, we describe an excerpt of the results it is fair to expect the control logic to be the least refined.
we collect based on 260+ hours of test flights performing Reactive control is still able to drastically improve the pitch
way-point navigation in the three environments.7 error, by a 32% (37%) factor with the quadcopter (hexacop-
Results. As an example, Figure 7(a) shows the average ter) in Arch.
improvements in pitch error; these are significant, rang- The improvements in attitude error translate into
ing from a 41% reduction with Cleanflight in Lab to a more accurate motion control and fewer attitude correc-
27% reduction with Ardupilot in Arch. We obtain similar tions. As a result, energy utilization improves. Figure 7(b)
results, sometimes better, for yaw and roll.7 Comparing shows the results we obtain in this respect. Reactive con-
this performance with earlier experiments, we confirm trol reaches up to a 24% improvement. This means fly-
that it is the ability to shift processing resources in time ing more than 27min instead of 22min with OpenPilot in
that enables more accurate control decisions.7 Not run- Arch. This figure is crucial for aerial drones; the improve-
ning the control loop unnecessarily frees resources, ments reactive control enables are thus extremely valu-
increasing their availability whenever there is actually the able. Most importantly, these improvements are higher
need to use them. In these circumstances, reactive control in the more demanding settings. Figure 7(b) shows
dynamically increases the rate of control, possibly beyond that the better resource utilization of reactive control
the pre-set rate. becomes more important as the environment is harsher.
Evidence of this is shown in Figure 8, showing an exam- Similarly, the quadcopter shows higher improvements
ple trace that indicates the average control rate at second than the hexacopter. The mechanical design of the lat-
scale using Ardupilot and the hexacopter. In Arch, reac- ter already makes it physically resilient. Differently, the
tive control results in rapid adaptations of the control rate quadcopter offers more ample margin to cope with the
in response to the environment influence, for example, environment influence in software.
wind gusts. On average, the control rate starts slightly
below the 400Hz used in time-triggered control and slowly 5. END-USER APPLICATIONS
increases. An anemometer we deploy in the middle of the The performance improvements of reactive control reflect
field confirms that the average wind speed is growing dur- in more efficient operation of end-user drone applications
ing this experiment. ranging from 3D reconstruction to search-and-rescue.18
In contrast, Figure 8 shows reactive control in Lab The latter is a paradigmatic example of active sensing
exhibiting more limited short-term adaptations. The aver- functionality, whereby data gathered by application-spe-
age control rate stays below the rate of time-triggered cific sensors guides the execution of the application logic,
control, with occasional bursts whenever corrections are which includes here the drone movements. We build a
needed to respond to environmental events, for example, prototype system to investigate the impact of reactive con-
when passing close to a ventilation duct. The trends in trol in this kind of applications.
Figure 8 demonstrate reactive control’s adaptation abili- System. Professional alpine skiers are used to carry
ties both in the short and long term. a device called Appareil de Recherche de Victimes en
Still in Figure 7(a), the improvements of reactive con- Avalanche (ARVA)20 during their excursions. ARVA is noth-
trol apply to the Y6 as well; in fact, these are highest in a ing but a 457KHz radio transmitter expressly designed
Quadcopter - Cleanflight
Quadcopter - Ardupilot Hexacopter - Cleanflight
Hexacopter - Ardupilot Quadcopter - OpenPilot
3DR Y6 - Ardupilot Hexacopter - OpenPilot
Flight time improvement (%)
Pitch error improvement (%)
50 25
40 20
30 15
20 10
10 5
0 0
Lab Rugby Arch Lab Rugby Arch
Test environment Test environment
(a) Pitch error improvement. (b) Flight time improvement.
900
Rate at second scale (Hz)
Technical Perspective
To view the accompanying paper,
visit doi.acm.org/10.1145/3264413 rh
THE MPI COMMUNITY recently celebrated munication, especially Put, is a very good
25 years since the start of the MPI stan- match to the capabilities of modern Net-
dardization effort. This early-1990s effort The following paper work Interface Controllers (NICs): They
was due to the emergence of commodity convincingly shows very often support remote direct mem-
clusters as a replacement to vector ma- ory access (rDMA) operations whereby
chines, in what was dubbed by Eugene that the potential local and remote NICs collaborate in
Brooks as “The attack of the killer mi- of MPI one-sided copying data from local memory to re-
cros.” Commodity clusters needed very mote memory with no software involve-
different software than vector systems, communication ment, aside from the call that initiates
and two efforts were started to satisfy this can be realized. the transfer at the source node. There-
need: The first effort, developed by High fore, one-sided communication has the
Performance Fortran Forum, was HPF— potential to significantly reduce the soft-
a data parallel extension to Fortran 90 ware overheads for communication.
that would provide portability across This is extremely important as the
vector, SIMD, and cluster systems. The One major extension to MPI has been next generation of networks and NICs
more modest second effort, developed the introduction of one-sided commu- will have the capability of handling
by the Message Passing Interface Forum, nication, first in MPI 2.0, and then, with tens or hundreds of millions of mes-
was MPI—a portable message-passing major additions, in MPI 3.0. The main sages per second: With current com-
library aimed specifically at clusters. communication paradigm for MPI point- munication protocols, this would
The MPI effort succeeded beyond the to-point communication has been two- mean that tens of GigaOps would be
dreams of the early forum members. To- sided communication, where a send call consumed by communication.
day, all large supercomputers are com- at the source is matched by a receive call at The following paper convincingly
modity clusters, all support MPI, and the destination. This paradigm has weak- shows that the potential of MPI one-
basically all large scientific application nesses: The complex matching rules of sided communication can be realized.
codes; as well as an increasing number sends to receives result in significant soft- It provides both a general framework
of data analytics codes, use MPI. The ware overheads, especially for receive op- for the efficient implementation of MPI
same will be true for the coming genera- erations; overlap of communication and one-sided communication on modern
tion of exascale systems. computation requires the presence of architectures, and an experimental proof
Early competitors to MPI, including an asynchronous communication agent that such an implementation can sig-
HPF, have disappeared. This success that can poll queues concurrently with nificantly reduce communication over-
has multiple reasons: Some good choic- ongoing computation; and send-receive heads and improve the performance of
es made in the MPI design, the relative communication either requires an extra large-scale applications. The paper is
ease of its implemention, the early avail- copying of messages (eager protocol) or timely and important for two reasons:
ability of high-quality implementations, extra handshakes between sender and First, users tend to avoid new features in
the confidence that an MPI library will receiver (rendezvous protocol). MPI (or other software) unless they have
continue to be available on future HPC One-sided communication requires a convincing proof of their advantages
systems, and the malleability of a library the involvement of only one process: and a solid implementation; the pa-
solution that can support multiple pro- the source process (for Put) or the des- per provides such a proof and provides
gramming styles. tination process (for Get). This already guidance for new releases of the MPI
One critical cause of this success enables a significant reduction of soft- library. Second, hardware vendors are
has been the continued evolution of ware overheads. It requires the involved often focused on optimizing their fu-
the MPI specification, in support of process to provide the location of both ture systems for past applications; NIC
evolving architectures and application the local and remote communication designers are focused on accelerating
needs: The MPI 1.1 specification, re- buffers; this is rarely a problem since the two-sided communication as it is cur-
leased in June 1995, was a document same association between local and re- rently the main communication para-
of 231 pages describing 128 functions; mote buffer tends to be reused multiple digm. The paper provides a timely warn-
the MPI 3.1 specification, released times. It separates between communica- ing that more attention must be devoted
June 2015, is an 836-page document tion and synchronization as only one of to one-sided communication.
describing 451 functions. Over time, the two communicating processes will
MPI came to accommodate threads, know the communication occurred; this Marc Snir is the Michael Faiman Professor in the
Department of Computer Science at the University of Illinois
parallel I/O, and an extensive set of is often an advantage as one synchroni- at Urbana-Champaign, IL, USA.
collective operations, including non- zation can cover multiple communica-
blocking ones. tions. Most importantly, one-sided com- Copyright held by author.
* RG performed much of the implementation during an internship at UIUC/ The original version of this paper was published in the
NCSA while the analysis and documentation was performed during a scientific Proceedings of the Supercomputing Conference 2013 (SC’13),
visit at ETH Zurich. RG’s primary email address is gerstenberger.robert@
Nov. 2013, ACM.
gmail.com.
Figure 1. An overview of MPI-3 RMA and associated cost functions. The figure shows abstract cost functions for all operations in terms of
their input domains. (a) Synchronization and (b) Communication. The symbol p denotes the number of processes, s is the data size, k is the
maximum number of neighbors, and o defines an MPI operation. The notation P: {p} → T defines the input space for the performance (cost)
function P. In this case, it indicates, for a specific MPI function, that the execution time depends only on p. We provide asymptotic cost
functions in Section 2 and parametrized cost functions for our implementation in Section 3.
(a) (b)
Figure 1 also shows abstract definitions of the perfor- Traditional Windows. These windows expose existing
mance models for each synchronization and communica- user-memory by specifying an arbitrary local base address.
tion operation. The performance model for each function All remote accesses are relative to this address. Traditional
depends on the exact implementation. We provide a detailed windows are not scalable as they require Ω ( p) storage on
overview of the asymptotic as well as exact performance prop- each of the p processes in the worst case. Yet, they are use-
erties of our protocols and our implementation in the next ful when the library can only access user-specified memory.
sections. The different performance characteristics of com- Memory addresses are exchanged with two MPI_Allgather
munication and synchronization functions make a unique operations: one for DMAPP and one for XPMEM.
combination of implementation options for each specific Allocated Windows. These windows allow the MPI library
use-case optimal. Yet, it is not always easy to choose this best to allocate window memory and thus use identical base
variant. The exact models can be used to design close-to-opti- addresses on all nodes requiring only O (1) storage. This can
mal implementations (or as input for model-guided autotun- be done with a system-wide symmetric heap or with the follow-
ing) while the simpler asymptotic models can be used in the ing POSIX-compliant protocol: (1) a leader process chooses a
algorithm design phase as exemplified by Karp et al.7 random address and broadcasts it to other processes in the
To support post-petascale computers, all protocols need window, and (2) each process tries to allocate the memory
to implement each function in a scalable way, that is, con- with this specific address using mmap(). Those two steps are
suming O (log p) memory and time on p processes. For the repeated until the allocation was successful on all the processes
purpose of explanation and illustration, we choose to discuss (this can be checked with MPI_Allreduce). This mechanism
a reference implementation as a use-case. However, all pro- requires O (log p) time (with high probability).
tocols and schemes discussed in the following can be used Dynamic Windows. Here, windows can be dynamically
on any RDMA-capable network. resized by attaching or detaching memory regions with local
MPI_Win_attach and MPI_Win_detach calls. They can be
2.1. Use-case: Cray DMAPP and XPMEM used in, for example, dynamic RMA-based data structures.
Our reference implementation used to describe RMA pro- In our implementation, the former call registers a memory
tocols and principles is called foMPI (fast one sided MPI). region and inserts the information into a linked list; the latter
foMPI is a fully functional MPI-3 RMA library implementation removes a region from the list. Both calls require O (1) memory
for Cray Gemini (XK5, XE6) and Aries (XC30)3 systems. In per region. The access to the list on a target is purely one sided.
order to maximize asynchronous progression and minimize We use a local cache to reduce the number of remote accesses;
overhead, foMPI interfaces to the lowest-level available a simple protocol uses gets to ensure the cache validity and to
hardware APIs. update local information if necessary.
For inter-node (network) communication, foMPI uses the Shared Memory Windows. These windows are only valid
RDMA API of Gemini and Aries networks: Distributed Memory for intra-node communication, enabling efficient load and
Application (DMAPP). DMAPP offers put, get, and a limited set store accesses. They can be implemented with POSIX shared
of atomic memory operations for certain 8 Byte datatypes. memory or XPMEM with constant memory overhead per
For intra-node communication, we use XPMEM,16 a portable core.5 We implement the intra-node case as a variant of allo-
Linux kernel module that allows to map the memory of one cated windows, providing identical performance and full
process into the virtual address space of another. All opera- compatibility with shared memory windows.
tions can be directly implemented with load and store instruc-
tions, as well as CPU atomics (e.g., using the x86 lock prefix). 2.3. Communication functions
foMPI’s performance properties are self-consistent (i.e., Communication functions map nearly directly to low-level
respective foMPI functions perform no worse than a combi- hardware functions, enabling significant speedups over mes-
nation of other foMPI functions that implement the same sage passing. This is a major strength of RMA programming.
functionality) and thus avoid surprises for users. We now pro- In foMPI, put and get simply use DMAPP put and get for
ceed to develop algorithms to implement the window cre- remote accesses or local memcpy for XPMEM accesses.
ation routines that expose local memory for remote access. Accumulates either use DMAPP atomics (for common integer
After this, we describe protocols for communication and operations on 8 Byte data) or fall back to a simple protocol
synchronization functions over RDMA networks. that locks the remote window, gets the data, accumulates it
locally, and writes it back. This fallback protocol ensures that
2.2. Scalable window creation the target is not involved in the communication for true pas-
An MPI window is a region of process memory that is made sive mode. It can be improved if we allow buffering (enabling
accessible to remote processes. We assume that communi- a space-time trade-off18) and active messages to perform the
cation memory needs to be registered with the communica- remote operations atomically.
tion subsystem and that remote processes require a remote We now show novel protocols to implement synchroniza-
descriptor that is returned from the registration to access tion modes in a scalable way on pure RDMA networks without
the memory. This is true for most of today’s RDMA inter- remote buffering.
faces including DMAPP and XPMEM.
2.4. Scalable window synchronization
foMPI can be downloaded from MPI defines exposure and access epochs. A process starts
http://spcl.inf.ethz.ch/Research/Parallel_Programming/foMPI. an exposure epoch to allow other processes access to its
data during the epochs. The MPI specification forbids Shared Counter Exclusive Bit Shared Counter Exclusive Counter
argument are present in its local list. The main complexity comp.+ fetched
comm. data
lies in the scalable storage of this neighbor list, needed for Proc 1 releases a
shared global lock
start, which requires a remote free-storage management MPI_Win_unlock_all()
add
scheme. The wait call can simply be synchronized with a
000 000
completion counter. A process calling wait will not return
MPI_Win_lock(EXCL, 1)
until the completion counter reaches the number of pro-
fetch-add
cesses in the specified group. To enable this, the complete
000 001 fetched
call first guarantees remote visibility of all issued RMA compare data
000 000
and swap
operations (by calling mfence or DMAPP’s gsync) and then 00000 1
increases the completion counter at all processes of the 00000 0
Figure 3. Microbenchmarks: (a) Latency comparison for put with DMAPP communication. Note that message passing (MPI-1) implies
remote synchronization while UPC, Fortran 2008 Coarrays, and MPI-2.2/3 only guarantee consistency. (b) Communication/computation
overlap for put over DMAPP, Cray MPI-2.2 has much higher latency up to 64 KB (cf. a), thus allows higher overlap. (c) Message rate for put
communication.
1
Overlap [%]
8 16 32 64 0.100
50
10
2.5
2.0
(a) Latency Inter-Node Put (b) Overlap Inter-Node (c) Message Rate Inter-Node
Figure 4. Performance of atomic accumulate operations and synchronization latencies. (a) Atomic Operation Performance, (b) Latency for
Global Synchronization, and (c) Latency for PSCW (Ring Topology).
1 2 4 intra-node intra-node
Latency [us]
Latency [us]
2.41 us
1000
100 10
Transport Layer 10
FOMPI SUM
10 Cray UPC aadd
FOMPI MIN
FOMPI CAS
1 Cray UPC CAS 1 1
1 8 64 512 4096 32768 262144 2 8 32 128 512 2k 8k 2 8 32 128 512 2k 8k 32k 128k
Number of Elements Number of Processes Number of Processes
(a) (b) (c)
4. ACCELERATING FULL CODES WITH RMA su3_rmd module, which is part of the SPEC CPU2006 and
To compare our protocols and implementation with the state SPEC MPI benchmarks, is included in the MILC code.
of the art, we analyze a 3D FFT code as well as the MIMD The program performs a stencil computation on a 4D
Lattice Computation (MILC) full production application rectangular grid and it decomposes the domain in all four
with several hundred thousand lines of source code that dimensions to minimize the surface-to-volume ratio. To
performs quantum field theory computations. Other appli- keep data consistent, neighbor communication is per-
cation case-studies can be found in the original SC13 paper, formed in all eight directions. Global allreductions are
they include a distributed hashtable representing many big done regularly to check the solver convergence. The most
data and analytics applications and a dynamic sparse data time-consuming part of MILC is the conjugate gradient
exchange representing graph traversals and complex modern solver which uses nonblocking communication overlapped
scientific codes such as n-body methods. with local computations.
In all the codes, we keep most parameters constant to com- Figure 6 shows the execution time of the whole appli-
pare the performance of PGAS languages, message passing, cation for a weak-scaling problem with a local lattice
and MPI RMA. Thus, we did not employ advanced concepts, of 43 × 8, a size very similar to the original Blue Waters
such as MPI datatypes or process topologies, which are not Petascale benchmark. Some computation phases (e.g.,
available in all designs (e.g., UPC and Fortran 2008). CG) execute up to 45% faster, yet, we chose to report
full-code performance. Cray’s UPC and foMPI exhibit
4.1. 3D fast Fourier transform essentially the same performance, while the UPC code
We now discuss how to exploit overlap of computation and uses Cray-specific tuning15 and the MPI-3 code is por-
communication in a 3D Fast Fourier Transformation. We use table to different architectures. The full-application
Cray’s MPI and UPC versions of the NAS 3D FFT benchmark. performance gain over Cray’s MPI-1 version is more
Nishtala et al.12 and Bell et al.1 demonstrated that overlap of than 15% for some configurations. The application was
computation and communication can be used to improve scaled successfully to up to 524,288 processes with all
the performance of a 2D-decomposed 3D FFT. We compare implementations. This result and our microbenchmarks
the default “nonblocking MPI” with the “UPC slab” decom- demonstrate the scalability and performance of our
position, which starts to communicate the data of a plane as protocols and that the MPI-3 RMA library interface can
soon as it is available and completes the communication as achieve speedups competitive to compiled languages
late as possible. For a fair comparison, our foMPI implemen- such as UPC and Fortran 2008 Coarrays while offering all
tation uses the same decomposition and communication of MPI’s convenient functionalities (e.g., Topologies and
scheme like the UPC version and required minimal code Datatypes). Finally, we illustrate that the new MPI-3 RMA
changes resulting in the same code complexity. semantics enable full applications to achieve significant
Figure 5 illustrates the results for the strong scaling class D speedups over message passing in a fully portable way.
benchmark (2048 × 1024 × 1024). UPC achieves a consistent Since most of those existing codes are written in MPI,
speedup over message passing, mostly due to the communi- a step-wise transformation can be used to optimize most
cation and computation overlap. foMPI has a some-what critical parts first.
lower static overhead than UPC and thus enables better over-
lap (cf. Figure 3b) and slightly higher performance. 5. RELATED WORK
PGAS programming has been investigated in the context of
4.2. MIMD lattice computation UPC and Fortran 2008 Coarrays. For example, an optimized
The MIMD Lattice Computation (MILC) Collaboration stud- UPC Barnes Hut implementation shows similarities to MPI-3
ies Quantum Chromodynamics (QCD), the theory of strong RMA programming by using bulk vectorized memory trans-
interaction.2 The group develops a set of applications, fers combined with vector reductions instead of shared
known as the MILC code, which regularly gets one of the pointer accesses.17 Highly optimized PGAS applications
largest allocations at US NSF supercomputer centers. The often use a style that can easily be adapted to MPI-3 RMA.
Figure 5. 3D FFT Performance. The annotations represent the Figure 6. Full MILC code execution time. The annotations represent
improvement of foMPI over message passing. the improvement of foMPI over message passing.
8%
%
.7
.8
Performance [GFlop/s]
45
13
400
%
%
.6
800
.2
39
3%
15
%
5.
.0
40
200
%
8%
400
.3
10
.
%
14
.8
23
4%
%
.
18
.2
13
%
5%
.3
100
9%
10
6.
200
7.
1024 4096 16384 65536 4k 8k 16k 32k 64k 128k 256k 512k
Number of Processes Number of Processes
˲˲ Computer Systems (including Networks, Cloud cal insurance, retirement and housing subsidy, engineering, computer science, economics, and/
Computing, IoT, Software Engineering, etc.) which are among the best in China. Salary and or statistical modeling methodologies. Candi-
˲˲ Cognitive Robotics and Autonomous Systems rank will commensurate with qualifications and dates with strong empirical training in econom-
˲˲ Cybersecurity (including Cryptography) experience. More information can be found at ics, behavioral science or computer science are
Applicants should have an earned Ph.D. de- http://talent.sustc.edu.cn/en. encouraged to apply. The appointed will be ex-
gree and demonstrated achievements in both We provide some of the best start-up packages pected to do innovative research in the OIT field,
research and teaching. The teaching language at in the sector to our faculty members, including to participate in the school’s PhD program, and
SUSTech is bilingual, either English or Putong- one PhD studentship per year, in addition to a to teach both required and elective courses in the
hua. It is perfectly acceptable to use English in all significant amount of start-up funding (which can MBA program. Junior applicants should have or
lectures, assignments, exams. In fact, our exist- be used to fund additional PhD students and post- expect to complete a PhD by September 1, 2019.
ing faculty members include several non-Chinese docs, research travels, and research equipments). While the Graduate School of Business will
speaking professors. To apply, please provide a cover letter iden- not be conducting any interviews at the INFORMS
As a State-level innovative city, Shenzhen has tifying the primary area of research, curriculum meeting in Phoenix, AZ, some members of the
identified innovation as the key strategy for its vitae, and research and teaching statements, and OIT faculty will be attending. Candidates who will
development. It is home to some of China’s most forward them to cshire@sustc.edu.cn. be presenting at INFORMS are strongly encour-
successful high-tech companies, such as Huawei aged to submit their CV, a research abstract and
and Tencent. SUSTech considers entrepreneur- any supporting information by October 28, 2018.
ship as one of the main directions of the univer- Stanford University We will continue to accept applications until No-
sity. Strong supports will be provided to possible Faculty positions in Operations, Information vember 15, 2018.
new initiatives. SUSTech encourages candidates and Technology Applicants should submit their applications
with experience in entrepreneurship to apply. electronically by visiting the web site http://www.
The Department of Computer Science and The Operations, Information and Technology gsb.stanford.edu/recruiting and uploading their
Engineering at SUSTech was founded in 2016. (OIT) area at the Graduate School of Business, curriculum vitae, research papers and publica-
It has 17 professors, all of whom hold doctoral Stanford University, is seeking qualified appli- tions, and teaching evaluations, if applicable,
degrees or have years of experience in overseas cants for full-time, tenure-track positions, start- on that site. For an application to be considered
universities. Among them, two were elected into ing September 1, 2019. All ranks and relevant complete, all applicants must submit a CV, a job
the “1000 Talents” Program in China; three are disciplines will be considered. Applicants are con- market paper and arrange for three letters of rec-
IEEE fellows; one IET fellow. The department is sidered in all areas of Operations, Information ommendation to be submitted by November 15,
expected to grow to 50 tenure track faculty mem- and Technology (OIT) that are broadly defined to 2018. For questions regarding the application
bers eventually, in addition to teaching-only pro- include the analytical and empirical study of tech- process, please send an email to Faculty_Recruit-
fessors and research-only professors. nological systems, in which technology, people, er@gsb.stanford.edu.
SUSTech is committed to increase the diversi- and markets interact. It thus includes operations, Stanford is an equal employment opportu-
ty of its faculty, and has a range of family-friendly information systems/technology, and manage- nity and affirmative action employer. All quali-
policies in place. The university offers competi- ment of technology. Applicants are expected to fied applicants will receive consideration for em-
tive salaries and fringe benefits including medi- have rigorous training in management science, ployment without regard to race, color, religion,
strong endorsements by referees of high interna- University of Toronto invites applications for up partment at search2018@ece.utoronto.ca.
tional standing. Evidence of excellence in teach- to four full-time tenure-stream faculty appoint- The University of Toronto is strongly com-
ing will be demonstrated by strong communica- ments at the rank of Associate Professor. The ap- mitted to diversity within its community and es-
tion skills; a compelling statement of teaching pointments will commence on July 1, 2019. pecially welcomes applications from racialized
submitted as part of the application highlighting Within the general field of electrical and com- persons / persons of colour, women, Indigenous /
areas of interest, awards and accomplishments, puter engineering, we seek applications from Aboriginal People of North America, persons with
and teaching philosophy; sample course syllabi candidates with expertise in one or more of the disabilities, LGBTQ persons, and others who may
and materials; and teaching evaluations, as well following strategic research areas: 1. Computer contribute to the further diversification of ideas.
as strong letters of recommendation. Systems and Software; 2. Electrical Power Sys- As part of your application, you will be asked
Eligibility and willingness to register as a Pro- tems; 3. Systems Control, including but not lim- to complete a brief Diversity Survey. This survey is
fessional Engineer in Ontario is highly desirable. ited to autonomous and robotic systems. voluntary. Any information directly related to you
Salary will be commensurate with qualifica- Applicants are expected to have a Ph.D. in is confidential and cannot be accessed by search
tions and experience. Electrical and Computer Engineering, or a relat- committees or human resources staff. Results
The Edward S. Rogers Sr. Department of Elec- ed field, and have at least five years of academic or will be aggregated for institutional planning pur-
trical and Computer Engineering at the Univer- relevant industrial experience. poses. For more information, please see http://
sity of Toronto ranks among the best in North Successful candidates will be expected to uoft.me/UP.
America. It attracts outstanding students, has maintain and lead an outstanding, independent, All qualified candidates are encouraged to
excellent facilities, and is ideally located in the competitive, innovative, and externally funded apply; however, Canadians and permanent resi-
middle of a vibrant, artistic, diverse and cosmo- research program of international calibre, and dents will be given priority.
politan city. to teach at both the undergraduate and gradu-
Additional information may be found at ate levels. Candidates should have demonstrated
http://www.ece.utoronto.ca. excellence in research and teaching. Excellence University of Zurich
Review of applications will begin after Sep- in research is evidenced primarily by sustained Assistant Professorship in Interacting with
tember 1, 2018, however, the position will remain and impactful publications in leading journals Data (Non-tenure Track)
open until November 29, 2018. or conferences in the field, awards and accolades,
As part of your online application, please presentations at significant conferences and The Faculty of Business, Economics and Infor-
include a cover letter, a curriculum vitae, a sum- a high profile in the field with strong endorse- matics of the University of Zurich invites applica-
mary of your previous research and future research ments by referees of high international stand- tions for an Assistant Professorship in Interact-
plans, as well as a teaching dossier including a ing. Evidence of excellence in teaching will be ing with Data (Non-tenure Track) starting in 2019.
statement of teaching experience and interests, demonstrated by strong communication skills, Candidates should hold a Ph.D. degree in
your teaching philosophy and accomplishments, a compelling statement of teaching submitted as Computer Science with specialization in Interac-
and teaching evaluations. Applicants must arrange part of the application highlighting areas of inter- tive Data Analysis, Visual Analytics, Information
for three letters of reference to be sent directly by est, awards and accomplishments, and teaching Visualization or related areas and have an excel-
the referees (on letterhead, signed and scanned), philosophy; sample course syllabi and materials; lent record of academic achievements in the rel-
by email to the ECE department at search2018@ and teaching evaluations, as well as strong letters evant fields. A strong motivation to teach both at
ece.utoronto.ca. Applications without any refer- of recommendation. the undergraduate and the graduate levels as well
ence letters will not be considered; it is your re- Eligibility and willingness to register as a Pro- as an interest in human and societal aspects of
sponsibility to make sure your referees send us the fessional Engineer in Ontario is highly desirable. managing data are highly beneficial.
letters while the position remains open. Salary will be commensurate with qualifica- The successful candidate is expected to estab-
You must submit your application online tions and experience. lish her or his research group within the Depart-
while the position is open, by following the sub- The Edward S. Rogers Sr. Department of Elec- ment of lnformatics, actively interface with the
mission guidelines given at http://uoft.me/how- trical and Computer Engineering at the Univer- other groups at the department and the faculty,
to-apply. Applications submitted in any other way sity of Toronto ranks among the best in North and seek collaboration with researchers across
will not be considered. We recommend combin- America. It attracts outstanding students, has ex- faculties within the Digital Society Initiative of
ing attached documents into one or two files in cellent facilities, and is ideally located in the mid- the University of Zurich.
PDF/MS Word format. If you have any questions dle of a vibrant, artistic, diverse and cosmopoli- Through its educational and research objec-
about this position, please contact the ECE de- tan city. Additional information may be found at tives, the University of Zurich aims at attracting
partment at search2018@ece.utoronto.ca. http://www.ece.utoronto.ca. leading international researchers who are willing
The University of Toronto is strongly com- Review of applications will begin after Sep- to contribute to its development and to strength-
mitted to diversity within its community and es- tember 1, 2018, however, the position will remain ening its reputation. The University of Zurich is
pecially welcomes applications from racialized open until November 29, 2018. an equal opportunity employer and strongly en-
persons / persons of colour, women, Indigenous / As part of your online application, please courages applications from female candidates.
Aboriginal People of North America, persons with include a cover letter, a curriculum vitae, a sum- Please submit your application at https://
disabilities, LGBTQ persons, and others who may mary of your previous research and future research www.facultyhiring.oec.uzh.ch/position/9633792
contribute to the further diversification of ideas. plans, as well as a teaching dossier including a before October 15, 2018.
As part of your application, you will be asked statement of teaching experience and interests, Documents should be addressed to Prof. Dr.
to complete a brief Diversity Survey. This survey is your teaching philosophy and accomplishments, Harald Gall; Dean of the Faculty of Business, Eco-
voluntary. Any information directly related to you and teaching evaluations. Applicants must arrange nomics and Informatics; University of Zurich;
is confidential and cannot be accessed by search for three letters of reference to be sent directly by Switzerland.
committees or human resources staff. Results the referees (on letterhead, signed and scanned), For further questions regarding the profile of
will be aggregated for institutional planning pur- by email to the ECE department at search2018@ the open position please contact Prof. Renato Pa-
poses. For more information, please see http:// ece.utoronto.ca. Applications without any refer- jarola (pajarola@ifi.uzh.ch)
uoft.me/UP. ence letters will not be considered; it is your re-
All qualified candidates are encouraged to sponsibility to make sure your referees send us the
apply; however, Canadians and permanent resi- letters while the position remains open.
dents will be given priority. You must submit your application online
while the position is open, by following the sub-
mission guidelines given at http://uoft.me/how-
University of Toronto to-apply. Applications submitted in any other way
Associate Professor, Tenure Stream will not be considered. We recommend combin-
ing attached documents into one or two files in
The Edward S. Rogers Sr. Department of Elec- PDF/MS Word format. If you have any questions
trical and Computer Engineering (ECE) at the about this position, please contact the ECE de-
[ C ONTI N U E D FRO M P. 120] coding was they are moving? Can you tell how
defined in the context of something many people there are? It turns out
called multicast. you can, because they are breathing…
Multicast is a communications pro- So what other physiological signals
tocol in which you deliver the same
information to a group of destinations
can you extract?
And these questions are extremely
Distinguished
simultaneously. intellectually interesting, but it’s not Speakers Program
But in networking, typically, that’s just that; they have very practical and
not how it works. In networking, you useful applications to people’s lives!
typically have unicast, where one send- http://dsp.acm.org
er transmits to a single destination. You’re now working, through a start-up
Even when you are sending something called Emerald, to commercialize the
like broadcast television over the In- technology and develop some of those
ternet, your broadcast is actually using applications—for instance, remotely
unicast. You have your server turning monitoring people’s health.
that traffic to all the individuals who We talk a lot about the smart home,
are interested in it. but really the smartest thing a home
What Muriel and I did was try to can do is to take care of us and our
take that really beautiful, elegant health. Our vision is to have a technol-
theory, and think about it in the con- ogy that disappears into the environ-
text of real networks. I felt wireless ment; I don’t have to enter information
networks, in particular, might be the about my heartrate, or put some device
right environment for this technol- on myself and remember to charge it.
ogy. Wireless is way more limited in I don’t need to change my behavior in Students and faculty
terms of data rate and bandwidth any way, but still there is a home that’s
than wired networks, and it’s also less watching over my health and keeping can take advantage of
reliable. So network coding is an ideal track of problems early on—or even be-
solution when you make an error in fore they occur—and alerting doctors ACM’s Distinguished
your transmission. or the hospital or a caregiver.
Speakers Program
In your recent work, you’ve used wire- That sounds promising. Where are you to invite renowned
less signals to track people’s mo- in your efforts?
tions—even through walls. How did At this early stage, our focus is to thought leaders in
you get that idea? work with healthcare providers, on the
When we began, it was really curi- one hand, and with the biotech and academia, industry
osity. Let’s say there is a room and you pharma industry, on the other. It turns
don’t have access to it. Can you tell if out there are many deep physiological
and government
there are people in the room? If you signals we can extract, so we need to to deliver compelling
can tell there are people, can you tell connect with people who understand
how many people? When we tried that, what those signals mean in the context and insightful talks
we didn’t really know whether or not it of diseases. I can tell you that my mom
was possible, and we certainly didn’t is walking well or that she fell—that’s on the most important
know what kind of application you’d the extent of it. I couldn’t tell you if the
use it for. All we knew is that we have patterns of information indicate we
topics in computing
been able to track people using their should change the dose on her Parkin- and IT today.
cellphones—so, using a wireless sig- son’s medication.
nal, but a wireless signal that is emitted ACM covers the cost
from a device. And we have some un- One of the most consistently cited fea-
derstanding of how wireless works in tures of your work is creativity. of transportation
an indoor environment and propagates In general, in almost all the stuff
through walls and materials. I do, I’m driven by curiosity. I’m al-
for the speaker
ways interested in trying something to travel to your event.
After your initial demonstrations were where I don’t know the answer, or
successful, the questions got more where I’m not sure whether the an-
complex, and practical applications swer is “yes” or “no.”
began to present themselves.
Once we started working with it, Leah Hoffmann is a technology writer based in Piermont,
NY, USA.
we began to have all these ideas—
why stop at just being able to see if
people are moving? Can you tell how © 2018 ACM 0001-0782/18/10 $15.00
Q&A
Reaping the Benefits
of a Diverse Background
Earlier this year, ACM named Dina Katabi of the Massachusetts Institute
of Technology’s Computer Science and Artificial Intelligence Laboratory recipient
of the 2017 ACM Prize in Computing for her creative contributions to wireless systems.
DINA KATABI, RECIPIENT of the 2017 ACM In some of your earliest work at MIT,
Prize in Computing, took a winding you collaborated with David Clark—the
road to computing, and it paid off. Now Internet’s chief protocol architect dur-
a professor of electrical engineering and ing most of the 1980s—on network con-
computer science at the Massachusetts trol, where one of the biggest problems
Institute of Technology (MIT), Katabi be- is managing transmissions when they
gan her career in medicine. Since mak- threaten to overwhelm the network.
ing the transition, she has made numer- At the time, the traditional method
ous creative contributions to wireless of congestion control was based on
network design. Today, she is helping to heuristics. It was more of an art than
develop medical applications for a tech- anything. But it was often not very effi-
nology she pioneered, which uses wire- cient—not very fair to different users—
less signals to sense humans and their because the Internet is just too big.
movements through walls—her early With my thesis, I tried to connect that
training coming full circle. art and intuition to the field of control
theory, which is a subfield in electri-
Your undergraduate degree is in elec- cal engineering that is typically used
trical engineering, but you began by to control plants and manufacturing
studying medicine. systems. So you can keep the intuition,
In Syria, after high school, there is to the U.S., I wanted to learn more but if you infuse into it some of the
a nationwide exam, and the expecta- about algorithms. mathematical models, you can achieve
tion is that the top people will go to much better results. You can make the
medical school. I took the exam, and Having experience with different fields network more stable and achieve more
I ranked very high. I also come from seems to have proven beneficial to your efficient systems.
a family of doctors. So I went to med work.
school, but after the first year, I de- I’ve benefited from having a very di- After you received your Ph.D., you stayed
cided I could not continue. I wanted verse background, which has enabled at MIT and began working with infor-
to do math and engineering, so I de- me to see beyond the field I am in. Par- mation theorist Muriel Médard on net-
cided to switch. ticularly when I was working on wire- work coding, a technique for increasing
less systems, my background gave me networks’ data capacity that was prom-
You then came to the U.S., and did your the expertise I needed to design the cir- ising in theory, but had not yet been
Ph.D. in computer science. cuit, the signal, and also the algorithm shown to work on a real network.
PHOTO BY JA NA AŠENBRENNEROVÁ
At the time, computer science (CS) that extracts information from that sig- When we began our work, network
was a very new field in Syria. In fact, nal. You can design many systems with coding had shown high gains in spe-
at the school I attended, there was electrical engineering, but the ability cific examples, but those examples
no such thing as a CS school or de- to add intelligence to them using CS is did not map to the way that networks
partment. But I was always fascinated much more powerful than if it was just really operate. For instance, the theory
with computers, and when I came pure signal processing. of network [C O NTINUED O N P. 119]
PAV I N G T H E W AY T O W A R D E X C E L L E N C E I N C O M P U T I N G E D U C AT I O N
ACM/SIGCSE Seek
Co-EDITORS-IN-CHIEF for ACM Inroads
ACM and the Special Interest Group on Computer Science Education (SIGCSE) seek
co-editors-in-chief (co-EICs) to lead its quarterly magazine ACM Inroads.
The magazine serves computing education professionals globally by fostering
dialogue, cooperation, and collaboration between educators worldwide. It achieves
this by publishing high-quality content describing, analyzing, and critiquing current
issues and practices affecting computer education now and in the future.
The magazine is written by and for educators, with each issue presenting thought-
provoking commentaries and articles that examine current research and practices
within the computing community.
For more about Inroads, see http://inroads.acm.org/
Job Description
The EIC position is a highly visible, hands-on volunteer position responsible for
leading, networking, and overseeing all editorial aspects of the magazine’s content
creation process, including but not limited to: soliciting articles from prospective
authors; managing the magazine’s editorial board and contributors to meet
quarterly publication deadlines; creating new editorial features, special sections,
columns; upholding a high bar for the content’s quality and diversity; assigning
manuscripts to associate editors for review; making final editorial decisions; setting
the overall direction and online strategy of the publication. Prior experience leading
or managing editorial projects a plus.
Eligibility Requirements
The co-EiCs search is open to applicants worldwide.
Applications are welcome from both individuals and from pairs wishing to serve as
co-EICs.
Applicants must be willing and able to make a 3-year commitment to this post.
To apply, please send your CV along with a 300-word vision statement expressing
the reasons for your interest in the position to: eicsearch@inroads.acm.org
The deadline for submissions is OCTOBER 15, 2018.
The editorship will commence on DECEMBER 1, 2018.
Organized by