Shift Magazine - May 2021 - 25

a research project between the aviation
manufacturer and French automakers to
better understand potential crossover
learnings. " The deep difference in their
capabilities is enormous. "
But even trained professionals can succumb to the boredom that automation
complacency breeds. In November 2009,
the flight crew of Northwest Airlines Flight
188 overshot their Minneapolis-St. Paul
airport destination by more than 100 miles
because they were distracted while using
a personal computer in the cockpit once
they enabled autopilot for cruise flight,
according to the NTSB. The flight landed
an hour late without further incident.
A similar situation in the automotive
realm had deadly consequences. On
March 18, 2018, a safety driver behind the
wheel of one of Uber's autonomous test
vehicles in Tempe, Ariz., was watching
" The Voice " on her phone when the vehicle struck and killed a pedestrian.
" Automation performs remarkably well
most of the time, and therein lies the problem, " NTSB member Bruce Landsberg
wrote in the agency's final report on the
collision.

NOT A PERFECT REMEDY

When automation performs in unexpected or confusing ways, problems can
spiral into catastrophic outcomes. Aboard
Air France Flight 447 in June 2009, an icedover sensor led the autopilot system to erroneously believe the plane was at too low an
altitude and needed to climb. As it climbed
at a high pitch and triggered an alert, information was not displayed in a way that the
human pilots could quickly understand the
situation, a subsequent investigation found.
In their confusion, the pilots exacerbated an
aerodynamic stall rather than make control
inputs that would ensure a recovery. The
plane crashed into the Atlantic Ocean, killing all 228 people aboard.
This " mode confusion " may crop up
in vehicles that have driver-assist systems, according to a 2014 study by Missy
Cummings, a Duke University professor
and director of the school's Humans and

GETTY IMAGES

Journalists review pieces of the wreckage of Air France Flight 447, which crashed into the
Atlantic Ocean in 2009. " Mode confusion " is thought to have been a factor in the crash.

Autonomy Laboratory. There's an inherent
danger in that.
" At precisely the time when the automation needs assistance, the operator could
not provide it, " she wrote. " We cannot
assume the operator to be always engaged,
always informed and always ready to intervene and make correct decisions when
required by the automation to do so. "
In many corners, the solutions to automation complacency and mode confusion
are to develop more sophisticated automated systems. Cummings notes that
the accident rate in U.S. commercial jet
operations dropped from approximately
4 per million departures to 1.4 per million
departures between 1959 and 2012, and
automation undoubtedly played a role in
that improvement.

On the ground, human error causes 94
percent of motor-vehicle crashes, according to federal regulators, a statistic that
self-driving vehicle companies often tout
as their raison d'etre. While automation
can lead to net safety benefits, transportation incidents that involve human-machine
interaction show it should not be considered a cure-all.
" People often mistake automation as a
'remedy for error,' " said human-machine
interaction researcher Liza Dixon. " But
adding automation to a human-machine
system doesn't remove the error. It displaces it, either by creating new problems
or by outsourcing the error to system
designers. This is perhaps one of the greatest takeaways from the aviation domain
which applies directly to automotive. " n

may 2021 * shift

S025_AN_20210524.indd 25

5/14/21 2:34 PM

Shift Magazine - May 2021

Table of Contents for the Digital Edition of Shift Magazine - May 2021