IEEE Technology and Society Magazine - June 2018 - 86

A different possible form of the wisdom of the crowd
can be the wisdom of the crowd of algorithms, enabled
by the existence of a plurality of algorithms and systems
devoted to govern the same concern. In essence, human
decisions might be based on some aggregation of the
inputs of a number of preferably independent algorithms. That is, we can take the principle of the "wisdom
of the many" to systems. However, when algorithms deal
with ethical choices, the involvement of humans may
become necessary. Should there be no time to involve
a human in the decision-making, one can consider that
some algorithmic decisions are premade by a human in
advance so that human accountability is retained, and
automatically adapted to the specific context once these
decisions have to become actions.
Lastly, when there are systematic failures, there
needs to be a way to "pull the plug." When an algorithm
it is found to deliver unfavorable outcomes or cause
problems, humans must be able to stop using the system. As a simple example, a user (as it is already
indeed the case in our homes) should always be able to
turn off an automatic home heating system and operate
the system manually. The mechanism for control over
these systems, and the way to turn them off, needs to
be obvious [24]. However, for critical systems, the look
and feel of the control switches needs to be different,
so that unintentional, potentially fatal, mistakes are not
made. As related in [24], control-room operators in a
nuclear power plant found that similar-looking knobs
could lead to a disastrous outcome, hence beer-keg
handles were placed over them. Putting humans in ultimate control of algorithms would seem sensible, but is
not without its own issues. For example, a human cannot simply switch off an autonomous vehicle when s/he
thinks it is not performing up to its requirements -
there needs to be a way to deliver control back to
humans safely, and once in control, for humans to safely control the algorithm.

From Context-Awareness to ContextControl-Awareness
Pervasive computing technologies enable algorithms to
make decisions that are context-aware, i.e., adapted to
the context in which they operate (the already mentioned personalization being a specific form of contextawareness). Context-awareness has been studied since
the 1990s in the area of pervasive computing [25], [26].
The last decade has seen dramatic progress in automatic recognition of context (including place - outdoors
and indoors, human activity, habit, preference, and available energy and resources, etc.) and the self-adaptation
of pervasive computing devices to the learned context.
Given the possible perils of algorithmic governance,
one possible research direction is "context-control-aware"

systems. That is, algorithms that can override control are
not given full access for adapting the devices under their
influence but are, instead, given a shared access control
with a network of socially connected devices, with humans as co-decision makers, for shared governance [27].
This allows pervasive computing systems to be more "considerate," as they are not only aware of their contexts (and
input to the systems), but also of how their influence and
control (and the output of the systems) can bring unintended consequences. Such shared control should also
account for safety, security, and privacy that - although
extensively researched so far mostly in a separate silo -
have yet to be integrated in context-control-aware systems
for safer smart environments.
The issue of control (also related to the previously
mentioned issue of "pulling the plug") involves that of
making Smart Cities and environments really usable. This
alludes to Steve Krug's attributes of usability of an interface [28]: useful, learnable, memorable, desirable, and
delightful. In particular, the studied contexts in contextcontrol-aware computing for Smart Cities and environments should also be provided with interfaces allowing
citizens and end users to voice any discomfort and displeasure with the systems. This would enable higher levels of interactivity with the governing algorithms, and
enable these systems to, e.g., "reverse action," "pull
back," and activate "dumb mode" when required. We
need Smart Cities to not just be "efficient." We also need
to make it possible for citizens to be more interactive with
the governing of cities, and vice versa [29].
There is a need for balance between a system that is
too obtrusive to be useful, in which the user is too often
involved, and a system that is too autonomous so that
algorithmic regulation becomes real. A question is, can
an algorithm be designed to compute this balance?
Can an algorithm "solve" the problem of algorithmic
dominance or governance, or of being context-aware,
including an algorithm being aware of itself being too
controlling? This is where potentially the pervasive computing community, in a close collaboration with other
fields in this truly complex multidisciplinary issue, can
contribute a solution.

Protecting the Individual Citizen
from the Algorithm
The obvious advantage of the increasing availability of
pervasive computing infrastructures is that they will
make our lives much easier. However, this evolution
also carries potential risks for individuals and for society as a whole. By blindly accepting the deployment of
algorithms that run our society, we could become a
"dumber" society or even lose control. In this context,
"algocracy" may be nontrivial to reconcile with democracy. Dealing with these issues will require deploying a

IEEE Technology and Society Magazine

∕

JUNE 2018

Table of Contents for the Digital Edition of IEEE Technology and Society Magazine - June 2018