Signal Processing - September 2017 - 133

infrastructure-based category at the 2016 and 2017 competitions. These teams were required to report 3-D locations.
Teams that relied on Wi-Fi were only allowed to leverage
the existing deployed networking infrastucture at the conference venue. In 2015, there were four Wi-Fi access points, and in
2016, three Wi-Fi access points. The only exception to this rule
was the 2014 competition, where the teams were able to use ten
Wi-Fi access points that were deployed by the organizers.
Note that the deployment of ten dedicated Wi-Fi access
points created a bias in favor of all systems in the infrastructure-free category. Given the relatively small size of the
evaluation area, deploying ten access points resulted into an
unusually high density of access points. Most areas today (i.e.,
malls) provide fewer numbers of access points that are also
mounted differently in the space (i.e., mounted on the ceiling).

Participating teams
All teams were classified into two categories: infrastructurefree and infrastructure-based, according to their hardware
deployment requirements. Teams in the infrastructure-free category did not require the deployment of any custom hardware
to compute indoor locations, apart from existing Wi-Fi infrastructure. Most of these approaches leverage existing Wi-Fi
signals and combine them with sensors, such as accelerometer, gyro, and compass, on existing commercially available
devices, such as phones and tablets. However, teams in the
infrastructure-based category required the deployment of custom hardware, such as Bluetooth beacons, magnetic resonators, ultrasound speakers, custom RF Tx, and more.
Tables 1 and 2 provide an overview of all the teams that
participated in the past four competitions (also see Figure 4).
The technology used by each team and the team's global rank
in terms of accuracy for that year are also listed.

Localization accuracy overview
Figure 5 shows the average localization error of every team that
was successfully evaluated in the last four years of the competition. Looking across all years and types of systems, lidar-based
technology seems to consistently achieve the lowest localization error (Sánchez et al. in 2015 with 0.2 m, Leica and also
Zhang et al. in 2016 with 0.05 m and 0.16 m, respectively, and
Zhang et al. in 2017 with 0.03 m). Even though extremely accurate, the cost as well as the size and power requirements of lidar
sensors prevents this technology from becoming a mainstream
indoor location solution. In fact, this type of sensor was accepted as exhibition systems, as the main motivation behind the
competition was to focus on technologies that can be massively
deployed in the near future to enable everyday devices, such as
phones and tablets, to be accurately localized indoors.

Infrastructure-free technologies
With this in mind, infrastructure-free approaches seem ideal, as
they don't require any hardware to be deployed. They solely
rely on Wi-Fi/magnetic fingerprints and the inertial sensors
already available on most phones and tablets. The best teams in
this category were able to achieve localization accuracy any-

where between 1.1 and 1.9 m. In 2014, Beder and Klepal and
Li et al. achieved 1.56 and 1.96-m errors, respectively, but in a
relatively small area with an unusually high density of Wi-Fi
access points (ten Wi-Fi access points were deployed in 2014,
as described in the "Variations Across Competitions" section).
In 2015, SPIRIT Navigation and Guimarães et al. were able to
achieve 1.9 m and 2.6 m, respectively, but in a much larger
open-space evaluation area with only three Wi-Fi access points.
Guimarães et al. also competed in 2016, achieving a localization error of 1.1 m, the lowest error across all teams leveraging
similar technology.
Surprisingly, the accuracy of infrastructure-free teams in
the 2017 competition decreased drastically. Even teams that had
performed very well in 2015 and 2016 were not able to get anywhere close to 1-2-m accuracy (i.e., the team from Portugal's
Fraunhofer Research Institute). Most of the teams in this category
attributed this discrepancy to three factors. First, they observed
that the Wi-Fi fingerprints in the evaluation area were fluctuating significantly over time, more than in any other location they
had previously tested their systems. Second, the magnetic fingerprints were completely unreliable, most probably because of
the 18-ft-tall escalators that were operating during both days of
the competition. Third, five of the total 20 evaluation points were
placed on the stairs in between the escalators, resulting in more
than usual localization error for this type of system.
The only systems that performed very well in this category
in 2017 did not rely on Wi-Fi and/or magnetic fingerprinting.
Instead, they depended on either the phone's sensors (Li et al.)
or on foot-mounted IMU sensors (Ju et al.).
When considering a realistic environment, such as a shopping mall or an office building, infrastructure-free technologies can achieve accuracies of approximately 2 m. However,
these results do not take into account the temporal variations of
the wireless signal fingerprints. During the competition, teams
train their systems one day before the evaluation, resulting in a
positively biased evaluation. When considering temporal variations, we expect the localization error to increase by 1 or 2 m,
resulting into an expected error of 3-4 m.
Another interesting observation is that even though most
of the infrastructure-free teams rely on the same signals and
sensors, the actual localization accuracy can vary widely. For
instance, in 2014, the localization error of the Wi-Fi-based
approaches ranged anywhere between 1.56 and 5.23 m, with
similar trends in 2015 and 2016. Even though Wi-Fi fingerprinting is a very-well-studied topic, the exact implementation
can significantly affect the localization accuracy.

Infrastructure-based technologies
When considering infrastructure-based approaches, there is a
much more diverse set of technologies on which both academia and industry have focused. Most of the systems that
have been evaluated so far focus on six core technologies:
sound/ultrasound ToF, UWB, visible light communication,
BLE beacons, magnetic resonators, and 2.4-GHz phase offset.
Across all of these technologies, UWB seems to rapidly rise
as the most accurate and popular infrastructure-based indoor

IEEE SIGNAL PROCESSING MAGAZINE

September 2017

133

Table of Contents for the Digital Edition of Signal Processing - September 2017