IEEE Aerospace and Electronic Systems Magazine - December 2020 - 48

Towards the Use of Artiﬁcial Intelligence on the Edge in Space Systems: Challenges and Opportunities
developed [35]. A benchmark between the different COTS
for DL is shown in[28], [29], and [30].

AN AI-FIRST APPROACH TO SPACE APPLICATIONS
CHALLENGES WHEN USING AI
While AI is being already successfully applied in space, it
is still confined to offline data processing and not adopted
" on the edge " inside the spacecraft themselves. The primary reason is the difficulty of porting DL networks to
hardware that predates the algorithms themselves and has
insufficient performance to do even basic inferencing. For
example, the weights and topology of models necessary to
provide sufficient accuracy are often too large for the
memory budget of satellites. In addition, the inference of
many AI models is computationally intensive [36], since
they require a high number of operations per second to
respect the latency requirements typical of many onboard
applications. This is generally not acceptable for many
applications owing to the power consumption constraints
due to the difficulty of heat dissipation and the low power
budget. Such problems could be mitigated by adopting
dedicated AI platforms, such as the Myriad 2, described in
section " State of the Art of COTS SoCs for DNN
Inference, " whose architecture and technology process
permits the implementation of complex CNN models with
excellent tradeoffs between model complexity, processing
speed, and power consumption. Furthermore, memory budget problems can be mitigated through an effective network
selection and design strategies by performing model compression. In that respect, research indicates that certain neural network models can be compacted without significant
loss of accuracy. For instance, knowledge distillation can
be applied to a model with a smaller size starting from a
pretrained model [37]. Furthermore, quantization and pruning techniques can be efficiently applied to compress the
model [35], even improving its accuracy compared to the
original one [32]. Depending on the arithmetic representation used, different hardware can be exploited. Indeed,
Jetson Nano and Myriad 2 natively support only 16-bit
floating point models. Myriad X also supports 8-bit fixed
point arithmetic. Google Coral uses an 8-bit fixed point systolic array. FPGAs can be exploited to implement both
floating point and fixed point architectures. However, better performances and a lower power per inference are
obtained using fixed point [32], [35], reaching their peak of
performances exploiting binarized convolutional neural
networks [30], [35]. A second reason that may have slowed
the adoption of AI for onboard applications is the lack of
confidence in the unpredictability of the approach. This
nondeterminism derives from the impossibility of de facto
testing the weights set resulting from training, performed
through a finite number of data, for all possible inputs.
48

Since safety is critical for space applications, owing to the
high cost of failure, more predictable approaches to AI are
generally preferred [24]. To minimize risks in EO missions,
eventual applications of AI could be limited to the payload
level to perform object detection/classification locally on sensor data [38]. In this case, eventual failures of AI would only
affect the quality of data for the single payload, without being
a risk to the entire satellite. For lower dependability applications, the use of COTS processors such as Myriad 2 or such
as COTS FPGAs (mostly high-performance Ultrascale+
series FPGA) can be envisaged in all those applications with
low-to-medium dependability requirements, provided that
the AI inference accelerator is supervised by a fault-tolerant
engine. The last issue involves the training of deep networks.
Indeed, a primary problem concerns the availability of datasets for training and model evaluation, especially for missions
featuring new equipment, including novel sensors, for which
a dataset for DNN training does not exist. Furthermore, in
view of its complexity, DNN training shall be performed on
ground by leveraging cloud-based GPUs or more specialized
training hardware such as TPUs. These aspects pose a real
concern on the usability of models trained before the launch
of satellites, whose training is not performed through the original satellite data. However, this problem is mitigated by the
possibility of reconfiguring models during the life of missions, enabled by the use of modern COTS ASICs and by the
reduced dimensions of files necessary for their programming,
which is becoming compatible with the uplink bandwidth of
small satellites, as described in details in the section
" Effecting Change Through the Application of AI. "

EFFECTING CHANGE THROUGH THE APPLICATION OF AI
The use of AI, in particular deep learning techniques, generally leads to better results in remote sensing than previous approaches [38]. Moreover, the benefits due to the
introduction of deep learning would not be linked only to
the results of the single payload, but improvements in flexibility would be possible for the entire satellite.
To better explain such a concept, consider the classical
architecture of an EO satellite, shown in Figure 1. Data are
generally accumulated in a mass-memory and transmitted
to ground when the satellite enters the coverage-area of a
dedicated ground station. Such an approach is called
" bent-pipe " communication paradigm, which involves
sending the data to the ground following a command
transmitted by the specific ground station [39]. In the classical approach, the raw image produced by the imager is
processed by an FPGA interface. The latter is used to convert the images into a proprietary format, which might be
exploited by the image compression system, generally
realized through an ASIC solution, owing to the required
performances. Compressed images are finally stored in
mass memory before being transmitted to a ground station.

IEEE A&E SYSTEMS MAGAZINE

DECEMBER 2020

IEEE Aerospace and Electronic Systems Magazine - December 2020

Table of Contents for the Digital Edition of IEEE Aerospace and Electronic Systems Magazine - December 2020