Aerospace and Electronic Systems - July 2019 - 53

Petkovic et al.
initial default (weak) model is learned on the whole dataset that minimizes the loss function L. Given that the gradient boosted method aims at optimizing the loss function,
it is important that L should be convex and differentiable.
A typical choice of L is L2 square-loss function [10]. In
turn, gradient boosting at each step m learns a new model
on the pseudo-residuals, i.e., the discrepancy value
between the true and the predicted value of the ensemble
in the previous iterations (line 4).
However, such straightforward approach can very easily overfit to the training data. In order to address the problem of overfitting, more sophisticated gradient boosting
methods implement two different mechanisms: a learning
rate and random data sampling procedure. The former regulates the influence of the prediction of each subsequent
model added in the ensemble set (line 6). The latter,
referred to as Stochastic Gradient Boosting [7], employs
additional random data sampling procedure: Each model
is learned and evaluated on different random subsamples
of the training data (line 3).

Algorithm 2: Stohastic Gradient Boosted Trees
(D TRAIN , M, L, h)
1: GBT; ¼ defaultModelðD TRAIN ; LÞ
2: for m ¼ 1; 2; . . .; M do
3: E ¼ randomSampleðD TRAIN Þ
@LðE; GBTmÀ1 ðEÞÞ
4: Rm ¼
@GBTmÀ1

// compute
pseudo-residuals

5: T ¼ induceðRm ; LÞ
6: GBTm ¼ GBTmÀ1 þ hT
7: return GBT
In this study, we employ XGBoost [6]-a recent efficient implementation of Stochastic Gradient Boosting [7]
that employs regression trees as proposed in [24] as base
models. In the paper, we denote the XGBoost ensembles
with XGB.

EXPERIMENTAL SETUP
PARAMETER INSTANTIATION
Granularity: The data granularity is defined by the length
Dt of the time interval that corresponds to one example in
the dataset. We construct the predictive models using
datasets with Dt 2 f1; 5; 10; 15; 30; 60g (measured in
minutes) where Dt ¼ 1 corresponds to the dataset used
in [4].
Historical features: For the dataset with finest
granularity Dt ¼ 1, we consider the following numbers H
of historical intervals from (1): H 2 f4; 16; 32; 64; 128g.
Consequently, the historical time span ranges from 4 min
to 128 min. The choices of the historical intervals in the
JULY 2019

Table 3.

Values of the Number of Historic Intervals H and the
Corresponding Historic Time Spans, for Different
Granularities. Dt
Dt

Values of H

Time spans

1

f4; 16; 32; 64; 128g

f4; 16; 32; 64; 128g

5

f1; 3; 6; 13; 25g

f5; 15; 30; 65; 125g

10

f1; 2; 3; 6; 13g

f10; 20; 30; 60; 130g

15

f1; 2; 3; 4; 9g

f15; 30; 45; 60; 135g

30

f1; 2; 3; 4; 5g

f30; 60; 90; 120; 150g

60

f1; 2; 3g

f60; 120; 180g

datasets with courser granularity are presented in Table 3.
In total, these values yield 35 features in the dataset with
Dt 30 and 21 features dataset with Dt ¼ 60.
RF parameters: To constrain the size of the trees in
the RFs, we specify a minimal number of examples in the
leaves mLEAF for each tree. Since the number of instances
in the datasets is inversely proportional to Dt, we set the
minimal number of instances for the Dt ¼ 1 experiments
to 500, while for the others we set them to
mLEAF ¼ 500=Dt. Additionally, we set the total number
of trees in the RFs to 200, where one quarter of the features is considered at every split when growing the trees,
i.e., f ¼ 0:25jF j in Algorithm 1.
XGBoost Parameters: Analogously, to constrain the
size of the trees, we set maximal depth of each tree in the
ensemble to 11. The learning rate parameter is set to 0.1.
Additionally, to address potential overfitting issues, for
every boosting iteration 60% of the features and 60% of
the examples are randomly chosen for training. The maximum number of boosting iteration (ensemble constituents)
is set to 200 with an early-stop option, i.e., if the newly
added trees in the ensemble do not improve the performance of the ensemble over five consecutive boosting iterations the algorithm stops.

EVALUATION PROCEDURE
The dataset D consists of examples ðx
x; yÞ where x is a
vector of feature values (features are described in
Section "DATA"), and y is a vector of 33 target values,
i.e., the electrical currents trough the heaters and coolers.
The dataset is divided into two parts: D TRAIN that
describes the state of the spacecraft throughout the first
three Martian years, and D TEST that describes the state of
the spacecraft throughout the fourth Martian year. All predictive models, i.e., the approximations y^ : x 7! y^ðx
xÞ of
the true mappings y : x 7! yðx
xÞ, were learned on D TRAIN .

IEEE A&E SYSTEMS MAGAZINE

53
Aerospace and Electronic Systems - July 2019

Table of Contents for the Digital Edition of Aerospace and Electronic Systems - July 2019
