IEEE Computational Intelligence Magazine - February 2023 - 46

time observation, the LDANE architectural approach will
increase the number of input and output neurons in the
autoencoder to adjust the input data size. Then, selective
retraining and expansion using the rehearsal principle will
retrain the weights affected by network changes. Autoencoder
split is conducted after the network embeddings are obtained
if needed. It aims to evaluate the stability of network embedding
over time by using the regularization method. The regularization
approach is employed to regulate the parameter if it
shifts too much from the original value and prevents forgetting.
Ifthe current features have significantly changed while
the training process is running, then the network needs to train
the weight again by splitting the autoencoder.
A multilabel classification case study is one of the
experiments that are used to calculate the performance of
LDANE. Some datasets are used, such as synthetic data
(SYN-1 and SYN-2) [104], DBLP [88], and Epinions.1
Some comparison methods for embedding models include
DeepWalk [85], DynGEM [40], and SDNE [105]. On multilabel
classification results, LDANE achieved the highest
average performance of micro-F1 0.93 and macro-F1 0.92
on the SYN-2 dataset. It performs better than other embedding
network models that do not employ lifelong learning
techniques.
C. TrafficStream
Chen et al. [44] proposed a framework of streaming traffic
flow forecasting called TrafficStream to address a specific
domain oftraffic flow that data structure is constantly evolving
in its nodes and data structure. Apart from the ways to enable a
lifelong learning setting, this type of method can be classified
as a task-specific method. Task-specific methods usually have
specific tasks that need to be achieved in order to get a good
performance in the desired domain. The model (or method) is
very focused on solving particular tasks and has no guarantee
to be applicable to other domains. In general lifelong learning,
examples of task-specific methods include NELL (never-ending
language learner) [13] and LSC (lifelong sentiment
classification) [106].
In order to address that challenge, the easiest way is to
retrain the model from scratch regularly at a particular time,
such as weekly, monthly, or yearly. However, it is considered
very costly and inefficient for computing resources because it
incorporates a massive amount of data when it comes sequentially.
Moreover, because ofnetwork expansions, the topology
and architecture of the previous ones will have different patterns,
so the spatial dependency ofprevious models will not be
relevant again. The previous insight from learned knowledge
needs to be consolidated and preserved to help increase the
model's performance. Based on those challenges, the TrafficStream
framework aims to efficiently and effectively capture
patterns for new graph structures and consolidate the
1www.epinions.com
knowledge between the previous and the new ones. In its
framework, TrafficStream uses a simple traffic flow forecasting
model (SurModel) to represent a complex graph data model in
the intelligent transportation system [53]. There are two component
processes that efficiently update the current network
pattern in the TrafficStream method: expanding network
training and evaluation pattern detection. Those are based on
the changes in network structures due to several conditions,
such as adding new sensors or stations to record traffic flow
data. The primary objective of expanding the training process
is to reduce the complexity of capturing dependency between
nodes in the graph. With vast amounts of attributes and a
number of nodes, it will increase the processing task in the
GNN layer. So that TrafficStream introduces the model to
generate a sub-graph by reducing the number of included
nodes to only 2-hops neighbors in a node. This mechanism
claims to be effective in increasing the speed time ofprocessing
in GNN. The second component is evolution pattern detection,
which aims to detect the different patterns on a graph
entirely different from the previous one. This process is beneficial
in recognizing the shifting ofpoint ofinterest (POI) so that
the model can selectively choose the nodes that have significant
changes to be learned.
The essence ofactivating lifelong learning in this method is
to consolidate knowledge to avoid catastrophic forgetting. For
this reason, the model tracks back to the previous data and
applies two strategies: information replay and smoothing
parameters. Information replay performs the training process
using sample data from previous information. The selection of
sample data is simple by selecting randomly from several nodes.
With the proposed JS divergence algorithm, the model chooses
the nodes with lowerJDS scores to be selected and used to
construct sub-graphs. The second strategy is parameter
smoothing, which aims to maintain shift information, resulting
in catastrophic forgetting. The method used to perform
parameter smoothing refers to the EWC method [19] as
defined below:
Lsmooth ¼
X
i
FiðCTðiÞCT1ðiÞÞ2;
(14)
where denotes the weight of the smoothing term, and Fi
refers to the importance of the i-th parameter in the previous
function model CTðiÞ that can be estimated as follow:
F ¼
jXT1j x2XT1
1 X
½gðCT1; xÞgðCT1; xÞT;
where g refers to first-order derivatives of the loss. XT1
(15)
is a
flow data of all nodes on previous graph GT1. Through that
mechanism, the weight with fewer essential parameters is
smaller, and those parameters can adapt to the new patterns.
On the other hand, more important parameters will have a
larger weight in the weight-smoothing process so that changes
can be limited to maintain previous knowledge.
46 IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE | FEBRUARY 2023
http://www.epinions.com
IEEE Computational Intelligence Magazine - February 2023

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - February 2023
