IEEE Computational Intelligence Magazine - February 2023 - 47

TABLE III Summarization of existing graph lifelong learning approaches.
METHODS
FGN [56]
APPLICATIONS
classification
HPNs [67]
ER-GNN [9]
LONC [36]
DiCGR [68]
GPIL [69]
TWP [70]
TKGE [71]
classification
classification
classification
GRAPH LEARNING MODULES SCENARIOS
METRICS
graph convolution, graph
attention
new instances & new
classes (NIC)
atomic structure embedding new instances & new
classes (NIC)
graph neural network
graph neural network
link prediction, node classification knowledge graph embedding,
graph attention
few-shot node classification
node and graph classification
knowledge graph enrichment,
link prediction, triple classification
ContinualGNN [10] classification
LDANE [72]
network representation enrichment,
graph reconstruction, link prediction,
node classification
TrafficStream [44] traffic flow forecasting
graph convolution
graph neural network
new instances & new
classes (NIC)
new classes (NC)
average accuracy, forgetting measure
average accuracy, forgetting measure
average accuracy, forgetting measure
average accuracy, forward transfer
new instances (NI) average top-10 ranked entities (H@
10) & average accuracy
new instances & new
classes (NIC)
new classes (NC)
average accuracy, dropping rate
average accuracy, forgetting measure
knowledge graph embedding new instances (NI) average top-10 ranked entities (H@
10)
graphSAGE
network embedding
graph neural network
The experiments are conducted to verify the model
with traffic forecasting scenarios. It uses a real-world dataset
called PEMS3-Stream [107]. Several traffic forecasting
methods use a retraining strategy as a comparison baseline,
STGCN [53], spatial-temporal synchronous graph convolutional
network (STSGCN) [108], gated recurrent unit
(GRU) [109], etc. The conclusion shows that TrafficStream
gets high prediction accuracy compared with the traditional
retraining strategies, and also it is proven to reduce training
complexity by using a continual learning strategy in graphbased
tasks.
VII. METHOD COMPARISONS
Comparing current methods of graph lifelong learning still
remains an open issue. Although graph lifelong learning
focuses on accommodating new knowledge, preventing catastrophic
forgetting, and considering graph representation
learning, current methods have various scenarios and applications.
Those scenarios can be grouped into three different
types: 1) only focusing on new instances (NI), 2) new classes
(NC), or 3) both new instances and new classes (NIC).
Moreover, each model has different applications, such as
node prediction, link prediction, node embedding, and
others.
This section summarizes the considerations for each categorization
of lifelong learning. For the architectural approach,
The models adjust their complexity by allocating prototypes
like HPNs [67], and expansions in LDANE [72]. The model
should consider memory allocations and efficiency aspects
with those expansion mechanisms to prevent a technical bottleneck.
In the regularization approach, the way to allocate
new knowledge is by managing the shared solution space to
new instances & new
classes (NIC)
average accuracy
new instances (NI) average precision
new instances (NI) mean errors
be applicable to all tasks. However, managing the constraints
properly on model weight should be considered because new
tasks are forced to adapt to the previous knowledge, resulting
in no maximum performance to address new tasks. For the
rehearsal approach, some samples from prior experiences are
replayed to strengthen memory and maintain performance to
complete previous tasks. Using maximum data resources
should be avoided because it will involve very complex data
and attributes in the graph. Therefore, considering computational
speed is very important when retraining past knowledge
so that the model does not require extensive time complexity.
The strategy ofselecting nice and small samples ofdata, nodes,
and topological structures in graph data involved through the
replay process should be well considered to minimize the time
complexity of the learning process. An automated algorithm
should be able to select the nice samples automatically to
enhance the quality ofthe retraining process [110]. Moreover, a
strategy to determine when to activate the replay process is also
needed not to run it every time to be more efficient. For the
hybrid method, even though it will take full advantage ofspecific
approaches ofarchitecture, regularization, and rehearsal and has a
more complex framework, it needs to address multiple considerations
across different approaches. All models that have been
reviewed have their unique characteristics based on the scenario
and its applications. Based on those unique characteristics, the
summary ofthe models' differences can be seen in Table III.
VIII. BENCHMARKS
This section briefly explains some of the available datasets,
experimental details, and how to calculate performance on
graph lifelong learning.
FEBRUARY 2023 | IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE 47
IEEE Computational Intelligence Magazine - February 2023

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - February 2023
