IEEE Computational Intelligence Magazine - February 2023 - 49

A. Uncertain Neighboorhoods
In learning the representation of a graph, every node relationship
should be converted into a lower-dimensional vector
space so that it can be further processed in solving downstream
tasks [26]. The learning representation process involves changing
node attributes' information with its neighbors. Referring
to predefined graphs with fixed structures, feature embedding
on nodes will be easier to conduct, so that neighborhood
information is more stable for the learning process as is done in
conventional GNNs methods [27], [35]. However, in graph
lifelong learning, although some approach explicitly addresses
the problem of the continual embedding process [68], [71],
[72], it still requires an optimal strategy for dynamic embedding
on graphs with uncertain neighborhood conditions.
Graph lifelong learning should be able to capture new relations
and nodes and consider when the right time to do the embedding
process to accommodate data instances in order to
improve the algorithms' scalability and efficiency.
B. Extreme Evolution
The evolution of graph data is complicated to understand.
Sometimes the data graph turns out to be more complex but
still retains most ofthe old structure. On the other hand, there
are types of graph networks with extreme evolution, which
means the graph structure can drastically evolve over time, and
most of the old forms are extinct from the graph data. Moreover,
when the graph data has an extreme evolution characteristic,
it is likely that the previously learned tasks no longer need
to be maintained. This factor in the graph also needs to be
considered when implementing a graph lifelong learning. The
model expects to judge whether it needs to minimize forgetting
the previous tasks or let the model forget the previous
tasks completely that are no longer relevant to the currently
observed graph representation.
C. Global Dependencies Learning
Learning global context or global dependencies in graph data is
very important to understand the entire relationships between
nodes. It can capture the information of two nodes even if
they stay apart in a large-scale graph. Current progress in learning
global dependency optimally in a static graph has achieved
state-of-the-art performances, for example, using transformer
mechanisms that elevated sequential learning with novel positional
encoding implemented in graph data [115]. In graph
lifelong learning, passing the information of new instances
across all the nodes and edges in learning the global context of
graphs incrementally is still challenging. Further improvement
is needed to learn global dependency efficiently in a continual
manner, especially in a large-scale graph, to improve graph
lifelong learning methods.
D. Comparative Benchmarking
Current graph lifelong learning strategies are applied in different
scenarios, such as focusing on adding new instances
(NI), new classes (NC), or considering both new instances
and classes (NIC). Moreover, it has different applications,
such as node classification, graph prediction, edge prediction,
or graph embedding enrichment. Nowadays, benchmarking
strategies are highly non-standard. It is essential to
be considered as further research in addition to the current
studies to bring fair comparations and improve graph lifelong
learning methods.
E. Model Training Parallelisation
Most graph lifelong learning techniques are run on singlepoint
computational resources. However, when looking at the
current practical implementation, most ofthe data comes from
a distributed architecture that is deployed on spread-edge devices
using cloud services. Collecting all data on a single point
computational resource sometimes brings other limitations on
security, performance, and costs. To address the emergence of
more intelligent services in a distributed architecture, another
open challenge of graph lifelong learning is to enable a federated
learning paradigm [116], [117] that can perform computation
in edge devices using local models parallelly and aggregate
them to the single-point device into a general model [118].
That concept will be beneficial because representations of
nodes or relations ofgraphs and new classes can appear in local
points ofdistributed architecture.
X. CONCLUSION
This survey article overviews graph lifelong learning, an
emerging field at the intersection of graph learning and lifelong
learning. The motivations to enable lifelong learning
settings on graph data and several potential applications of
graph lifelong learning in several fields, such as social networks,
traffic prediction, recommender systems, and anomaly
detection, are discussed in depth. This paper covers the
state-of-the-art methods in graph lifelong learning, which
are categorized into several groups: architectural approach,
rehearsal approach, regularization approach, and hybrid
approach. It also presents method comparisons by considering
some computational aspects in each categorization. All
methodsare compared based oneachscenarioand application.
Moreover, explanations of benchmarking settings and
candidate datasets for testing graph lifelong learning models
are also provided. Apart from the catastrophic forgetting
problem, several open issues of graph lifelong learning, such
as uncertain neighborhoods, extreme evolution, global
dependencies learning, fair benchmarking, class imbalance,
and model training parallelization, are discussed as future
research directions. It is worth noting that many common
challenges facing general machine learning are also applicable
to graph lifelong learning, including, e.g.,fairnessand
bias, explainability/interpretability, privacy and security, and
data trust/quality. Overall, our survey work facilitates new
interests and works in the design and development of innovative
theory, models/algorithms, and applications of graph
lifelong learning.
FEBRUARY 2023 | IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE 49
IEEE Computational Intelligence Magazine - February 2023

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - February 2023
