IEEE Computational Intelligence Magazine - February 2023 - 39

TABLE II Graph lifelong learning method comparison.
METHODS
APPROACHES
Feature Graph Networks (FGN) [56]
Hierarchical Prototype Networks (HPNs) [67]
Experience Replay GNN Framework (ER-GNN) [9]
Lifelong Open-world Node Classification (LONC) [36]
Disentangle-based Continual graph Representation
learning (DiCGRL) [68]
Graph Pseudo Incremental Learning (GPIL) [69]
Topology-aware Weight Preserving (TWP) [70]
Translation-based Knowledge Graph Embedding
(TKGE) [71]
ContinualGNN [10]
Lifelong Dynamic Attributed Network Embedding
(LDANE) [72]
TrafficStream [44]
ARCHITECTURAL REHEARSAL REGULARIZATION
Yes
Yes
No
No
No
No
Yes
Yes
No
No
No
No
No
Yes
No
lifelong learning works covered in this survey paper are
divided into four categories: architectural, regularization,
rehearsal, and hybrid, as shown in Fig. 3. The specification of
each method is described in Table II.
III. ARCHITECTURAL APPROACH
In the general lifelong learning setting, this approach focuses
on modifying the specific architecture of networks, activation
functions, or layers ofalgorithms to address a new task and prevent
the forgetting of previous tasks [15], [21], [22]. In the
graph lifelong learning scenario, this approach relies on changing
the graph structure, expanding more units, and performing
compression techniques. Some examples are FGN [56], which
convert graph data architecture into regular learning problems,
and HPNs [67], which extract different level abstractions of
prototypes to accommodate new knowledge.
A. Feature Graph Networks
In a graph learning process, the full adjacency matrix or
the entire graph topology information is needed to propagate
node features across all layers. Sampling techniques
No
No
No
No
Yes
Yes
Yes
No
No
No
No
Yes
Yes
Yes
Yes
Yes
Yes
Yes
CODE LINKS
https://github.com/wang-chen/LGL
-
-
https://github.com/lgalke/lifelonglearning
https://github.com/KXY-PUBLIC/DiCGRL
https://github.com/zhen-tan-dmml/gfcil
https://github.com/hhliu79/TWP
-
https://github.com/Junshan-Wang/
ContinualGNN
-
https://github.com/AprLie/TrafficStream
have
been proposed to scale GCNs to large graphs [35],
[73], however GCNs still require a costly pre-processing
step that includes the entire graph dataset, which is not
suitable for incremental learning in lifelong learning. In
particular, the topology of graphs provides unique challenges
for lifelong learning. This has motivated the proposal
of a new graph topology that is suitable for general
lifelong learning approaches.
Wang et al. [56] developed the FGN approach to convert
graph architectures into a representation that can be
ingested by regular learning architectures. Fig. 4 shows
how FGN converts regular graph representations into feature
graphs. In particular, FGN represents nodes from the
original graph as independent graphs, and the features from
the original graph as nodes. This technique enables lifelong
learning approaches based on CNNs architectures to be
applied to solve GNN tasks.
FIGURE 3 Graph lifelong learning categorization.
FIGURE 4 Illustration of FGN. It changes the structure of graphs. In the
regular graph representation shown in (a), the node a has neighbors
NðaÞ¼fa; b; c; d; eg and feature vector xa ¼½1; 0; 0; 1. FGN converts the
regular graph so that the features of node a will be represented as
nodes fa1; a2; a3; a4g in the Feature Graph shown in (b), connected to
each other via cross-correlation establishment. [56]. (a) Regular graph.
(b) Feature graph.
FEBRUARY 2023 | IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE 39

https://github.com/wang-chen/LGL https://github.com/lgalke/lifelong-learning https://github.com/lgalke/lifelong-learning https://github.com/KXY-PUBLIC/DiCGRL https://github.com/zhen-tan-dmml/gfcil https://github.com/hhliu79/TWP https://github.com/Junshan-Wang/ContinualGNN https://github.com/Junshan-Wang/ContinualGNN https://github.com/AprLie/TrafficStream

IEEE Computational Intelligence Magazine - February 2023

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - February 2023