IEEE Computational Intelligence Magazine - February 2023 - 33

I. INTRODUCTION
Addressing catastrophic forgetting is one of the most
G
raph data is ubiquitous in areas such as social networks,
biological networks, road networks, and
computer networks [1], [2]. Graph learning has
shown promising performance in deriving inferences
from such graph-structured data [3], and in modelling
and understanding problems such as drug side effects [4],
knowledge graphs [5], social network analysis [6], recommender
systems [7], and physical movement prediction [8].
Despite their success, current graph learning approaches
are limited in the sense that they can only accommodate
predefined tasks [9], [10]. A challenge emerges in how to
address new tasks that arise sequentially in graph-based
problems, such as the addition of new nodes as the graph
grows over time and the inclusion of new class labels in
classification tasks. Consider, for example, the problem of
classifying the topic of each paper in a dataset of scientific
publications. Initially, we have a collection of papers covering
a fixedset of topics that canbelinkedtoeachother via
their shared mutual information and thereby represented as
a graph. Over time, the number of nodes (which represent
papers) and their relations can expand, and new class labels
(which represent the topic of the paper) can emerge. When
this happens, the model'sexistingknowledge cannotbe
used to classify the newly appeared classes because those
emerging instances and classes are completely new. A more
advanced technique is thus needed to accommodate knowledge
from new data and new class labels at different points
in time. Another example is traffic flow prediction. Once
new locations and new roads appear on a map, a more
advanced method is needed to accommodate the new
instances and refine the previous experiences to result in
good traffic flow prediction in both old and new representations.
The focus of this survey paper is a learning model,
called graph lifelong learning, with the ability to learn new
knowledge sequentially in graph-based problems.
Lifelong learning, also known as continual learning [11],
incremental learning [12], and never-ending learning [13],
is a learning characteristic that aims to imitate the human
ability to perform continual learning and transfer, acquire,
and refine knowledge throughout one's lifetime [14]. A lifelong
learning model should be able to recognize the similarity
between upcoming and previous knowledge. When a
new task is very similar to the prior experience, the learning
agent should improve the existing model to perform better.
Meanwhile, when a new task has little similarity or correlation
with the previous experiences, the system can transfer
the knowledge to address the new tasks [15]. Most popular
deep learning algorithms do not have the ability to incrementally
learn incoming new information, and this leads to
catastrophic forgetting or interference problems [16], [17],
[18]. In particular, new information interferes with previous
knowledge, causing a decrease in the model'sperformance
on the old tasks because the new one is overwriting it, as
shown in Fig. 1.
important goals of lifelong learning. Prior works have been
proposed to enable lifelong learning and address catastrophic
forgetting in problems where the input is in the form ofa regular
structure that has clear order of data, such as images or
text data [15], [19], [20], [21], [22], [23], [24], [25]. However,
general lifelong learning approaches used in these domains do
not directly generalize to graphs. This is because graphs don't
have a unique node ordering or a canonical vector representation,
which makes it challenging to define a distance measure
between components [3]. Graph representation learning [26]
as a canonical framework aims to address this issue by capturing
the structure of nodes, edges, and subgraphs and converting
them into low-dimensional vectors. While general
lifelong learning can accommodate new knowledge added to
the model, it does not address the mechanisms of representation
learning, which is the main component of graph learning
frameworks. There are also challenges in adapting the existing
graph representation approaches to lifelong learning. Since
conventional approaches such as graph GCNs [27] require the
entire graph to perform representation learning, this can
introduce efficiency problems when we force such approaches
to perform incremental learning.
In order tosuccessfully applylifelonglearninginthe
graph domain, existing lifelong learning approaches must
be adapted into the specific graph lifelong learning methods
that consider the irregular graph structure and use
complete components. This requires three main points of
uniqueness to be addressed when compared to the traditional
lifelong learning:
❏ The ability to detect new instances or tasks in the graph.
❏ The ability to accommodate new knowledge in graphs
based on additional data instances and/or classes.
❏ Performing graph representation learning efficiently and
accommodating new knowledge without requiring the use
ofthe entire graph structure.
FIGURE 1 Illustration of catastrophic forgetting of Task 1 when Tasks 2
and 3 are added.
FEBRUARY 2023 | IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE 33
IEEE Computational Intelligence Magazine - February 2023

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - February 2023
