IEEE Computational Intelligence Magazine - February 2021 - 57

transition from the potential to the kinetic energy by the energy conversation law and exhibits a cyclic increment or decrement in sine forms. This analogy additionally reveals a hidden
relationship between stability and plasticity that can inspire
other suitable engineering mechanisms. Thereafter, we can
incorporate Hebbian and homeostatic plasticity mechanisms,
studied extensively in neuroscience [16], to regulate the
Dem-AI systems.
B. Self-Organizing Hierarchical Structuring Mechanism

Fig. 5(b) shows the self-organizing hierarchical structuring
mechanism that helps in constructing and maintaining the
structure of many levels of specialized groups. This structuring
process can be divided into three stages:
❏❏ Early-Stage (Hierarchical structure construction):
The lowest-level groups are created by grouping the agents
who perform a common learning task and have similar
characteristics in their learning models. A new level can
be created when the measure of distances among current
groups is greater than a threshold pre-defined for each level.
The structure can reach the maximum number of levels
which is defined in the Dem-AI meta-law. Alternatively, we
can extend existing hierarchical clustering algorithms [23],
clustering mechanisms for FL [15], [27] or game-theoretic
mechanisms [28]-[30].
❏❏ Adaptation Stage: The adaptation stage allows the learning agents to change their groups. When the level of group
plasticity is high, the measured distances among specialized
groups are short, and, as such, agents can move among these
groups.
❏❏ High Specialization Stage: The Dem-AI system allows
micro-adjustments in low-level groups due to the wellseparated and stable specialized groups that are al--
ready formed.
New learning agents first join in a suitable specialized group
in the top-level of the learning system. In time, these agents will
be admitted to lower-level specialized groups with whom they
share similarities in the learning characteristics. Consequently,
these agents leverage the existing group knowledge to speed up
their learning process. Furthermore, with the availability of new
data, the agents can contribute their valuable personalized
knowledge to improve the generalization capability of the
groups. Note that the metric of measured distance in the structuring mechanism can be derived from the differences in the
characteristics of learning agents or specialized groups (e.g., in
FL, the metric can be a Euclidean distance of model parameters,
gradients, or momentum of learning agents or groups). Therefore, the policy for the system to group different agents and
change groups can be defined by the threshold metrics, measured as the differences between the learning agents and between
the groups. The recent work in [31] provides a promising
approach to analyze the similarity of layers in the neural network
representation based on centered kernel alignment (CKA).
In swarm intelligence system, e.g., swarm robotics, the selforganized behaviors of a large number of robots can coordinate

with each other to design robust, scalable, and flexible collective
behaviors [18], which can be instrumental for our mechanism
design. Similarly, in addition to the development process of a
social structure, the other suitable mechanisms such as the
growth process in a biological cell can be incorporated. In biology, the solid complex composited structures such as DNA can
be separated after the initial coincidence period in the cell division process. For well-separated groups that are formed, the
agents who become excessively different through personalized
learning (e.g., different gradient and personalized model
parameters) or those who function poorly can be eliminated
from their groups. Analogously, such mechanisms can be found
in immune responses that destroy unhealthy cells (e.g., cancer
or virus-infected cells) [32]. However, we can also consider that
these agents behave as new learning agents and move towards
other suitable groups.
C. Specialized Learning Mechanism

Specialized learning facilitates the personalized and specialized
group learning capability using existing hierarchical generalized
knowledge that is represented in Fig. 6(a). For this mechanism,
we discuss the general design of personalized learning and specialized group learning, as well as related problems.
Personalized learning problem of learning agents: In
Dem-AI, a personalized learning problem can be constructed
for each learning agent with a personalized learning objective
(PLO) that comprises: 1) a personalized learning goal (PLG),
and 2) a reusable generalized knowledge (GK). For example,
PLG is the learning loss function, and GK is the regularizer
defined as the difference between the new model parameters
and the model parameters of the higher-level specialized
groups (e.g., FEDL [5], FedProx [13]), i.e.,
PLO = a t PLG + b t GK

= a t PLG + b t / 1(k) GK ^N (gk)h,
N
g
k

(1)

where N (gk) and GK (N (gk)) are the number of agents and the
generalized knowledge in the specialized group level k, respectively. The higher levels of generalized knowledge are less
important when solving any specific learning task of the agents
than the lower-level specialized knowledge. Since the specialized process is more important when improving the specialized
capability of the personalized learning model, the weight
parameter b t must be decreased in order to reduce the plasticity while a t must be increased according to the meta-law
design. If the learning agents cannot directly incorporate the
generalized knowledge (e.g., they do not have the same model
parameters), a special integration mechanism for the hierarchical structure of knowledge is required. Moreover, computing,
communication resource and delay constraints also need to be
considered in the learning problem. An example of the specialized learning problem using the proximal term to constrain the
local learning model, such that it be closer to the learning
model at the higher-level groups, is defined as follows:

FEBRUARY 2021 | IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE

IEEE Computational Intelligence Magazine - February 2021

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - February 2021