IEEE Computational Intelligence Magazine - August 2020 - 51

EIR
y1

y2

yT

DLC/DLR

FCN

Time Distributed FCN

Shared LSTM Space

+
: DLR/DLC Features
: Shared Features
: EIR Features

h1

h2

h3

hT

Shared LSTM

DAIC-WOZ Input

CMU-MOSEI Input

FIGURE 1 Fully-shared multi-task model (FS MT.). FCN stands for fully connected network, EIR for emotion intensity regression and DLC
(resp. DLR) for depression level classification (resp. depression level regression).

and dropout layers, whose output is fed to the task-specific
output layer.
This architecture, presented in Figure 2, improves over the
fully-shared multi-task architecture by providing an infrastructure that has separate spaces for task-specific and shared features. But this too may have drawbacks. The shared feature
space could contain some unnecessary task-specific features,
while some shared features could also be mixed with the private space, thus suffering from feature redundancy as shown on
the right side of Figure 2.
Adversarial Shared-Private Multi-Task (ASP MT.). Inspired by
the results obtained by [14], [28], we design a similar architecture
with two modifications. The adversarial shared-private multi-task
architecture consists of three LSTM networks, that is, two taskspecific and one shared, all of which have the same number of
units.The input of a task is fed to the task-specific and the shared
LSTM networks. The outputs from the task-specific and the
shared LSTM layers are then fused using the attention fusion
mechanism, oppositely to [14], [28], who use concatenation.
The output from the shared LSTM layer is also fed to a network N D of fully-connected dropout and softmax layers. This
network outputs the task label (for example, if there are two
tasks T1 and T2, the task label for T1 is [1, 0], the task label for
T2 is [0, 1]). The shared LSTM layers and N D act as an adversarial network, the shared LSTM layer acting as the generator
and N D acting as the discriminator.

Finally, a L diff loss function acts as an orthogonality constraint between private and shared layers and differs from the
one used in [14], [28]. It is defined in Equation 1, where $ 1
is the L1 norm, H and S are two matrices, whose rows are each
unit output of the task-specific LSTM network and the shared
LSTM network, respectively, and m and n are the first and second dimensions of H < S respectively. This definition of L diff
was empirically settled after testing other definitions. The architecture is shown in Figure 3.
L diff =

H 


IEEE Computational Intelligence Magazine - August 2020
Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - August 2020
Contents


IEEE Computational Intelligence Magazine - August 2020 - Cover1

IEEE Computational Intelligence Magazine - August 2020 - Cover2

IEEE Computational Intelligence Magazine - August 2020 - Contents

IEEE Computational Intelligence Magazine - August 2020 - 2

IEEE Computational Intelligence Magazine - August 2020 - 3

IEEE Computational Intelligence Magazine - August 2020 - 4

IEEE Computational Intelligence Magazine - August 2020 - 5

IEEE Computational Intelligence Magazine - August 2020 - 6

IEEE Computational Intelligence Magazine - August 2020 - 7

IEEE Computational Intelligence Magazine - August 2020 - 8

IEEE Computational Intelligence Magazine - August 2020 - 9

IEEE Computational Intelligence Magazine - August 2020 - 10

IEEE Computational Intelligence Magazine - August 2020 - 11

IEEE Computational Intelligence Magazine - August 2020 - 12

IEEE Computational Intelligence Magazine - August 2020 - 13

IEEE Computational Intelligence Magazine - August 2020 - 14

IEEE Computational Intelligence Magazine - August 2020 - 15

IEEE Computational Intelligence Magazine - August 2020 - 16

IEEE Computational Intelligence Magazine - August 2020 - 17

IEEE Computational Intelligence Magazine - August 2020 - 18

IEEE Computational Intelligence Magazine - August 2020 - 19

IEEE Computational Intelligence Magazine - August 2020 - 20

IEEE Computational Intelligence Magazine - August 2020 - 21

IEEE Computational Intelligence Magazine - August 2020 - 22

IEEE Computational Intelligence Magazine - August 2020 - 23

IEEE Computational Intelligence Magazine - August 2020 - 24

IEEE Computational Intelligence Magazine - August 2020 - 25

IEEE Computational Intelligence Magazine - August 2020 - 26

IEEE Computational Intelligence Magazine - August 2020 - 27

IEEE Computational Intelligence Magazine - August 2020 - 28

IEEE Computational Intelligence Magazine - August 2020 - 29

IEEE Computational Intelligence Magazine - August 2020 - 30

IEEE Computational Intelligence Magazine - August 2020 - 31

IEEE Computational Intelligence Magazine - August 2020 - 32

IEEE Computational Intelligence Magazine - August 2020 - 33

IEEE Computational Intelligence Magazine - August 2020 - 34

IEEE Computational Intelligence Magazine - August 2020 - 35

IEEE Computational Intelligence Magazine - August 2020 - 36

IEEE Computational Intelligence Magazine - August 2020 - 37

IEEE Computational Intelligence Magazine - August 2020 - 38

IEEE Computational Intelligence Magazine - August 2020 - 39

IEEE Computational Intelligence Magazine - August 2020 - 40

IEEE Computational Intelligence Magazine - August 2020 - 41

IEEE Computational Intelligence Magazine - August 2020 - 42

IEEE Computational Intelligence Magazine - August 2020 - 43

IEEE Computational Intelligence Magazine - August 2020 - 44

IEEE Computational Intelligence Magazine - August 2020 - 45

IEEE Computational Intelligence Magazine - August 2020 - 46

IEEE Computational Intelligence Magazine - August 2020 - 47

IEEE Computational Intelligence Magazine - August 2020 - 48

IEEE Computational Intelligence Magazine - August 2020 - 49

IEEE Computational Intelligence Magazine - August 2020 - 50

IEEE Computational Intelligence Magazine - August 2020 - 51

IEEE Computational Intelligence Magazine - August 2020 - 52

IEEE Computational Intelligence Magazine - August 2020 - 53

IEEE Computational Intelligence Magazine - August 2020 - 54

IEEE Computational Intelligence Magazine - August 2020 - 55

IEEE Computational Intelligence Magazine - August 2020 - 56

IEEE Computational Intelligence Magazine - August 2020 - 57

IEEE Computational Intelligence Magazine - August 2020 - 58

IEEE Computational Intelligence Magazine - August 2020 - 59

IEEE Computational Intelligence Magazine - August 2020 - 60

IEEE Computational Intelligence Magazine - August 2020 - 61

IEEE Computational Intelligence Magazine - August 2020 - 62

IEEE Computational Intelligence Magazine - August 2020 - 63

IEEE Computational Intelligence Magazine - August 2020 - 64

IEEE Computational Intelligence Magazine - August 2020 - 65

IEEE Computational Intelligence Magazine - August 2020 - 66

IEEE Computational Intelligence Magazine - August 2020 - 67

IEEE Computational Intelligence Magazine - August 2020 - 68

IEEE Computational Intelligence Magazine - August 2020 - 69

IEEE Computational Intelligence Magazine - August 2020 - 70

IEEE Computational Intelligence Magazine - August 2020 - 71

IEEE Computational Intelligence Magazine - August 2020 - 72

IEEE Computational Intelligence Magazine - August 2020 - 73

IEEE Computational Intelligence Magazine - August 2020 - 74

IEEE Computational Intelligence Magazine - August 2020 - 75

IEEE Computational Intelligence Magazine - August 2020 - 76

IEEE Computational Intelligence Magazine - August 2020 - Cover3

IEEE Computational Intelligence Magazine - August 2020 - Cover4


https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202311

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202308

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202305

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202302

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202211

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202208

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202205

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202202

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202111

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202108

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202105

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202102

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202011

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202008

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202005

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202002

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201911

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201908

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201905

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201902

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201811

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201808

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201805

https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201802

https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter17

https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall17

https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer17

https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring17

https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter16

https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall16

https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer16

https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring16

https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter15

https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall15

https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer15

https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring15

https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter14

https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall14

https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer14

https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring14

https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter13

https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall13

https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer13

https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring13

https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter12

https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall12


https://www.nxtbookmedia.com