IEEE Computational Intelligence Magazine - August 2020 - 51
EIR
y1
y2
yT
DLC/DLR
FCN
Time Distributed FCN
Shared LSTM Space
+
: DLR/DLC Features
: Shared Features
: EIR Features
h1
h2
h3
hT
Shared LSTM
DAIC-WOZ Input
CMU-MOSEI Input
FIGURE 1 Fully-shared multi-task model (FS MT.). FCN stands for fully connected network, EIR for emotion intensity regression and DLC
(resp. DLR) for depression level classification (resp. depression level regression).
and dropout layers, whose output is fed to the task-specific
output layer.
This architecture, presented in Figure 2, improves over the
fully-shared multi-task architecture by providing an infrastructure that has separate spaces for task-specific and shared features. But this too may have drawbacks. The shared feature
space could contain some unnecessary task-specific features,
while some shared features could also be mixed with the private space, thus suffering from feature redundancy as shown on
the right side of Figure 2.
Adversarial Shared-Private Multi-Task (ASP MT.). Inspired by
the results obtained by [14], [28], we design a similar architecture
with two modifications. The adversarial shared-private multi-task
architecture consists of three LSTM networks, that is, two taskspecific and one shared, all of which have the same number of
units.The input of a task is fed to the task-specific and the shared
LSTM networks. The outputs from the task-specific and the
shared LSTM layers are then fused using the attention fusion
mechanism, oppositely to [14], [28], who use concatenation.
The output from the shared LSTM layer is also fed to a network N D of fully-connected dropout and softmax layers. This
network outputs the task label (for example, if there are two
tasks T1 and T2, the task label for T1 is [1, 0], the task label for
T2 is [0, 1]). The shared LSTM layers and N D act as an adversarial network, the shared LSTM layer acting as the generator
and N D acting as the discriminator.
Finally, a L diff loss function acts as an orthogonality constraint between private and shared layers and differs from the
one used in [14], [28]. It is defined in Equation 1, where $ 1
is the L1 norm, H and S are two matrices, whose rows are each
unit output of the task-specific LSTM network and the shared
LSTM network, respectively, and m and n are the first and second dimensions of H < S respectively. This definition of L diff
was empirically settled after testing other definitions. The architecture is shown in Figure 3.
L diff =
H
IEEE Computational Intelligence Magazine - August 2020
Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - August 2020
Contents
IEEE Computational Intelligence Magazine - August 2020 - Cover1
IEEE Computational Intelligence Magazine - August 2020 - Cover2
IEEE Computational Intelligence Magazine - August 2020 - Contents
IEEE Computational Intelligence Magazine - August 2020 - 2
IEEE Computational Intelligence Magazine - August 2020 - 3
IEEE Computational Intelligence Magazine - August 2020 - 4
IEEE Computational Intelligence Magazine - August 2020 - 5
IEEE Computational Intelligence Magazine - August 2020 - 6
IEEE Computational Intelligence Magazine - August 2020 - 7
IEEE Computational Intelligence Magazine - August 2020 - 8
IEEE Computational Intelligence Magazine - August 2020 - 9
IEEE Computational Intelligence Magazine - August 2020 - 10
IEEE Computational Intelligence Magazine - August 2020 - 11
IEEE Computational Intelligence Magazine - August 2020 - 12
IEEE Computational Intelligence Magazine - August 2020 - 13
IEEE Computational Intelligence Magazine - August 2020 - 14
IEEE Computational Intelligence Magazine - August 2020 - 15
IEEE Computational Intelligence Magazine - August 2020 - 16
IEEE Computational Intelligence Magazine - August 2020 - 17
IEEE Computational Intelligence Magazine - August 2020 - 18
IEEE Computational Intelligence Magazine - August 2020 - 19
IEEE Computational Intelligence Magazine - August 2020 - 20
IEEE Computational Intelligence Magazine - August 2020 - 21
IEEE Computational Intelligence Magazine - August 2020 - 22
IEEE Computational Intelligence Magazine - August 2020 - 23
IEEE Computational Intelligence Magazine - August 2020 - 24
IEEE Computational Intelligence Magazine - August 2020 - 25
IEEE Computational Intelligence Magazine - August 2020 - 26
IEEE Computational Intelligence Magazine - August 2020 - 27
IEEE Computational Intelligence Magazine - August 2020 - 28
IEEE Computational Intelligence Magazine - August 2020 - 29
IEEE Computational Intelligence Magazine - August 2020 - 30
IEEE Computational Intelligence Magazine - August 2020 - 31
IEEE Computational Intelligence Magazine - August 2020 - 32
IEEE Computational Intelligence Magazine - August 2020 - 33
IEEE Computational Intelligence Magazine - August 2020 - 34
IEEE Computational Intelligence Magazine - August 2020 - 35
IEEE Computational Intelligence Magazine - August 2020 - 36
IEEE Computational Intelligence Magazine - August 2020 - 37
IEEE Computational Intelligence Magazine - August 2020 - 38
IEEE Computational Intelligence Magazine - August 2020 - 39
IEEE Computational Intelligence Magazine - August 2020 - 40
IEEE Computational Intelligence Magazine - August 2020 - 41
IEEE Computational Intelligence Magazine - August 2020 - 42
IEEE Computational Intelligence Magazine - August 2020 - 43
IEEE Computational Intelligence Magazine - August 2020 - 44
IEEE Computational Intelligence Magazine - August 2020 - 45
IEEE Computational Intelligence Magazine - August 2020 - 46
IEEE Computational Intelligence Magazine - August 2020 - 47
IEEE Computational Intelligence Magazine - August 2020 - 48
IEEE Computational Intelligence Magazine - August 2020 - 49
IEEE Computational Intelligence Magazine - August 2020 - 50
IEEE Computational Intelligence Magazine - August 2020 - 51
IEEE Computational Intelligence Magazine - August 2020 - 52
IEEE Computational Intelligence Magazine - August 2020 - 53
IEEE Computational Intelligence Magazine - August 2020 - 54
IEEE Computational Intelligence Magazine - August 2020 - 55
IEEE Computational Intelligence Magazine - August 2020 - 56
IEEE Computational Intelligence Magazine - August 2020 - 57
IEEE Computational Intelligence Magazine - August 2020 - 58
IEEE Computational Intelligence Magazine - August 2020 - 59
IEEE Computational Intelligence Magazine - August 2020 - 60
IEEE Computational Intelligence Magazine - August 2020 - 61
IEEE Computational Intelligence Magazine - August 2020 - 62
IEEE Computational Intelligence Magazine - August 2020 - 63
IEEE Computational Intelligence Magazine - August 2020 - 64
IEEE Computational Intelligence Magazine - August 2020 - 65
IEEE Computational Intelligence Magazine - August 2020 - 66
IEEE Computational Intelligence Magazine - August 2020 - 67
IEEE Computational Intelligence Magazine - August 2020 - 68
IEEE Computational Intelligence Magazine - August 2020 - 69
IEEE Computational Intelligence Magazine - August 2020 - 70
IEEE Computational Intelligence Magazine - August 2020 - 71
IEEE Computational Intelligence Magazine - August 2020 - 72
IEEE Computational Intelligence Magazine - August 2020 - 73
IEEE Computational Intelligence Magazine - August 2020 - 74
IEEE Computational Intelligence Magazine - August 2020 - 75
IEEE Computational Intelligence Magazine - August 2020 - 76
IEEE Computational Intelligence Magazine - August 2020 - Cover3
IEEE Computational Intelligence Magazine - August 2020 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202311
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202308
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202305
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202302
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202211
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202208
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202205
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202202
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202111
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202108
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202105
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202102
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202011
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202008
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202005
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202002
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201911
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201908
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201905
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201902
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201811
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201808
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201805
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201802
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter12
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall12
https://www.nxtbookmedia.com