IEEE Computational Intelligence Magazine - May 2021 - 37

where W o ! R Ri = 1 Fi # no and b o ! R no respectively denote the
weight matrix and bias of the classifier.
Finally, the cross-entropy loss, L, that is used for network
training, is calculated by the prediction yt and the label y:
B

L = CE (y, yt ) = - / y (b) log yt (b), (5)

b=1

where B and CE($ , $) respectively denote the mini-batch sizes
and the cross-entropy loss function, and yt (b) and y (b) denote
the prediction and ground-truth label for the b-th training
sample in the mini-batch1.
IV. Experiments

In this section, we describe the datasets used for performance
evaluation, our experimental settings, and baseline settings. Furthermore, we present the performance of our method and
competing methods.
A. Datasets and Preprocessing

In this study, we used five different publicly available datasets to
validate the proposed method on four different EEG data paradigms. Table I shows a brief summary of the datasets used in
this work.
1) Motor Imagery (subject-dependent)
First, we used two big datasets for MI EEGs, GIST-MI [40]2
and KU-MI [8]3,4. The GIST-MI dataset consists of two different MI tasks: left-hand and right-hand MI that are acquired
from 52 subjects. All EEG signals were recorded from 64 Ag/
AgCl electrode channels according to the standard 10-20 system, sampled at 512 Hz. Each class contained 100 or 120 trials,
and each trial was a 3 sec long MI task. Since this dataset is not
separated into training and test samples, we conducted a fivefold cross-validation for a fair evaluation. Additionally, the
KU-MI dataset is acquired across two sessions from 54 subjects,
recorded from 62 Ag/AgCl electrodes according to the stan1

All codes used in our experiments are available at 'https://github.com/DeepBCI/
Deep-BCI/tree/master/1_Intelligent_BCI/Multi_Scale_Neural_Network_for_EEG
_Representation_Learning_in_BCI.'
2
Available at http://gigadb.org/dataset/100295
3
Available at http://gigadb.org/dataset/100542
4
Experimental results of the KU-MI dataset are reported in Supplementary B.

dard 10-20 system, and sampled with 1000 Hz. Each class of
this dataset contains 50 trials with 4 sec long of MI. The
KU-MI dataset is originally separated by the training and testing samples, thus we used it directly as training and testing sets.
For both MI datasets, we preprocessed signals by applying a
large Laplacian filtering,5 baseline correction by subtracting the
mean value of the fixation signal from each MI trial, and bandpass filtering between 4 and 40 Hz with 4th order Butterworth
filter. Then, we removed parts in the first and last 0.5 sec from
each trial, and finally applied Gaussian normalization. We
applied the same mean and standard deviation values for normalization to the test samples. The multi-channel EEG signals
were only shifted and scaled by their respective channel-wise
mean and standard deviation values. Thus, inter-channel correlations inherent in the data were preserved.
2) Motor Imagery (subject-independent)
In this work, we mostly focused on developing a subjectdependent BCI method. However, since spontaneous EEG, e.g.,
MI signals, from different subjects are highly variable [27], [29],
developing a deep learning-based subject-independent MIbased BCI method has gained lots of interest recently by pioneering research [17], [27].
In this respect, we conducted extra experiments in the subject-independent manner over KU-MI dataset [8]. In detail, by
following [27], we used a leave-one-subject-out cross-validation
for the performance evaluation. We trained our proposed
MSNN using all source subjects' samples and evaluated on the
target subject's test samples in session 2.
3) Steady-State Visually Evoked Potentials
We also used the KU-SSVEP dataset [8]3 for SSVEP decoding
experiments in this study. This KU-SSVEP dataset was
acquired from 54 subjects and recorded from 62 Ag/AgCl
electrode channels using the 10-20 system. The KU-SSVEP
dataset contains four EEG classes with 4 sec long from target
stimuli at 5.45, 6.67, 8.57, and 12 Hz, and each class has 25
EEG trials of training and testing samples for each session. As it
is reported that the length of used data has a significant influence on the performance of the SSVEP detection, we have also
5

When the target channel does not have four nearest neighbors, we just used available
channels and their average value to filter the target channel.

TABLE I A brief summary of datasets used in our work. Note that CHB-MIT [12] dataset has different numbers of records for each
subject, thus the mean of records are reported at '# sessions.' For fair comparison, we evaluated GIST-MI [40] and SEED-VIG [36] by
5-fold cross-validation (CV) and CHB-MIT [12] by leave-one-record-out (LORO)-CV. Finally, KU-MI and KU-SSVEP [8] datasets are
provided with disjoint training/testing samples, so we used them directly.
DATASET

# SUBJECTS

# SESSIONS

# TOTAL
TRIALS

# ELECTRODES

SAMPLING
RATE [HZ]

TRIAL
TIME [SEC]

EVALUATION
STRATEGY

GIST-MI [40]

100 or 120

512

5-FOLD CV

KU-MI & SSVEP [8]

200

1000

TRAIN/TEST

SEED-VIG [36]

885

200

5-FOLD CV

CHB-MIT [12]

26.58

256

LORO-CV

MAY 2021 | IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE

https://github.com/DeepBCI/Deep-BCI/tree/master/1_Intelligent_BCI/Multi_Scale_Neural_Network_for_EEG_Representation_Learning_in_BCI https://github.com/DeepBCI/Deep-BCI/tree/master/1_Intelligent_BCI/Multi_Scale_Neural_Network_for_EEG_Representation_Learning_in_BCI https://github.com/DeepBCI/Deep-BCI/tree/master/1_Intelligent_BCI/Multi_Scale_Neural_Network_for_EEG_Representation_Learning_in_BCI http://www.gigadb.org/dataset/100295 http://www.gigadb.org/dataset/100542

IEEE Computational Intelligence Magazine - May 2021

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - May 2021