IEEE Computational Intelligence Magazine - May 2021 - 44

E. Early Seizure Detection

Early detection [37] of seizures is one of the most important
potential practical applications for this work. Hence, we also
validated the benefits of the proposed method in early seizure
detection. Specifically, in the training phase, the MSNN was
trained using normal and ictal EEG samples with binary labels
(e.g., 0: normal and 1: seizure) similar to a conventional training
framework. In the testing phase, we input the EEG samples
using a sliding window with a 1/256 stride. Then, we observed
the change in the output probability values to determine the
character of the input (normal or ictal).
Additionally, we visualized these changes in Fig. 5(d) (We
used the first subject's third EEG trial in the CHB-MIT dataset
[12] for the visualization). In Fig. 5(d), magenta-colored dotdashed lines denote the seizure onset and offset. Colored solid
lines denote the probability change of various methods. In this
visualization, we observed that the proposed method is more
stable for detecting seizures. Specifically, the proposed method
detects the seizure EEG signal as a seizure state with a strong
probability (almost 1), whereas the other methods have low
confidence values (Shoeb and Guttag [14]'s method and ESTCNN [26]) or even make incorrect decisions regarding the seizure state (Shallow ConvNet [3], Deep ConvNet [3], RSTNN
[18], and EEGNet [4]).
F. Towards EEG Representation Method

In our work, a distinguished strength of the proposed MSNN
is a reasonable discriminative ability in various paradigms. It is
worth noting that this strength can be effectively used in newlydesigned BCI protocols or paradigms for which we do not have
any existing reasonably-empowered models for classification or
known neurophysiological background useful for model building. Additionally, to suppress heuristic approaches in the studies,
combining the proposed network with automated machine
learning techniques [43] can maximize its usefulness.
To show this, we designed and conducted additional experiments. First, we used our MSNN for a P300-based online
home application control BCI system [44]. The P300 dataset
consists of two different tasks: target and non-target P300 that
TABLE IV Performance evaluation on the P300 dataset [44].
The Method column denotes a baseline [44] and the
proposed method. * denotes evaluation performances taken
from [44].
METHOD

MEAN±STD.

MAX

MIN

PCA + SVM [44]

.83±.18

1.00

.40*

MSNN (PROPOSED)

.86±.17

1.00

.43

TABLE V Performance evaluation on the imagined speech
dataset. The Method row denotes baselines and the
proposed method.
METHOD

SHALLOW
CONVNET [3]

DEEP
CONVNET [3] EEGNET [4]

MSNN
(PROPOSED)

ACCURACY

.51±.07

.64±.12

.67±.09

.63±.08

IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE | MAY 2021

are acquired from 30 subjects, recorded from 31 electrode
channels according to the standard 10-20 system. In the experiment, the MSNN was trained with raw EEG signals of 10 repetitions. As summarized in Table IV, the proposed network
outperformed the baseline proposed by the original work [44].
Second, we also used the MSNN for an imagined speech EEG
classification task. The imagined speech paradigm lacks neurophysiological finds, thus there is not enough neurophysiological
prior knowledge [45]. We exploited BCI Competition V-3
dataset14, which consists of five different imagined speech:
'hello,' 'help me,' 'stop,' 'thank you,' and 'yes' that are acquired
from 15 subjects, recorded from 64 electrode channels according to the standard 10-20 system. In this experiment, our proposed network was also trained with raw imagined speech
EEG samples. As summarized in Table V, the proposed network not only outperformed the other baselines but also
showed promising performance for five-class imagined speech
classification task. Based on these preliminary experimental
results, we expect that the MSNN can be regarded as a starting
point or a backbone architecture to a variety of BCI systems
including newly-designed protocols and paradigms.
VI. Conclusion

In this work, we proposed a novel and compact deep multiscale neural network which can learn multi-scale EEG signal
features. In our experiments, we validated our novel architecture's effectiveness over diverse EEG paradigms, MI, SSVEP, seizure, and drowsy EEG signals. Even though our proposed
MSNN cannot achieve superior performance in every task, it
still shows reasonably high performance comparable to the best
one without major structural modification. Thus, it is expected
that our MSNN can be used as a general EEG representation
learner for various EEG-based BCI. Furthermore, we inspected
the relevance scores to demonstrate the benefits of the multiscale feature extraction ability, investigated activation pattern
maps to understand what types of neurophysiological phenomena were learned by our CNN model, and visualized the t-SNE
of learned features to examine the ability of our method to differentiate feature classes. Finally, we also demonstrated that the
proposed method can be used for precise drowsiness detection
and early seizure detection. In all these respects, we concluded
that the proposed deep multi-scale neural network offers significant potential for interpreting EEG signals. Additionally, because
the proposed network is clearly generalizable to various EEG paradigms, it is expected to bring promising benefits that can be
applied to neural architecture search methods [46], thereby making a deep learning-based BCI adaptable to different paradigms.
From a practical standpoint, there are still source limitations
with regard to the inter-subject variation [29] in performance
or BCIs with multiple classes. In this work, the datasets used in
our experiments have a small number of classes, which are definitely not enough for practical use. However, it is important for
a BCI system to be robust for multi-class settings. In this regard,
14

Available at: https://osf.io/pq7vb/

https://www.osf.io/pq7vb/

IEEE Computational Intelligence Magazine - May 2021

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - May 2021