IEEE Circuits and Systems Magazine - Q4 2020 - 65

Feature

Â©SHUTTERSTOCK/RASHAD ASHUR

Rethinking Motivation of
Deep Neural Architectures

Weilin Luo, Jinhu LÃ¼, Fellow, IEEE,
Xuerong Li, Lei Chen, and Kexin Liu

Abstract
N owadays, deep neural architectures have acquired great
achievements in many domains, such as image processing and
natural language processing. In this paper, we hope to provide
new perspectives for the future exploration of novel artificial neural architectures via reviewing the proposal and development of
existing architectures. We first roughly divide the influence domain of intrinsic motivations on some common deep neural architectures into three categories: information processing, information transmission and learning strategy. Furthermore, to illustrate
how deep neural architectures are motivated and developed, motivation and architecture details of three deep neural networks,
namely convolutional neural network (CNN), recurrent neural
network (RNN) and generative adversarial network (GAN), are
introduced respectively. Moreover, the evolution of these neural
Digital Object Identifier 10.1109/MCAS.2020.3027222
Date of current version: 12 November 2020

FOURTH QUARTER 2020

1531-636X/20Â©2020IEEE

architectures are also elaborated in this paper. At last, this review
is concluded and several promising research topics about deep
neural architectures in the future are discussed.

I. Introduction and Background
ince the concept of artificial intelligence (AI) was
proposed in the Dartmouth workshop, AI techniques have given rise to great reforms in various
aspects of modern society, such as public opinion analysis in social media, recommender system of e-commerce
and intelligent manufacturing in industrial production. In
the whole AI community, artificial neural network (ANN)
is one class of the most critical algorithms with profound
influence. Unlike general networks [1], [2], ANN consists
of many connected artificial neurons that model the

S

IEEE CIRCUITS AND SYSTEMS MAGAZINE

65

IEEE Circuits and Systems Magazine - Q4 2020

Table of Contents for the Digital Edition of IEEE Circuits and Systems Magazine - Q4 2020

Contents
IEEE Circuits and Systems Magazine - Q4 2020 - Cover1
IEEE Circuits and Systems Magazine - Q4 2020 - Cover2
IEEE Circuits and Systems Magazine - Q4 2020 - Contents
IEEE Circuits and Systems Magazine - Q4 2020 - 2
IEEE Circuits and Systems Magazine - Q4 2020 - 3
IEEE Circuits and Systems Magazine - Q4 2020 - 4
IEEE Circuits and Systems Magazine - Q4 2020 - 5
IEEE Circuits and Systems Magazine - Q4 2020 - 6
IEEE Circuits and Systems Magazine - Q4 2020 - 7
IEEE Circuits and Systems Magazine - Q4 2020 - 8
IEEE Circuits and Systems Magazine - Q4 2020 - 9
IEEE Circuits and Systems Magazine - Q4 2020 - 10
IEEE Circuits and Systems Magazine - Q4 2020 - 11
IEEE Circuits and Systems Magazine - Q4 2020 - 12
IEEE Circuits and Systems Magazine - Q4 2020 - 13
IEEE Circuits and Systems Magazine - Q4 2020 - 14
IEEE Circuits and Systems Magazine - Q4 2020 - 15
IEEE Circuits and Systems Magazine - Q4 2020 - 16
IEEE Circuits and Systems Magazine - Q4 2020 - 17
IEEE Circuits and Systems Magazine - Q4 2020 - 18
IEEE Circuits and Systems Magazine - Q4 2020 - 19
IEEE Circuits and Systems Magazine - Q4 2020 - 20
IEEE Circuits and Systems Magazine - Q4 2020 - 21
IEEE Circuits and Systems Magazine - Q4 2020 - 22
IEEE Circuits and Systems Magazine - Q4 2020 - 23
IEEE Circuits and Systems Magazine - Q4 2020 - 24
IEEE Circuits and Systems Magazine - Q4 2020 - 25
IEEE Circuits and Systems Magazine - Q4 2020 - 26
IEEE Circuits and Systems Magazine - Q4 2020 - 27
IEEE Circuits and Systems Magazine - Q4 2020 - 28
IEEE Circuits and Systems Magazine - Q4 2020 - 29
IEEE Circuits and Systems Magazine - Q4 2020 - 30
IEEE Circuits and Systems Magazine - Q4 2020 - 31
IEEE Circuits and Systems Magazine - Q4 2020 - 32
IEEE Circuits and Systems Magazine - Q4 2020 - 33
IEEE Circuits and Systems Magazine - Q4 2020 - 34
IEEE Circuits and Systems Magazine - Q4 2020 - 35
IEEE Circuits and Systems Magazine - Q4 2020 - 36
IEEE Circuits and Systems Magazine - Q4 2020 - 37
IEEE Circuits and Systems Magazine - Q4 2020 - 38
IEEE Circuits and Systems Magazine - Q4 2020 - 39
IEEE Circuits and Systems Magazine - Q4 2020 - 40
IEEE Circuits and Systems Magazine - Q4 2020 - 41
IEEE Circuits and Systems Magazine - Q4 2020 - 42
IEEE Circuits and Systems Magazine - Q4 2020 - 43
IEEE Circuits and Systems Magazine - Q4 2020 - 44
IEEE Circuits and Systems Magazine - Q4 2020 - 45
IEEE Circuits and Systems Magazine - Q4 2020 - 46
IEEE Circuits and Systems Magazine - Q4 2020 - 47
IEEE Circuits and Systems Magazine - Q4 2020 - 48
IEEE Circuits and Systems Magazine - Q4 2020 - 49
IEEE Circuits and Systems Magazine - Q4 2020 - 50
IEEE Circuits and Systems Magazine - Q4 2020 - 51
IEEE Circuits and Systems Magazine - Q4 2020 - 52
IEEE Circuits and Systems Magazine - Q4 2020 - 53
IEEE Circuits and Systems Magazine - Q4 2020 - 54
IEEE Circuits and Systems Magazine - Q4 2020 - 55
IEEE Circuits and Systems Magazine - Q4 2020 - 56
IEEE Circuits and Systems Magazine - Q4 2020 - 57
IEEE Circuits and Systems Magazine - Q4 2020 - 58
IEEE Circuits and Systems Magazine - Q4 2020 - 59
IEEE Circuits and Systems Magazine - Q4 2020 - 60
IEEE Circuits and Systems Magazine - Q4 2020 - 61
IEEE Circuits and Systems Magazine - Q4 2020 - 62
IEEE Circuits and Systems Magazine - Q4 2020 - 63
IEEE Circuits and Systems Magazine - Q4 2020 - 64
IEEE Circuits and Systems Magazine - Q4 2020 - 65
IEEE Circuits and Systems Magazine - Q4 2020 - 66
IEEE Circuits and Systems Magazine - Q4 2020 - 67
IEEE Circuits and Systems Magazine - Q4 2020 - 68
IEEE Circuits and Systems Magazine - Q4 2020 - 69
IEEE Circuits and Systems Magazine - Q4 2020 - 70
IEEE Circuits and Systems Magazine - Q4 2020 - 71
IEEE Circuits and Systems Magazine - Q4 2020 - 72
IEEE Circuits and Systems Magazine - Q4 2020 - 73
IEEE Circuits and Systems Magazine - Q4 2020 - 74
IEEE Circuits and Systems Magazine - Q4 2020 - 75
IEEE Circuits and Systems Magazine - Q4 2020 - 76
IEEE Circuits and Systems Magazine - Q4 2020 - 77
IEEE Circuits and Systems Magazine - Q4 2020 - 78
IEEE Circuits and Systems Magazine - Q4 2020 - 79
IEEE Circuits and Systems Magazine - Q4 2020 - 80
IEEE Circuits and Systems Magazine - Q4 2020 - 81
IEEE Circuits and Systems Magazine - Q4 2020 - 82
IEEE Circuits and Systems Magazine - Q4 2020 - 83
IEEE Circuits and Systems Magazine - Q4 2020 - 84
IEEE Circuits and Systems Magazine - Q4 2020 - 85
IEEE Circuits and Systems Magazine - Q4 2020 - 86
IEEE Circuits and Systems Magazine - Q4 2020 - 87
IEEE Circuits and Systems Magazine - Q4 2020 - 88
IEEE Circuits and Systems Magazine - Q4 2020 - Cover3
IEEE Circuits and Systems Magazine - Q4 2020 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2023Q3
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2023Q2
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2023Q1
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2022Q4
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2022Q3
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2022Q2
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2022Q1
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2021Q4
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2021q3
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2021q2
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2021q1
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2020q4
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2020q3
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2020q2
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2020q1
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2019q4
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2019q3
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2019q2
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2019q1
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2018q4
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2018q3
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2018q2
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2018q1
https://www.nxtbookmedia.com