Computational Intelligence - May 2017 - 65

0.2
iTTC (s−1)

Acceleration (m/s2)

5

0

−5

0

50
Time (s)
(a)

0

−0.2

100

Brake (par)

Throttle (%)

50

0

0

50
Time (s)
(b)

100

0

50
Time (s)
(d)

100

20

100

0

50
Time (s)
(c)

10

0

100

MFOC

LQR

PID

2

300

1

0

0

50
Time (s)
(a)

100

1

Performance Index

Comfortness Index

Figure 13 Trajectory during car following scenario: (a) Acceleration; (b) iTTC; (c) Throttle
opening; (d) Braking pressure.

Safety Index

Based on the running trajectory, the
state transition data are sampled and then
expanded to the state space of intelligent
cruise control system. Hence, the training
data set D = " (s t, a t, s t +1) , is obtained.
The reward rt can be calculated for each
element in the data set according to Eq. (8).
The cr itic network inputs s =
6d, v h, Dv, a h@ and the control policy a,
outputs the approached state-action
value function Qt , with a structure of
5 - 45 - 1. The action network inputs
s = 6d, v h, Dv, a h@ a n d o u t p u t s t h e
approached control policy rt, with a
structure of 4 - 10 - 1. Their weights
are generated randomly with a normal
distribution N (0, 2/n i), where n i is the
number of nodes in its previous layer.
The action network is trained to regress
a feasible initial control policy a =
-[- 0.0249, - 0.1307, 0.0499, 0.7163] s
in advance. This initial control policy is
obtained experimentally. Based on the
data D, the critic network is trained to
evaluate the control policy according to
the objective function Eq. (20), and then
the action network is trained according to
the objective function Eq. (21) in order to
improve the control policy. Thus a new
control policy (called the first policy) is
yielded.The critic network and the action
network are trained alternatively until the
weights converge. The converged action
network is called the final policy.
The trajectories of the host car steered
by the initial policy, the first policy and
the final policy are shown in Figure 9
and Figure 10. At time 0 s, the intelligent
cruise control system starts. After a period
of time, it detects a preceding car with
velocity 72 km/h in the 130 m ahead.
The preceding car speeds up to 90 km/h
with an acceleration of 1 m/s 2 at 50 s.
This velocity is kept steady until 85 s, and
it speeds down to 72 km/h with an
acceleration of -1 m/s 2 . Its velocity
curve is shown as the dashed line in Fig-
ure 9 (b). Correspondingly, the driver's
desired distance is shown as the dashed
line in Figure 9 (a). The trajectories
steered by the initial control policies are
fluctuant. It takes much time to track the
preceding car.While the first policy shows
a great improvement over the initial poli-
cy: it tracks the preceding car quickly and

200

100

0

-1

0

50
Time (s)
(b)

0

100

MFOC

0

LQR

50
Time (s)
(c)

100

PID

Figure 14 Performance index during car following scenario: (a) Comfort index; (b) Safety
index; (c) Backward value function.

keeps the clearance as the driver's desired
distance and also decreases the amplitude
of the fluctuation. The final policy pro-
duces the best performance: it tracks the
preceding car quickly and smoothly with
little fluctuation.
Figure 11 (a) and (b) show the com-
fort performance index and the safety per-
formance index calculated by Eq. (5) and

Eq. (7), respectively. The lower the value,
the better the performance will be. Fig-
ure 11 (a) shows the comfort performance
becomes better and better during the
learning process, which indicates the com-
fort improves. Figure 11 (b) shows the car
steered by the initial policy exceeds the
safety distance several times and causes the
index greater than zero. After the learning

may 2017 | IEEE ComputatIonal IntEllIgEnCE magazInE

65



Table of Contents for the Digital Edition of Computational Intelligence - May 2017

Computational Intelligence - May 2017 - Cover1
Computational Intelligence - May 2017 - Cover2
Computational Intelligence - May 2017 - 1
Computational Intelligence - May 2017 - 2
Computational Intelligence - May 2017 - 3
Computational Intelligence - May 2017 - 4
Computational Intelligence - May 2017 - 5
Computational Intelligence - May 2017 - 6
Computational Intelligence - May 2017 - 7
Computational Intelligence - May 2017 - 8
Computational Intelligence - May 2017 - 9
Computational Intelligence - May 2017 - 10
Computational Intelligence - May 2017 - 11
Computational Intelligence - May 2017 - 12
Computational Intelligence - May 2017 - 13
Computational Intelligence - May 2017 - 14
Computational Intelligence - May 2017 - 15
Computational Intelligence - May 2017 - 16
Computational Intelligence - May 2017 - 17
Computational Intelligence - May 2017 - 18
Computational Intelligence - May 2017 - 19
Computational Intelligence - May 2017 - 20
Computational Intelligence - May 2017 - 21
Computational Intelligence - May 2017 - 22
Computational Intelligence - May 2017 - 23
Computational Intelligence - May 2017 - 24
Computational Intelligence - May 2017 - 25
Computational Intelligence - May 2017 - 26
Computational Intelligence - May 2017 - 27
Computational Intelligence - May 2017 - 28
Computational Intelligence - May 2017 - 29
Computational Intelligence - May 2017 - 30
Computational Intelligence - May 2017 - 31
Computational Intelligence - May 2017 - 32
Computational Intelligence - May 2017 - 33
Computational Intelligence - May 2017 - 34
Computational Intelligence - May 2017 - 35
Computational Intelligence - May 2017 - 36
Computational Intelligence - May 2017 - 37
Computational Intelligence - May 2017 - 38
Computational Intelligence - May 2017 - 39
Computational Intelligence - May 2017 - 40
Computational Intelligence - May 2017 - 41
Computational Intelligence - May 2017 - 42
Computational Intelligence - May 2017 - 43
Computational Intelligence - May 2017 - 44
Computational Intelligence - May 2017 - 45
Computational Intelligence - May 2017 - 46
Computational Intelligence - May 2017 - 47
Computational Intelligence - May 2017 - 48
Computational Intelligence - May 2017 - 49
Computational Intelligence - May 2017 - 50
Computational Intelligence - May 2017 - 51
Computational Intelligence - May 2017 - 52
Computational Intelligence - May 2017 - 53
Computational Intelligence - May 2017 - 54
Computational Intelligence - May 2017 - 55
Computational Intelligence - May 2017 - 56
Computational Intelligence - May 2017 - 57
Computational Intelligence - May 2017 - 58
Computational Intelligence - May 2017 - 59
Computational Intelligence - May 2017 - 60
Computational Intelligence - May 2017 - 61
Computational Intelligence - May 2017 - 62
Computational Intelligence - May 2017 - 63
Computational Intelligence - May 2017 - 64
Computational Intelligence - May 2017 - 65
Computational Intelligence - May 2017 - 66
Computational Intelligence - May 2017 - 67
Computational Intelligence - May 2017 - 68
Computational Intelligence - May 2017 - 69
Computational Intelligence - May 2017 - 70
Computational Intelligence - May 2017 - 71
Computational Intelligence - May 2017 - 72
Computational Intelligence - May 2017 - Cover3
Computational Intelligence - May 2017 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202311
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202308
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202305
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202302
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202211
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202208
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202205
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202202
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202111
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202108
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202105
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202102
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202011
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202008
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202005
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202002
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201911
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201908
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201905
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201902
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201811
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201808
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201805
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201802
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter12
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall12
https://www.nxtbookmedia.com