IEEE - Aerospace and Electronic Systems - December 2019 - 36

Cognitive Radar for Classiﬁcation
Table 2.

Table 1.

Reward and cost for the classifying actions.

Sensor modes 1 and 2 of the multifunction radar.

Mode
1

Observation

C1

C2

C3

1

0.9000

0.9000

0.1000

2

0.1000

0.1000

0.9000

Reward

0

0

0

Time

1

1

1

0.5

0.5

0.5

1

0.8500

0.2000

0.1000

2

0.1500

0.8000

0.9000

0

0

0

0.5

0.5

0.5

1

1

1

Energy
Mode
2

Observation
Reward
Time
Energy

Rp ¼

1
X

gt

X
s

t¼0

Rðs; pðbt ÞÞbt ðsÞ

Classify as
C2

(9)

pðbt Þot

where bt ¼ b
for t > 0 is the t À th belief state following strategy p and observing o1 ; . . . ; ot . The value
function VRp ðbÞ ¼ EðRp jb0 ¼ bÞ is the expected reward.
The optimal policy pÃ maximizes the expected reward.
POMDPs are much harder to solve than MDPs, but the
modeling capabilities of POMDPs are much wider, since
most real-world problems are not fully observable. If one
wants to model more than one objective, constrained
POMDPs (CPOMDPs) are a suitable method. A CPOMDP
imposes additional restrictions on the optimal policy by
adding costs to actions. When action ak is carried out, we
not only get a reward, but also have costs cl ðsi ; ak Þ for
l ¼ 1; . . . ; L, with a nonnegative function cl . ck are the
immediate cost functions and cÃk are the desired bounds for
the expected cost which are defined in the same way as
the expected reward as Vcpk ðbÞ ¼ Eðcpk jb0 ¼ bÞ where
cpk ¼

1
X
t¼0

gt

X
s

ck ðs; pðbt ÞÞbt ðsÞ

(10)

are the returned cost. Hence, an admissible strategy p
to a CPOMDP has to satisfy
Vcpk ðb0 Þ

cÃk :

(11)

For a given policy, we calculate the expected reward and
the expected costs.
As constraint, the expected costs have to be below a
bound cÃl for every l ¼ 1; . . . ; L. An optimal solution to a
CPOMDP is the policy which has the maximum reward
while bounding the expected costs. Algorithms for solving
CPOMDPs are for example the CPBVI-algorithm [30] or
the CALP-algorithm [31].
36

Classify as
C1

Classify as
C3

C1

C2

C3

0.5

À10

À10

Time

0

0

0

Energy

0

0

0

Reward

Reward

À10

0.5

À10

Time

0

0

0

Energy

0

0

0

Reward

À10

À10

0.5

Time

0

0

0

Energy

0

0

0

CLASSIFYING WITH RESTRICTIONS
The following illustrates a simple example for multistage
classification motivated by Castanon [32].
The task at hand is to classify as accurate as possible a
number of targets which belong to three classes C ¼ f1;
2; 3g (positive reward for correct declarations and negative
for wrong ones) in a scenario with an a priori probability
p1 ¼ 13 ; p2 ¼ 13, and p3 ¼ 13, while keeping the use of time
and energy resources constrained. We solve the following
optimization problem:
maxp

EðRðpÞÞ
subject to:
EðtimeðpÞÞ cÃ1 ¼ 3
EðenergyðpÞÞ cÃ2 ¼ 3:

The used radar can choose one out of two modes (a1 ; a2 ).
As shown in Table 1, the first mode is good at distinguishing between classes 1,2 and class 3, while using one time
unit per measurement and 0.5 energy units per measurement. The second mode is still quite good at differentiating between classes 2,3 and class 1 and consumes half a
time unit and one energy unit per measurement.
In addition to the two modes, there are also classification actions. The classification is final, i.e., there will be
no further actions carried out on this target. Since the final
declaration is the sole purpose of the given task, the measurements itself do not contribute to the reward. The
reward for correct classification will be 0.5 while picking
the wrong class will result in a reward of À10, or differently put in a penalty of 10. Classifying does not consume
any of the resources time or energy. In Table 2, the classifying actions are summarized.
Figure 10 sketches a decision tree for the given problem. When mode 1 or 2 is used, the tree splits into two

IEEE A&E SYSTEMS MAGAZINE

DECEMBER 2019
IEEE - Aerospace and Electronic Systems - December 2019

Table of Contents for the Digital Edition of IEEE - Aerospace and Electronic Systems - December 2019
