Signal Processing - November 2017 - 150
The mismatched ML estimator
The aim of this section is to present the second milestone of
the estimation theory under model misspecification: the mismatched ML (MML) estimator. As discussed in the section "A
Formal Theory of Statistical Inference Under Misspecified
Models," the theoretical framework supporting the existence
and the convergence properties of the MML estimator were
developed by Huber [20] and later by White [46]. Here, our
goal is to summarize their main findings from an SP standpoint. As detailed in the "Description of a Misspecified
Model Problem" section, assume we have a set x = {x m} mM= 1
of M i.i.d. measurement vectors distributed according to a
true, but unknown or inaccessible, pdf p X (x m). So the loglikelihood function for the data x under a generally misspecified parametric pdf fX ^x m ih ! F is given by
M
l M (i) _ M -1 / m = 1 ln fX (x m i). Following the classical
definition, the MML estimate is the vector that maximizes
the (misspecified) log-likelihood function
it MML (x) _ argmax l M (i) = argmax
i!H
i!H
M
/
ln fX (x m i),
m =1
(10)
where x m + p X (x m). The definition of the MML estimator
given in (10) is clear and self-explanatory. Furthermore, it is
consistent with the classical "matched" ML estimator. But
what is the convergence point of it MML (x)? As proved in [20]
and [46], under suitable regularity conditions, the MML estimator converges [almost surely (a.s.)] to the pseudotrue
parameter vector i 0 defined in (1). This is a desirable result
since it shows that the MML estimator converges to the
parameter vector that minimizes the distance, in the KLD
sense, between the misspecified and the true pdfs (see
"Variance Estimation" and "Power Estimation in Correlated
Data"). In addition, Huber and White investigated the asymptotic behavior of the MML estimator, and their valuable findings can be summarized in the f- ollowing theorem.
For Theorem 2 [20], [46], under suitable regularity conditions,
it can be shown that
a.s.
it MML (x) M"
i 0 . (11)
"3
Moreover,
M ^it MML (x)- i 0h M+
N ^0, C i 0h, (12)
"3
d.
d.
where M+
indicates the convergence in distribution and
"3
C i 0 _ A i-01 B i 0 A i-01 , where the matrices A i 0 and B i 0 have
been defined in (2) and (3), respectively. Matrix C i 0 is sometimes referred to as Huber's sandwich covariance. Two comments are in order:
1) The MML estimator is asymptotically MS-unbiased, and
its asymptotic error covariance is equal to the MCRB, i.e.,
it is an efficient estimator w.r.t. the MCRB. The analogy
with the classical matched ML estimator is completely transparent. In particular, if the model F is correctly specified, i.e., there exists a parameter vector ir ! H such that
a.s.
p X (x m) = fX (x m ir ), then it MML (x) M"
ir with an asymp"3
150
A consistent sample estimate of the MCRB
In this section, we go back to an issue raised before, i.e., the
calculation of the MCRB when the true model is completely
unknown. In fact, from (5), to obtain a closed form expression
of the MCRB, we need to analytically evaluate i 0, A i 0 , and
B i 0 . As shown in (1)-(3), these quantities involve the analysis
of the expectation operator taken w.r.t. the true pdf p X (x m). If
p X (x m) is completely unknown, we will not be able to evaluate these expectations in closed form, but, as an alternative, we
could obtain sample estimates of them. More formally, we
define the matrices [46]:
6A M (i)@ij _ M -1
6B M (i)@ij _ M -1
M
/
m =1
M
/
m =1
2 2 ln fX ^x m ih
, (13)
2i i 2 i j
2 ln fX ^x m ih 2 ln fX ^x m ih
,(14)
$
2i i
2i j
C M (i) _ [A M (i)] -1 B M (i) [A M (i)] -1 . (15)
Remarkably, it can be shown (see the proof in [46, Theorem
3.2]) that
a.s.
C M (it MML) M"
C i 0 = MCRB (i 0). (16)
"3
Theorem 2
totic error covariance matrix given by the classical CRB,
which is the inverse of the FIM B ir =-A ir .
2) Theorem 2 represents a very useful result for practical
applications. In fact, it tells us that, when we do not have
any a priori information about the true data model, the ML
estimator derived under a possibly misspecified model is
still a reasonable choice among other MS-unbiased
-mismatched estimators, since it converges to the parameter
vector that minimizes the KLD between the true and the
assumed model and it has the lowest possible error covariance (at least asymptotically).
In other words, (16) assures us that we can obtain a strongly
consistent estimate of the MCRB by evaluating the sample
counterpart of A i 0 and B i 0 , i.e., A M (i) and B M (i), at the
value of the MML estimator. This result has strong practical
implications, since it provides an estimate of the MCRB
when we do not have any prior knowledge of the true pdf
p X (x m). Hence, it widens areas of applicability of the MCRB.
This, of course, requires the data to be stationary over some
reasonable period to allow sufficient averaging (as is required
in numerous SP applications). This result can also be used to
design statistical tests to detect model misspecification [46],
[47, p. 218].
Generalization to the Bayesian setting
The Bayesian philosophy adopts the notion that one has some
prior knowledge (a belief or perhaps a guess) about the values a
desired parameter will assume before an experiment. Once data
are observed, then one can update that prior knowledge based on
the information provided by the data measurements. Thus, the
IEEE SIGNAL PROCESSING MAGAZINE
|
November 2017
|
Table of Contents for the Digital Edition of Signal Processing - November 2017
Signal Processing - November 2017 - Cover1
Signal Processing - November 2017 - Cover2
Signal Processing - November 2017 - 1
Signal Processing - November 2017 - 2
Signal Processing - November 2017 - 3
Signal Processing - November 2017 - 4
Signal Processing - November 2017 - 5
Signal Processing - November 2017 - 6
Signal Processing - November 2017 - 7
Signal Processing - November 2017 - 8
Signal Processing - November 2017 - 9
Signal Processing - November 2017 - 10
Signal Processing - November 2017 - 11
Signal Processing - November 2017 - 12
Signal Processing - November 2017 - 13
Signal Processing - November 2017 - 14
Signal Processing - November 2017 - 15
Signal Processing - November 2017 - 16
Signal Processing - November 2017 - 17
Signal Processing - November 2017 - 18
Signal Processing - November 2017 - 19
Signal Processing - November 2017 - 20
Signal Processing - November 2017 - 21
Signal Processing - November 2017 - 22
Signal Processing - November 2017 - 23
Signal Processing - November 2017 - 24
Signal Processing - November 2017 - 25
Signal Processing - November 2017 - 26
Signal Processing - November 2017 - 27
Signal Processing - November 2017 - 28
Signal Processing - November 2017 - 29
Signal Processing - November 2017 - 30
Signal Processing - November 2017 - 31
Signal Processing - November 2017 - 32
Signal Processing - November 2017 - 33
Signal Processing - November 2017 - 34
Signal Processing - November 2017 - 35
Signal Processing - November 2017 - 36
Signal Processing - November 2017 - 37
Signal Processing - November 2017 - 38
Signal Processing - November 2017 - 39
Signal Processing - November 2017 - 40
Signal Processing - November 2017 - 41
Signal Processing - November 2017 - 42
Signal Processing - November 2017 - 43
Signal Processing - November 2017 - 44
Signal Processing - November 2017 - 45
Signal Processing - November 2017 - 46
Signal Processing - November 2017 - 47
Signal Processing - November 2017 - 48
Signal Processing - November 2017 - 49
Signal Processing - November 2017 - 50
Signal Processing - November 2017 - 51
Signal Processing - November 2017 - 52
Signal Processing - November 2017 - 53
Signal Processing - November 2017 - 54
Signal Processing - November 2017 - 55
Signal Processing - November 2017 - 56
Signal Processing - November 2017 - 57
Signal Processing - November 2017 - 58
Signal Processing - November 2017 - 59
Signal Processing - November 2017 - 60
Signal Processing - November 2017 - 61
Signal Processing - November 2017 - 62
Signal Processing - November 2017 - 63
Signal Processing - November 2017 - 64
Signal Processing - November 2017 - 65
Signal Processing - November 2017 - 66
Signal Processing - November 2017 - 67
Signal Processing - November 2017 - 68
Signal Processing - November 2017 - 69
Signal Processing - November 2017 - 70
Signal Processing - November 2017 - 71
Signal Processing - November 2017 - 72
Signal Processing - November 2017 - 73
Signal Processing - November 2017 - 74
Signal Processing - November 2017 - 75
Signal Processing - November 2017 - 76
Signal Processing - November 2017 - 77
Signal Processing - November 2017 - 78
Signal Processing - November 2017 - 79
Signal Processing - November 2017 - 80
Signal Processing - November 2017 - 81
Signal Processing - November 2017 - 82
Signal Processing - November 2017 - 83
Signal Processing - November 2017 - 84
Signal Processing - November 2017 - 85
Signal Processing - November 2017 - 86
Signal Processing - November 2017 - 87
Signal Processing - November 2017 - 88
Signal Processing - November 2017 - 89
Signal Processing - November 2017 - 90
Signal Processing - November 2017 - 91
Signal Processing - November 2017 - 92
Signal Processing - November 2017 - 93
Signal Processing - November 2017 - 94
Signal Processing - November 2017 - 95
Signal Processing - November 2017 - 96
Signal Processing - November 2017 - 97
Signal Processing - November 2017 - 98
Signal Processing - November 2017 - 99
Signal Processing - November 2017 - 100
Signal Processing - November 2017 - 101
Signal Processing - November 2017 - 102
Signal Processing - November 2017 - 103
Signal Processing - November 2017 - 104
Signal Processing - November 2017 - 105
Signal Processing - November 2017 - 106
Signal Processing - November 2017 - 107
Signal Processing - November 2017 - 108
Signal Processing - November 2017 - 109
Signal Processing - November 2017 - 110
Signal Processing - November 2017 - 111
Signal Processing - November 2017 - 112
Signal Processing - November 2017 - 113
Signal Processing - November 2017 - 114
Signal Processing - November 2017 - 115
Signal Processing - November 2017 - 116
Signal Processing - November 2017 - 117
Signal Processing - November 2017 - 118
Signal Processing - November 2017 - 119
Signal Processing - November 2017 - 120
Signal Processing - November 2017 - 121
Signal Processing - November 2017 - 122
Signal Processing - November 2017 - 123
Signal Processing - November 2017 - 124
Signal Processing - November 2017 - 125
Signal Processing - November 2017 - 126
Signal Processing - November 2017 - 127
Signal Processing - November 2017 - 128
Signal Processing - November 2017 - 129
Signal Processing - November 2017 - 130
Signal Processing - November 2017 - 131
Signal Processing - November 2017 - 132
Signal Processing - November 2017 - 133
Signal Processing - November 2017 - 134
Signal Processing - November 2017 - 135
Signal Processing - November 2017 - 136
Signal Processing - November 2017 - 137
Signal Processing - November 2017 - 138
Signal Processing - November 2017 - 139
Signal Processing - November 2017 - 140
Signal Processing - November 2017 - 141
Signal Processing - November 2017 - 142
Signal Processing - November 2017 - 143
Signal Processing - November 2017 - 144
Signal Processing - November 2017 - 145
Signal Processing - November 2017 - 146
Signal Processing - November 2017 - 147
Signal Processing - November 2017 - 148
Signal Processing - November 2017 - 149
Signal Processing - November 2017 - 150
Signal Processing - November 2017 - 151
Signal Processing - November 2017 - 152
Signal Processing - November 2017 - 153
Signal Processing - November 2017 - 154
Signal Processing - November 2017 - 155
Signal Processing - November 2017 - 156
Signal Processing - November 2017 - 157
Signal Processing - November 2017 - 158
Signal Processing - November 2017 - 159
Signal Processing - November 2017 - 160
Signal Processing - November 2017 - 161
Signal Processing - November 2017 - 162
Signal Processing - November 2017 - 163
Signal Processing - November 2017 - 164
Signal Processing - November 2017 - 165
Signal Processing - November 2017 - 166
Signal Processing - November 2017 - 167
Signal Processing - November 2017 - 168
Signal Processing - November 2017 - 169
Signal Processing - November 2017 - 170
Signal Processing - November 2017 - 171
Signal Processing - November 2017 - 172
Signal Processing - November 2017 - 173
Signal Processing - November 2017 - 174
Signal Processing - November 2017 - 175
Signal Processing - November 2017 - 176
Signal Processing - November 2017 - Cover3
Signal Processing - November 2017 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_201809
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_201807
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_201805
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_201803
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_201801
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1117
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0917
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0717
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0517
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0317
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0117
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1116
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0916
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0716
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0516
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0316
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0116
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1115
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0915
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0715
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0515
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0315
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0115
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1114
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0914
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0714
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0514
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0314
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0114
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1113
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0913
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0713
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0513
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0313
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0113
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1112
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0912
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0712
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0512
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0312
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0112
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1111
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0911
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0711
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0511
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0311
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0111
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1110
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0910
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0710
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0510
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0310
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0110
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1109
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0909
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0709
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0509
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0309
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0109
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1108
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0908
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0708
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0508
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0308
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0108
https://www.nxtbookmedia.com