Signal Processing - January 2017 - 96
p (x) = x, i.e., Eve has no additional
role here. Two specific examples are
as follows:
- Alice wants to transmit as much
information as possible, relevant to
Bob, while consuming minimum
resources on bandwidth/storage.
-■ Alice wants to transmit as much
information as possible, relevant
to Bob, while preventing the original data from being reconstructed
or leaked.
The objective is to design the query
y, which maximizes the utility gain,
prescribed by u (x), while keeping the
data bandwidth below a cer tain bound.
■■ Privacy funnel (PF) [2]. In this scenario, it is assumed that u (x) = x,
i.e., Bob has no role here. Now Alice
wants to transmit the original data, as
much as possible, while preventing
leaking sensitive data to Eve. (Recall
that Eve may perform any adversarial
inference attack, based on the query
y, to intrude the privacy prescribed
by p (x) .) Therefore, we must design
a query y that can minimize the privacy leakage while assuring a guaranteed level of information on the
original data is being conveyed.
Most analysis regarding IB [4] and
PF [2], especially those pertaining to
the convex optimization, may be naturally extended to the general case when
u (x) ! x and p (x) ! x, i.e., both Bob
and Eve have their own specific goals.
Gauss-Markov estimation theorem
The assumption of Gaussian distribution
of the data is vital to our development
of the CP theory. More specifically,
it allows us to make use of the results
that 1) the amount of information can
be quantified as a log function of its
variance and 2) the difference of variances (before-and-after-query) can be
derived via the classic Gauss-Markov
estimation theorem [3]. As proven next,
the Gaussian assumption leads us to a
simple (eigenvalue-based) optimal solution in closed form.
Now, as depicted in Figure 2, the
design will be optimized over two competing linear vector spaces:
■■ Utility subspace. The utility function
is represented by u = U T x, where
96
■■
U ! 0 M # n is the projection matrix
characterizing the utility subspace.
Privacy subspace. The cost (i.e.,
privacy) function is represented by
p = P T x, where P ! 0 M # o is the
matrix characterizing the privacy
subspace. (Here we shall simply
assume the subspace projection
matrices U and P as given, leaving
their learning/estimation strategies a
later discussion.)
Gaussian distribution
with linear optimal query
The linear query is represented by
y = f T x + f,
(1)
where we assume that f = 1 and f
is an independent random noise, with
variance v 2f = v 2, all without loss of
generality.
The amount of information contained in x (as well as u and p) can be
quantitatively measured by its entropy
H (x) / -
#
p (x) log p (x) dx,
where the integration is taken over the
M-dimensional vector space. Assume
that x has a Gaussian distribution
N ^xt 0, R x h , where xt 0 and R x are the
mean and covariance matrix of x. The
entropy and covariance matrix are
closely connected
H (x) = 1 log 2 ^ R x h + M log 2 (2re) .
2
2
Effect of query on covariance matrix
Assume that E [x] = xt 0 and E [(x xt 0) (x - xt 0) T ] = R x . Before query, the
initial estimate of x is xt 0 , and the initial error-covariance-matrix (ECM) is
E [(x - xt 0) (x - xt 0) T ] = R x . Now that
we are given the knowlege of the query,
y = f T x + f. According to the well-known
Gauss-Markov theorem; cf. [3, ch.
15], the optimal estimation of x is
xt = E [x | y]
-1
= xt 0 + v -2 ^R -x 1 + v -2 f f T h f [y - f T xt 0] .
Let xu / xt - x denote the updated
estimation error (given the query y)
with its corresponding ECM as follows:
IEEE Signal Processing Magazine
|
January 2017
|
R xu = E [xu xu T | y]
-1
= ^R x-1 + v -2 f f T h
= R x - R x f 6v 2 + f T R x f @ f T R x .
-1
(2)
This leads to a postquery Gaussian distribution for x, denoted by N ^xt , R xu h .
For multiquery cases, just change f "
F ! R M # m and v 2 " / e ! R m # m .
Effect on the estimation error
of the original feature vector
Conventionally, we would like to maximally preserve the fidelity, i.e., to best
reconstruct the original data. In this
case, natural formulation for the optimal
query vector(s) is as follows:
argmax f {trace (R xu )}
(3)
whose optimal solution lies exactly on
the principal component analysis (PCA)
eigen-subspace.
Effect on the utility and
privacy entropies
It is obvious that the additional query
knowledge can only reduce the entropy of u and p. The utility and privacy
functions are linear functions of the state
vector x, so they are Gaussian distributed with the utility covariance matrices
(before and after the query) given as
R u / U T R x U and R uu = U T R xu U (4)
and the privacy covariance matrices as
R p / P T R x P and R pu / P T R xu P. (5)
Example 1. The double income problem (DIP)
Here, a two-dimensiona l vector
x = [x 1 x 2] T represents the two individual incomes of a couple.
■■ From the utility perspective, to
assess the family's total income, the
utility function should be set as
u = u (x) = x 1 + x 2 .
■■ From the privacy perspective, the
query should not pry into the income
disparity between the couple. To protect such privacy, the privacy function
is set as p = p (x) = x 1 - x 2 .
Suppose that the initial covariance
matrix of x is
8 -6
E.
Rx = ;
- 6 10
Table of Contents for the Digital Edition of Signal Processing - January 2017
Signal Processing - January 2017 - Cover1
Signal Processing - January 2017 - Cover2
Signal Processing - January 2017 - 1
Signal Processing - January 2017 - 2
Signal Processing - January 2017 - 3
Signal Processing - January 2017 - 4
Signal Processing - January 2017 - 5
Signal Processing - January 2017 - 6
Signal Processing - January 2017 - 7
Signal Processing - January 2017 - 8
Signal Processing - January 2017 - 9
Signal Processing - January 2017 - 10
Signal Processing - January 2017 - 11
Signal Processing - January 2017 - 12
Signal Processing - January 2017 - 13
Signal Processing - January 2017 - 14
Signal Processing - January 2017 - 15
Signal Processing - January 2017 - 16
Signal Processing - January 2017 - 17
Signal Processing - January 2017 - 18
Signal Processing - January 2017 - 19
Signal Processing - January 2017 - 20
Signal Processing - January 2017 - 21
Signal Processing - January 2017 - 22
Signal Processing - January 2017 - 23
Signal Processing - January 2017 - 24
Signal Processing - January 2017 - 25
Signal Processing - January 2017 - 26
Signal Processing - January 2017 - 27
Signal Processing - January 2017 - 28
Signal Processing - January 2017 - 29
Signal Processing - January 2017 - 30
Signal Processing - January 2017 - 31
Signal Processing - January 2017 - 32
Signal Processing - January 2017 - 33
Signal Processing - January 2017 - 34
Signal Processing - January 2017 - 35
Signal Processing - January 2017 - 36
Signal Processing - January 2017 - 37
Signal Processing - January 2017 - 38
Signal Processing - January 2017 - 39
Signal Processing - January 2017 - 40
Signal Processing - January 2017 - 41
Signal Processing - January 2017 - 42
Signal Processing - January 2017 - 43
Signal Processing - January 2017 - 44
Signal Processing - January 2017 - 45
Signal Processing - January 2017 - 46
Signal Processing - January 2017 - 47
Signal Processing - January 2017 - 48
Signal Processing - January 2017 - 49
Signal Processing - January 2017 - 50
Signal Processing - January 2017 - 51
Signal Processing - January 2017 - 52
Signal Processing - January 2017 - 53
Signal Processing - January 2017 - 54
Signal Processing - January 2017 - 55
Signal Processing - January 2017 - 56
Signal Processing - January 2017 - 57
Signal Processing - January 2017 - 58
Signal Processing - January 2017 - 59
Signal Processing - January 2017 - 60
Signal Processing - January 2017 - 61
Signal Processing - January 2017 - 62
Signal Processing - January 2017 - 63
Signal Processing - January 2017 - 64
Signal Processing - January 2017 - 65
Signal Processing - January 2017 - 66
Signal Processing - January 2017 - 67
Signal Processing - January 2017 - 68
Signal Processing - January 2017 - 69
Signal Processing - January 2017 - 70
Signal Processing - January 2017 - 71
Signal Processing - January 2017 - 72
Signal Processing - January 2017 - 73
Signal Processing - January 2017 - 74
Signal Processing - January 2017 - 75
Signal Processing - January 2017 - 76
Signal Processing - January 2017 - 77
Signal Processing - January 2017 - 78
Signal Processing - January 2017 - 79
Signal Processing - January 2017 - 80
Signal Processing - January 2017 - 81
Signal Processing - January 2017 - 82
Signal Processing - January 2017 - 83
Signal Processing - January 2017 - 84
Signal Processing - January 2017 - 85
Signal Processing - January 2017 - 86
Signal Processing - January 2017 - 87
Signal Processing - January 2017 - 88
Signal Processing - January 2017 - 89
Signal Processing - January 2017 - 90
Signal Processing - January 2017 - 91
Signal Processing - January 2017 - 92
Signal Processing - January 2017 - 93
Signal Processing - January 2017 - 94
Signal Processing - January 2017 - 95
Signal Processing - January 2017 - 96
Signal Processing - January 2017 - 97
Signal Processing - January 2017 - 98
Signal Processing - January 2017 - 99
Signal Processing - January 2017 - 100
Signal Processing - January 2017 - 101
Signal Processing - January 2017 - 102
Signal Processing - January 2017 - 103
Signal Processing - January 2017 - 104
Signal Processing - January 2017 - 105
Signal Processing - January 2017 - 106
Signal Processing - January 2017 - 107
Signal Processing - January 2017 - 108
Signal Processing - January 2017 - 109
Signal Processing - January 2017 - 110
Signal Processing - January 2017 - 111
Signal Processing - January 2017 - 112
Signal Processing - January 2017 - 113
Signal Processing - January 2017 - 114
Signal Processing - January 2017 - 115
Signal Processing - January 2017 - 116
Signal Processing - January 2017 - Cover3
Signal Processing - January 2017 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_201809
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_201807
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_201805
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_201803
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_201801
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1117
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0917
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0717
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0517
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0317
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0117
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1116
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0916
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0716
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0516
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0316
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0116
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1115
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0915
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0715
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0515
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0315
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0115
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1114
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0914
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0714
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0514
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0314
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0114
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1113
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0913
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0713
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0513
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0313
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0113
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1112
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0912
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0712
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0512
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0312
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0112
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1111
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0911
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0711
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0511
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0311
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0111
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1110
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0910
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0710
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0510
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0310
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0110
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1109
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0909
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0709
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0509
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0309
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0109
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1108
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0908
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0708
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0508
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0308
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0108
https://www.nxtbookmedia.com