Professional Documents
Culture Documents
http://ocw.mit.edu
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
Problem Set 5
Due April 26, 2007
Problem 1
Decision Boundaries: Two-dimensional Gaussian Case
The optimal Bayesian decision rule can be written:
(
)
1 ; pp10 (x
> PP01
(x)
(x) =
0 ; otherwise
It is sometimes useful to express the decision in the log domain, or equivalently
(
1 ; ln (p1 (x)) ln (p0 (x)) > ln PP10
(x) =
0 ; otherwise
The decision boundary is defined as the locus of points, x, where the ratios are equal,
that is
P0
ln (p1 (x)) ln (p0 (x)) = ln
P1
If x = [x1 , x2 ] is a two-dimensional Gaussian variable, its PDF is written:
1
1
T
1
exp (x mi ) i (x mi )
pi (x) =
2
2 |i |1/2
where mi , i are the class-conditional means and covariances, respectively. Plugging this
into the log form of the decision boundary above yields:
1
1
P0
1
|0 |
T
T
1
1
(x m1 ) 1 (x m1 ) + (x m0 ) 0 (x m0 ) + ln
= ln
2
2
2
|1 |
P1
Suggestion: You may want to do part (d) of this problem first as a way of
checking your answers to the first three parts although it is not necessary to do
so.
a) Suppose
1
x1
1
1
P1 = P0 =
m0 =
x=
m1 =
x0
1
0
2
100
90
19
1 9
1
19
19
1
=
=
1 = 0 = 9 10
|1 | = |0 | =
90
100
1
0
1
100
10
19
19
express the decision boundary in the form x2 = f (x1 ).
Cite as: Julie Greenberg, Bertrand Delgutte, William (Sandy) Wells, John Fisher, and Gari Clifford. Course materials
for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007. MIT OpenCourseWare
(http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
1 r
r 1
where |r| < 1 (which is simply a constraint to ensure i is a valid covariance matrix)
keeping all other relevant terms from part (a). How does this change the decision
boundary as compared to the result of part (a)?
d) Now let
0 =
1
9
10
9
10
1
0
=
100
19
90
19
90
19
100
19
|0 | =
19
100
setting all other parameters, except P1 and P0 , the same as in part (a). Use matlab
contour function to plot the decision boundary as a function of the ratio of prior
probabilities of each class for the values P0 /P1 = [1/4, 1/2, 1, 2, 4]. Here is some of
the code you will need (where function is the left side of the decision boundary
equation, ln (p1 (x)) ln (p0 (x))):
[x1,x2] = meshgrid(-4:0.1:4,-4:0.1:4);
d = function(x1,x2);
[c,h] = contour(x1,x2,d,log([1/4,1/2,1,2,4]));
clabel(c,h);
Cite as: Julie Greenberg, Bertrand Delgutte, William (Sandy) Wells, John Fisher, and Gari Clifford. Course materials
for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007. MIT OpenCourseWare
(http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
Problem 2
Suggestion: read the entire question, the answer can be stated in one sentence
with no calculations.
Suppose you have a 3-dimensional measurement vector x = [x1 , x2 , x3 ] for a binary classification problem where 0 < P1 < 1 (i.e. it is strictly greater then 0 and less then 1). Recall
that the class-conditional marginal distribution of x1 , x2 is
Z
pi (x1 , x2 ) =
pi (x1 , x2 , x3 ) dx3
Z
=
pi (x1 , x2 |x3 ) pi (x3 ) dx3
and that the unconditioned marginal density of any single measurement is
p (xk ) =
1
X
Pi pi (xk )
i=0
where k = 1, 2, or 3.
Now consider 2 different decision functions. The first (x1 , x2 , x3 ) is the optimal classifier
using the full measurement vector [x1 , x2 , x3 ], while the second (x1 , x2 ) is the optimal
classifier using only [x1 , x2 ]. In general the probability of error using (x1 , x2 , x3 ) will be
lower then when using (x1 , x2 ) (i.e. when we ignore the third measurement). State a
condition under which both classifiers will achieve the same probability of error.
Cite as: Julie Greenberg, Bertrand Delgutte, William (Sandy) Wells, John Fisher, and Gari Clifford. Course materials
for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007. MIT OpenCourseWare
(http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].