Professional Documents
Culture Documents
CMPUT 615
Applications of Machine Learning
in Image Analysis
K-means Overview
A clustering algorithm
An approximation to an NP-hard combinatorial
optimization problem
It is unsupervised
K stands for number of clusters, it is a user
input to the algorithm
From a set of data points or observations (all
numerical), K-means attempts to classify them
into K clusters
The algorithm is iterative in nature
K-means Details
Nk
where
mk is the mean vector of the kth cluster
Nk is the number of observations in kth cluster
k 1
x m
C (i )k
K-means Algorithm
For a given assignment C, compute the cluster means
mk:
xi
mk
i:C ( i ) k
Nk
, k 1, , K .
C (i ) arg min xi mk , i 1, , N
1 k K
An image (I)
Matlab code:
I = double(imread('));
J = reshape(kmeans(I(:),3),size(I));
Summary
K-means converges, but it finds a local minimum
of the cost function
Works only for numerical observations (for
categorical and mixture observations, K-medoids
is a clustering method)
Fine tuning is required when applied for image
segmentation; mostly because there is no
imposed spatial coherency in k-means algorithm
Often works as a staring point for sophisticated
image segmentation algorithms
(1979)
Otsus Method
Histogram (and the image) are bimodal.
No use of spatial coherence, nor any other
notion of object structure.
Assumes uniform illumination (implicitly), so
the bimodal brightness behavior arises from
object appearance differences only.
Theweightedwithinclassvarianceis:
2
1
2
2
Wheretheclassprobabilitiesareestimatedas:
t
q1 (t) P(i)
q2 (t)
i 1
P(i)
i t 1
Andtheclassmeansaregivenby:
t
iP(i)
1 (t)
i 1 q1 (t)
iP(i)
2 (t)
i t 1 q2 (t )
Finally,theindividualclassvariancesare:
t
P(i)
(t) [i 1 (t)]
q1 (t)
i1
2
1
P(i)
(t) [i 2 (t)]
q2 (t)
i t 1
2
2
Now,wecouldactuallystophere.Allweneedtodoisjust
runthroughthefullrangeoftvalues[1,256]andpickthe
w2 (t)
valuethatminimizes.
Buttherelationshipbetweenthewithinclassandbetween
classvariancescanbeexploitedtogeneratearecursion
relationthatpermitsamuchfastercalculation.
Finally...
Initialization...
Recursion...
q1 (t 1) q1 (t) P(t 1)
q1 (t 1)1 (t 1)
2 (t 1)
1 q1 (t 1)
Aftersomealgebra,wecanexpressthetotalvarianceas...
2
w
Withinclass,
frombefore
2
Betweenclass, B (t)
Sincethetotalisconstantandindependentoft,theeffectof
changingthethresholdismerelytomovethecontributionsof
thetwotermsbackandforth.
So,minimizingthewithinclassvarianceisthesameas
maximizingthebetweenclassvariance.
Thenicethingaboutthisisthatwecancomputethequantities
2
inrecursivelyaswerunthroughtherangeoftvalues.
B (t)
Binary image
by Otsus method
0.06
0.05
Matlab code:
0.04
I = double(imread('));
0.03
0.02
I = (I-min(I(:)))/(max(I(:))-min(I(:)));
0.01
0
50
100
150
200
250
300
J = I>graythresh(I);