Cluster Analysis Example:

SAS program (in blue) and output (in black)

interleaved with comments (in red)

Title Cluster Analysis for Hypothetical Data;

data t;
input cid $ 1-2 income educ;
cards;
c1 5 5
c2 6 6
c3 15 14
c4 16 15
c5 25 20
c6 30 19
run;

proc cluster simple noeigen method=centroid rmsstd rsquare nonorm out=tree;

id cid;
var income educ;
run;

The SIMPLE option displays simple, descriptive statistics.

The NOEIGEN option suppresses computation of eigenvalues. Specifying the

NOEIGEN option saves time if the number of variables is large, but it should
be used only if the variables are nearly uncorrelated or if you are not
interested in the cubic clustering criterion.

The METHOD= specification determines the clustering method used by the

procedure. Here, we are using CENTROID method.

The RMSSTD option displays the root-mean-square standard deviation of each

cluster.

The RSQUARE option displays the R2 and semipartial R2 to evaluate cluster

solution.

The NONORM option prevents the distances from being normalized to unit mean or
unit root mean square with most methods.

The values of the ID variable identify observations in the displayed cluster

history and in the OUTTREE= data set. If the ID statement is omitted, each
observation is denoted by OBn, where n is the observation number.

The VAR statement lists numeric variables to be used in the cluster analysis.
If you omit the VAR statement, all numeric variables not listed in other
statements are used.

proc tree data=tree out=clus3 nclusters=3;

id cid;
copy income educ;

1
The TREE procedure produces a tree diagram, also known as a dendrogram or
phenogram, using a data set created by the CLUSTER procedure. The CLUSTER
procedure creates output data sets that contain the results of hierarchical
clustering as a tree structure. The TREE procedure uses the output data set to
produce a diagram of the tree structure.

The NCLUSTERS= option specifies the number of clusters desired in the OUT=
data set.

The ID variable is used to identify the objects (leaves) in the tree on the
output. The ID variable can be a character or numeric variable of any length.

The COPY statement specifies one or more character or numeric variables to be

copied to the OUT= data set.

proc sort; by cluster;

proc print; by cluster;

var cid income educ;
title2 '3-cluster solution';
run;

The above commands yield the following SAS output:

Cluster Analysis for Hypothetical Data 1

The CLUSTER Procedure

Centroid Hierarchical Cluster Analysis

Variable Mean Std Dev Skewness Kurtosis Bimodality

income 16.1667 9.9883 0.2684 -1.4015 0.2211

educ 13.1667 6.3692 -0.4510 -1.8108 0.2711

Root-Mean-Square Total-Sample Standard Deviation = 8.376555

Cluster History

RMS Centroid
NCL -Clusters Joined-- FREQ STD SPRSQ RSQ Distance

5 c1 c2 2 0.7071 0.0014 .999 1.4142

4 c3 c4 2 0.7071 0.0014 .997 1.4142
3 c5 c6 2 2.5495 0.0185 .979 5.099
2 CL4 CL3 4 5.5227 0.2409 .738 13
1 CL5 CL2 6 8.3766 0.7378 .000 19.704

The statistics above provide information about the cluster solution. RMSSTD
is the pooled standard deviation of all the variables forming the cluster.
Since the objective of cluster analysis is to form homogeneous groups, the
RMSSTD of a cluster should be as small as possible. SPRSQ (semipartial R-

2
sqaured) is a measure of the homogeneity of merged clusters, so SPRSQ is the
loss of homogeneity due to combining two groups or clusters to form a new
group or cluster. Thus, the SPRSQ value should be small to imply that we are
merging two homogeneous groups. RSQ (R-squared) measures the extent to which
groups or clusters are different from each other (so, when you have just one
cluster RSQ value is, intuitively, zero). Thus, the RSQ value should be high.
Centroid Distance is simply the Euclidian distance between the centroid of the
two clusters that are to be joined or merged. So, Centroid Distance is a
measure of the homogeneity of merged clusters and the value should be small.

Cluster Analysis for Hypothetical Data 2

3-cluster solution

CLUSTER=1

Obs cid income educ

1 c1 5 5
2 c2 6 6

CLUSTER=2

Obs cid income educ

3 c3 15 14
4 c4 16 15

CLUSTER=3

Obs cid income educ

5 c5 25 20
6 c6 30 19

3
 20

D
i
s
t
a
n
c
e 15

B
e
t
w
e
e
n 10

C
l
u
s
t
e
r
5
C
e
n
t
r
o
i
d
0
s
c1 c2 c3 c4 c5 c6

cid

4
Title Non-Hierarchical Cluster Analysis of Hypothetical Data;

data t2;
input cid $ 1-2 income educ;
cards;
c1 5 5
c2 6 6
c3 15 14
c4 16 15
c5 25 20
c6 30 19
run;

proc fastclus radius=0 replace=full maxclusters=3 maxiter=20 list distance;

id cid;
var income educ;
run;

You must specify either the MAXCLUSTERS= or the RADIUS= argument in the PROC
FASTCLUS statement

The RADIUS= option establishes the minimum distance criterion for selecting
new seeds. No observation is considered as a new seed unless its minimum
distance to previous seeds exceeds the value given by the RADIUS= option. The
default value is 0.

The MAXCLUSTERS= option specifies the maximum number of clusters allowed. If

you omit the MAXCLUSTERS= option, a value of 100 is assumed.

The REPLACE= option specifies how seed replacement is performed.

FULL
requests default seed replacement.

PART
requests seed replacement only when the distance between the observation
and the closest seed is greater than the minimum distance between seeds.

NONE
suppresses seed replacement.

RANDOM
selects a simple pseudo-random sample of complete observations as
initial cluster seeds.

The MAXITER= option specifies the maximum number of iterations for recomputing
cluster seeds. When the value of the MAXITER= option is greater than 0, each
observation is assigned to the nearest seed, and the seeds are recomputed as
the means of the clusters.

The LIST option lists all observations, giving the value of the ID variable
(if any), the number of the cluster to which the observation is assigned, and
the distance between the observation and the final cluster seed.

5
The DISTANCE option computes distances between the cluster means.

The ID variable, which can be character or numeric, identifies observations on

the output when you specify the LIST option.

The VAR statement lists the numeric variables to be used in the cluster
analysis. If you omit the VAR statement, all numeric variables not listed in
other statements are used.

Non-Hierarchical Cluster Analysis of Hypothetical Data 1

The FASTCLUS Procedure

Replace=FULL Radius=0 Maxclusters=3 Maxiter=20 Converge=0.02

Initial Seeds

Cluster income educ

1 5.00000000 5.00000000
2 30.00000000 19.00000000
3 16.00000000 15.00000000

Minimum Distance Between Initial Seeds = 14.56022

Iteration History

Relative Change in Cluster Seeds

Iteration Criterion 1 2 3

1 1.5811 0.0486 0.1751 0.0486

2 1.1180 0 0 0

Convergence criterion is satisfied.

Here, the cluster solution at the second iteration is the final cluster
solution because the change in cluster seeds at the second iteration is less
than the convergence criterion. Note that a zero change in the centroid of
the cluster seeds for the second iteration implies that the reallocation did
not result in any reassignment of observations.

Cluster Listing

Distance
from
Obs cid Cluster Seed

1 c1 1 0.7071
2 c2 1 0.7071
3 c3 3 0.7071
4 c4 3 0.7071
5 c5 2 2.5495
6 c6 2 2.5495

Criterion Based on Final Seeds = 1.1180

6
 Cluster Summary

Maximum Distance
RMS Std from Seed Radius Nearest
Cluster Frequency Deviation to Observation Exceeded Cluster

1 2 0.7071 0.7071 3
2 2 2.5495 2.5495 3
3 2 0.7071 0.7071 2

Non-Hierarchical Cluster Analysis of Hypothetical Data 2

The FASTCLUS Procedure

Replace=FULL Radius=0 Maxclusters=3 Maxiter=20 Converge=0.02

Cluster Summary

Distance Between
Cluster Cluster Centroids

1 13.4536
2 13.0000
3 13.0000

The statistics used for the evaluation of the cluster solution are the same as
in the hierarchical cluster analysis.

Statistics for Variables

Variable Total STD Within STD R-Square RSQ/(1-RSQ)

income 9.98833 2.12132 0.972937 35.950617

educ 6.36920 0.70711 0.992605 134.222222
OVER-ALL 8.37655 1.58114 0.978622 45.777778

The cluster solution can also be evaluated with respect to each lustering
variable. If the measurement scales are not the same, then for each variable
one should obtain the ration of the respective Within STD to the Total STD,
and compare this ration across the variables.

Pseudo F Statistic = 68.67

Approximate Expected Over-All R-Squared = .

Cubic Clustering Criterion = .

WARNING: The two values above are invalid for correlated variables.

Cluster Means

Cluster income educ

1 5.50000000 5.50000000
2 27.50000000 19.50000000
3 15.50000000 14.50000000

7
 Cluster Standard Deviations

Cluster income educ

1 0.707106781 0.707106781
2 3.535533906 0.707106781
3 0.707106781 0.707106781

Non-Hierarchical Cluster Analysis of Hypothetical Data 3

The FASTCLUS Procedure

Replace=FULL Radius=0 Maxclusters=3 Maxiter=20 Converge=0.02

Distance Between Cluster Centroids

Nearest
Cluster 1 2 3

1 . 26.07680962 13.45362405
2 26.07680962 . 13.00000000
3 13.45362405 13.00000000 .