SCIENTIFIC PROCEEDINGS OF RIGA TECHNICAL UNIVERSITY

Computer Science Information Technology and Management Science

_____________________________________________________________________________________2002

METHODS OF FUZZY PATTERN RECOGNITION

R. Grekovs

Keywords: pattern recognition, fuzzy sets, composition of fuzzy relations, fuzzy rules, fuzzy clustering

1. Introduction. Technique of learning fuzzy rules

The first approaches to learning fuzzy rules from data were based on neural network concepts
(perceptron-based methods) [1]. The underlying principal of all these models is that they need
an evaluation of the output that the fuzzy rules represented by the neural network produce for
a given input. This can be an error measure or simply a binary evaluation. Then the models
aim is at improving the output by changing the fuzzy sets appearing in the rules in a suitable
way. This change is usually derived by some back propagation technique or simply by
heuristic algorithm. They are usually designed for the adaptation of the fuzzy sets, but
sometimes also involve a simple strategy for choosing an initial set of rules, adding or
deleting rules while learning. They strongly rely on the simple architecture that allows
identify how changes in the fuzzy sets effect the output. Therefore, they are quite suited for
tuning the fuzzy sets in this type of rules, but are not designed for learning rules.
Another popular approach to learning fuzzy rules is based on evolutionary algorithms.
Evolutionary algorithms are a quite general technique for parameter optimisation that is
motivated by concepts of biological evolution. It starts with an initial 'population' of possible
solutions of the optimisation problem that is usually randomly generated. Then changes some
of the values of the population ('mutation') and apply 'biological' operators like crossover that
exchanges values between different solutions. Then the best solutions are selected to form the
next generation which is treated in the same way until a maximum number of generations is
computed or a solution with a desired quality appears. Evolutionary algorithms require a
parameterised formulation of the problem with a quite limited number of parameters.
Therefore, they can be applied to fuzzy systems where the number of possible rules is strictly
limited.
An alternative approach to learning fuzzy rules is fuzzy clustering. Each fuzzy rule
specifies a vague point of the graph of the described function. Many fuzzy clustering
algorithms are pursuing the following strategy: A fuzzy cluster is represented by a typical
element – usually the cluster centre – and the membership degree of a datum to the cluster is
decreasing with increasing, sometimes transformed distance to the cluster centre. Many fuzzy
clustering approaches characterize each cluster by a set of parameters - prototype. Given a set
X = {x1, …, xN} ⊂ IRp of sample data, the aim is to determine prototypes minimizing the
objective function:
c N
J ( X ,U , v) = ∑∑ (u ik ) m d 2 (vi , x k ) ,
i =1 k =1
where uik - the membership degree of datum xk to cluster i;
d(vi,xk) - the distance of datum xk to the cluster i, represented by the prototype vi;
с - the number of clusters.
To minimize data points xk with a small distance d(vi,xk), to the cluster i should be assigned a
high membership degree whereas data with larger distances should have low membership
degrees. Clustering algorithm begins with a random initialisation and updates uik and vi in an
iterative procedure.
 SCIENTIFIC PROCEEDINGS OF RIGA TECHNICAL UNIVERSITY
Computer Science Information Technology and Management Science
_____________________________________________________________________________________2002

It is obvious that fuzzy clustering relies on the interpretation of the rules and is
therefore well suited for learning such rules. However, fuzzy clustering does not yield optimal
fuzzy sets due to the loss of information caused by the projection clusters.

2. Feature analysis in fuzzy pattern classification

To select most effective elements of the space a concept of composition of shadows of fuzzy
sets into 2-dimensional subspaces is introduced in [2]. As a criterion of evaluation a power of
reduction, which is received during the transformation of an initial feature set, is used.
Orthogonal shadows of these sets on co-ordinate planes will be as follows:
µ S A ( x, y ) = sup µ A ( x, y, z ), ∀( x, y ) ∈ X × Y ,
XY
z∈Z

µ S ( x, z ) = sup µ B ( x, y, z ), ∀( x, z ) ∈ X × Z
B
XZ
y∈Y

If binary shadows S A
XZ and S ZY B
are used as arguments of composition P°Q, then
reduction factors δ will serve as separation power of parameter Z when classifying patterns A
and B. Thus, two compositions could be constructed by one and the same parameter and per
each composition two reduction factors are obtained. The total reduction power could be
defined as δ Z = (δ XZ
A
∧ δ ZY
B
) ∨ (δ XZ B
∧ δ ZYA
) , where operations ∨ and ∧ are used as expressions
x ∨ y = x + y - xy; x ∧ y = x· y.
Parameter Z is not able to separate classes A and B, if the degree of an object’s
membership to classes does not depend on that parameter:
µ S XZA ( x, z ) = µ S XA ( x), ∀( x, z ) ∈ X × Z 

L 
µ S B ( z , y ) = µ S B ( y ), ∀( z , y ) ∈ Z × Y 
ZY Y 
After operations of composition are performed S XZ = S XZ A A
, S ZYB = S ZY
B
. From this it
follows that δ XZ = δ ZY = 0 and δ ZY = δ XZ = 0. Thus, δ Z = 0.
A B A B

Parameter Z has maximum separation power in comparison with parameters X and Y,

if it enables correct object classification whereas the other parameters are not able to perform
it.
The paper [3] introduces the idea of a graph of composition of fuzzy relations. As an
example, two fuzzy binary relations are used: P and Q are defined as follows:
P ⊂ X × Z, Q ⊂ Z × Y.
Relations P and Q are set through the membership functions:
µP : X × Z → [0, 1], µQ : Z × Y → [0, 1].

x1 ... xn−1 ... xn X = {x1 ,..., xn }

... ... P⊂X×Z

G z1 ... zk −1 ... zk Z = {z1 ,..., z k }

... ... Q⊂Z×Y

y1 ... y ... y j Y = { y ,..., y

m 1 m }

Fig. 1. Graph of composition P º Q

 SCIENTIFIC PROCEEDINGS OF RIGA TECHNICAL UNIVERSITY
Computer Science Information Technology and Management Science
_____________________________________________________________________________________2002

The composition of relations P º Q, will be a binary relation R = P º Q ⊂ X × Y with

the membership function as follows:
µ R ( x, y ) = sup{µ P ( x, z ) ∧ µ Q ( z, y )}, ∀( x, y ) ∈ X × Y .
z∈Z
Composition graph G, is obtained through the aggregation of fuzzy graphs of relations P and
Q over their common set Z (Fig. 1).
For each pair (x, y) ∈ X × Y there is some non-empty subset Z* ⊂ Z, in which for each
of its element z* the following holds:
µ P ( x, z * ) ∧ µ Q ( z * , y ) = µ R ( x, y ) = sup{µ P ( x, z ) ∧ µ Q ( z, y )}. (1)
z∈Z
More formally, it could be written as follows:
(∀( x, y ) ∈ X × Y )(∃Z * ⊂ Z )(∀z * ∈ Z * )
µ P ( x , z * ) ∧ µ Q ( z * , y ) = µ R ( x, y )
For any composition a mapping of X×Y into Z could be constructed that possesses property
(1):
Γ : X × Y → Z. (2)
Γ(x,y) determines some subset Z with the following properties:
∀z * ∈ I ( x, y ) µ P ( x, z * ) ∧ µ Q ( z * , y ) = µ R ( x, y ) .
For all elements z* of set Z \ I(x,y) (т.е. z* ∈ Z ∧ z* ∉ I(x,y)) the following holds:
∀z * ∉ I ( x, y ) µ P ( x, z * ) ∧ µ Q ( z * , y ) < µ R ( x, y ) .
Projections of mapping (2) into sets X and Y that correspond to IX and IY could be constructed:
I X ( x) = ∪ I ( x, y ), ∀x ∈ X ;
y∈Y

I Y ( y ) = ∪ I ( x, y ), ∀y ∈ Y .
x∈ X

Next, reduced arguments of composition - P̂ and Q̂ are determined with the graph Gˆ . It can
be seen that reduction leads to the decrease in cardinality of composition arguments P and Q.
The cardinality of fuzzy relation P could be found by summing up the membership degrees
for all the elements of relation P: P = ∑∑ µ P ( x, z)
x∈ X z∈Z

Next, a factor that evaluates the decrease in sets P and Q is introduced. Value | P̂ | will vary
within limits | P̂min | and | P̂max |. It is obvious that | P̂max | = |P|. Let’s find a possible value for
| P̂min |:
 
Pˆ = ∑∑ µ Pˆ
( x, z ) = ∑ ∑µ Pˆ
( x, z ) + ∑µ Pˆ
( x, z )  .
x∈ X z∈Z  z∈ΓX ( x ) x∈ X z∈Z \ ΓX ( x ) 
As µ Pˆ ( x, z ) = 0 for each elements z ∈ Z \ ΓX ( x) , that
Pˆ = ∑ ∑µ
x∈ X z∈ΓX ( x )
Pˆ
( x, z ) .

Value | P̂ | → min, if |ΓX(x)| → min. In the case of the strongest reduction |ΓX(x)|min = 1 the
reduction index δP ∈ [0,1] = 0, but when there’s no reduction – 1 (Fig.2).
A B C

D E F
Fig.2. Mapping of a set of values | P̂ | into a set of reduction degrees δP
 SCIENTIFIC PROCEEDINGS OF RIGA TECHNICAL UNIVERSITY
Computer Science Information Technology and Management Science
_____________________________________________________________________________________2002

The lower horizontal axis represents an axis of reduction degree δP. The upper axis is an axis
of power. Point D corresponds to the maximum value of reduction | P̂max |, but point F –
minimum value of reduction | P̂min |. In point Е a value of reduction is equal to | P̂ |. If А = 0, В
= δP, С = 1, that the proportionality expression will appear:
BC EF 1−δ P Pˆ − Pˆmin Pˆmax − Pˆ
= or = or δP = .
AB DE δP Pˆmax − Pˆ Pˆmax − Pˆmin
Thus,
∑µ
( x , z )∈ X × Z
P ( x, z ) − ∑µ
( x , z )∈ X × Z
Pˆ
( x, z )
δP = .
∑µ
( x , z )∈ X × Z
P ( x, z ) − min ∑ µ Pˆ ( x, z )
z∈Z
x∈ X

The reduction index δQ for relation Q, could be calculated by analogy.

3. Using a fuzzy logic in qualitative modelling

The method of qualitative modeling is subdivided into two parts [4]: fuzzy modeling and
linguistic approximation. Qualitative modeling is not very popular method, but its conception
is defined by fuzzy modeling or linguistic (qualitative) modeling. Before entering the main
subject, let’s overview a fuzzy modeling in fuzzy theory and qualitative reasoning in AI.
There are many interpretations of fuzzy modeling. For instance, we can consider a
fuzzy set as a fuzzy model, which uses a description language based on fuzzy logic with fuzzy
predicates. In a broader sense we can interpret the fuzzy modeling as a qualitative modeling
scheme by which we qualitatively describe system behavior using a natural language.
Qualitative model is a generalized fuzzy model consisting of linguistic explanations about
system behavior. Thus, to describe control rules, linguistic were used.
There are small distinctions and big similarities between fuzzy modeling and
qualitative reasoning. One distinction is that fuzzy modeling starts from the fact that a precise
mathematical model of a complex system cannot be obtained, whereas qualitative reasoning
starts from the fact that, although a complete may be available, it cannot provide insight into
the system; a description based on deep knowledge is needed. Similarities are found in the
confidence of the advantage of qualitative expressions, in the goals, and in some parts of
description languages for modeling. Qualitative reasoning makes use of a quantity space on
which “landmarks” are defined. Usually one mark “0” is set, and then three values similar to
fuzzy {+, 0, -} are used.
Qualitative modeling is a linguistic model. Linguistic model is a model that is
described using linguistic terms instead of mathematical equations with numerical values or
conventional logical formula with logical symbols.
Identification of fuzzy models can be divided into two types – structure identification
and parameter identification. Structure identification has to solve two problems: find input
variables and find input-output relations. It is necessary to find input candidates (often
heuristically) and input variables, which affect the output. Also, it is necessary to find the
number of rules in a fuzzy model and the division of an input space. Parameters are
coefficients in a model of functional system (for the fuzzy model – parameters of membership
function).
A method of qualitative modeling consists of two steps: review of a fuzzy model in
terms of fuzzy numbers and linguistic interpretation of fuzzy model. First, define input
variables. From the set of possible candidates input variables are selected heuristically and
 SCIENTIFIC PROCEEDINGS OF RIGA TECHNICAL UNIVERSITY
Computer Science Information Technology and Management Science
_____________________________________________________________________________________2002

those affected the output are finally selected. If take a model with four possible input
variables, then search algorithm will be as follows: on the basis of a fuzzy model with one
input four other models are created – everyone for the defined input. Then regulation criterion
is calculated (RC) for each model and one model is selected with one input to minimize RC.
Next we fix the one input selected above and add another input to our fuzzy model from
among the remaining three candidates. Fuzzy model has two inputs at this stage. We select
the second input as at the first step, according to the value of RC. This process is repeated
until RC is increasing. The search of a division of the space goes with FCM algorithm (the
fuzzy c-means method). As a result each output y is associated with a membership degree to
the fuzzy cluster В. Fuzzy cluster А in the input set can be induced by projections of cluster А
into co-ordinate axis x1 and x2 (Fig. 3).
Output
space
B
y
x
Input space
2
A2
A

A1 x
1
Fig. 3. Projection of fuzzy cluster

Thus, the following rule can be derived: if x1 is А1 and x2 is А2, then y is В. In this case fuzzy
division of input space is the direct result of fuzzy clustering. Finding the number of clusters
is the most important aspect of clustering. The following criterion is used to solve this
problem:
n c
S (c) = ∑∑ ( µ ik ) m ( x k − vi
2 2
− vi − x ) ,
k =1 i =1
where n: number of data to be clustered;
c: number of clusters, с ≥ 2;
xk: kth data, usually vector;
x: average of data: x1, x2, ..., xn;
v i: vector expressing the center of ith cluster;
⋅: norm;
µik: grade of kth data belonging to ith cluster;
m: adjustable weight (usually m = 1.5 ~ 3).

To get qualitative model on a basis of a fuzzy model the method of linguistic

approximation of fuzzy sets is used. It is necessary to find corresponding word or phrase from
initial set of words for linguistic approximation. After this procedure we will get qualitative
model with linguistic rules.

4. Fuzzy clustering and fuzzy rules

Clustering algorithm begins with random initialization and interactively changes

membership degrees uik and prototypes vi. The most simple algorithm is the fuzzy c-means
algorithm whose prototypes are simply the cluster centers in the form of vectors vi ∈ IRp, and
the distance d(vi, xk) is the Euclidean distance between xk and cluster center. Gustafson and
 SCIENTIFIC PROCEEDINGS OF RIGA TECHNICAL UNIVERSITY
Computer Science Information Technology and Management Science
_____________________________________________________________________________________2002

Kessel enriched each prototype with symmetric, positive definite matrix Сi and compute the
distance as:
d 2 (vi , x k ) = (det C i )1 / p ⋅ ( x k − vi ) Τ C −1 ( x k − vi ).
The method designed by Gath and Geva introduces an additional Pi for each prototype that
allows the algorithm to adopt to clusters of different sizes.
The principal idea to apply fuzzy clustering in order to derive fuzzy rule is that each
cluster induces a rule by projecting the cluster to the corresponding coordinate spaces. The
projection of a cluster to the ith domain is obtained by taking the ith coordinate of each data
point and assigning to it the membership degree of the original data point to the cluster. Thus,
cluster induces the rule
If ξ1 is µ1 and ... and ξр-1 is µр-1, then ξр is µр,
where µi denotes ith projection of the cluster and ξ1, ..., ξр-1 – input variables and ξр – output
variable. Therefore, fuzzy sets are defined by the projections of the cluster, whereas the
conclusion must be linear function in the input variables.
However, fuzzy clustering is not very precise deriving the rules. From one hand the
shape of a membership function is quite unusual, from the other – fuzzy clustering is
developed to divide product space, but not a simple domain. One approach is based on
developing of Takagi-Sugeno type controllers from the data with constant functions in the
conclusions of the rules. Thus the parameters that have to be determined are the constant
output values in the conclusions of the rules and the tips of the triangular membership
functions in each input domain. The good results of this approach motivated to develop a
fuzzy clustering algorithm that aims at finding fuzzy partitions for the single domains on the
basis of multidimensional data. To find fuzzy partitions it is necessary to determine a suitable
grid in the multi-dimensional space. The membership degree of a point to the cluster
represented by a grid point as the minimum of the membership degrees of the triangular
membership functions whose tips are the projections of the grid point. Fuzzy clustering starts
with equidistant triangular membership functions on the domains. We compute the
projections of the data and the membership degrees of these projections to the triangular
membership functions. Then we update the triangular membership functions by computing
new tips as the cluster centers:
∑ µ (x ) ⋅ x
n m
k =1
(v)
t old k ,v k ,v
t (v)
= ,
∑ µ (x )
new n m
k =1
(v)
t old k ,v

(v) (v)
where t new and t old actualized, respectively old tip and xk, v denotes vth projection of datum xk,
µt is the triangular fuzzy set with its tip at told
(v)
(v)
.
old

An advantage of this method is that clusters do not have an infinite range, thus data
points that are covered by other clusters far away from one cluster do not have any influence
on this cluster.

5. Fuzzy KORA-Ω algorithm

Fuzzy KORA-Ω algorithm is a fuzzy variant of КОRА-3 algorithm, introduced by

М.М.Bongard [6] in 1966. КОRА-3 is developed to solve classification problem in geology
area. Algorithm works with two disjoint classes with objects described in terms of Boolean
variables and has three stages – learning, relearning and classification.
Fuzzy KORA-Ω works in an analogous way to the KORA-3, and the essential
differences are based on the following fundamentals:
 SCIENTIFIC PROCEEDINGS OF RIGA TECHNICAL UNIVERSITY
Computer Science Information Technology and Management Science
_____________________________________________________________________________________2002

Let R={x1, …, xn} – the feature set used to describe the objects.
O1, …, Om – the objects of the training sample МА.
~ ~
K i' - the objects of the class K i in МА, i = 1, …, r.
Ci:Mi×Mi→G – a comparison criterion of values for feature xi, where Mi is the value
set associated to xi.
β:(Mi1 × …× Mis)2→G – partial similarity measure for any {xi1, …, xis} ⊂ R.
ΩO – a partial description of an object О using only the features of Ω = {xi1, …, xis} ⊆
R.
~
µi(O) – object's membership degree to K i .
A value combination a = (ai1, …, aip) for the corresponding features Ω = {xi1, …, xip}
~
form a fuzzy δi-complex feature (а, Ω) for K i' , with degree µi((а, Ω)), i = 1, …, r, if:
m
~
1. ∃O ∈ K i' | xi1 (O) = ai1 ^...^ xip (O) = aip 2. ∑ β (ΩO , a)µ (O
j =1
j i j ) ≥ δi
m

m ∑ β (ΩO , a)µ (O j i j )
3. ∑ β (ΩO j , a )(1 − µ i (O j )) ≤ δ i
'
4. µ i ((a, Ω)) =
j =1
m
,
j =1
∑ µ (O
j =1
i j )

where δi > 0 and δi' > 0 are thresholds. In the hard case µ i ((a, Ω)) = 1 . In case when β
is an equality function, Ω = 3 and µ i (O j ) ∈ {0,1} , this definition coincides with one for the
КОРА-3.
~ ~ ~
Let RC( K i' ) – the set of δi-complex feature for K i' . The set of all objects O ∈ K i' is
that

∑ β (ΩO, a) < η
~
i
( a ,Ω )∈RC ( K i' )
~
is called fuzzy ηi–remains and is denoted by R( K i' ).
A value combination a = (ai1, …, aip) for the corresponding features Ω = {xi1, …, xip}
~
form a fuzzy δi-complex feature (а, Ω) for K i' , with degree µi((а, Ω)), i = 1, …, r, if:
~
1. ∃O ∈ K i' | xi1 (O) = ai1 ^...^ xip (O) = aip 2. ∑ β (ΩO, a)µ (O) ≥ δ
~
i ij
O∈r ( K i' )

m ∑ β (ΩO, a)µ (O)

~
i

3. ∑ β (ΩO , a)(1 − µ (O
j i j )) ≤ δ '
ij 4. µ i ((a, Ω)) =
O∈r ( K i' )
m
,
j =1
∑ µ (O
j =1
i j )

where δij > 0 and δij' > 0 are thresholds, j iteration number. In the hard case
µ i ((a, Ω)) = 1 . In case when β is equality, Ω = 3 and µ i (O j ) ∈ {0,1} , this definition coincides
with one for the КОРА-3.
Let (а, Ω) be a fuzzy δi-complex feature. Its informational weight is calculated as:
P ((a, Ω)) = ∑ P( x k ) ∑ β (ΩO, a) Pi (O j ) ,
~
xk ∈Ω O j ∈K i'

where P(xk) and Pi(Oj) are the informational weights of feature xk and Oj object in the
~'
class K i , respectively.
 SCIENTIFIC PROCEEDINGS OF RIGA TECHNICAL UNIVERSITY
Computer Science Information Technology and Management Science
_____________________________________________________________________________________2002

When a new object О is going to be classified, it is compared with all fuzzy δi-
complex features of each class (complementary or not). The membership degree of О for the
~
class K i' is given by:
∑ β (ΩO, a)µ ((a, Ω)) P((a, Ω))
~
i
( a , Ω )∈RC ( K i' )
µ i (O) = max i , where max i = max ( µ i (O)) .
∑ β (ΩO, a) P((a, Ω))
~
~'
O∈K i
( a ,Ω )∈RC ( K i' )

The proposed Fuzzy KORA-Ω algorithm can be used to solve supervised

classification problems with many classes (fuzzy and hard classes – disjount or not), with
participations of objects described with any kind and number of features, any kind of
comparison criterion for each feature, any kinds of similarity measure and any set of feature
subsets to find complex features.

References
1. Klawonn F., Keller A. Learning Fuzzy Rules from Data. In:
http://citeseer.nj.nec.com/221423.html.
2. Ozols Y., Borisov A. A Comparative Analysis of the Features in the Fuzzy Patterns
Classification // In: Proc. Fourth European Congress on Intelligent Techniques and Soft
Computing, EUFIT’96 - Aachen, Germany, September 2-5, 1996 – p. 1690-1695.
3. Ozols Y., Borisov A. Pattern Classification and Feature Extraction on the Basis of
Composition of Fuzzy Relations // In: Proc. Seventh International Fuzzy Systems
Association World Congress, IFSA’97 - Prague, Czech Republic, June, 25-29, 1997 – p.
129-134.
4. Sugeno M., Yasukawa T. A Fuzzy-Logic-Based Approach to Qualitative Modelling // In:
IEEE Transactions on Fuzzy Systems, Vol.1, No.1, February 1993.
5. Klawonn F., Keller A. Fuzzy Clustering and Fuzzy Rules // In: Proc. Seventh
International Fuzzy Systems Association World Congress, IFSA’97 - Prague, Czech
Republic, June, 25-29, 1997.
6. Комплексная интерпретация геологических и геофизических данных на
вычислительных машинах. Под ред. Ш.А.Губермана - Москва, 1966.
7. De-la-Vega-Doria L.A., Carrasco-Ochoa J.A., Ruiz-Shulcloper J. Fuzzy KORA-Ω
Algorithm // In: EUFIT ’98, Aachen, Germany, September 7-10, 1998.

Romans Grekovs, Ph.D. student in the Institute of Information Technology at the Technical University of Riga.
Address: Decision Support Systems Group, Institute of Information Technology, Technical University of Riga, 1
Kalkyu Street, Riga LV-1658, Latvia. E-mail: romans.grekovs@navigator.lv.

Grekovs R. Izplūdušo tēlu atpazīšanas metodes

Šis raksts izskata dažādus darbus izplūdušo tēlu atpazīšanas jomā. Atpazīšanas metodes tiek klasificēti. Arī
aprakstīti nēperceptroniskas tēlu atpazīšanas metodes kā arī metodes, kuri izmanto izplūdušas projekcijas tēlu
atpazīšanā un klasteru analīzē.

Grekovs R. Methods of Fuzzy Pattern Recognition

This paper is dedicated to the review of some papers on the theme of fuzzy pattern recognition. There’s some
classification of recognition methods. Described non-perceptron based methods of pattern recognition,
including those, which use fuzzy projections in pattern recognition and cluster analysis.

Греков Р. Методы нечеткого распознавания образов

Данная статья посвящена обзору работ по теме распознавания нечетких образов. Приведена
классификация методов распознавания. Рассмотрены неперцептронные методы распознавания образов,
включая методы, использующие нечеткие проекции в распознавании образов и анализе кластеров.