Professional Documents
Culture Documents
56 / 114
REPORT
1
2
3
Marcos E. Cintra , Maria C. Monard and Heloisa A. Camargo
1
Exact and Natural Sciences Department Federal Territorial University of the Semi-Arid - UFERSA
Av. Francisco Mota, 572, 59.625-900 - Mossor - RN - Brazil
2
Math. and Computer Science Institute - University of So Paulo - USP, Av.
Trabalhador So-
Base (FRB) and a Fuzzy Data Base (FDB). The FRB has
the rules that form the core of the system. These rules are
of the system, stored in the FDB. The FDB and FRB are
easily understand.
Regarding DTs, the ID3 [11], CART [1], and C4.5 [13]
Fu-
rules.
In this work, we describe our fuzzy version of C4.5, na-
med FuzzyDT [3], which has been applied for the induc-
1. Introduction
of the coee rust disease in Brazilian crops [4]. This paper includes the experimental evaluation of FuzzyDT and
The
Also,
induction process.
Fuzzy systems, on the other hand, have also been successfully applied in many areas covered by machine learning. Fuzzy systems can handle uncertainty and imprecision by means of the fuzzy logic and fuzzy sets theories,
producing interpretable models.
E = {e1 , e2 , ..., en },
also named
examples or cases,
whi-
be expressed by
57 / 114
Rk
Class = Ci
X1 , ..., Xm
i.e., the
Ci C
is the
examples [12].
To understand the process of DT pruning, assume
training examples are covered by a leaf,
rectly. This way, the error rate for this leaf is dened by
However, it
CF ,
UCF (E, N ).
As the upper
is CF/ .
UCF (E, N )
3. Decision Trees
of them incor-
Some of the
25%.
they
are
usually
competitive
with
more
costly
approaches;
1
2
3
4
5
DT algorithm. A fuzzy version of the classic C4.5 algorithm was proposed in [8] (in Japanese). In this work, we
present our fuzzy version of C4.5, named FuzzyDT [3].
The classic C4.5 algorithm uses the information gain
minal node.
respective tests.
the
# At1
1 15
2 17
3 19
At2
0.6
0.9
0.4
At3 Class
3.1 Ca
2.2 Cb
2.3 Cc
...
n 16 0.8 2.9 Ca
# At1
1 Small
2 High
3 High
...
n Low
At2
Low
High
Low
58 / 114
High
High
Ca
At3
Small
At3
Class
High
Ca
Low
Cb
Low
Cc
High
At2
At1
Small
Ca
High
Cb
Small
High
Cc
At1
Small
Cc
Original dataset
High
Cb
Fuzzyfied dataset
Induced Tree
i.e., crispy features, since the continuous features are dened in terms of fuzzy sets and the training set is fuzzyed
before the DT induction takes place. Algorithm 1 details
this fuzzycation step.
At2 ,
and
At3 ),
examples, 3 at-
Cb ,
and
Cc ).
attribute. The fuzzyed version of this dataset is presented in the second block. This fuzzyed set of examples is
used to induce the nal DT, illustrated in the last block
of Figure 1.
It follows a detailed comparison between classic and
fuzzy DTs.
119.
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
Compactness :
Classic and fuzzy DTs, although sharing the same basic idea of building a tree like structure by partitioning
Compactness 82).
comparisons.
59 / 114
process. This way, the number determined by the DT algorithm might be dierent from the number of divisions
Petal
width
Petal
width
<= 0.6
> 0.6
Virginica
<= 4.9
thermore, even if this information is not available, automatic methods for the generation of fuzzy partitions can be
Compactness
On the
Inference As previously stated, a special issue regarding classic DTs is the fact that they can be seen as
a set of disjunct rules in which only one rule is red to
classify a new example.
the classic model, two branches are usually red simultaneously, each one with a degree of compatibility with an
input example. This characteristic is illustrated in Figure
2, which presents the partition of attribute
Atn
on the left,
dened by fuzzy sets S1, S2, and S3, and the membership
and
y2
Versicolor
Setosa
Figure 4 presents the fuzzy sets (Small and Large) dethe input values (
generated rules.
y1
Setosa
Large
degrees
Small
> 4.9
Versicolor
same rule.
Petal
length
Virginica
Large
Petal
length
the same attribute (81, 82, 84, 89, 95, 102, 106, and 109),
(such as feature
Small
for input
x1 ,
IF
Virginica.
Petal Width
is
0.6
THEN
Class
Petal
Length Small = 0.66; Petal Length Large = 0.34; Petal
Width Small = 0.79; Petal Width Large = 0.21) are used
degrees of the input example, shown in Figure 4 (
t-norm,
Large
Small
0.66
0.79
0.34
0.21
Large
0
1
4.9
0.1
0.6
Petal Length
y1
S1
S2
S1
S3
S2
Atm
y2
Atn
2.5
Petal Width
Atn
S3
Ato
a x1
b
c
Partition defining Attribute Atn
1.
2.
Length
For this example, using the classic fuzzy reasoning method (best rule), the class of the rst rule, which has higNotice that for an input value
x1 ,
y2 ,
y1
and
to
b,
For
to
c,
Virginica.
example and the last one is quite small (0.01). This way,
we are likely to believe, intuitively, the class of such similar examples should be the same. Nevertheless, the classic
sicolor
60 / 114
Ver-
the fuzzy sets dening the tree tests, still classies this
Examples
Features (c d)
ME
Dataset
Breast
682
(9
Credit
653
15
(6
Cylinder
277
32
Diabetes
769
Gamma
Features (c d)
ME
0)
65.10
Iono
351
34
(34
9)
45.33
Iris
150
(4
0)
35.90
0)
(19
13)
35.74
Liver
345
66.67
(7
0)
(8
0)
34.90
Segment
210
57.97
19
(19
0)
19020
10
(10
0)
64.84
Spam
85.71
4601
57
(57
0)
Glass
220
(9
0)
65.46
60.60
Steel
1941
29
(29
0)
Haberman
306
(3
0)
32.82
73.53
Vehicle
846
18
(18
0)
Heart
270
13
(13
0)
74.23
44.44
Wine
178
13
(13
0)
59.74
i.e.
than C4.5,
tasets.
results.
Dataset
FuzzyDT
C4.5
Dataset
FuzzyDT
C4.5
Breast
1.49
(0.00)
5.13
(3.03)
Ionosphere
3.99
(7.35)
11.40
(3.83)
Credit
7.81
(0.00)
13.00
(2.98)
Iris
8.00
(2.67)
5.33
(5.81)
Cylinder
6.16
(4.51)
30.65
(5.91)
Liver
36.76
(4.65)
32.74
(6.57)
Diabetes
21.84
(1.27)
25.52
(2.63)
Segmentation
12.38
(2.33)
2.86
(1.12)
Gamma
21.13
(0.70)
15.02
(0.58)
Spam
28.98
(0.00)
7.87
(1.35)
Glass
39.13
(0.00)
30.37
(6.87)
Steel
20.78
(0.93)
23.13
(2.48)
Haberman
26.67
(0.00)
29.09
(6.10)
Vehicle
25.37
(1.80)
27.07
(4.10)
Heart
14.44
(5.60)
23.11
(5.94)
Wine
5.00
(6.43)
7.25
(6.59)
6. Experiments
Gamma
61 / 114
rules for FuzzyDT against 328.70 for C4.5. No other discrepancies are present in the results regarding the average
number of rules of the induced models.
FuzzyDT
Dataset
Rules
Breast
Credit
FuzzyDT
C4.5
C4.5
SD
Rules
SD
Cond.
SD
Cond.
SD
15.00
(0.00)
12.30
(3.32(
50.00
(0.00)
52.70
(19.83)
7.80
(1.83)
19.30
(6.48)
21.40
(5.50)
90.90
(33.76)
Cylinder
45.80
(4.87)
42.80
(9.45)
198.50
(26.16)
248.50
(102.69)
Diabetes
13.40
(5.50)
23.60
(7.55)
42.60
(22.00)
150.20
(64.15)
Gamma
43.00
(0.00)
328.70
(31.36)
228.00
(0.00)
3,634.80
(435.18)
(20.35)
Glass
24.00
(0.00)
24.10
(2.17)
99.00
(0.00)
137.80
Haber
4.60
(1.20)
3.10
(1.64)
8.30
(2.10)
6.90
(4.18)
Heart
22.40
(2.84)
23.60
(3.67)
78.90
(13.87)
95.70
(23.08)
Iono
21.00
(0.00)
13.90
(1.45)
89.00
(0.00)
72.40
(13.15)
Iris
5.00
(0.00)
4.60
(0.66)
9.00
(0.00)
12.10
(3.08)
Liver
2.40
(1.28)
24.50
(4.92)
3.70
(2.00)
139.90
(34.60)
Segment
28.80
(1.89)
41.80
(2.96)
127.30
(13.52)
314.70
(35.73)
Spam
55.00
(0.00)
100.40
(10.43)
660.00
(0.00)
1,045.50
(142.89)
Steel
225.60
(2.85)
159.90
(6.74)
1,761.70
(24.48)
1,909.00
(125.86)
Vehicle
82.00
(6.47)
66.30
(7.20)
530.50
(62.02)
503.00
(79.81)
Wine
15.40
(0.80)
5.10
(0.30)
48.20
(4.40)
12.50
(1.20)
7. Conclusions
Credit, Gamma,
and
Liver
data-
dit,
the
Liver
Gamma,
Cre-
dataset.
62 / 114
panded previous experiments. We also provided a thorough comparison of some relevant issues regarding the classic
and the fuzzy models, and discussed the use of FuzzyDT
for feature subset selection. FuzzyDT was experimentally evaluated and compared to C4.5 using 16 datasets and
a 10-fold cross-validation strategy.
FuzzyDT obtained
smaller error for 10 datasets and was also able to induce models with less rules and less conditions in the rules
when compared to C4.5.
As future work we intend to compare FuzzyDT with
other fuzzy DTs, as well as with other classic DTs.
We
Repository, 2010.
[7] C. Z. Janikow.
Fid 4.1:
an overview.
In Proc. of
Faji Shisutemu
References
Wadsworth &
[12] J.
Ross
Quinlan.
Learning).
Feature sub-
C4.5:
Programs
for
Machi-
1993.
Int.
In
Fuzzy sets.
8:338353, 1965.