Discovering new knowledge with advanced data mining tool

Simon Kocbek1, Primoz Kosec2, Peter Kokol2, Mitja Lenic2 and Matjaz Debevc2
1
University of Maribor, Faculty of Health Care, Zitna ulica 15, 2000 Maribor, Slovenia
E-mail: simon.kocbek@uni-mb.si
2
University of Maribor, Faculty of Electrical Engineering and Computer Science,
Smetanova ulica 17, 2000 Maribor, Slovenia
E-mail: pkosec@uni-mb.si; kokol@uni-mb.si; mitja.lenic@uni-mb.si; matjaz.debevc@uni-mb.si

Abstract project. A special preparation was devoted to deaf and

In this paper we present the results of an intelligent
analysis of database which was gathered during e-
learning project called DISNET [1] in which a random
sample of around 300 unemployed people
collaborated. The intelligent data analysis using
advanced methods for decision tree construction was
used in order to try to find the main factors that
influence a trainee’s progress in knowing how to use
e-materials. We also analysed how trainee’s gender
influences on the usage of e-materials.
1. Introduction
Nowadays the importance of data mining is rapidly
increasing, which could be crucial for all the learners
who try to digest large and various amounts of data. It
is the job of data mining to help these learners by
leveraging sophisticated techniques in data analysis,
restructuring, and organization.
Data mining can also be extremely useful in the
field of education, manly because educators can
identify and monitor all students with special education
needs, who are increasingly integrated into regular
curricula. Even though the process of tracking their
progress is even more difficult, data mining is able to
provide improved situation awareness for educators.
Our goal was to find out how more successful
trainees using e-learning system defer from less
successful ones, and how each group of trainees feels
about e-learning system. We also wanted to research
what male and female trainees think about usability of
e-learning materials.
More than 300 trainees were included in the
educational process that took place in a form of
courses following the method of blended learning.
People with disabilities were also included in the
heard of hearing people for which the adopted e-
learning materials, including video interpreter, were
developed. For the purpose of evaluation and
additional improvement of e-materials, the efficiency
of learning, and trainees’ satisfaction with e-learning
materials and platform was tested.
The collection of data has been performed in the
framework of European PHARE 2003 project
DISNET entitled “Improvement of computer literacy
of unemployed adults” in the year 2006.
2. Method
Traces of machine learning community could
already be found in mid 1960s. Trough time different
approaches have evolved [2]. Most of the strength was
and is concentrated in finding a way to extract
generalized knowledge from the examples.
The selection of appropriate method for analysis of
data can be crucial for success.
2.1. Existing algorithms
Decision trees [3] are understandable to humans
and can be used even without a computer, but they
have difficulties expressing complex nonlinear
problem.
There are many other approaches, like
representation of the knowledge with rules, rough-sets,
case based reasoning, support vector machines,
different fuzzy methodologies, ensemble methods [4]
and they all try to answer the question: How to find
optimal solution, i.e. learn how to learn.
Evolutionary approaches (EA) to knowledge
extraction are also a good alternative, because they are
not inherently limited to local solution. They are based
on evolutionary ideas of natural selection and genetic

Seventh IEEE International Conference on Advanced Learning Technologies (ICALT 2007)

0-7695-2916-X/07 $25.00 © 2007
 processes of biological organisms. Genetic algorithms
are able to evolve solutions to real-world problems, if
they have been suitably encoded [5].
Hybrid approaches rest on the assumption that only
in the synergetic combination of single models can
unleash their full power [6]. Each of the single
methods has its advantages, but also limitations and
disadvantages. Therefore, the logical step is to
combine different methods to overcome disadvantages
and limitations of a single method.
2.2. Multimethod approach
While studying those approaches, we were inspired
by the idea of hybrid approaches and evolutionary
algorithms. Both approaches are very promising in
achieving the goal to improve the quality of knowledge
extraction, and are not inherently limited to sub-
optimal solutions. We also noticed that almost all
attempts combining different methods use loose
coupling approach. The methods work almost
independent of each other, and therefore a lot of luck is
needed to make them work as a team.
Multimethod approach introduces the idea of
population of different intelligent systems that can
produce multiple comparable good solutions, which
are incrementally improved using the EA approach. In
order to enable knowledge sharing between different
methods the support for transformation between each
individual method is provided. Initial population of
intelligent systems is generated using different
methods. In each generation different operations
appropriate for individual knowledge representation
are applied to improve existing and also to create new
intelligent systems. That enables incremental
refinement of extracted knowledge, with different
aspects of a given problem. For example, using
different induction methods - such as different purity
measures - can be simply combined into the decision
tree. As long as the knowledge representation is the
same, a combination of different methods is not a big
obstacle. The main problem is how to combine
methods that use different knowledge representations
(for example neural networks and decision trees). In
such cases we provide two alternatives: (1) to convert
one knowledge representation into another, using
different (already known) methods or (2) to combine
both knowledge representations into a single intelligent
system.
The first alternative requires implementation of
knowledge transmutators (for example conversion of a
neural network into a decision tree). Such conversions
are not perfect and some of the knowledge is normally
Seventh IEEE International Conference on Advanced Learning Technologies (ICALT 2007)
0-7695-2916-X/07 $25.00 © 2007
lost, but conversions can produce different aspect on
the presented problem that can lead to better results.
The second alternative requires some cut-points
where knowledge representations can be merged. In
the decision tree internal nodes or decision leafs
represent such cut points (Fig.1), i.e. a condition can
be replaced by another intelligent system (for ex-ample
support vector machine - SVM). Such trees are called
hybrid decision trees.
GA
Gini ID3, GA
GA GA
… ID3
…
Figure 1- An example of a decision tree induced using
multimethod approach. Each node is induced with
appropriate method (GA – genetic algorithm, ID3,
Gini, Chi-square, J-measure, SVM, neural network,
etc.)
3. Preparing the data
In order to evaluate the efficiency of e-learning, the
following steps had to be taken:
1. Pre-exam: before e-learning of each module a
trainee was tested for his/hers prior knowledge.
2. E-learning of one module. Four modules were
included: Internet, Information Technology (IT),
Microsoft Word and Microsoft Windows.
3. Post-exam: after the trainee had finished the course
he/she took a new test.
Each trainee had one week for learning each
module. The pre-exam results were compared to post-
exam results in order to establish how much did the
trainees learn in one week. We calculated the average
progress of each trainee and we classified them into
two different classes: (1) Class 1 – The average
trainee’s progress was below 23% and (2) Class 2 –
The average trainee’s progress was above or equal to
23%.
In order to evaluate the usability of e-learning
course the trainees answered the SUMI (Software
Usability Measurement Inventory) questionnaire [8].
We decided to create two types of datasets. Single
sample in both types had the same input values: the
answers of SUMI questionnaire and the information
 about disability. The output for the first type was the
efficiency learning class and the output for the second
type was the gender of the trainee. After clearing the
data, 169 samples were in each dataset, from which
there were: (a) 81 trainees from efficiency learning
class 1 and 88 trainees from efficiency learning class 2,
and (b) 60 female and 53 male trainees. Altogether,
there were 116 females, but we randomly reduced that
number to avoid wrong conclusions. Both datasets
have also included 5 trainees with partial and 8
trainees with full deafness.
Although 169 datasets may seem small amount of
data, the analysis of this kind of data by human would
consume too much time and energy. Because of that
reason, we decided to use data mining techniques.
4. Results
In order to perform the intelligent data analysis a
complete dataset was randomly divided into training
and testing set at a ratio 2 to 1 by multimethod tool.
In the first experiment first type of dataset was
used, where the output was the efficiency learning
class. We tested the data on several different methods
and the accuracy of classification was 75%. The most
important attribute was the following question
“Downloading and uploading data files into e-material
is difficult”. Those participants which agreed with the
statement fell into Class 1. Another interesting
attribute was the statement “I frequently need help
when I am using e-material”. Trainees which agreed
with the statement fell into the same class. In the next
experiment we used the second type dataset, which had
the trainee’s gender as the output. Again, we tested the
data on several different methods. The accuracy of
classification was around 71%. The results showed
that the most important attribute in induced decision
trees was the question “Did you often feel tense using
e-material?” Female participants answered “yes” more
often comparing to male participants.
More than half of all the females also agreed with
the statement “Sometimes I am not sure that I use the
right command for working with e-material”. On the
other hand, male trainees did not have that problem,
and most of them agreed with the statement “When I
use this e-material I feel like I have total control of it”.
5. Conclusion
We presented the results of intelligent data analysis
used in education. Advanced multimethod approach
was performed for classification of trainee’s progress
and gender. One of the most frequently shown patterns
Seventh IEEE International Conference on Advanced Learning Technologies (ICALT 2007)
0-7695-2916-X/07 $25.00 © 2007
was that men were more comfortable in using e-
materials than women. According to the average
trainee’s progress distribution the most significant
factor that influenced participant’s progress was
managing data files. Consequently, they needed help
assistance, which was crucial for defining less
successful trainees (Class 1). The results of searching
for new patterns in educational database obtained with
intelligent data analysis based on multimethod decision
tree induction approach turned out to be very
interesting. The induced decision trees were highly
accurate in the terms of total accuracy, and also
average class accuracy. However, some new
interesting patterns, which can have some influence,
were also shown but they should be more carefully
investigated and expertly evaluated. Presented results
also show that our multimethod approach for decision
tree induction can be used for knowledge discovery in
educational databases.
6. References
[1] M. Debevc, M. Verlic, P. Povalej, P. Kokol,
“Designing and implementation of ECDL e-learning
material for deaf and hard of hearing”, 34.
Österreichischen Linguistiktagung, Universität
Klagenfurt, 8.12.2006.
[2] Thrun S, Pratt L. Learning to Learn. Kluwer
Academic Publishers 1998.
[3] Quinlan JR. C4.5: Programs for Machine Learning. San
Mateo: Morgan Kaufmann publishers, 1993.
[4] Dietterich TG. Ensemble Methods in Machine Learning.
In: First International Workshop on Multiple Classifier
Systems, Lecture Notes in Computer Science. New York:
Springer-Verlag, 2000: 1-15.
[5] Goldberg DE. Genetic Algorithms in Search,
Optimization, and Machine Learning. Addison Wesley
Reading MA 1989.
[6] Iglesias CJ. The Role of Hybrid Systems in Intelligent
Data Management: The Case of Fuzzy/neural Hybrids,
Control Engineering Practice 1996; 4 (6): 839-845.
[7] M. Lenic, P. Kokol, “Combining classifiers with
multimethod approach”. In: Ajith et al. ABRAHAM. Soft
computing systems: design, management and applications
2002, Frontiers in artificial intelligence and applications.
Amsterdam: IOS Press, 2002; 87: 374-383.
[8] J. Kirakowski, M. Corbett, “SUMI: The Software
Usability Measurement Inventory”, British Journal of
Educational Technology, 24 (3), 1993, pp. 210-212.