Advanced Review

Advances in data stream mining

Mohamed Medhat Gaber

Mining data streams has been a focal point of research interest over the past
decade. Hardware and software advances have contributed to the significance of
this area of research by introducing faster than ever data generation. This rapidly
generated data has been termed as data streams. Credit card transactions, Google
searches, phone calls in a city, and many others\are typical data streams. In many
important applications, it is inevitable to analyze this streaming data in real time.
Traditional data mining techniques have fallen short in addressing the needs of
data stream mining. Randomization, approximation, and adaptation have been
used extensively in developing new techniques or adopting exiting ones to enable
them to operate in a streaming environment. This paper reviews key milestones
and state of the art in the data stream mining area. Future insights are also be
presented. C 2011 Wiley Periodicals, Inc.

How to cite this article:

WIREs Data Mining Knowl Discov 2012, 2: 7985 doi: 10.1002/widm.52

INTRODUCTION the last category contributes to a generic controller

that could be used on top of any stream mining
D ata streams as a concept is defined as high-speed
generated instances of data that challenge our
computational systems to store, process, and reason
algorithm:

about.1,2 However, streaming data, if analyzed, is an
important source of knowledge that enables us to take
extremely important decisions in real time. The area
has attracted attention of the data mining commu-
nity over the last decade to develop new techniques
or adopt existing ones aiming to realize the many
important applications of data stream mining. Busi-
ness, scientific, and security applications have been
discussed extensively in the literature.3,4
The last decade has witnessed an active research
in the data stream mining. Hundreds of techniques
have been proposed to address the research issues of
analyzing rapidly arrived data streams in real time.
Out of the large body of literature, we can iden-
tify four different categories that have contributed in
shaping this area of research as follows. Other cate-
gories of techniques can be identified. For example, a
large body of one-pass techniques do exist in the data
stream mining literature.1 However, the impact of
the following four categories have been widely recog-
nized. The first three categories represent approaches
to building learning algorithms. On the other hand,
1. Two-phase techniques
2. Hoeffding bound-based techniques
3. Symbolic approximation-based techniques
4. Granularity-based techniques
This paper will discuss the main principles be-
hind each of the above categories and how this princi-
ple has been applied to different techniques. This dis-
cussion will be followed by presenting new directions
in the area. Finally, future insights will be given.
NOTABLE TECHNIQUES IN DATA
STREAM MINING
This section will provide a discussion of the four
identified categories of data stream mining techniques
listed in the introductory section.
Two-Phase Techniques
The two-phase techniques have been introduced by
Aggarwal et al.5 The general idea for this category of
techniques is to maintain an online summary of data

Correspondence to: mohamed.gaber@port.ac.uk using what has been termed as microclusters. Micro-
School of Computing, University of Portsmouth, Portsmouth, clustering has extended the data structure proposed
Hampshire, UK by Zhang et al.6 to develop the balanced iterative
DOI: 10.1002/widm.52 reducing and clustering using hierarchies (BIRCH).

Volume 2, January/February 2012 

c 2011 John Wiley & Sons, Inc. 79
Advanced Review
The maintenance of the online microclusters is fol-
lowed by a second phase that is done offline. This
second phase differs from one technique to another
according to whether the ultimate objective is running
a supervised or an unsupervised technique.
On the basis of the two-phase strategy, a frame-
work for clustering data streams termed as CluS-
tream has been proposed.5 The proposed technique
divides the clustering process into two components:
online and offline. The online component stores sum-
mary statistics about the data streams and the of-
fline one performs clustering on the summarized data
according to a number of user preferences such as
the time frame and the number of clusters. In an im-
portant milestone to the two-phase techniques, Ag-
garwal et al.7 have proposed an extension to CluS-
tream termed as HPStreama projected clustering
for high-dimensional data streams. HPStream has
outperformed CluStream in a number of case stud-
ies. The main motivation behind the development of
HPStream is that CluStream has not performed ef-
fectively with high-dimensionality streaming informa-
tion.
Aggarwal et al.8 have adopted the idea of mi-
croclusters introduced in CluStream5 in on-demand
classification. CluStream, as described earlier, divides
the clustering process into the two components: of-
fline and online.
On-demand classification8,9 uses clustering re-
sults to classify data using statistics of class distri-
bution. The main motivation behind the technique is
that the classification model should be used over a
time period according to the application. The tech-
nique uses microclustering for each class in the data
stream. This initialization is followed by a nearest
neighbor classification of the unlabeled data. The mi-
croclusters are the key of the proposed technique
which is the subtractive property. This property en-
ables the extraction of the needed microclusters over
the required time period.
Hoeffding Bound-Based Techniques
Domingos and Hulten 10,11
have proposed a generic
strategy for scaling up machine learning algorithms
termed very fast machine learning (VFML). This strat-
egy depends on determining an upper bound for the
learners accuracy loss as a function in the number of
examples/data records in each step of the algorithm.
Hoeffding bound12 has been the key for the develop-
ment of the VFML techniques. Hence, we have coined
this group of techniques as Hoeffding bound-based
techniques. It states that with probability 1 , the
), where (r)
true mean (r) is at least ((r) is the esti-
80 
c 2011 John Wiley & Sons, Inc.
wires.wiley.com/widm
mated mean value expressed as:

R2 ln(1/)
= ,
2n
where R is the range of the estimated number and n is
the number of points. This generic method has been
applied to an extension of the traditional K-means
clustering algorithm, VFKM, and decision tree classi-
fication, very fast decision trees (VFDT), techniques.
Unlike K-median that has been used extensively
in data stream clustering, the K-means algorithm com-
putes the cluster centers by using the mean values
of the data records assigned to the cluster under ex-
amination. VFKM13 uses Hoeffding bound to deter-
mine the number of examples needed in each step
of K-means algorithm. VFKM runs as a sequence of
K-means executions with each run uses more data
records than the previous one until the calculated sta-
tistical Hoeffding bound is satisfied.
Domingos and Hulten10,11 have developed
VFDT, which is a decision tree learning system based
on Hoeffding trees. It splits the tree using the current
best attribute taking into consideration that the num-
ber of examples/records used satisfies the Hoeffding
bound. VFDT is an extended version of Hoeffding
tree algorithm that addresses the research issues of
data streams. These research issues are as follows:
Ties of attributes: occur when two or more
attributes have close values of the splitting
criteria such as information gain.
High speed nature of data streams: represents
an inherent feature of data streams.
Bounded memory: the tree can grow till the
algorithm runs out of memory.
Accuracy of the output: is an issue in all data
stream mining algorithms.
The extension of Hoeffding trees in VFDT has
been done using the following techniques:
Ties of attributes have been overcome using
a user-specified threshold of acceptable error
measure for the output. That way the algo-
rithm running time will be reduced and it
overcomes the risk of infinite running time
of the algorithm.
The high speed nature of the streaming infor-
mation has been addressed using batch pro-
cessing. The computation of the splitting cri-
teria is done in a batch processing rather than
online processing. This significantly reduces
the time of recalculating the criteria for all
Volume 2, January/February 2012
 WIREs Data Mining and Knowledge Discovery
the attributes with each incoming record of
the stream.
Bounded memory has been addressed by de-
activating the least promising leaves and ig-
noring the poor attributes. The calculation of
these poor attributes is done through the dif-
ference between the splitting criteria of the
highest and lowest attributes. If the difference
is greater than a prespecified value, the at-
tribute with the lowest splitting measure will
be removed from memory saving the memory
of the data stream computing environment.
The accuracy of the output has been taken
into consideration using multiple scans over
the data streams in the case of low data rates,
and by using an accurate initialization of the
tree using a different, more accurate technique
to build an initial decision tree.
All of the above improvements have been
tested using synthetic data sets. The experiments
have proved efficiency of these improvements. The
VFDT has been extended to address the prob-
lem of concept drift in evolving data streams by
Hulten et al.11 The new framework has been termed
as CVFDT. It is mainly running VFDT over fixed
sliding windows in order to have the most updated
classifier. The change occurs when the splitting crite-
ria change significantly across the input attributes. It
is worth pointing out here the work by Masud et al.14
is tackling concept drifts with emerging classes.
SAX-Based Techniques
Symbolic ApproXimation (SAX) is a time series rep-
resentation that has been introduced by Keogh and
his colleagues.15 SAX has proved to be the state-of-
the-art technique in time series representation. Time
series data is a typical streaming source with a tem-
poral dimension.
In addition of being used in traditional data min-
ing techniques such as clustering, classification, and
indexing, it has achieved important breakthroughs in
finding the most different subsequence in a time series
termed discord16 and the most frequent subsequence
in a time series termed motif.17 Numerous applica-
tions have used SAX representation with notable suc-
cess. Some examples can be recalled here. It has been
reported16 that a premature ventricular contraction
could be accurately identified using discord detection
in the time series of electrocardiogram (ECG). Li and
Nallela18 have used motif discovery with SAX repre-
sentation to successfully find patterns of water level.
Volume 2, January/February 2012 
c 2011 John Wiley & Sons, Inc.
Advances in data stream mining
SAX follows three major steps in converting a
time series from its numerical form to its symbolic
form. The first step is Piecewise Aggregate Approxi-
mation (PAA). This is done by converting a time series
of size n to an arbitrarily size w using the following
equation:
n
wi
w 
Ci = Cj ,
n
j= wn (i1)+1
where Ci is the ith time point in the approximated
time series.
The second step is symbolic discretization. This
is done via producing equal areas under the curve
of the Gaussian distribution and setting respective
breakpoints. Each breakpoint represents a step from
one letter to another when replacing the approxi-
mated values produced by the PAA process by its
approximated symbolic values. The final step uses a
distance measure between each two characters that
are stored in a lookup table to find out the accumu-
lated distance between any two subsequences of times
series.
Granularity-Based Techniques
Granularity-based approach has been introduced by
Gaber et al.1921 Having noted that stream mining
techniques may fall short when running on resource-
constrained devices such as smart phones and sen-
sor nodes, the granularity-based approach works on
adapting the mining techniques to change their re-
source consumption patterns over time according to
availability of resources.
Resource consumption patterns represent the
change in resource consumption over a period of time
which is termed as time frame. The algorithm gran-
ularity settings are the input, output, and processing
settings of a mining algorithm that can vary over time
to cope with the availability of resources and current
data stream arrival rate. The following are definitions
of each of these settings:
Algorithm input granularity (AIG): AIG rep-
resents the process of changing the data
stream arrival rates that feed the algorithm.
Examples of techniques that could be used in-
clude sampling, load shedding, and creating
data synopsis. Sampling has been the choice
used in developing the granularity-based data
mining techniques.
Algorithm output granularity (AOG): AOG
is the process of changing the output size of
the algorithm in order to preserve the limited
81
Advanced Review
memory space. We refer to this output as the
number of knowledge structures. For exam-
ple, we may refer to number of clusters or
rules.
Algorithm processing granularity (APG):
APG is the process of changing the algorithm
parameters in order to consume less pro-
cessing power. Randomization and approx-
imation techniques represent the strategies of
APG.
It should be noted that there is a collective in-
teraction among the above three settings. AIG mainly
affects the data rate and it is associated with band-
width consumption and battery. On the other hand,
AOG is associated with memory and APG is asso-
ciated with processing power. However, the change
in any of them affects the other resources. The pro-
cess of enabling resource awareness should be very
lightweight in order to be feasible in a streaming en-
vironment characterized by its scarcity of resources.
Accordingly, the algorithm granularity settings only
consider direct interactions.
The algorithm granularity requires continuous
monitoring of the computational resources. This is
done over fixed time intervals/frames that we denote
as TF. According to this periodic resource monitoring,
the mining algorithm changes its parameters to cope
with the current consumption patterns of resources.
These parameters are AIG, APG, and AOG settings
discussed briefly in the previous section. It has to be
noted that setting the value of TF is a critical pa-
rameter for the success of the running technique. The
higher the TF is, the lower the adaptation overhead
will be, but at the expense of risking a high consump-
tion of resources during the long time frame.
The use of algorithm granularity as a general
approach for mining data streams will require us to
provide some formal definitions and notations. The
following are definitions that we will use in our dis-
cussion:
R: set of computational resources R = {r1 , r2 ,
. . ., rn };
TF: time interval for resource monitoring and
adaptation;
ALT: application lifetime;
ALT : time left to last the application lifetime;
NoF(ri ): number of time frames to consume the re-
sources ri , assuming that the consumption
pattern of ri will follow the same pattern of
the last time frame;
82 
c 2011 John Wiley & Sons, Inc.
wires.wiley.com/widm
AGP(ri ): algorithm granularity parameter that af-
fects the resource ri .
According to the above, the main rule to be used
to use the algorithm granularity approach is as fol-
lows:

IF ALT > NoF (ri )
TF
THEN SET AGP(ri )
ELSE SET AGP(ri ) +
Where AGP(ri ) + achieves higher accuracy at
the expense of higher consumption of the resource ri ,
and AGP(ri ) achieves lower accuracy at the advan-
tage of lower consumption of the resource ri .
This simplified rule could take different forms
according to the monitored resource and the algo-
rithm granularity parameter applied to control the
consumption of this resource. Interested readers are
referred to Ref 19 for applying the above rule in con-
trolling a data stream clustering algorithm termed as
RA-Cluster.
Interested practitioners can use the following
procedure for enabling resource awareness and adap-
tation for their data stream mining algorithms. The
procedure follows the following steps:
1. Identify the set of resources that mining al-
gorithm will adapt accordingly (R);
2. Set the application lifetime (ALT) and time
interval/frame (TF);
3. Define AGP(ri ) + and AGP(ri ) for every ri
R;
4. Run the algorithm for TF;
5. Monitor the resource consumption for every
ri R;
6. Apply AGP(ri ) + or AGP(ri ) to every ri

R according to the ratio ALT
TF
: NoF (ri ) and
the rule given;
7. Repeat the last three steps.
Applying the above procedure is all what is
needed to enable resource awareness and adaptation,
using the algorithm granularity approach, to stream
mining algorithms.
On the basis of the Granularity-based approach,
a number of data stream mining algorithms have
been developed. For a complete list of techniques,
the reader is advised to review the recent tutorial by
Gama et al.22
Volume 2, January/February 2012
 WIREs Data Mining and Knowledge Discovery
NEW DIRECTION IN DATA STREAM
MINING
Data stream mining has evolved as a new form of
online data analysis that has also challenged the com-
putational capabilities of our state-of-the-art data
processing facilities. However, advances in the com-
putational power of small computational devices
including personal digital assistants (PDAs), smart
phones and sensor nodes have realized an unpreceded
opportunity to perform ubiquitous data stream min-
ing. We can broadly categorize this area to mining
sensor data streams and mobile data mining. Recent
achievements in these areas are discussed in the fol-
lowing subsections.
Mining Sensor Data Streams
Many important applications coupled with the in-
crease of the computational power of wirelessly con-
nected sensor nodes have given birth to this new re-
search direction in the data stream mining area.
Mining data streams originated from sensor
nodes has witnessed notable success in the last few
years. Research issues associated with this area have
been detailed in Ref 3. Differences between data
stream mining in sensor networks and other platforms
as detailed in Ref 3 are as follows:
Duplication of data in densely deployment
of wireless sensor networks introduces a new
challenge.
Multilevel data mining is important in wire-
less sensor networks given that individual sen-
sors can generate local models that need to be
integrated.
Real-time data cleansing given that sensory
streaming data is likely to be noisy.
Adaptation to availability of resources is
inevitable given the limited resources that
each sensor node has. It is worth men-
tioning the success of granularity-based
approach in developing stream mining tech-
niques that are able to operate in wireless sen-
sor networks.20,23
The field is concerned with benefiting from the
large deployment of small computational devices that
are able to communicate wirelessly and have increas-
ing sensing capabilities. This rich source of streaming
data is a key to the success of many important secu-
rity, scientific, and industrial applications. Examples
of these applications could be found in Refs 3,4,24.
Volume 2, January/February 2012 
c 2011 John Wiley & Sons, Inc.
Advances in data stream mining
Mobile Data Stream Mining
The number of mobile users is in continuous in-
crease. Mobile data mining users are not an ex-
ception. Academic prototypes such as Open Mobile
Miner (OMM)25 and commercial products such as
MineFleet26 have already found their way to users.
We can date back the early start of the area of
mobile data mining to MobiMine system developed
by Karguta et al.27 Although the system targets mobile
brokers in the stock exchange area, the data mining
process has been performed on a server conserving the
scarce resources of the mobile device, a PDA in this
case. Few years later, Karguta et al.28 have developed
VEDAS system for distributed data stream mining
of a fleet of vehicles, analyzing both the drivers be-
havior and the vehicles health. The system has used
mobile devices running different data stream mining
techniques. This has been a result of the advances in
computational capabilities of our mobile devices.
Mobility of the user, connectivity problems, and
availability of computational resources are the major
research issues in this promising area of research. The
granularity-based approach has proved to be a suc-
cessful solution when running stream mining tech-
niques on mobile devices with limited resources.21
The OMM tool25 has adopted the granularity-based
approach.
Future Insights
We can state the future directions and insights in this
growing area of research:
Online medical, scientific, and biological data
stream mining using data generated from
medical, biological instruments, and various
tools employed in scientific laboratories;
Hardware solutions to small devices emitting
or receiving data streams in order to enable
high-performance computation on small de-
vices;
Developing software architectures that serve
data streaming applications;
Situation aware data stream mining that re-
calls the models built in similar situations
rather than building a new model;
Online text mining for opinion discovery with
the notable use of Web 2.0 technologies.
Conclusion
This review paper has highlighted the major strategies
and techniques used in data stream mining. We have
identified four categories of techniques: (1) two-phase
83
Advanced Review
techniques, (2) Hoeffding bound-based techniques,
(3) symbolic approximation-based techniques, and
(4) granularity-based techniques. Details of each cat-
egory have been discussed.
New directions and future insights in this grow-
ing area of research have been presented. Two re-
REFERENCES
1. Gaber MM, Zaslavsky A, Krishnaswamy S. Mining
data streams: a review. ACM SIGMOD Rec 2005,
34:1826.
2. Babcock B, Babu S, Datar M, Motwani R, Widom J.
Models and issues in data stream systems. In: Proceed-
ings of PODS. 2002.
3. Gama J, Gaber MM, eds. Learning from Data Streams:
Processing Techniques in Sensor Networks. Springer
Verlag; 2007.
4. Ganguly A, Gama J, Omitaomu O, Gaber MM, Vat-
savai RR, eds. Knowledge Discovery from Sensor
Data. Berlin, Germany: CRC Press; 2008.
5. Aggarwal CC, Han J, Wang J, Yu PS. A framework for
clustering evolving data streams. In: Proceedings of the
29th VLDB Conference. Berlin; 2003, 8192.
6. Zhang T, Ramakrishnan R, Livny, M. BIRCH: an ef-
ficient data clustering method for very large databases.
SIGMOD Rec. New York: ACM Press; 1996, 25:103
114.
7. Aggarwal CC, Han J, Wang J, Yu P. A framework for
high dimensional projected clustering of data streams.
In: Proceedings of the VLDB Conference. 2004.
8. Aggarwal CC, Han J, Wang J, Yu P. On demand clas-
sification of data streams. In: Proceedings of the ACM
KDD Conference. Seattle, WA; 2004, 503508.
9. Gaber MM, Zaslavsky A, Krishnaswamy S. A survey
of classification methods in data streams. In: Aggarwal
C, ed. Data Streams: Models and Algorithms. Springer
Verlag; 2007, 3959.
10. Domingos P, Hulten G. Mining high-speed data
streams. In: Proceedings of the 6th ACM SIGKDD In-
ternational Conference on Knowledge Discovery and
Data Mining. New York: ACM Press; 2000, 71
80.
11. Hulten G, Spencer L, Domingos P. Mining time-
changing data streams. In: Proceedings of the 7th
ACM SIGKDD International Conference on Knowl-
edge Discovery and Data Mining. New York: ACM
Press; 2001, 97106.
12. Hoeffding W. Probability inequalities for sums of
bounded random variables. J Am Stat Assoc 1963,
58:1330.
13. Domingos P, Hulten G. A general method for scaling
up machine learning algorithms and its application to
84 
c 2011 John Wiley & Sons, Inc.
wires.wiley.com/widm
search directions have been discussed. The first con-
cerns mining data originated from sensor networks.
Mobile data stream mining represents the second
area. Finally, future insights by the author have been
enumerated giving the reader some potential direction
for research.
clustering. In: Proceedings of the 18th International
Conference on Machine Learning. Williams College,
Williamstown, MA, USA, 2001, 106113.
14. Masud MM, Gao J, Khan L, Han J, Thuraisingham
BM. Integrating novel class detection with classifica-
tion for concept-drifting data streams. In: Proceed-
ings of the European Conference on Machine Learning
and Principles and Practice of Knowledge Discovery in
Databases. Bled, Slovenia, 2009, 7994.
15. Lin J, Keogh E, Lonardi S, Chiu B. A symbolic repre-
sentation of time series, with implications for streaming
algorithms. In: Proceedings of the 8th ACM SIGMOD
Workshop on Research Issues in Data Mining and
Knowledge Discovery. San Diego, CA; 2003, 211.
16. Keogh E, Lin J, Fu A. HOT SAX: efficiently finding
the most unusual time series subsequence. In: Pro-
ceedings of the 5th IEEE International Conference on
Data Mining (ICDM 2005). Houston, TX; 2005, 226
233.
17. Chiu B, Keogh E, Lonardi S. Probabilistic discovery of
time series motifs. In: Ninth ACM SIGKDD Interna-
tional Conference on Knowledge Discovery and Data
Mining. Washington D.C.; 2003, 493498.
18. Li L, Nallela S. Probabilistic discovery of motifs in wa-
ter level. In: IEEE International Conference on Infor-
mation Reuse and Integration. Las Vegas, NV; 2009,
388393.
19. Gaber MM, Yu PS. A holistic approach for resource-
aware adaptive data stream mining. J New Gen Com-
put 2006, 25:95115.
20. Phung ND, Gaber MM, Rohm U. Resource-aware on-
line data mining in wireless sensor networks. In: Pro-
ceedings of the IEEE Symposium on Computational
Intelligence and Data Mining. IEEE Symposium Series
on Computational Intelligence. Honolulu, HI; 2007,
139146.
21. Gaber MM. Data stream mining using granularity-
based approach. In: Abraham A, Hassanien A,
Carvalho A, Snase V, eds. Foundations of Computa-
tional Intelligence. Vol. 6. Berlin/Heidelberg: Springer;
2009, 4766.
22. Gama J, Gaber MM, Krishnaswamy S. Data stream
mining: from theory to applications and from sta-
tionary to mobile. In: Twenty-Fifth Symposium On
Applied Computing. Sierre, Switzerland. Available
at: http://www.csse.monash.edu.au/shonali/ACM-
Volume 2, January/February 2012
 WIREs Data Mining and Knowledge Discovery
SAC10-DS-Tutorial/Tutorial-SAC10-Final.pdf (Ac-
cessed October 20, 2011.)
23. Gaber MM, Shiddiqi AM. Distributed data stream
classification for wireless sensor networks. In: Proceed-
ings of the 2010 ACM Symposium on Applied Com-
puting (SAC). Sierre, Switzerland: ACM Press; 2010,
16291630.
24. Gaber MM, Vatsavai R, Omitaomu O, Gama J,
Chawla N, Ganguly A, eds. Knowledge Discovery
from Sensor Data. Lecture Notes in Computer Science.
Vol. 5840. Las Vegas, Berlin, Germany, NV: Springer;
2010.
25. Krishnaswamy S, Gaber MM, Harbach M, Hugues C,
Sinha A, Gillick B, Haghighi PD, Zaslavsky A. Open
Volume 2, January/February 2012 
c 2011 John Wiley & Sons, Inc.
Advances in data stream mining
Mobile Miner: a toolkit for mobile data stream mining.
ACM Knowl Discov Databases 2009.
26. Agnik. MineFleet description. Available at: http://
www.agnik.com/minefleet.html (Accessed October 17,
2011.)
27. Kargupta H, Park B, Pittie S, Liu L, Kushraj D, Sarkar
K. MobiMine: monitoring the stock market from a
PDA. ACM SIGKDD Explor 2002, 3:3746.
28. Kargupta H, Bhargava R, Liu K, Powers M, Blair P,
Bushra S, Dull J, Sarkar K, Klein M, Vasa M, Handy D.
VEDAS: a mobile and distributed data stream mining
system for real-time vehicle monitoring. In: Proceed-
ings of the SIAM International Data Mining Confer-
ence. Orlando, FL; 2004, 300311.
85