IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, VOL. 14, NO.
1, MARCH 2013
A Study on the Method for Cleaning and
Repairing the Probe Vehicle Data
Zhaosheng Zhang, Diange Yang, Tao Zhang, Qiaochu He, and Xiaomin Lian
Abstract—Probe vehicle data are being increasingly applied
in urban dynamic traffic data collection. However, the mobility
and scale limit of probe vehicles may lead to incomplete or in-
accurate data and thus influence the measurement of the state
of traffic. At present, probe vehicle data are usually repaired by
linear interpolation or a historical average method, but the repair
accuracy is relatively low. To address the given problems, the
multithreshold control repair method (MTCRM) was proposed
to clean and repair the probe vehicle data. The MTCRM adopts
threshold control and a rule based on the approximate normal-
ization transform to clean abnormal traffic data and to fill in the
missing data by a weighted average method and an exponential
smoothing method. In this approach, we combine topological road
network characteristics to fill in the missing data from data for
neighboring road sections and repair noisy data by reconstructing
the principal components. This paper mainly focuses on analyzing
the component of the recurring pattern of probe vehicle data,
which can provide guidelines for the subsequent traffic forecasts.
The findings of data repair for different grades of road in Beijing,
China, demonstrate that the mean repair error may meet the
requirements of traffic-state measurement, demonstrating that
MTCRM can effectively clean probe vehicle data.
Index Terms—Data cleaning, data repair, multithreshold con-
trol repair method (MTCRM), normalization transform, probe
vehicle data, reconstruction of principal components.
I. I NTRODUCTION
W ITH THE rapid development of vehicle navigation sys-
tems, dynamic traffic data are being more widely ap-
plied [1]–[3]. Probe vehicles are widely used to collect dynamic
traffic data because of their wide coverage, high precision,
and excellent real-time performance [4], [5]. However, not
all road sections can be covered by enough probe vehicles
because of their high mobility and limited number [6]. Wireless
communication may also cause loss of data; therefore, probe
vehicle data may be incorrect or incomplete and thus affect the
accuracy of traffic-state measurement. Reference [7] points out
that 50% of the collected traffic data have problems such as data
Manuscript received June 18, 2012; revised August 19, 2012; accepted
August 26, 2012. Date of publication September 20, 2012; date of current
version February 25, 2013. This work was supported by the National High-tech
R&D Program (863 Program) under project (2012AA111901). The Associate
Editor for this paper was J. A. Miller.
Z. Zhang, D. Yang (Corresponding author), T. Zhang, and X. Lian
(Corresponding author) are with Department of Automotive Engineering,
Tsinghua University, Beijing 100084, China (e-mail: zzs08@mails.
tsinghua.edu.cn; ydg@mail.tsinghua.edu.cn; zhang-t@mails.tsinghua.edu.cn;
lianxm@tsinghua.edu.cn).
Q. He is with Department of Industrial Engineering and Operation
Research, University of California, Berkeley, CA 94709 USA (e-mail:
heqc0425@berkeley.edu).
Color versions of one or more of the figures in this paper are available online
at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TITS.2012.2217378
1524-9050/$31.00 © 2012 IEEE
419
error and data loss, which may increase the risk of instability
when these data are used in transportation systems. Therefore,
data cleaning and repair are very important tasks for obtaining
accurate dynamic traffic information [8].
II. R ELATED W ORK
Probe vehicle data cleaning involves identifying errors in the
raw data and then filtering or removing them. The removed
data are regarded as missing information and are completed
by data repair. At present, although methods for traffic flow
data have been well developed, few studies have investigated
the cleaning and repair of probe vehicle data. Jarema et al. [9]
proposed treating missing data by a historical average method
that replaces the incorrect or missing data with historical data
from the same period. Then, Jiang et al. [10] improved the
method by considering historical traffic data, average values of
adjacent time periods, and data of the adjacent road sections
to correct the missing data. Jacobson et al. [11] suggested
the threshold control algorithm, which identifies abnormal data
based on the idea that the values of traffic flow parameters for
a certain time interval should be within a reasonable range.
According to this principle, traffic parameters outside this range
are recognized as incorrect data. However, in this method, it
is difficult to accurately determine the threshold values for the
data range. Coifman [12] developed a method for determining
whether the data are reasonable using three parameters (vehicle
speed, flow, and occupancy). However, the application of their
method is limited because this method requires traffic flow
conditions to be determined and a valid range of average vehicle
length given. Some improved methods, such as analyses of
traffic flow data by clustering methods, have been suggested
in [13] and [14].
All the given studies are based on traffic flow data, which
include several types of information such as vehicle speed, flow,
and occupancy. In cases of missing single-property data, the
traffic data can be completed by the other data. However, probe
vehicle data only include vehicle speed information and cannot
be cleaned by methods developed based on traffic flow. With
the wide use of probe vehicle data [15]–[23], processing these
data is becoming increasingly important. Yu et al. [24] filled the
missing data (less than four data points) by linear interpolation
while deleting the source data for days, missing more than four
data points. Lv et al. [25] developed a nonparametric regression
method for repairing missing data, although their approach
requires a large amount of historical data under various traffic
conditions and is thus unsatisfactory when sufficient data are
unavailable. In summary, methods based on traffic flow cannot
be applied to repairing probe vehicle data; therefore, probe
420 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, VOL. 14, NO. 1, MARCH 2013

vehicle data are currently repaired by simple linear interpo-

lation or historical average methods. As a result, the repair
precision is low, which may influence the accuracy of traffic-
state measurement.
In this paper, based on the periodicity and spatial relation-
ships of probe vehicle data, we develop a new method for
cleaning incorrect data by thresholding and a 3σ rule method
based on a pseudonormalization transformation. Missing data
are completed by numerical analysis taking into account the
topological characteristics of the road network. Additionally,
noisy data are filtered by principal component reconstruction
to further improve the accuracy of the data repair. It should Fig. 1. Changes of vehicle speed (z-axis, in km/h) with time (x-axis) and date
be noted that the content of this paper converged into single- (y-axis).
property data (speed) process, for which, in general, these
probe vehicle data combined with speed, occupancy, and flow
together cannot be directly provided by the traffic data provider.

III. T RAFFIC DATA C LEANING

As a result of the regularity of urban traveling, vehicle speed
shows periodicities and trends. The data collected on a road
section over multiple days can be expressed as a matrix X in
⎡x x ··· x ⎤
1,1 1,2 1,N
⎢ x2,1 x2,2 ··· x2,N ⎥
X=⎢
⎣ .. .. .. .. ⎥
⎦ (1)
. . . .
xM,1 xM,2 ··· xM,N
where xi,j represents the vehicle speed at moment j on
day i, M is the number of days of data collection, and
N is the number of data collected per day. The row vec-
tor Xi = xi,1 , xi,j , . . . , xi,N  of X represents the vehi-
cle speeds recorded at different moments on day i and
is called the date vector; analogously, the column vector
Xj = x1,j , xi,j , . . . , xM,j  of X represents the vehicle speeds
recorded at the same time point but on different days and is
called the moment vector.
If collection interval is shorter, then the vehicle speed varia-
tion and the noise will be greater, which may add to the difficul-
ties of data processing. Conversely, a longer acquisition interval
offers a smoother curve but does not reflect the real-time
conditions of traffic because of the lower acquisition frequency.
The Highway Capacity Manual of the USA recommends that
an acquisition interval of 5 min is appropriate. Therefore, in
our probe vehicle system, a 5-min collection interval is used,
which gives 288 data points per day.
Fig. 1 shows the changes of vehicle speed with time and
date. The speed varies slightly from day to day, but the general
trend remains similar. In this paper, missing or incorrect probe
vehicle data are repaired based on the characteristics of time
series and similarity analysis of adjacent roads.
A. Raw Data Screening
Before further careful screening, raw data are rough-screened
by the following three steps to remove invalid data.
1) Filtering negative data. Negative vehicle speed data are
obviously errors; therefore, these data are removed and
replaced with 0.
Fig. 2. Removal of abnormal traffic data. The symbol +shows historical
average data.  shows the upper threshold.  shows the lower threshold.
× shows measurement data. The ellipse indicates detected abnormal data.
2) Conditions of excessive missing data. When large
amounts of data are missing, the characteristics of traf-
fic operation cannot be accurately described by the es-
tablished traffic model; therefore, these data should be
deleted. If the missing data volume for one day reaches
10% of the total data or the continuously missing data
exceeds 5%, the data for the entire day are discarded.
3) Identification and filtering of abnormal data. The col-
lected vehicle speed data may deviate from the normal
speed data owing to factors such as traffic accidents and
weather factors. Although the deviation may be reason-
able considering that road traffic is unavoidable, as seen
from a long historical trend, such speed flow is occasional
and uncertain. This paper mainly focuses on analyzing the
component of the recurring pattern of the probe vehicle
data, which can provide guides for the subsequent traffic
forecasts; however, this uncertainties may interfere with
the study of the characteristics of vehicle speed and traffic
planning. Thus, these data were removed from this paper.
Fig. 2 shows the calculated mean value X j for each moment
vector Xj , where the standard deviation of Xj is σj . The
85% confidence interval of the moment vector Xj is [X j −
1.44σj , X j + 1.44σj ]. For the date vector Xi , if 5% of the
data are consecutively outside of the confidence interval, then
the vector contains abnormal data.
ZHANG et al.: STUDY ON THE METHOD FOR CLEANING AND REPAIRING THE PROBE VEHICLE DATA 421

Fig. 3. Q–Q plots showing normality of the data before (left) and after (right) pseudonormalization transformation.

B. Abnormal Data Cleaning

Instantaneous data are considered abnormal if they greatly
deviate from the center of the distribution. In this paper, a
probability model is established for the moment vector Xj , and
the data are then cleaned based on a 3σ rule.
The 3σ rule is applicable to data that follow a normal distri- Fig. 4. Schematic showing spatial distribution of road sections in a local
bution. Therefore, the vector Xj is first checked for normality area. Black circles represent road junctions. Lines represent roads. Numbers
with a quantile–quantile (Q–Q) plot. If it does not satisfy represent IDs of roads. Arrows point in the direction of traffic.
normality, the data in the vector should be transformed to a
IV. T RAFFIC DATA R EPAIR
pseudonormal distribution using a modified power function, as
shown in A. Missing Data Repair
 xγ −1
(γ) γ , γ = 0 Missing data are very common in data collection and may be
x = (2) either isolated or consecutive. In this paper, the missing data are
ln(x), γ = 0.
processed, as explained in the following.
For the measured data values x1 , x2 , . . . , xM , Box and Cox For isolated missing data xt , a weighted average method is
[26] give a method for calculating the optimal exponential γ used for data repair. Contrary to the mean value repair method,
in (3), which makes the equation to give the maximum value, using a weighted average takes advantage of trends over time,
as follows: thus reducing the influence of variations in the adjacent data, as

1
 (γ) 2
M shown in the following:
M
l(γ) = max − ln x −x (γ)
M i=1 i 1

2 T
x̂t = wk · xt+k (k = 0). (5)

M
W
+ (γ − 1) ln(xi ) (3) k=−T

i=1
where
1
xγ − 1
M
x(γ) = . (4)
M i=1 γ
To reduce the computational complexity, γ should be within
the interval (0, 5). Fig. 3 shows data before and after this trans-
formation. The closer the sample data approach the straight
line, the more the samples comply with a normal distribution.
It is shown in Fig. 3 that the transformed data (x(γ) ) fol-
low a pseudonormal distribution. Thus, the confidence inter-
val of x(γ) can be determined using the 3σ rule: [x(γ) −
3σ (γ) ), x(γ) + 3σ (γ) )], where σ (γ) represents the standard
deviation of x(γ) , and abnormal data are deleted based on the
confidence interval. After the data identified as abnormal are
In (5), x̂t is the repaired missing data, wk is the weighting
coefficient, W is the sum of all weighted coefficients, and
T is the maximum interval for repairing data. Note that wk
decreases further from the missing measurement point. The
maximum interval T of the neighboring data is set to 3, and
the corresponding weight coefficients wk are set to 0.7, 0.2 and
0.1, respectively. Considering that vehicle speed–time curves
usually display clear trends of increase or decrease, continuous
missing data are repaired by a secondary exponential smoothing
method as follows:
x̂t+r = at + bt r (r = 1, 2, . . .) (6)
where at and bt are intermediate variables, which are given in
the following:
 (1) (2)
at = 2Qt  − Qt 
removed, the missing data are repaired by a weighted average (1) (2) (7)
method or an exponential smoothing method.
α
bt = 1−α Qt − Qt .
422 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, VOL. 14, NO. 1, MARCH 2013

TABLE I
C ORRELATION C OEFFICIENTS OF N EIGHBORING ROAD S ECTIONS

(1) (2)
In (7), Qt is the primary exponential smoothing value, Qt In this paper, reconstruction of principal components is used to
is the secondary exponential smoothing value, and α ∈ (0, 1) is remove high-frequency noisy data. Compared with other noise
the smoothing coefficient, i.e., eliminating methods, reconstruction of principal components is
able to process the data for several days, and the regularity and
(1) (1)
Qt = αxt + (1 − α)Qt−1 trends of time series are also used, thereby reducing the volume
(2) (1) (2) of data processing and improving the precision at the same time.
Qt = αQt + (1 − α)Qt (8)
For a sampling data matrix X, each date vector corresponds
to a variable, and each moment vector corresponds to a sample.
Additionally, the quality of data repair can be improved by
The covariance matrix S for recording samples is
taking into account spatial similarities in traffic information,
such as traffic data for the upstream and downstream road 1

N
sections and nearby road sections. The experiment described S= (Xj − X j )(Xj − X j )T (11)
N − 1 j=1
in Fig. 4 provides the correlation between neighboring road
sections at some time.
Consider segment 23010 in Table I. The correlation coeffi- where T is the transpose of the matrix, λ1  λ2  · · ·  λM
cient with the neighboring road sections is relatively large but are the eigenvalues of S, and U{uj |j = 1, 2, . . . , M } indicates
decreases as the distance increases. Specifically, the correlation the corresponding orthogonal unit eigenvector matrix. The prin-
between the upstream and downstream road sections and seg- cipal component matrix Y of the data matrix X is given as
ment 23010 is relatively large, whereas the correlation between ⎡y y ··· y ⎤
1,1 1,2 1,M
segment 23010 and the parallel segments is relatively small; ⎢ y2,1 y2,2 ··· y2,M ⎥
therefore, the correlation coefficient can be calculated by the Y = XT U = ⎢
⎣ .. .. .. .. ⎥
⎦. (12)
adjacent road section. . . . .
The speed on road section 23010 can be calculated based on yN,1 xN,2 ··· yN,M
the similarities with adjacent road section, as expressed in the The mth main component contribution rate Zm of the principal
following: component matrix Y is shown as

Hω

M
x̂ω
t = βhω · xω
h (h) (9) Z m = λm λi (m = 1, 2, . . . , M ). (13)
h=1 i=1

where ω is the ID number of road that missing data, H ω is the Then, the contribution rate of the first m principal compo-
total number of adjacent segments of road ω, h is the sequence s
nents, Zm is
number of adjacent road, xω h (h) is the vehicle speed on adjacent

m
road h at time t, and βhω is a weight factor given in λi
s i=1
rhω Zm = . (14)
βhω = . (10) M

Hω λi
rτω i=1
τ =1
The principal components of matrix Y are used to recon-
In (10), rhω is the coefficient of correlation between the road struct the original data matrix X. Taking the first p principal
section that is missing data and its neighboring road section. In components with contribution rates Zm s
 95%, the recon-
general, the correlation coefficients are different for different 
structed matrix X is given as
segment at each day; thus, for the data without correlation ⎡y
coefficient, rhω was set to the average value of all the correlation 1,1 y y
1,2 ··· 0⎤
1,p
coefficients at those days earlier than the appointed day. ⎢ y2,1 y2,2 y2,p ··· 0⎥
X = ⎢
⎣ .. .. .. .. .. ⎥
⎦U
−1
(15)
. . . . .
B. Repair Method for Noisy Data yN,1 xN,2 yN,p ··· 0

In traffic flow analyses, dimension reduction is frequently
used to isolate the important information in the data. Principal
component analysis (PCA) is a main tool for dimensionality
reduction. Earlier research has suggested that traffic flow can
be classified into an eigenflow plus noise [27]. These noise data
refer to those data that cannot reflect the traffic characteristic.
where U −1 represents the inverse of U.
The effect of noise reduction using this PCA-based method
is shown in Fig. 5. After the high-frequency noise in the data is
reduced, the curve is smoother, and the fluctuations are smaller,
assuming that the transient characteristics of speed data remain
normal.
ZHANG et al.: STUDY ON THE METHOD FOR CLEANING AND REPAIRING THE PROBE VEHICLE DATA 423

March 31, 2011 and are computationally repaired using the

methods described earlier.
The experimental results were evaluated using the following
indicators.
relative error (rerr):
x̂ − x
rerr =
x
mean absolute relative error (mrerr):
n  
1
 x̂i − xi 
mrerr =
n i=1  xi 

max absolute relative error (marerr):

 
 x̂i − xi 
marerr = max   
xi 
Fig. 5. Comparison of traffic data before and after PCA noise reduction. The
symbol ∗ shows raw data before noise reduction. + shows PCA data after noise coefficient of equivalence (EC):
reduction. − shows noise. 
n
(x̂i − xi )2
i=1
EC = 1 −   .

n
2 
n
(x̂i ) + (xi )2
i=1 i=1

Fig. 6. Standardized residual and autocorrelation function of traffic data after
PCA noise-reduction treatment.
The standard deviations of the residual and autocorrelation
function are shown in Fig. 6. Almost all (96.5%) of the standard
residual deviations fall in the range (−2, 2), and the autocorre-
lation with time lag approaches zero, indicating that the residual
is a normally distributed white noise.
V. E XPERIMENTAL VALIDATION
Different types of roads are associated with different den-
sities of probe vehicles and, therefore, different data qualities.
We validated the above data cleaning/repair methods with data
collected on four types of roads in Beijing, China: freeways,
express roads, arterial roads, and minor arterial roads. Probe
vehicle data are provided by traffic service providers; it col-
lected from 23 000 vehicles, and these data covered 85% of
the minor arterial roads and superior roads in Beijing with an
interval of 5 min between January 1, 2011 and March 30, 2011.
As the missing and incorrect data were removed, the vehicle
speed data within 81 days were left. Missing and incorrect
data were artificially introduced to information recorded on
EC reflects the match error between the actual data and the
repair data. A value of EC > 0.9 was considered to indicate an
excellent repair.
When repairing the data with the weighted average method,
the maximum interval T of the neighboring data is set to 3, and
the corresponding weight coefficients wk were set to 0.7, 0.2
and 0.1, respectively. For consecutive missing data, the expo-
nential smoothing coefficient (α) was empirically determined
as 0.5, and the neighboring segments in speed estimation by
neighbor segment spatial similarity were H = 5. Expressway
data repair by the given method is shown in the following.
In Fig. 7, the data marked with stars are raw data, the data
marked with round solid points are modified data, and the
data marked with crosses are man-made abnormal traffic data.
The right side of Fig. 7 shows a magnified portion of the left
side of Fig. 7. Noise is removed from the modified data by
reconstruction of principal components. Compared with the
raw data, the processed vehicle speed not only maintains the
transient characteristics but also eliminates the high-frequency
noise in the data, thus smoothing the speed curve and laying a
foundation for subsequent traffic-state data identification. The
traffic abnormal data in different times (one day) were artifi-
cially randomly introduced and were processed by MTCRM,
conventional linear interpolation, and history average methods,
respectively. The results are compared and shown in Fig. 8.
The errors of the repaired data are shown in Table II. As
shown in Table II, compared with the conventional historical
average and linear interpolation method, MTCRM shows sig-
nificant advantages in precision and optimality. The 89% of the
relative error for the expressway is within 3%, and the average
absolute relative error is 2.3%. The repair error statistics for
different types of abnormal data are shown in Table III, where
more than two consecutive abnormal data points are called
consecutive abnormal data.
424 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, VOL. 14, NO. 1, MARCH 2013

Fig. 7. Expressway data repair.

Fig. 8. Repaired data comparison between MTCRM and conventional methods.

TABLE II TABLE III
E RROR C OMPARISON FOR D IFFERENT R EPAIR M ETHODS S TATISTICS OF D IFFERENT T YPES OF A BNORMAL DATA E RRORS

data, the repair precision of the algorithm is slightly lower,

As shown in the given table, discrete abnormal data were and the maximum error of consecutive abnormal data is 4.1%.
repaired relatively accurately, with a maximum relative error of The repair results for minor arterial roads, arterial roads, and
2.5%. In cases of multiple consecutive neighboring abnormal freeways are shown in Fig. 9.
ZHANG et al.: STUDY ON THE METHOD FOR CLEANING AND REPAIRING THE PROBE VEHICLE DATA 425

Fig. 9. Repair of vehicle speed data recorded on different types of roads. (a) Minor arterial road data repair. (b) Minor arterial road repair error. (c) Arterial road
data repair. (d) Arterial road repair error. (e) Freeway data repair. (f) Freeway repair error.

As shown in Fig. 9, the relative error of minor arterial road data
repair is relatively large, whereas the error associated with the
freeway is smaller, with a relative error within ±1.5% for 95%
of the time, and within ±3% even for time periods with obvious
speed fluctuation. The difference can be mainly attributed to dif-
ferent vehicle speeds and the number of probe vehicles on the two
types of roads. Compared with freeways, there are fewer probe
vehicles, and the vehicle speed is lower on minor arterial roads.
Figs. 8 and 9 also clearly indicate morning and evening rush
hours on the four types of roads. The vehicle speeds are sub-
stantially lower during rush hours and higher during the night.
Additionally, the rush hour patterns were consistent on the
four types of roads, and such periodicity and consistency can
facilitate the identification and repair of traffic data. Table IV
lists indicators of repair error other than errors resulting from
historical averaging.
426 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, VOL. 14, NO. 1, MARCH 2013
TABLE IV
E RROR I NDEXES FOR D IFFERENT ROAD T YPES
The errors calculated for different road types are shown in
Table IV. For the repair of data from express roads, for which
there is smaller data fluctuation, the mean absolute error was
only 0.76%, and the EC was 0.96. The quality of probe vehicle
data is significantly improved without requiring more probe
vehicles or additional data processing equipment, indicating
that MTCRM can effectively clean probe vehicle data.
VI. C ONCLUSION
The precision of the current repair and cleaning method for
probe vehicle data is low, which may influence the accuracy
of traffic-state prediction. In this paper, we have developed
new numerical analysis methods for cleaning probe vehicle
data. A new method is presented for data repair and cleaning
using principal component reconstruction based on the traffic
information and topological characteristics of the road network.
Experimental validation shows that abnormal data can be
effectively identified by a 3σ rule based on an approximate
normalization transform. Principal component reconstruction
can effectively exploit the periodicity and trends in traffic data
to reduce the influence of noise on probe vehicle data and
improve the data accuracy. This paper has focused on the single-
property data (speed) process in probe vehicle data as those
combined with speed, occupancy, and flow together cannot be
directly provided by the traffic data provider; nevertheless, we
believe our newly developed method has great potential for
future transportation research.
R EFERENCES
[1] T. Zhang, D. G. Yang, T. Li, K. Q. Li, and X. M. Lian, “An improved
virtual intersection model for vehicle navigation at intersections,” Transp.
Res. C, Emerging Technol., vol. 19, no. 3, pp. 413–423, Jun. 2011.
[2] J. W. Ding, C. F. Wang, F. H. Meng, and T. Y. Wu, “Real-time vehicle
route guidance using vehicle-to-vehicle communication,” IET Commun.,
vol. 4, no. 7, pp. 870–883, Apr. 2010.
[3] F. Dion, J. S. Oh, and R. Robinson, “Virtual testbed for assessing probe
vehicle data in IntelliDrive systems,” IEEE Trans. Intell. Transp. Syst.,
vol. 12, no. 3, pp. 635–644, Sep. 2011.
[4] J. E. Naranjo, F. Jiménez, F. J. Serradilla, and J. G. Zato, “Comparison
between floating car data and infrastructure sensors for traffic speed es-
timation,” in Proc. 13th Int. IEEE Conf. Intell. Transp. Syst., Workshop
Emergent Coop. Technol. Intell. Transp. Syst., 2010.
[5] Z. Yang, “Research and implementation of large-scale FCD processing,”
M.S. thesis, Dept. Pattern Recogn. Intell. Syst., Univ. Sci. Technol. China,
He Fei, China, 2010.
[6] J. E. Naranjo, F. Jiménez, F. J. Serradilla, and J. G. Zato, “Floating car
data augmentation based on infrastructure sensors and neural networks,”
IEEE Trans. Intell. Transp. Syst., vol. 13, no. 1, pp. 107–114, Mar. 2012.
[7] M. Zhong, P. Lingras, and S. Sharma, “Estimation of missing traffic
counts using factor, genetic, neural, and regression techniques,” Transp.
Res. C, Emerging Technol., vol. 12, no. 2, pp. 139–166, Apr. 2004.
[8] X. Y. Wang, J. L. Zhang, and X. Y. Yang, The Theoretical Approaches of
Traffic Flow Data Cleaning and State Identify as well as Optimize Control.
Beijing, China: Science, 2011, pp. 13–18.
[9] F. Jarema, C. Dahlin, and R. Gillmann, “FHWA study tour for european
traffic monitoring programs and technologies,” Federal Highway Admin.,
U.S. Dept. Transp., Washington, DC, 1997.
[10] G. Y. Jiang, L. H. Gang, X. D. Zhang, and J. F. Wang, “Malfunction
identifying and modifying of dynamic traffic data,” J. Traffic Transp. Eng.,
vol. 4, no. 1, pp. 121–125, Jan. 2004.
[11] L. N. Jacobson, N. L. Nihan, and J. D. Bender, “Detecting erroneous loop
detector data in a freeway traffic management system,” Transp. Res. Rec.,
vol. 1287, pp. 151–166, Mar. 1990.
[12] B. Coifman, “Improved velocity estimation using single loop detectors,”
Transp. Res. A, Policy Pract., vol. 35, no. 10, pp. 863–880, Dec. 2001.
[13] X. Y. Gong, “Traffic flow data filtering algorithms based on data mining,”
in Proc. Nat. ITS Syst. Traffic Inf. Collect. Integr. Tech., Hangzhou, China,
2003, pp. 163–173.
[14] L. Sun and J. Zhou, “Development of multi regime speed-density rela-
tionships by cluster analysis,” Transp. Res. Rec., vol. 1934, pp. 64–71,
Feb. 2005.
[15] A. Simroth and H. Zähle, “Travel time prediction using floating car data
applied to logistics planning,” IEEE Trans. Intell. Transp. Syst., vol. 12,
no. 1, pp. 243–253, Mar. 2011.
[16] J. F. Ehmke, S. Meisel, and D. C. Mattfeld, “Floating car based travel
times for city logistics,” Transp. Res. C, Emerging Technol., vol. 21, no. 1,
pp. 228–352, Apr. 2012.
[17] B. Mehran, M. Kuwahara, and F. Naznin, “Implementing kinematic wave
theory to reconstruct vehicle trajectories from fixed and probe sensor
data,” Transp. Res. C, Emerging Technol., vol. 20, no. 1, pp. 144–163,
Feb. 2012.
[18] Q. Ou, R. L. Bertini, J. W. C. van Lint, and S. P. Hoogendoorn, “A theoret-
ical framework for traffic speed estimation by fusing low-resolution probe
vehicle data,” IEEE Trans. Intell. Transp. Syst., vol. 12, no. 3, pp. 747–
756, Sep. 2011.
[19] J. J. V. Díaz, D. F. Llorca, A. B. R. González, R. Q. Mínguez, Á. L.
Llamazares, and M. Á. Sotelo, “Extended floating car data system: Exper-
imental results and application for a hybrid route level of service,” IEEE
Trans. Intell. Transp. Syst., vol. 13, no. 1, pp. 25–35, Mar. 2012.
[20] S. Breitenberger, K. Bogenberger, M. Hauschild, and K. Laffkas, “Ex-
tended floating car data—An overview,” in Proc. World Congr. Intell.
Transp. Syst., Madrid, Spain, Nov. 2003.
[21] L. Lin, T. Osafune, and M. Lenardi, “Floating car data system enforce-
ment through vehicle to vehicle communications,” in Proc. 6th Int. Conf.
ITS Telecommun., Jun. 2006, pp. 122–126.
[22] S. Maerivoet and S. Logghe, “Validation of travel times based on cellular
loating vehicle data,” in Proc. Eur. Congr. Intell. Transp. Syst., Aalborg,
Denmark, Jun. 2007.
[23] S. Messelodi, M. Modena, M. Zanin, F. G. B. De Natale, F. Granelli,
E. Betterle, and A. Guarise, “Intelligent extended floating car data col-
lection,” Expert Syst. Appl. Int. J. Arch., vol. 36, no. 3, pp. 4213–4227,
Apr. 2009.
[24] L. Yu, L. Yu, Y. Qi, and H. Wen, “Traffic incident detection algorithm for
urban expressways based on probe vehicle data,” J. Trans. Syst. Eng. Inf.
Technol., vol. 8, no. 4, pp. 36–41, Aug. 2008.
[25] W. F. Lv, Y. Liang, T. Y. Zhu, and D. D. Wu, “An FCD compensation
model based on traffic condition trends matching,” in Proc. 4th ICCIT,
2009, pp. 1201–1206.
[26] G. E. P. Box and D. R. Cox, “An analysis of transformations,” J. R. Stat.
Soc. B, Methodol., vol. 26, no. 2, pp. 211–252, Apr. 1964.
[27] A. Lakhina, K. Papagiannaki, M. Crovella, C. Diot, E. D. Kolaczyk, and
N. Taft, “Structural analysis of network traffic flows,” in Proc. SIGMET-
RICS, 2004, pp. 61–72.
ZHANG et al.: STUDY ON THE METHOD FOR CLEANING AND REPAIRING THE PROBE VEHICLE DATA
Zhaosheng Zhang was born in Hezhe, China, in
1984. He received the B.S. degree in automotive en-
gineering from Hunan University, Changsha, China,
in 2008. He is currently working toward the Ph.D.
degree in automotive engineering with the Depart-
ment of Automotive Engineering, Tsinghua Univer-
sity, Beijing, China.
His research interests include data processing, ve-
hicle navigation, and path planning.
Diange Yang received the B.S. and Ph.D. degrees
in automotive engineering from Tsinghua University,
Beijing, China, in 1996 and 2001, respectively.
He is currently an Associate Professor with the
Department of Automotive Engineering, Tsinghua
University. His research interests include intelligent
transport systems, vehicle electronics, and vehicle
noise measurement.
Dr. Yang received the Second Prize from the
National Technology Invention Rewards of China
in 2010 and the Award for Distinguished Young
Science and Technology Talent of the China Automobile Industry in 2011.
Tao Zhang received the B.S. and Ph.D. degrees in
automotive engineering from Tsinghua University,
Beijing, China, in 2005 and 2010, respectively.
He is currently a Postdoctoral Researcher with the
Department of Automotive Engineering, Tsinghua
University. His research interests include vehicle
navigation and electronic control.
427
Qiaochu He received the B.S. degree in automo-
tive engineering from Tsinghua University, Beijing,
China, in 2011. He is currently working toward the
Ph.D. degree in operation research with the De-
partment of Industrial Engineering and Operation
Research, University of California, Berkeley.
His research interests include convex optimiza-
tion, stochastic processes, and their applications in
service operation management.
Xiaomin Lian received the B.S., M.S., and Ph.D.
degrees in automotive engineering from Tsinghua
University, Beijing, China, in 1982, 1986, and 1997,
respectively.
He is currently a Professor with the Department of
Automotive Engineering, Tsinghua University. His
research interests include vehicle Global Positioning
System navigation, vehicle electronics, and vibration
control.