Advanced Engineering Informatics 22 (2008) 493–503

Contents lists available at ScienceDirect

Advanced Engineering Informatics

journal homepage: www.elsevier.com/locate/aei

Automated construction activity monitoring system

Danijel Rebolj *, Nenad Čuš Babič, Aleš Magdič, Peter Podbreznik, Mirko Pšunder
University of Maribor, Faculty of Civil Engineering, Construction Informatics Centre, Smetanova 17, SI 2000 Maribor, Slovenia

a r t i c l e i n f o a b s t r a c t

Article history: Building activities are inadequately monitored during construction projects when attempting to satisfy
Received 30 October 2007 the need for information, especially in those cases of unforeseen events and conditions on-site. Many
Received in revised form 4 June 2008 IT-supported methods have been introduced, but so far none has been able to deliver satisfactory and
Accepted 7 June 2008
reliable information. This paper presents a combined method, consisting of three components: an auto-
Available online 8 August 2008
mated activity tracking subsystem based on image recognition, an automated material tracking subsys-
tem, and a mobile computing supported communication environment. All subsystems are based on the
Keywords:
same 4D model of a building.
Progress monitoring
Resource tracking
Ó 2008 Elsevier Ltd. All rights reserved.
Project platform

1. Introduction
Delays are a very common and costly problem encountered dur-
ing construction projects [1–3]. The cause lies either in unrealistic
planning or unforeseen site conditions [4]. In the first case, the
solution is clearly to prepare more realistic plans but, regrettably,
unforeseen events and conditions are a fact on every construction
site and often result in changing the duration of the affected activ-
ities. The only solution, in such cases, is timely reaction in order to
minimize or even avoid delaying the whole project. Management
information systems [5,6] and new approaches, such as dynamic
planning and control [7], help to improve project control, but any
chosen system or methodology depends on reliable and relevant
information [8].
In practice, data collection and activity-progress monitoring is
still mainly based on traditional methods, which are slow, inaccu-
rate, and expensive [9,10]. If on-site activities progress according
to plan, then the time-interval between an on-site event and the
moment data is fed to a control system should be insignificant.
However, if a critical-path activity is being delayed, the whole
activity plan has to be rescheduled immediately in order to mini-
mize any negative impacts on the project. According to experience,
the monitoring of activity start-times is problematic on the daily
level [4].
Construction companies are gradually recognizing the problem
of timely information and are putting lots of efforts into capturing
and analyzing activity data [11]. Regrettably, in real situations, too
many construction projects sooner or later erode the benefits of
carefully prepared activity plans and degenerate into improvisa-
* Corresponding author. Tel.: +386 2 2294 381.
E-mail address: danijel.rebolj@uni-mb.si (D. Rebolj).
tion. The only solution for ensuring a consistent flow of relevant
information seems to be the automation of data collection [12].
Many attempts have already been made, using various approaches,
in order to control construction project performance [4,12]. They
have been based on indirect indicators, such as labor productivity
[13,14], use of equipment [15], materials’ flow [10,16], or directly
measured activity progress, like some recent methods based on
site image recognition [17,18]. Further attempts have been re-
ported where mobile devices have been used by workers to sup-
port faster and more reliable data collection [19–21].
In addition to coordination (planning) and control, there is a
third perspective regarding project execution: communication
[22]. Many methodologies have been published and applied to sup-
port and improve communication in both the strategic and opera-
tional aspects of project management [23,21,24,7].
In our approach we take into account all three perspectives:
coordination, control and communication. Activity progress is mon-
itored directly by using a combination of data collection methods.
Such an integrated approach has already been suggested by some
authors to improve data reliability (e.g. Navon [4]), but we have
found no evidence, in literature, regarding implementation. In the
paper we first present the overall concept of the system, then the
components and system integration, and finally a case study.
2. Concept of the automated activity monitoring system
The goal of an automated monitoring system is to acquire data,
convert it into information and deliver that information on-time in
regard to better project performance. In our approach we focus on
activities as the main entities in the construction information loop,
which includes activity plans (schedule plan, 3D model), on-site
activity progress, and activity reports (Fig. 1).

1474-0346/$ - see front matter Ó 2008 Elsevier Ltd. All rights reserved.
doi:10.1016/j.aei.2008.06.002
494 D. Rebolj et al. / Advanced Engineering Informatics 22 (2008) 493–503
Fig. 1. Construction activity information loop.
The automated data collection methods of the system are based
on the building information model (BIM), especially on the 4D
model of the building [25]. Due to the important role of activities
during the monitoring process, these have to be defined consis-
tently in a way that bears a clear relation between activities and
elements of the 3D model [26]. The first automated method is
based on site images of the building, which are then compared to
the 4D graphic representation of the building within the same
time-frame. This method is further presented, in detail, in the sec-
tion about system components (4D-ACT). The main task of this
component is to discover any differences between the planned
and built elements of the building.
The second data collection method is based on automated
material tracking. The method itself is described in detail in the
Material resource tracking section. It can be used as a stand-alone
system, but when in combination with the 4D-ACT, it has an addi-
tional function of improving the reliability of activity data. Data
from both components are cross-checked regarding consistency
using the activity plan. In the case of a clear link between those
material resources needed for an activity, and a specific material
ID in the material-flow (for example unique prefabricated ele-
ments), the 4D-ACT results can be confirmed with high precision.
However, even if material resources have no unique link with a sin-
gle activity (for example concrete), their consumption can be
checked against those resources needed to perform an activity, as
detected by the 4D-ACT subsystem.
In the case of a one-to-one relation an activity is complete when
the related element is detected on-site while, in case of one-to-
many relations it is even possible to determine the percentage
completed, by counting the detected elements. In the cases of
many-to-one or many-to-many relations between activities and
BIM elements, however, combined information is necessary in or-
der to determine a completed activity (as for example, activities for
building a column need various resources such as reinforcement,
panels, concrete, which can be monitored together with the image
of the column).
The third component of the automated construction activity
monitoring system is the Dynamic Communication Environment
(DyCE), which adds expert information to each activity being per-
formed on-site. This information is delivered by the on-site staff,
by using mobile computers to communicate project-related
information.
All real-time on-site activity-related information is synthesized
and cross-checked before any activity plan re-scheduling is sug-
gested by the system. After the activity plan is re-scheduled by
the responsible manager, the 4D model is reformed and the new cy-
cle starts; a single pass of the monitoring loop is shown on Fig. 2.
3. System components
The automated construction activity monitoring system con-
sists of three main components. Each of them can be used as a
stand-alone system, but the combination significantly raises the
reliability of the system as a whole. This system is not, as yet, fully
functional or fully automated because the components are not yet
fully developed. We believe, however, that the current stage of
development already proves the concept. Each component is de-
scribed, in detail, in the following sections.
3.1. Automated construction activity tracking system (4D-ACT)
Information technologies enable a combination of different
types of information into a consistent structure called a 4D model
[27]. The 4D model contains a product and process model and,
thus, integrates information about geometry and building activi-
ties. We propose a solution and have developed a 4D model-based
Automated Construction activity Tracking system (4D-ACT) for the
effective detection of differences between as-planned and as-built
situations, which uses logical, temporal and spatial information
from a 4D model, and images of the construction site. The system
performs a real-time comparison between site images and those
images extracted from the 4D model. 4D-ACT contains application
modules (4D tool, segmentation, calibration, recognition, etc),
which have been separately tested on different real cases. The inte-
grated system has been tested within an experimental environ-
ment, as well as on-site case studies.
 D. Rebolj et al. / Advanced Engineering Informatics 22 (2008) 493–503
Fig. 2. Conceptual scheme of the automated construction activity monitoring system.
3.1.1. Introduction
Image segmentation, pattern recognition, and computer vision
have a wide range of applications over different engineering areas.
In civil engineering, these methods have already been used in (i)
3D reconstruction of building elements and building identification
[28–36], (ii) image reasoning for search of jobsite pictures (iii) ter-
rain modeling using aerial images [34,37], (iv) automatic detection
of cracks [38] and surface defects [39] on bridge constructions, (v)
automated detection of cracks in buried concrete pipes [40], (vi)
robotized inspection of cracks in tunnels [41], (vii) building object
recognition based on color [42] and material properties [43,44],
(viii) road detection from aerial images [45] and (ix) automated
detection of tooling defects on polished stone [46].
3.1.2. System architecture
3.1.2.1. 4D tool. When constructing the 4D model, the 4D tool
needs to contain two inputs: the IFC product model and the sche-
dule plan in MS Project format. Any connections between the
building elements of the geometry model and activities are per-
formed by manually selecting elements and corresponding activi-
ties (Fig. 3). The main motivation for developing our own 4D tool
was to obtain full control over the data structure in the 4D model.
The most important feature of the 4D tool is the 3D reference
model, which presents the 3D model at a defined point of time dur-
ing the building process. A model-view image of the 3D geometry
(total or partial) is generated from the 3D reference model based
on the parameters of a virtual camera. This virtual camera has
the same parameters (position, orientation and viewing direction)
as the camera on the building site, but is observing a virtual build-
ing – the 3D reference model (Fig. 4).
3.1.2.2. Image segmentation. Images contain a lot of information. In
order to establish the recognition process of the building elements,
495
it is necessary to extract various levels of information (colors, gra-
dients, textures etc.) for which segmentation is the most common
method. The region-growing [47] was chosen as the most suitable
method for the segmentation of noisy building site images. The
segmentation process is based on finding areas of pixels with sim-
ilar predefined features.
The algorithm establishes criteria from a learning set before
starting the segmentation process. The user marks those small
pieces of image which are members of the area he wants to seg-
ment. These pieces define the learning set. The result of segmenta-
tion is an extracted image area, which has a certain level of
similarity regarding the learning set. In this way, those parts of
the image which do not belong to the building (for example tem-
porary equipment) are filtered out.
The segmentation module was tested using images of an
experimental wooden model and images from a real building site.
Fig. 5b presents successful segmentation of the wooden model
image (Fig. 5a), where all conditions were optimal. On the build-
ing site image, however (Fig. 5c), problems with image exposure
and temporary equipment led to unsatisfactory segmentation
(Fig. 5d). In order to alleviate this kind of problem, we plan to
use multiple calibrated cameras, which can merge camera views
into a union view space.
3.1.2.3. Camera calibration. When a segmented shape extracted
from a site image matches with a shape from the 3D refer-
ence model image, then the building element is easily rec-
ognized. However, parts of building objects are usually
covered by temporary equipment and, thus, collection of
information about observed objects are impractical. For this
reason, the images are captured from multiple cameras at
different positions and orientation. Merging data from multi-
ple cameras is possible after they are calibrated. Calibration
496 D. Rebolj et al. / Advanced Engineering Informatics 22 (2008) 493–503

Fig. 3. 4D tool enables 4D model construction using product and process models. Only the facade panels were visible for easier element selection.

Fig. 4. Virtual camera’s view, generated from the 3D reference model (a) and the segmented experimental site image – a wooden model (b).

can be performed by various methods [48–50] such as:
eight-point algorithm, LMedS, RANSAC, M-estimator, etc. The
M-estimator calibration method was chosen for use in 4D-
ACT.
3.1.2.4. Recognition of building elements. Segmented site and
model-view images both show the same elements in the same per-
spective, by considering that the parameters of the virtual and
building-site cameras are the same.
Comparison between the segmented site image and the mod-
el view image is done an automated recognition algorithm,
based on minimum differences between element features from
both images [51]. If a difference is under a predefined threshold,
then the element from the segmented image has the highest
probability of being identified as an element on the model-view
image.
Different scenarios can be expected during the recognition pro-
cess. Successful matching of all elements from the segmented im-
age is the best scenario and means
 the learning set has been marked optimally,
 images from the building site were successfully segmented, and
 activities on-site match with the planned activities.
In those cases of unsuccessful matching, 4D-ACT identifies and
lists non-matched elements as either missing elements or un-
known elements (intruders).
Various reliability levels for building element recognition can be
reached, depending on building complexity, camera system, the
building site, or building process technology. The segmented area
on the site image (Fig. 4b) is compared with elements on the
view-image. In order for a particular element to be recognized, it
has to be visible on both images. The 4D-ACT successfully recog-
nized all visible building elements from the site image of the
experimental wooden model, using a single camera. For these par-
ticular building elements, differences between ‘‘as-planned” and
‘‘as-built” can be confirmed. Further development of the system
should be oriented towards recognizing obscured building
elements by using multiple cameras.
 D. Rebolj et al. / Advanced Engineering Informatics 22 (2008) 493–503
Fig. 5. Input, and the results of the segmentation process in an experimental environment (a and b), and on a real building site (c and d).
3.2. Material resource tracking
Previous research has shown [52,53,4] that the visibility and
tracking of material resources in construction projects plays an
important role in the total project’s success. For this reason, the
construction industry has adopted supply chain management
(SCM) methods used by manufacturing industries to ensure better
control and, thus, better flow of materials. The problem is not as
yet solved to the extent required by the industry [52,4]. Current
practice shows that manual data collection at construction sites
is not an approach, which satisfies the information requirements
for today’s projects. Data quality is low and it requires too much
time and effort, resulting in infrequent and incomplete project con-
trol [54]. However, SCM implementations already exist at con-
struction sites and, nevertheless, represent an important source
of information for decision makers at the project management
level.
In our efforts, we went beyond material management only. The
main aim of our work was in the field of automated project perfor-
mance control. The hypothesis was that the spending of material
resources can effectively serve as an indirect measure of a project’s
progress. The idea is based on the fact that it is necessary to estab-
lish an automated material resource tracking system in order to
achieve better productivity, avoid project delays, and reduce waste.
Existing data gathered through supply-chain management activi-
ties, when properly integrated into a project information system
based on BIM, can be further used to derive information about a
project’s progress.
The material management system is based on the 4D model.
Links between material resources and BIM elements are estab-
lished through the activity plan of the 4D model, where each activ-
ity includes information about necessary resources. The status of a
material has to be propagated in both directions between the mod-
el and the SCM system. In general, implementation can follow dif-
ferent supply chain management approaches. For example,
implementation of ‘‘Materials management and control model”,
497
as defined by Navon [4], would satisfy these criteria via its input
and tracking units.
Fig. 6 shows how supply chain management is mapped to the
main contractor’s information system, in order to achieve addi-
tional benefits for the purpose of project monitoring. Building
information is available in the 4D model, where building elements
are linked to those activities contained in the project’s plan. For
every point-in-time during the project, material quantities and
material scheduling requirements can be derived for prefabrication
and material procurement. These requirements are more or less
detailed depending on planning methods, such as incremental
planning, and the project’s progress. On the other hand, material
flow is registered on the construction site. Basically this informa-
tion describes which material has been delivered to the site and
when it was used during building activities. Information is propa-
gated from the construction site back to the information system of
the main contractor.
The site manager initiates detailed planning based on the
existing 4D model. This plan defines successive activities, and
the model shows the accomplished work, and the available parts
or materials on-site. The site manager, therefore, has immediate
information about possible material inadequacies, and can act in
a timely manner. The plan is propagated across the organization
and generates more-detailed needs for procurement and
prefabrication.
In addition to the process described above, project tracking also
includes activities regarding direct progress monitoring, such as
on-site inspection by a site manager or capturing images of a con-
struction site and using image recognition systems. These activities
confirm or reject the validity of any assumed progress indicators.
3.3. Dynamic communication environment (DyCE)
A construction process or phase of a construction project is
characterized by a complex interactivity, interdependence and
communication between different project partners. In this
498 D. Rebolj et al. / Advanced Engineering Informatics 22 (2008) 493–503
orders
Material supply
Prefabrication (external)
Prefabrication (internal)
deliveries
Receiving
spent
Construction on site
site manager
Fig. 6. 4D-based supply chain management and coordination.
constantly reactive production environment, handling problem sit-
uations results in natural communication patterns that are dy-
namic, spontaneous and informal [24]. Additionally, a vast
potential for on-site problem-solving lies in the knowledge, expe-
rience and skills of the site staff themselves, and a proper mobile
interpersonal communication network, which can link together
the rich knowledge and experience of site staff and facilitate infor-
mation exchange for problem solving [55].
A pragmatic DyCE approach focuses on enhancing individual
project communication networks and is based on an assumption
that the construction process should be modeled as a socio-techni-
cal system, and should focus on human’s work [56]. DyCE extends
the scope of common (mainly hierarchically oriented and inflexi-
ble) information and communication systems by using mobile
computing potentials. It can cover the whole organization, build
a uniform information system, and dynamically adapt to relevant
users, projects, and tasks by using context information. Thus the
main features of DyCE are context sensitivity and context adapt-
ability (data filtering), instant access to process information, peer
availability (current status and location), and ad-hoc problem-ori-
ented team building and communication. From the functionality
aspect, DyCE forms a personal workspace (environment) providing
the necessary information on an ‘‘as-needed” basis – at the right
time, and the right place. Further details can be found in [55].
Since DyCE uses a project task or activity as the basic entity
around which information is allocated and communicated, it is
an ideal component to complement the automated activity moni-
toring system.
4. System integration
The Automated construction activity monitoring system is com-
plex but manageable, due to the modular system architecture (Fig.
2). The common point of the described components is the 4D mod-
orders
Procurement
needed
ERP
available/needed
progress
4D project manager
el or BIM in general, where activities are the basic elements. Each
component can be used to signal activity plan inconsistencies inde-
pendently, by applying its own inference rules. By combining infor-
mation coming from each component, it significantly raises the
level of reliability. If 4D-ACT recognizes a building element as
being built, but the material tracking system still detects the pre-
fabricated element to be on the repository, then one piece of infor-
mation is obviously untrue. The status then has to be checked
manually, whereby the additional activity information gathered
through the DyCE communication system raises the efficiency level
when solving the obvious information inconsistency. In case all
subsystems deliver consistent information, its reliability then be-
comes crucial.
We have developed a tracking information control-loop pro-
gram in order to support information consistency checking, with
the following input:
1. original project schedule (task begin, end, resource
information),
2. task schedules from the image recognition based tracking sys-
tem 4D-ACT (task begin, end),
3. material in-use time from the material resource tracking
system.
By a simple cross-check of input data, the control program pro-
vides the following output:
1. confirmation or negation of input data consistency,
2. if input is consistent then relevant activities status in the pro-
ject schedule is updated and the responsible manager
notified,
3. if input is inconsistent then the differences between planned
and tracked information is shown and the site manager is asked
to manually check and input the correct activity status,
 D. Rebolj et al. / Advanced Engineering Informatics 22 (2008) 493–503
4. in the case where the detected activity status is different from
the plan, the manager is asked to update the activity plan and
the 4D model consecutively.
In regard to case 3, there is a need for an adequate time-win-
dow, because there is always a time-gap between the moment
when the resource tracking system detects a certain resource at
the building area gate, and the time when the corresponding activ-
ity is detected by the activity-tracking system. The time gap is set
according to the building technology for each type of activity.
5. Case study
Based on the concepts described above, we conducted a pilot
implementation of the system in a company during an industrial-
ized construction process. The company is medium-sized and pri-
marily produces storage buildings, industrial units, and large
storage buildings, in Slovenia and abroad. The buildings consist
of load-bearing steel or concrete structures, with metal roofs and
facade elements. In addition to construction projects, the company
also manufactures roof and facade elements for the market.
Our work started with an analysis of the project’s workflow
after the project contract has been signed, given that pre-sales
and pure manufacturing processes are well organized and ade-
quately supported by ICT tools. Fig. 7a illustrates the initial situa-
tion at the company. From the project viewpoint, the work
processes at the company can be divided into three groups: de-
tailed design, prefabrication, and construction site activities. De-
tailed design is the basis for manufacturing and it also produces
blueprints for construction. The work is well-supported by CAD
(a) Existing construction project IT support
Project plans
building
components
production
ERP
(b) Updated construction project IT support
Project plans
building
components
production
Document
ERP
management
Fig. 7. The current (a) and the planned state (b) in the company where the system components were tested.
499
tools. Prefabrication is organized as mass-production and is highly
automated. Industrialized production of building elements is inte-
grated with other business activities through an Enterprise Re-
source Planning (ERP) system. Construction site activities are
project-oriented and include organizing the construction site, con-
struction work, project progress monitoring and management
activities, and the tracking of material flow to and within the con-
struction sites. The analysis showed that this group of activities is
insufficiently supported by ICT tools. Detailed planning and data
collection concerning the performed construction work was incon-
sistently handled, with several software tools depending on each
individual project manager.
During the process analysis we identified two main improve-
ments, which could significantly influence overall performance.
Firstly, integration of building design and industrialized production
should be achieved, and secondly, on-site project management and
project documentation related activities should be consolidated
and integrated within the manufacturing. The first stage of imple-
menting the proposed system focused on the project progress mon-
itoring and tracking of building elements production and delivery.
5.1. Image recognition
A single camera was used to test the 4D-ACT system in the de-
scribed case-study. Fig. 8a shows a picture taken from the camera
at a specific point-in-time. This picture has been segmented by the
segmentation module (Fig. 8b) and then used, together with the
adequate image extracted from the 3D reference model (Fig. 8c),
as input for the building elements recognition module. In the pre-
sented case, the module correctly identified the roof element (Fig.
design
Modelers
&
calculations
Drawings
&
blueprints construction
site
design
Modelers
&
calculations
4D
BIM
construction
site
video
monitoring
&
4D explorer
&
project portal
500 D. Rebolj et al. / Advanced Engineering Informatics 22 (2008) 493–503

Fig. 8. (a) site image of the building describes the as-built situation (b) segmentation of the site image (c) image of the 3D reference model depicts the as-planned situation
(d) the difference between as-planned and as-built models.

8d) as the difference between both images, and thus the difference
between the existing and planned situation.
The vehicle in Fig. 8a has been correctly filtered-out during the
segmentation process as it was not recognized as a member of the
learning set (it differs in gradient, texture and color).
5.2. Tracking of materials
In regard to material tracking, a lightweight 4D browser was
implemented (Fig. 9) with functionality that supports all four tasks
as presented in Fig. 6: an overview of available material, detailed
planning, reporting on delivered materials and material’s con-
sumption. The site manager is constantly informed about the cur-
rent state of the model. The building structure can be displayed in
2D or 3D and enables the site manager to explore every activity of
the project plan and its mapping onto the building model. For each
activity, the building elements are presented with detailed infor-
mation regarding material needs and availability status such as
‘‘planned”, ‘‘in fabrication”, ‘‘in stock”, ‘‘on-site”, ‘‘spent” and
‘‘built-in”. Based on this information, the site manager performs
detailed planning of the successive construction activities. On the
other hand, possible problems with required materials can be eas-
ily recognized and acted upon. This tool enables reporting of mate-
rial needs to the main contractor’s office and pre-fabrication
facilities.
Insights into the project plan via the 4D model also proved to be
useful for the other side of the supply chain. One example is pack-
aging of building elements at a pre-fabrication facility. When pre-
paring package plans, planners can actually see elements’ positions
on the building and the approximate mounting schedule. It is then
easier to decide which elements should be delivered within the
same package.
Our pilot implementation particularly focused its attention on
the tracking of prefabricated material. When a material unit is

Fig. 9. Material ordering and monitoring at the construction site handled by 4D browser.
 D. Rebolj et al. / Advanced Engineering Informatics 22 (2008) 493–503
delivered to the construction site, it is assigned as ‘‘received” sta-
tus, and when the package is opened, the ‘‘spent” status. At pres-
ent, registration of deliveries and consumption is performed
manually using the same 4D browser as described above. However,
we are performing extensive tests of RFID in this regard, with re-
spect to automate registration of a material’s status. The proximity
based method as described by Song [57] is particularly interesting
in the context of tracking pre-fabricated elements.
This tool is a networked application with data caching capabil-
ities. At present it is used on a site managers’ or foremen’s’ laptops.
Special care has to be taken about internet connectivity to mini-
mize the burden of the user. Our experience shows that for users
to accept an application in everyday work it is necessary to avoid
any additional administration operations such as connecting to
the network. At present, internet connection at the construction
site is implemented via UMTS/GPRS network, which actually
proved to be available at all construction sites of our test-bed
company.
Caching enables disconnected work in cases where internet
connection is unavailable, and it is necessary to finish an operation.
In the cases of processes that should be handled without delay,
such as material delivery registration, this is particularly useful.
All collected information is synchronized with the 4D server where
information from material management is merged with informa-
tion from other sources and the status of activities is updated.
5.3. Communication support and information consistency checking
Unfortunately it was impossible to implement the DyCE com-
munication system in the same case study to avoid the introduc-
tion of too many new tools to the staff at the same time. DyCE
has been used on another building site with another construction
company and we can reliably expect to provide useful additional
information for information checking.
In the current phase of the case study, we were mostly inter-
ested in the reliability of two components, the automated activ-
ity-tracking system and the material-tracking system. We
observed and analyzed the outputs of both systems indepen-
dently and were disinterested in automated integrated informa-
tion consistency checking, which is a rather simple program
that merges and reduces the output of all components into any
optimal information required for decision makers in the process.
This part will be added in the following stages of system imple-
mentation after the basic components are thoroughly tested and
accepted.
6. Discussion
According to the current case-study, the automated activity
tracking concept and the 4D-ACTsystem are fulfilling expecta-
tions. So far the system has been mainly tested under optimal
conditions (weather, light and visibility, perspective, type of con-
struction) because we have only used a single camera. But we are
aware that the system has to be reliable under difficult condi-
tions. Further improvements of algorithms and simultaneous
use of multiple cameras integrated into a common view space
should improve overall reliability, and also address problems of
hidden construction elements. Activities that are performed in-
side buildings can be observed using inside cameras or even
moving cameras. There is a still lot of research to be done in this
area.
The main goal of the whole system is to automatically monitor
and update the activity plan. Under the above stated conditions
this goal has been reached at specific points. So far the system only
notifies the user about activity status in the form of a simple list.
We plan to link the monitoring system with a project management
501
system in the next stage of our research. Integration with a mate-
rial tracking system should further improve the reliability of the
integrated, automated construction activity monitoring system.
Another feature of this integration will be the possibility of identi-
fying those individual activities that are linked to the same BIM
elements (many-to-many relation) according to those resources
related to each activity.
Integration of supply chain management techniques with the
4D model of the construction object increases the visibility of
material resources flow across the supply chain. From the perspec-
tive of the four roles of supply chain management in construction,
as identified by Vrihjhoef and Koskela [52], our proposed concept
targets ‘‘Role 1: improving the interface between site activities
and the supply chain” and ‘‘Role 4: integration of site and supply
chain”. Suppliers are more directly linked to the construction site
and are better informed about the needs and changes in plans.
Coordination and communication among participants is improved.
This is especially valid for industrialized construction where build-
ing parts are prefabricated.
Improved transparency of material flows also improves flexibil-
ity during detailed planning. Site managers have a clear picture of
incoming material flow, so they can customize short-term plans
according to available resources. On the other hand, as a risk con-
tingency measure, they can proactively communicate their urgent
needs to direct suppliers.
In a broader project management sense, the concept and the
tool are steps towards efficient implementation of the Last Planner
system introduced by Ballard [58]. It supports all four main catego-
ries: should, can, will and did. A site manager, as the last planner,
gets a tool for planning short term activities (should), he is always
aware about available resources for the time-frame he is dealing
with (can), a decision can be made on what will be performed in
the next step and this decision is propagated up the management
ladder (will), and when work is performed the status is updated
(did).
Our implementation can be compared with the shipment track-
ing based approach reported by Ala-Risku [53], which registers
incoming and outgoing flows of material resources at defined
points in the supply chain. In our pilot implementation, the track-
ing unit is a package that contains a certain number of building ele-
ments. We are now exploring different levels of detail of material
tracking (package level, main building element item level, support-
ing material item level, etc).
The crucial part of implementation is the streamlined data col-
lection of received and spent material at a construction site. We are
extensively exploring RFID technology to support the automation
of shipment tracking regarding incoming and outgoing flows. From
the technological viewpoint, we are working on problems relating
to tracking metal building elements, where the traditional RFID ap-
proach experiences significant problems [59]. From the business
point of view, additional research is performed with regard to
the configuration of construction sites, and on-site material repos-
itory organization.
As a difference to the existing shipment tracking approach, we
introduce 4D model as a reference point for data integration. This
links material resources directly to the building and the building
schedule. The benefits of 4D are various. Implementation of a 4D
model can be executed using certain widely recognized standards
(such as IFC for the BIM part of the model), which allow the heter-
ogeneous partners of a construction project to exchange informa-
tion in a standardized way. This reduces the cost of tool
implementation and broadens opportunities for SMEs, which are
very common in the construction industry.
Secondly, material needs are directly linked to project schedule
on whatever level of detail the schedule is developed. Stakeholders
in the supply-chain can get a better overview of project needs.
502 D. Rebolj et al. / Advanced Engineering Informatics 22 (2008) 493–503
Visibility of spent material status from the 4D model view-
point introduces a ‘‘supposed to be finished” status of project
activities. This brings information about project progress to pro-
ject management level faster than existing progress reporting.
Our experience shows that site managers on small to medium
sized projects are responsible for several construction sites and
are unable to monitor every site on a daily basis, in fact the con-
trol period can extend to a week or even more. Automated col-
lection of project progress information is therefore valuable.
Despite the fact that ‘‘supposed to be finished” status is just
an assumption of real activity progress, our pilot shows that
timely information outperforms problems introduced by false
alarms. The approach reduces latency between the occurrence
of a certain event and acknowledgment of the same event at
project management level. It brings more control to the project
and reduces the possibility of heavy consequences regarding pro-
ject schedule and budget.
According to observations during the field tests, the person-ori-
ented DyCE approach changed the way the project team communi-
cated. The permanent availability of the key project’s participants
in the virtual workspace established cross-functional/cross-organi-
zational communication and circumvented the traditional chain of
command. Collaborative problem solving was facilitated and solu-
tions were more effective through the instant creation of ad-hoc
teams and tacit knowledge exchange. From the aspect of a project,
this means a much smoother flow of information and, thus, a high-
er level of quality. While the technology itself showed positive ef-
fects for the improvement of project communications, there is an
evident need for promotion to minimize the clash between culture
and technology.
7. Conclusion
This paper responds to the problem of information bottle-
necks in the information-loop during the construction phase of
a building. The problem of gathering data on-time to support
those decisions which are necessary for better project perfor-
mance has been recognized by many authors. Our decision to fo-
cus on activity monitoring was supported by the TRIMO
construction company, with which we collaborated in finding a
consistent solution for effective construction project manage-
ment. Knowing the needs of the industry and the potentials of
IT we chose a combination of computer vision, material identifi-
cation, and mobile computing. We have further developed func-
tional components and integrated them into a system for
automated activity monitoring.
Although the system does help the project and site managers in
the early detection of project failures regarding the execution of
construction activities, there is still much to be refined. We recog-
nize the following on-going problems:
1. image based activity recognition (4D-ACT) is not yet fully devel-
oped and has to reach a higher level of reliability; a multiple
camera should solve the problems of obscured elements
2. BIM technology has to be used to its full extent, but has not as
yet had sufficient up-take throughout the industry
3. in practice definitions of activities are inadequately related to
BIM elements; a method of consistent activity definition has
to be developed
4. material resources are not always adequately related to activi-
ties, one of the reasons being in unclearly defined and identifi-
able material units
5. use of the system requires a good understanding of its concepts
and components by all participants; additional education is
necessary
Although the system has not yet been fully implemented, par-
tial results prove that the concept is capable of ensuring on-time
information for adequately timed reactions to unexpected events
on-site and, thus, better project performance.
References
[1] S. Alkass, M. Mazerolle, E. Tribaldos, F. Harris, Computer aided construction
delay analysis and claims preparation, Construction Management and
Economics 13 (1995) 335–352.
[2] P.E. Josephson, Y. Hammarlund, The causes and costs of defects in
construction: a study of seven building projects, Automation in Construction
8 (1999) 681–687.
[3] S.M. Levy, Project Management in Construction, McGraw-Hill, 2002.
[4] R. Navon, R. Sacks, Assessing research issues in Automated Project
Performance Control (APPC), Automation in Construction 16 (2007) 474–484.
[5] S. Shahid, T. Froese, Project management information control systems,
Canadian Journal of Civil Engineering 25 (1998) 735–754.
[6] J. Li, O. Moselhi, S. Alkass, Internet-based database management system for
project control, Engineering, Construction and Architectural Management 13
(2006) 242–253.
[7] S.H. Lee, F. Peña-Mora, M. Park, Dynamic planning and control methodology
for strategic and operational construction project management, Automation in
Construction 15 (2006) 84–97.
[8] G.A. Sears, S. Sears, R.H. Clough, Construction Project Management, Willey,
2000.
[9] I.N. Davidson, M.J. Skibniewski, Simulation of automated data collection in
buildings, ASCE Journal of Computing in Civil Engineering 9 (1995) 9–20.
[10] M.Y. Cheng, J.C. Chen, Integrating barcode and GIS for monitoring construction
progress, Automation in Construction 11 (2002) 23–33.
[11] B. McCullouch, Automating field data collection in construction organizations,
in: 4th ASCE Construction Congress, Minneapolis, ASCE, 1997.
[12] S. Kiziltas, B. Akinci, E. Ergen, P. Tang, Technological assessment and process
implications of field data capture technologies for construction and facility/
infrastructure management, Electronic Journal of Information Technology in
Construction. 13 (2008) 134–154. http://www.itcon.org/data/works/att/
2008_10.content.07637.pdf.
[13] C. Stauffer, W.E.L. Grimson, Learning patterns of activity using real-time
tracking, IEEE Transactions on Pattern Analysis and Machine Intelligence. 22
(2000) 747–757.
[14] R. Navon, E. Goldschmidt, Monitoring labor inputs: automated-data-collection
model and enabling technologies, Automation in Construction 12 (2002) 185–
199.
[15] R. Sacks, R. Navon, A. Shapira, I. Brodetsky, Monitoring construction
equipment for automated project performance control, in: 19th ISARC,
Gaithersburgh, MD, 2002, pp. 161–166.
[16] E. Ergena, B. Akinci, R. Sacks, Tracking and locating components in a precast
storage yard utilizing radio frequency identification technology and GPS,
Automation in Construction 16 (2007) 354–367.
[17] P. Podbreznik, D. Rebolj, Automatic comparison of site images and the 4D
model of thebuilding, in: R. Scherer, P. Katranuschkov, S.E. Schapke (Eds.), CIB
W78 22nd Conference on Information Technology in Construction, Technische
Universität, Dresden, 2005. pp. 235–239.
[18] H. Kim, N. Kano, Comparison of construction photograph and VR image in
construction progress, Automation in Construction 17 (2008) 137–143.
[19] J.H. Garrett Jr., J. Sunkpho, Issues in delivering mobile IT systems to field users,
in: Int. Kolloquium ueber die Anwendung der Informatik und Mathematik in
Architektur und Bauwesen (IKM), Weimar, 2000.
[20] M. Ward, T. Thorpe, A. Price, C. Wren, Implementation and control of wireless
data collection on construction sites, ITcon 9 (2004) 297–311.
[21] S. Bowden, A. Dorr, T. Thorpe, C. Anumba, Mobile ICT support for construction
process improvement, Automation in Construction 15 (2005) 664–676.
[22] M.C. Tidwell, Microcomputer Application for Field Construction projects,
McGraw-Hill, New York, 1992.
[23] N. Čuš Babič, D. Rebolj, A. Magdič, M. Radosavljević, Mobile Computing as a
means for supporting information flow in construction processes, Concurrent
Engineering Research and Applications 11 (2003) 37–46.
[24] A. Dainty, D. Moore, M. Murray, Communication in Construction: Theory and
Practice, Taylor & Francis, Oxon, 2006.
[25] K. McKinney, M. Fischer, Generating, evaluating and visualizing construction
schedules with CAD tools, Automation in Construction 7 (1998) 433–447.
[26] B. Koo, M. Fisher, Feasibility study of 4D CAD in commercial construction,
Journal of Construction Engineering and Management 128 (2000) 274–275.
[27] K.W. Chau, M. Anson, J.P. Zhang, Four-dimensional visualization of
construction scheduling and site utilization, Journal of Construction
Engineering and Management 130 (2004) 598–606.
[28] R.T. Collins, C.O. Jaynes, Y.O. Cheng, X. Wang, F. Stolle, E.M. Riseman, A.R.
Hanson, The ascender system: automated site modeling from multiple aerial
images, Computer Vision and Image Understanding 72 (1998) 143–162.
[29] M. Cord, D. Declercq, Three-dimensional building detection and modeling
using a statistical approach, IEEE Transactions on Image Processing 10 (2001)
715–723.
 D. Rebolj et al. / Advanced Engineering Informatics 22 (2008) 493–503
[30] O. Faugeras, R. Luc, S. Laveau, G. Csurka, C. Zeller, C. Gauclin, I. Zoghlami, 3-D
reconstruction of urban scenes from image sequences, Computer Vision and
Image Understanding 69 (1998) 292–309.
[31] A. Fischer, T.H. Kolbe, F. Lang, A.B. Cremers, W. Frstner, L. Plmer, V. Steinhage,
Extracting buildings from aerial images using hierarchical aggregation in 2D
and 3D, Computer Vision and Image Understanding 72 (1998) 185–203.
[32] M. Fradkin, H. Matre, M. Roux, Building detection from multiple aerial images
in dense urban areas, Computer Vision and Image Understanding 82 (2001)
181–207.
[33] C. Jaynes, E. Riseman, A. Hanson, Recognition and reconstruction of buildings
from multiple aerial images, Computer Vision and Image Understanding 90
(2003) 68–98.
[34] Z. Kim, R. Nevatia, Automatic description of complex buildings from multiple
images, Computer Vision and Image Understanding 96 (2004) 60–95.
[35] N. Paparoditis, M. Jordan, M. Cord, J.P. Cocquerez, Building detection and
reconstruction from mid- and high-resolution aerial imagery, Computer Vision
and Image Understanding 72 (1998) 122–142.
[36] L. Soibelman, J. Wu, C. Caldas, I. Brilakis, K.Y. Lin, Management and analysis of
unstructured construction data types, Advanced Engineering Informatics 22
(2008) 15–27.
[37] D.J.M. Sales, R. Bastos, C. Joo, V. Rodrigo, Semiautomatic 3D reconstruction of
urban areas using epipolar geometry and template matching, Computer Aided
Civil and Infrastructure Engineering 21 (2006) 466–485.
[38] I. Abdel-Qader, O. Abudayyeh, M.E. Kelly, Analysis of edge detection
techniques for crack identification in bridges, Journal of Computing in Civil
Engineering 17 (2003) 255–263.
[39] S. Lee, L.M. Chang, M. Skibniewski, Automated recognition of surface defects
using digital color image processing, Automation in Construction 15 (2006)
540–549.
[40] S.K. Sinha, P.W. Fieguth, Automated detection of cracks in buried concrete pipe
images, Automation in Construction 15 (2006) 58–72.
[41] S.N. Yu, J.H. Jang, C.S. Han, Auto inspection system using a mobile robot for
detecting concrete cracks in a tunnel, Automation in Construction 16 (2007)
255–261.
[42] Y.S. Wang, B.J. Griffiths, B.A. Wilkie, A novel system for coloured object
recognition, Computers in Industry 32 (1996) 69–77.
[43] I.K. Brilakis, L. Soibelman, Y. Shinagawa, Construction site image retrieval
based on material cluster recognition, Advanced Engineering Informatics 20
(2006) 443–452.
[44] O. Henricsson, The role of color attributes and similarity grouping in 3-d
building reconstruction, Computer Vision and Image Understanding 72 (1998)
163–184.
503
[45] K. Treash, K. Amaratunga, Automatic road detection in grayscale aerial images,
Journal of Computing in Civil Engineering 14 (2000) 60–69.
[46] J.R.J. Lee, M.L. Smith, L.N. Smith, P.S. Midha, Robust and efficient automated
detection of tooling defects in polished stone, Computers in Industry 56 (2005)
787–801.
[47] B. Potočnik, D. Zazula, Automated analysis of a sequence of ovarian ultrasound
images. Part 1: segmentation of single 2D image, Image and Vision Computing
20 (2002) 217–225.
[48] D.A. Forsyth, J. Ponce, Computer Vision – A Modern Approach, Prentice Hall,
2002.
[49] R.I. Hartley, A. Zisserman, Multiple View Geometry in Computer Vision,
Cambridge University Press, 2004.
[50] Z. Zhang, Determining the epipolar geometry and its uncertainty: a review,
International Journal of Computer Vision 27 (1998) 161–198.
[51] J. Bigun, Vision with Direction: A Systematic Introduction to Image Processing
and Computer Vision, Springer, 2006.
[52] R. Vrijhoef, L. Koskela, The four roles of supply chain management in
construction, European Journal of Purchasing & Supply Management 6
(2000) 169–178.
[53] T. Ala-Risku, M. Kärkkäinen, Material delivery problems in construction
projects: a possible solution, International Journal of Production Economics
1 (2006) 19–29.
[54] E. Ergen, B. Akinci, R. Sacks, Life-cycle data management of engineered-to-
order components using radio frequency identification, Advanced Engineering
Informatics 21 (2007) 356–366.
[55] A. Magdič, D. Rebolj, N. Šuman, Effective control of unanticipated on-site
events: a pragmatic, human-oriented problem solving approach, Electronic
Journal of Information Technology in Construction 9 (2004) 409–418. http://
www.itcon.org/cgi-bin/papers/Show?2004_29.
[56] Z. Turk, B. Lundgren, Communication workflow perspective on engineering
work, in: M. Hannus, M. Salonen, A. Samad Kazi (Eds.), Concurrent Engineering
in Construction, Challenges for the New Millennium, CIB publication 236,
Espoo, 1999.
[57] J. Song, C.T. Haas, C.H. Caldas, A proximity-based method for locating RFID
tagged objects, Advanced Engineering Informatics 21 (2007) 367–376.
[58] G. Ballard, The Last Planner System of Production Control, Doctoral thesis,
University of Birmingham, 2000.
[59] ERABUILD, Review of the Current State of Radio Frequency Identification
(RFID) Technology. Its Use and Potential Future Use in Construction, Final
Report, 2006.