Tracking Hardware Evolution
Author Name1, Author Name2, Author Name2
1
Company One, 123 Main Street, Cincinnati, OH USA
2
Company Two, 42 Gerbrekarstrasse, Brussels, Belgium
1
E-mail: name@domain.com
Abstract
Software evolution is the term used to describe the
process of developing and updating software systems.
Software repositories such as versioning systems, and bug
tracking systems are used to manage the evolution of
software projects. Mining this information is used to support
predictions and improve design and reuse. Integrated circuit
development can also benefit from these techniques.
Nowadays, both software and hardware development use
repositories and bug tracking systems. There are many
hardware open source projects such as SUN's OpenSparc
and designs at Opencores.org. We propose a methodology to
track specific HDL metrics in order to improve design
quality. Our results present a case study that correlates HDL
metrics to bug proneness of Verilog HDL modules. We also
present EyesOn, an open source framework designed to
automate historical and complexity metrics tracking of HDL
projects.
Keywords
HDL, design flow, quality, metrics, validation
1. Introduction
Verifying industrial scale designs is still a challenging
task that consumes most part of total development efforts.
International Technology Roadmap for Semiconductors
reported that verification stage is a bottleneck that has
actually reached crisis proportions [1]. Considering that
there is no formal or simulation tool capable to stimulate all
possible states in an industrial scale design, verification team
must work smarter, wisely spending their resources. In order
to handle the still increasing design complexity,
improvements in current verification methodologies are
needed. A step in this direction is to benefit from software
engineering experience. Nowadays hardware and software
development methodologies have many aspects in common:
requirements specification; use of description languages;
identification of functional design errors; use of versioning
systems and bug tracking tools.
Software evolution is a mature and well established
research area in software engineering [2,3]. Software
evolution research relies on extracting and processing
information from:
• Version Control Systems (VCS) repositories: These
systems store all revisions of the software source
code. Freely available platforms are Subversion
(SVN) [4] and Concurrent Version System (CVS)
[5];
1-xxxx-xxxx-4/10/$25.00 ©2010 IEEE
• Bug Tracking Systems (BTS): Store information
about all reported software bugs. Popular open-
source systems are Bugzilla [6] and Mantis [7];
• E-mail lists or software documentation.
In the last years, software evolution research focused
mainly on [8]: estimation of developer effort, impact of
changes in source code and defect prediction. In order to
facilitate software evolution research, several frameworks
were proposed [9]. These frameworks are suitable to track
historical and software complexity metrics, but unfortunately
do not offer support to extract and visualize Hardware
Description Language (HDL) specific metrics.
Software evolution techniques can be used to
complement and deeply improve hardware verification
methodologies. For example, large industrial System-on-a-
chips (SoCs) often reuse IP (Intellectual Property) blocks.
Many high-value IP blocks are developed over long periods
of time, being expected to live on for a decade [10]. Such
systems are designed by large teams, generating repositories
that contain millions of RTL lines of code and thousands of
bug reports. Open source hardware also plays an important
role on making hardware evolution research possible.
Designs like SUN’s Opensparc [11] and OpenRisc [12] offer
repositories that have plenty of information about how these
designs where implemented and verified.
Tracking hardware evolution is a new paradigm that
offers many advantages. As happens in software
construction, hardware development can also benefit from
hardware evolution. It allows hardware bug prediction,
mitigating hardware development complexity and
verification issues. Furthermore, it may provide hardware
module complexity profile, facilitating development
management.
The contributions of this work are threefold. First, we
propose the use of information available in HDL repositories
to track hardware evolution. Second, we provide EyesOn, an
open source, extensible framework designed to extract, store,
visualize, and export HDL complexity and historical metrics.
Third, we present scenarios where hardware evolution
tracking is inserted in traditional integrated design flows,
providing accurate and visual information.
This paper is outlined as follows. Section II reviews the
main software evolution tracking tools. In Section III, we
present the methodology to track hardware evolution, while
in Section IV details of the framework are provided. In
Section V some hardware evolution key visualization
possibilities are presented and a case study is discussed.
Finally, in Section VI, we conclude our remarks and present
future work directions.
12th Int’l Symposium on Quality Electronic Design
2. Related work While these tools have their own metrics and capabilities,
While software evolution has been studied for at least the EyesOn is the first to integrate environments for hardware
last three decades [13, 14], research on hardware evolution evolution research.
is still incipient. Therefore, there are no tools or frameworks
3. Methodology
designed to track HDL evolution. A preliminary work was
In this Section, we describe the methodology to track
proposed by [15]. In this work, some historical metrics and
hardware evolution. Basically, two different application
bug reports of HDL repositories are retrieved and stored on scenarios are described. These application scenarios are
a database. The main contribution of that work is to define a
illustrated on Figure 1.
language to describe bug reports. This language is used to
In the first scenario, hardware evolution tracking, the
attach bug reports on the commit messages, making possible
metrics are retrieved from the Repository and displayed on
to automatically retrieve and display this information. The
different formats that include:
most complete tools and frameworks designed process
repositories are legacy from software evolution research • Bar and line charts: present metrics evolution
community. The most importants are CVSAnalY, Release overcommits or periods of time;
History Database (RHDB), Kenyon, and Alitheia. • Kiviat diagrams: useful to compare the evolution of
Unfortunately, none of them are capable to handle HDL metrics on different axis and periods of time;
specific complexity metrics. • Heatmap diagrams: help the visualization of
CVSAnalY [16] is a repository data extractor. It was module’s specific metrics, such as modification rate
initially built to collect revision data from CVS repositories. and bug proneness.
They made GNOME project analysis as their case study and
presented statistics about inactivity, commiters, commits,
lines changed, lines modified, lines removed, first and last
commit. Based on these statics, inferences were presented
about modules and commiters.
RHDB [17] joined historical information from CVS and
Bugzilla, correlating their relationships. The experiments
were based on Mozilla web browser, a well known open
source project. Internal function calls were tracked and
mapped to call graphs, registering their evolution with
collected modification and bug reports.
Kenyon [18] is a software evolution tracker. It was
designed to facilitate software evolution research providing
an environment with repositories connectors, metric
extractors, and API points that could be extended. The
collected data is stored on a relational database through
Object-Relational Mapping.
Alitheia [19] tool aggregates similar functionalities, but
is designed to be a service oriented architecture and focused
in having good performance on large scale data extraction.
They are the most complete tools designed to facilitate
software evolution tracking. A detailed work comparing
different software evolution trackers is presented in [9].
The great number of metrics, the large scale of objects
being analyzed, and the context where they could be
Figure 1: Hardware evolution application scenarios.
observed require different viewpoints. RelVis [20] is a
visualization approach that presents multiple metric data
It’s important to note that hardware evolution tracking
about software entities and their relationships during the
can include also metrics extracted by commercial Electronic
software releases. It uses kiviat diagrams, also known as
Design Automation (EDA) tools.
radar charts, that are good to depict different values.
GEVOL [21] is a system that use graphs and colors to The second application scenario facilitates the use of
represent Java classes evolution and the contribution of their advanced data mining and statistics techniques. This is
developers. Different viewpoints like simple bar and pie where hardware evolution research starts, experimenting the
charts, and more sophisticated diagrams like network, matrix effectiveness of different software evolution techniques on
and cityscape views were presented in [22]. HDL hardware projects.

1-xxxx-xxxx-4/10/$25.00 ©2010 IEEE 12th Int’l Symposium on Quality Electronic Design


4. Framework architecture
Finding integrated environments for hardware evolution
research is a key problem. Having that in mind, we designed
and implemented EyesOn, an open source, extensible
framework. Our main contribution allows extracting, storing,
visualizing and exporting HDL complexity metrics,
facilitating hardware development and verification. Our
framework is organized as illustrated on Figure 2.
Figure 2: Framework architecture.
There are two storage components, namely Database and
Repository. In addition, we have modules for data
manipulation (extractors, parsers, metrics, visualization,
exporter).
The architecture was designed to allow the integration
among repositories and metric extractors, facilitating the
researchers work. During the hardware development process,
designers insert into the repository messages about every
module revision. From those revisions, data are extracted by
the Repository Extractor and inserted into the Database. The
framework provides support for researchers to retrieve
project files and extract desired metrics. Those can be
visualized through the Data Visualization module, which
offers many options, such as bars, lines, heat map and radar.
Through a GUI data visualization, it is also possible to build
queries and to exchange data.
Next, we explain the main modules and features
presented in the framework architecture.
4.1. Repository extractor
Repository Extractor is a module that connects to the
most popular VCS tools (CVS, Subversion, and Git) and
retrieves data about the revisions of the HDL designs.
Although VCS tools store revision information in different
ways, key data like tags, commits, comments, and authors
are standardized.
1-xxxx-xxxx-4/10/$25.00 ©2010 IEEE
The most important function performed by Repository
Extractor is the first step of VCS preprocessing. During this
stage, repository revision information is retrieved and stored
in a database. The acquired revision information will guide
metrics extraction which is the second EyesOn preprocessing
step.
4.2. Metrics extractor
Metrics Extractor is the module responsible to extract
and store different design metrics: HDL, External, Bug, and
Repository. With revision information
gathered by Repository Extractor it is
possible to observe project specific
state points on timeline.
4.2.1. HDL metrics.
In order to extract static metrics
from source code, the desired version
is retrieved on demand by Source
Extractor. A revision file and
directory structure is mounted on a
temporary file system path, providing
an environment that is a mirror of the
design’s evolution state.
Despite the fact that some metrics
are simple to implement, as occurs
with LoC (lines of code), others, such
as complexity and metrics that
quantify internal structures, depend of
HDL parsers.
EyesOn provides a rich framework with grammars and
HDL parsers. The metrics taken are mostly based on source
code analysis. At this version, we developed HDL parsers
that extract:
• McCabe [23]: Also known as Cyclomatic
Complexity, relies on the premise that source code
complexity does not depend on its size, but in its
structure. It measures the number of independent
program’s execution paths.
• NPath [24]: Counts the acyclic execution paths
through a function. This metric was originally
proposed to measure maintainability effort.
• Halstead [25]: Composed by a set of complexity
measures derived from the amount of operands and
operators.
• Other simpler metrics like LoC, statements, I/O
signals, operators, and operands.
4.2.2. External metrics.
In order to increase EyesOn flexibility, it is possible to
include metrics extracted by external tools. Any commercial
EDA tools offer statistics about design’s modules like flip-
flops, equivalent gate count, finite state machines, latches,
registers, etc.
12th Int’l Symposium on Quality Electronic Design
4.2.3. Bug metrics. also the case in hardware evolution. We provide a set of
In addition, EyesOn is also capable of extracting bug data charts, allowing the data visualization in many formats.
from BTS and VCS repositories. Simple VCS transaction
4. Results
detection heuristics are provided based on similar commit
As a case study, we used revision history information
messages and intersections of VCS with BTS modification
from a proprietary IP core. This core was divided into 17
intervals.
modules that were developed during a semester. All the
4.2.4. Repository metrics. modifications on these modules were reported and stored at
Repository metrics are useful to contextualize the other the repository as commit messages.
metrics in time. This information provides support to study In order to observe when bugs were discovered during
the relation among different classes of events like commit or the development process, we built a bi-weekly chart depicted
bug reports. Examples of Repository Metrics are: in Figure 3. The bars represent the accumulated found bugs
• Commit comments and its type; by the time interval.
• Contribution by author; Figure 4 depicts extracted metrics changing as hardware
• Revision differences; designers commit new versions of Module #1. Keywords,
operands and Lines of Code present a similar behavior. One
• Transactions;
can notice variations when source refactoring occurs (at
• Coupling among modifications. commit intervals [9:11] and [17:21]). The Lines per
4.3. Extension points Comment values reduced as the module was being
Our framework extensibility relies on its capability to developed. It happened because some commented
adapt to new tools, metrics and to accept parameterized data instructions were also removed from source code as it was
from external applications. In order to support these new being implemented. Despite refactoring, the module
features we provided extension points for metrics,
repositories, and external extractors.
4.4. Flexible database
Despite making metric extraction more agile, other
motivation to maintain a database is to facilitate queries. Due
to the capabilities of Hibernate, an Object-Relational
Mapping (ORM) framework, different Database
Management System (DBMS) can be configured to store the
manipulated data. Hibernate Query Language (HQL) is used
to retrieve database information based on mapped classes.

cyclomatic complexity grew, but, at the end, diminished a

little. It could be an indicative for future modifications.
Figure 4: Metrics changing between commits.
A simplified view of metrics evolution is shown in Figure
5. This chart is composed by LoC, Cyclomatic Complexity
and Accumulated Fixed Errors. The values were normalized.
This visualization is fragmented on time, showing only three
evolution steps. They were composed of commits #13, #20
and #26, respectively colored by red, blue and green. It
presents the growth of these metrics.
Figure 3: Number of errors per time interval.
4.5. Data Visualization
The data visualization module is a GUI workspace that
supports users to build queries and to exchange extracted
data. The usage of plug-in metrics classes on query builder
are facilitated by Java language Reflection API that discover
available mapped attributes on new registered metrics.
Efficient data visualization to track files and software
evolution has become a specific research area [26]. This is

1-xxxx-xxxx-4/10/$25.00 ©2010 IEEE 12th Int’l Symposium on Quality Electronic Design


Figure 5: Radar visualization.
Figure 6 shows a heat map that presents modules that are
colored according to the error prone value. Green (light gray
in grayscale copies) means low error proneness, while read
(dark gray) means high chance of error. Each module was
represented by a rectangle where its area is proportional to
its LoC size. The adopted prediction model was based on the
Most Frequently Fixed cache replacement algorithm
presented on [28]. It was incorporated by EyesOn as an
external application. As observed on the Figure, among 17
modules, 3 were subject to high error prone. They are not
necessarily the 3 largest ones.
Figure 6: Radar visualization.
5. Conclusion and future work
Tracking hardware evolution offers many benefits. It can
improve the quality of electronic design. We proposed the
use of information available in HDL repositories to track
hardware evolution. We presented EyesOn, an extensible,
open-source framework designed to automate hardware
evolution tracking. We presented a case study where
hardware evolution tracking were used to correlate HDL
metrics and bug proneness of Verilog HDL modules.
1-xxxx-xxxx-4/10/$25.00 ©2010 IEEE
As future work, we intent to extract HDL metrics in order
to build HDL quality predictor models. We are also
improving the set of EyesOn supported metrics as well as
visualization options.
6. References
[1] “International technology roadmap for semiconductors
- design,”. 2009. [Online]. Available:
http://www.itrs.net
[2] H. Kagdi, M. L. Collard, and J. I. Maletic, “A survey
and taxonomy of approaches for mining software
repositories in the context of software evolution,” J.
Softw. Maint. Evol., vol. 19, no. 2, pp. 77–131, 2007.
[3] K. H. Bennett and V. T. Rajlich, “Software
maintenance and evolution: a roadmap,” in ICSE ’00:
Proceedings of the Conference on The Future of
Software Engineering. New York, NY, USA: ACM,
2000, pp. 73– 87.
[4] B. Collins-Sussman, B. W. Fitzpatrick, and C. M.
Pilato, Version Control with Subversion. O’Reilly
Media, 2004.
[5] P. Cederqvist, Version Management with CVS. Free
Software Foundation, Inc., 2004.
[6] The Bugzilla Guide, 2010. [Online]. Available:
http://www.bugzilla.org
[7] Mantis Bug Tracker, 2010. [Online]. Available:
http://www.mantisbt.org
[8] M. DAmbros, H. C. Gall, M. Lanza, and M. Pinzger,
Software Evolution. Springer, 2008, ch. Analyzing
software repositories to understand software evolution,
pp. 37–67.
[9] G. Gousios, “Tools and methods for large scale
software engineering research,” Ph.D. dissertation,
Athens University of Economics and Business, jul
2009.
[10] W. Stapleton and P. Tobin, “Verification problems in
reusing internal design components,” in DAC ’09:
Proceedings of the 46th Annual Design Automation
Conference. New York, NY, USA: ACM, 2009, pp.
209–211.
[11] D. L. Weaver, OpenSPARCTMInternals, 2008.
[12] OpenRISC 1000 Architecture Manual, 2004. [Online].
Available: http://www.opencores.org
[13] “I J. P. Cavano and J. A. McCall, “A framework for
the measurement of software quality,” in Proceedings
of the software quality assurance workshop on
Functional and performance issues, 1978, pp. 133–139.
[14] V. R. Basili, L. C. Briand, and W. L. Melo, “A
validation of objectoriented design metrics as quality
indicators,” IEEE Trans. Softw. Eng., vol. 22, no. 10,
pp. 751–761, 1996.
[15] “Blind review.”
[16] G. Robles, S. Koch, and J. M. Gonzalez-Barahona,
“Remote analysis and measurement of libre software
systems by means of the CVSAnalY tool,” in
Proceedings of the 2nd ICSE Workshop on Remote
Analysis and Measurement of Software Systems
12th Int’l Symposium on Quality Electronic Design
 (RAMSS), Edinburg, Scotland, UK, 2004. [Online].
Available:
http://libresoft.urjc.es/html/downloads/cvsanaly-
icse.pdf
[17] M. Fischer, M. Pinzger, and H. Gall, “Analyzing and
relating bug report data for feature tracking,” in WCRE
’03: Proceedings of the 10th Working Conference on
Reverse Engineering. Washington, DC, USA: IEEE
Computer Society, 2003, p. 90.
[18] J. Bevan, E. J. Whitehead, Jr., S. Kim, and M. Godfrey,
“Facilitating software evolution research with kenyon,”
in ESEC/FSE-13: Proceedings of the 10th European
software engineering conference held jointly with 13th
ACM SIGSOFT international symposium on
Foundations of software engineering. New York, NY,
USA: ACM, 2005, pp. 177– 186.
[19] G. Gousios and D. Spinellis, “Alitheia core: An
extensible software quality monitoring platform,”
Software Engineering, International Conference on,
vol. 0, pp. 579–582, 2009.
[20] M. Pinzger, H. Gall, M. Fischer, and M. Lanza,
“Visualizing multiple evolution metrics,” in
Proceedings of the 2005 ACM symposium on Software
visualization. ACM, 2005, pp. 67–75.
[21] C. Collberg, S. Kobourov, J. Nagra, J. Pitts, and K.
Wampler, “A system for graph-based visualization of
the evolution of software,” in Proceedings of the 2003
ACM symposium on Software visualization. ACM,
2003.
[22] S. Eick, T. Graves, A. Karr, A. Mockus, and P.
Schuster, “Visualizing software changes,” IEEE
Transactions on Software Engineering, vol. 28, no. 4,
p. 412, 2002.
[23] T. J. McCabe, “A complexity measure,” in ICSE ’76:
Proceedings of the 2nd international conference on
Software engineering. Los Alamitos, CA, USA: IEEE
Computer Society Press, 1976, p. 407.
[24] B. A. Nejmeh, “Npath: a measure of execution path
complexity and its applications,” Commun. ACM, vol.
31, no. 2, pp. 188–200, 1988.
[25] M. H. Halstead, Elements of Software Science. New
York: Elsevier, 1977.
[26] L. Voinea, A. Telea, and M. Chaudron, “Version
centric visualization of code evolution,” in Proceedings
of Eurographics/IEEE-VGTC Symposium on
Visualization. Citeseer, 2005.
[27] A. Hassan and R. Holt, “The top ten list: Dynamic fault
prediction,” in Software Maintenance, 2005. ICSM’05.
Proceedings of the 21st IEEE International Conference
on, 2005, pp. 263–272.

1-xxxx-xxxx-4/10/$25.00 ©2010 IEEE 12th Int’l Symposium on Quality Electronic Design