VISUALISATION AND DESCRIPTION

IN
SOFTWARE ENGINEERING

Rudolf J. Vernik

A thesis submitted to the

School of Computing and Information Science
Faculty of Applied Science and Technology
University of South Australia
for the degree of
Doctor of Philosophy (Computer and Information Science)

April 1996
 Table of Contents

TABLE OF CONTENTS................................ ................................ ................................ ................. ii

LIST OF FIGURES................................ ................................ ................................ ...................... viii

LIST OF TABLES................................ ................................ ................................ ........................... ix

TABLE OF ABBREVIATIONS................................ ................................ ................................ .......x

SUMMARY................................ ................................ ................................ ................................ .....xii

DECLARATION................................ ................................ ................................ ........................... xiv

ACKNOWLEDGMENTS................................ ................................ ................................ ...............xv

1. INTRODUCTION................................ ................................ ................................ ........................ 1

1.1 GENERAL ..................................................................................................................................... 1

1.2 CONTEXT ..................................................................................................................................... 1
1.3 MOTIVATION................................................................................................................................ 3
1.4 MAIN PROPOSALS ANDIDEAS ....................................................................................................... 4
1.5 AIMS OF THESIS............................................................................................................................ 5
1.6 READERSHIP................................................................................................................................. 5
1.7 PRESENTATION............................................................................................................................. 6
1.7.1 Structure and Contents........................................................................................................... 6
1.7.2 Key Sections and Threads...................................................................................................... 6
1.7.3 Layout, Terminology and Notations....................................................................................... 7

2. BACKGROUND................................ ................................ ................................ .......................... 9

2.1 PRELIMINARYSTUDIES................................................................................................................. 9
2.1.1 Approach..............................................................................................................................10
2.1.2 Summary of Results..............................................................................................................10
2.1.2.1 Problems Related to Particular Sources of Information
...................................................11
2.1.2.2 Underlying Issues and Problems
.....................................................................................12
2.2 OVERVIEW ANDDISCUSSION OFRELATEDLITERATURE...............................................................14
2.2.1 General Problems and Issues.................................................................................................15

ii
 2.2.1.1 Problems of Software Visibility.
.....................................................................................16
2.2.1.2 Information Logistics.....................................................................................................18
2.2.1.3 Supporting Task-Based Information Needs.
....................................................................18
2.2.2 Issues Related to Specific Sources of SE Information
............................................................20
2.2.2.1 Technical Documentation...............................................................................................20
2.2.2.2 Software Metrics............................................................................................................23
2.2.2.3 Tools and Environments.................................................................................................26
2.2.3 Use of Visualisation Approaches
...........................................................................................29
2.2.3.1 Computer-Based Visualisations
......................................................................................29
2.2.3.2 Software Visualisation...................................................................................................30
2.2.3.3 Interaction Issues...........................................................................................................32
2.2.3.4 Extending the Scope of Software Visualisation
...............................................................33
2.2.3.5 Summary.......................................................................................................................36
2.2.4 Cognitive Science Issues.......................................................................................................36
2.2.4.1 Understanding and Defining Information Needs
..............................................................36
2.2.4.2 Models of Cognition.......................................................................................................37
2.2.4.3 Cognitive Issues Related to Representation.
....................................................................39
2.2.4.4 Building Cognitive Bridges
.............................................................................................40
2.3 ANALYSIS, PERSPECTIVES ANDCONJECTURES.............................................................................43
2.3.1 Analysis of Problems and Issues
............................................................................................43
2.3.2 Effectiveness Criteria............................................................................................................48
2.3.3 Conjectures...........................................................................................................................48

3. RESEARCH APPROACH................................ ................................ ................................ .........50

3.1 RESEARCHOBJECTIVES...............................................................................................................50
3.2 SOFTWAREENGINEERINGRESEARCH ..........................................................................................50
3.2.1 Difficulties and Problems......................................................................................................51
3.2.2 A Scientific Approach...........................................................................................................52
3.2.3 Experimentation in Software Engineering
..............................................................................53
3.3 RESEARCHSTRATEGY.................................................................................................................55
3.4 RESEARCHPROCESS....................................................................................................................56

4. CONCEPTS AND MODELS................................ ................................ ................................ ......59

4.1 ESTABLISHING THECONTEXT......................................................................................................60

4.1.1 Contextual Model.................................................................................................................60
4.1.2 Software Engineering Process...............................................................................................61
4.1.3 Software Engineering Information
.........................................................................................61

iii
 4.1.4 Project Information Resource................................................................................................62
4.1.5 Information Flows.................................................................................................................63
4.2 CLASSIFICATION ANDMODELLING OFINFORMATIONPRODUCTS..................................................64
4.3 USING INFORMATION FORDESCRIPTIONPURPOSES......................................................................67
4.3.1 Descriptive Information
.........................................................................................................67
4.3.2 Measurement and Description...............................................................................................68
4.3.3 Use of Descriptive Information
..............................................................................................70
4.3.4 Effective Descriptions...........................................................................................................71
4.4 THE DESCRIPTIONPROCESS.........................................................................................................72
4.4.1 Provision and Use Processes.................................................................................................73
4.4.2 The Mediation Process..........................................................................................................74
4.4.3 The Facilitation Process........................................................................................................75
4.5 DESCRIPTION INSOFTWAREENGINEERING PROCESS FRAMEWORK.............................................75
4.5.1 Model Dynamics...................................................................................................................75
4.5.2 Use of the DSEPF.................................................................................................................76
4.6 MODEL-BASED DESCRIPTIONS.....................................................................................................77
4.6.1 Systems Concepts.................................................................................................................77
4.6.2 Modelling.............................................................................................................................78
4.6.3 Using Models as the Basis of Descriptions
.............................................................................80
4.7 PRODUCT ORIENTEDDESCRIPTIONS............................................................................................82
4.7.1 Concept of a Software Product..............................................................................................82
4.7.2 Software Product Model.......................................................................................................83
4.7.3 Provision and Use of Product Oriented Descriptions
..............................................................85
4.8 SUMMARY...................................................................................................................................86

5. INTEGRATED VISUALISATION OF SOFTWARE SYSTEMS

................................ ............87

5.1 OVERVIEW OFINTEGRATEDVISUALISATIONAPPROACH..............................................................87

5.2 INTEGRATEDVISUALDESCRIPTIONS............................................................................................88
5.2.1 General Characteristics.........................................................................................................88
5.2.2 Primary View Visualisation Principles
...................................................................................90
5.3 CONCEPTUALMODEL OF ANIVES...............................................................................................93
5.3.1 Modelling and Integration.....................................................................................................94
5.3.2 Access..................................................................................................................................95
5.3.3 Filtering................................................................................................................................95
5.3.4 Representation and Presentation............................................................................................96
5.3.5 Control.................................................................................................................................96
5.3.6 IVES Usage..........................................................................................................................97

iv
5.4 SEE-ADA: A PRACTICALIMPLEMENTATION................................................................................97
5.4.1 Overview..............................................................................................................................98
5.4.2 SEE-Ada Views..................................................................................................................101
5.4.2.1 Primary Views.............................................................................................................101
5.4.2.2 Peripheral Views..........................................................................................................103
5.4.2.3 Subsystem View...........................................................................................................104
5.4.2.4 Presentational Integration.............................................................................................104
5.4.3 Describing Relationships.....................................................................................................105
5.4.4 Customisation and Tailoring...............................................................................................106
5.4.5 Mapping and Integrating Attribute Information
....................................................................108
5.4.6 Support for the Facilitation Process.....................................................................................111
5.4.7 SEE-Ada Usage..................................................................................................................112

6. EXPERIMENTATION................................ ................................ ................................ .............115

6.1 SUMMARY OFEARLY STUDIES, TRIALS, AND EXPERIENCES.......................................................115

6.2 INDUSTRY CASE STUDY ............................................................................................................117
6.2.1 Overview............................................................................................................................117
6.2.2 Purpose and Goals..............................................................................................................118
6.2.3 Case Study Setting..............................................................................................................119
6.2.4 Scope.................................................................................................................................120
6.2.4.1 Team Leader Focus......................................................................................................120
6.2.4.2 Potential Limitations....................................................................................................120
6.2.5 Strategy and Approach........................................................................................................121
6.2.5.1 Type of Case Study......................................................................................................121
6.2.5.2 Use of a Modified GQM approach
................................................................................121
6.2.5.3 Method........................................................................................................................122
6.2.6 Establishing Goal-based Information Requirements
.............................................................123
6.2.7 Experimental Setup.............................................................................................................124
6.2.7.1 Overview.....................................................................................................................124
6.2.7.2 SEE-Ada......................................................................................................................125
6.2.7.3 Usage Monitor.............................................................................................................126
6.2.7.4 Access and Filtering.....................................................................................................129
6.2.7.5 SPM Capture and Update Mechanism..........................................................................131
6.2.7.6 Recording and Analysis Tools......................................................................................132
6.2.8 Results...............................................................................................................................133
6.2.8.1 Goal 1: Availability of Project Information
...................................................................133
6.2.8.2 Goal 2: Use of Descriptive Information
.........................................................................139

v
 6.2.8.3 Goal 3: Setup and Support Requirements
......................................................................142
6.2.8.4 Goal 4: Use of an IVES................................................................................................ 146
6.3 SUMMARY.................................................................................................................................155

7. DISCUSSION................................ ................................ ................................ ............................ 157

7.1 SUMMARY OFCONCEPTS ANDCONJECTURES............................................................................157

7.2 SYNOPSIS OFRESEARCHAPPROACH..........................................................................................159
7.2.1 Use of a Scientific Approach...............................................................................................159
7.2.2 Characteristics and Limitations of Results
...........................................................................160
7.3 FINDINGSRELATED TOKEY ISSUES...........................................................................................162
7.3.1 Understanding and Supporting User Information Needs
.......................................................162
7.3.1.1 Description Concepts and Models
.................................................................................162
7.3.1.2 Understanding and Defining SE Tasks
..........................................................................164
7.3.1.3 Defining and Understanding Task-Based Information Needs
.........................................165
7.3.2 Availability of and Access to Information
............................................................................166
7.3.2.1 Availability of Underlying Project Information
..............................................................166
7.3.2.2 Direct Use of Underlying Information
...........................................................................168
7.3.2.3 Availability of and Access to Information using SEE-Ada
.............................................169
7.3.3 Assimilation and Integration of Information
.........................................................................170
7.3.4 Use of Appropriate Representation and Presentation Techniques
..........................................173
7.3.5 Customisation and Adaptation of Information
......................................................................174
7.3.6 Cost of Provision and Use...................................................................................................175
7.4 OTHER FINDINGS.......................................................................................................................177
7.4.1 Measurement and Metrics Programmes
...............................................................................177
7.4.2 Reporting and Recording.....................................................................................................179
7.4.3 Impact on Documentation Needs
.........................................................................................179
7.5 POTENTIALBENEFITS ANDIMPLICATIONS..................................................................................180
7.5.1 Implications for SE Practice................................................................................................ 181
7.5.2 Implications for CASE Tool Developers
..............................................................................182
7.5.3 Implications for SE Researchers
..........................................................................................183
7.6 ONGOING ANDFUTURE WORK...................................................................................................184
7.7 ORIGINALCONTENT ANDCONTRIBUTIONS.................................................................................186

8. SUMMING UP................................ ................................ ................................ .......................... 188

8.1 SUMMARY.................................................................................................................................188
8.2 ACHIEVEMENTS.........................................................................................................................189
8.3 LIMITATIONS.............................................................................................................................190

vi
8.4 FURTHER WORK........................................................................................................................191
8.5 POTENTIALIMPACT...................................................................................................................191

REFERENCES................................ ................................ ................................ ............................. 193

APPENDIX A: PERSPECTIVES ON SOFTWARE PROJECTS AND PROCESSES

............204

APPENDIX B: PRELIMINARY EFFECTIVENESS CRITERIA

................................ ............208

APPENDIX C: SEE-ADA DETAILS................................ ................................ .......................... 210

APPENDIX D: FIRE CONTROL SYSTEM PROJECT................................ ........................... 213

APPENDIX E: CASE STUDY GOALS AND QUESTIONS

................................ ..................... 218

APPENDIX F: CASE STUDY INFORMATION REQUIREMENTS

................................ .......219

APPENDIX G: CLASSES OF PERSISTENT SOFTWARE INFORMATION PRODUCTS

..221

APPENDIX H: TEAM LEADER TASK DEFINITIONS

................................ .......................... 225

APPENDIX I: EXAMPLES OF ATTRIBUTES ACCESSIBLE FROM SEE-ADA

.................228

vii
 List of Figures

FIGURE 1-1 STRUCTURALDESCRIPTION OF THETHESIS..................................................................... 8

FIGURE 2-1 COGNITIVEMODEL OF VISUALINTERACTION.................................................................38
FIGURE 2-2 ADAPTINGINFORMATION TOUSER NEEDS .....................................................................47
FIGURE 3-1 OVERVIEW OFRESEARCHPHASES ANDRESULTS...........................................................57
FIGURE 4-1 CONTEXTUALMODEL....................................................................................................60
FIGURE 4-2 CONCEPTUALMODEL OF INFORMATIONUSE .................................................................71
FIGURE 4-3 INCLUDING THEDESCRIPTIONPROCESS.........................................................................72
FIGURE 4-4 DESCRIPTIONPROCESSES...............................................................................................73
FIGURE 4-5 DESCRIPTION INSOFTWAREENGINEERINGPROCESS FRAMEWORK.................................76
FIGURE 4-6 REPRESENTATION OFSIMPLE COMPOSITEMODEL..........................................................80
FIGURE 4-7 SIMPLE DESCRIPTION USINGATOMIC MODEL.................................................................80
FIGURE 4-8 EVOLUTION OF THESOFTWAREPRODUCT MODEL .........................................................84
FIGURE 5-1 INTEGRATEDVISUALDESCRIPTION................................................................................89
FIGURE 5-2 IVES CONCEPTUALMODEL...........................................................................................94
FIGURE 5-3 SEE-ADA ENVIRONMENT..............................................................................................98
FIGURE 5-4 DESCRIBINGENTITY RELATIONSHIPS...........................................................................102
FIGURE 5-5 SEE-ADA INTEGRATEDVIEWS.....................................................................................102
FIGURE 5-6 CUSTOMISATION ANDTAILORING OFINFORMATION.....................................................107
FIGURE 5-7 VIEWINGATTRIBUTE ANDRELATIONSHIPINFORMATION..............................................107
FIGURE 5-8 VIEWINGATTRIBUTESBETWEEN VIEWS ......................................................................110
FIGURE 5-9 SEE-ADA SCRIPT MODE ..............................................................................................110
FIGURE 6-1 SETUP FOR INDUSTRY CASE STUDY.............................................................................124
FIGURE 6-2 SEE-ADA USAGE MONITOR.........................................................................................127
FIGURE 6-3 MONITORINGSCRIPT ANDUSER INTERACTION.............................................................127
FIGURE 6-4 TOTAL IP CLASSES BYCLASS CATEGORY....................................................................134
FIGURE 6-5 TOTAL IP CLASSESOVER TIME ...................................................................................135
FIGURE 6-6 TOTAL IP INSTANCESOVER TIME................................................................................136
FIGURE 6-7 INSTANCES BYCLASS CATEGORY FORSNAPSHOT24 ...................................................137
FIGURE 6-8 PRODUCT SOURCE CODE INSTANCES..........................................................................138
FIGURE 6-9 USAGE BY TASKS.........................................................................................................147

viii
 List of Tables

TABLE 2-1 SUMMARY OFPROBLEMSRELATED TO THEUSE OF DESCRIPTIVEINFORMATION.............44

TABLE 2-2 SUMMARY OFKEY ISSUES TO BEADDRESSED.................................................................45
TABLE 4-1 EXAMPLES OFSE INFORMATIONPRODUCTS ...................................................................66
TABLE 6-2 DETAILS OF USAGE SESSIONS.......................................................................................149

ix
 Table of Abbreviations

AMD: Active Missile Decoy

ASIS: Ada Semantic Interface Specification

AWADI: AWA Defence Industries

CASE: Computer Aided Software Engineering

CDIF: Common Data Interchange Format

CSU: Computer Software Unit

COTS: Commercial Off The Shelf

DIANA: Descriptive Intermediate Attributed Notation for Ada

DSE: Description in Software Engineering

DSEPF: Description in Software Engineering Process Framework

DSTO: Defence Science and Technology Organisation

FCS: Fire Control System

HCI: Human Computer Interface

IP: Information Product

IPSE: Integrated Project Support Environment

IVD: Integrated Visual Description

IVES: Integrated Visualisation Environment for Software

IV&V: Independent Verification and Validation

PDR: Preliminary Design Review

PIR: Project Information Resource

x
PIRM: Project Information Resource Model

PIRMAS: Project Information Resource Modelling and Analysis System

SDD: Software Design Document

SDF: Software Development File

SE: Software Engineering

SLOC: Source Line of Code

SMAS: SPM Measurement and Analysis System

SPM: Software Product Model

SRS: Software Requirements Specification

SV: Software Visualisation

UMAS: Usage Monitoring and Analysis System

UniSA: University of South Australia

xi
 Summary

Large software projects produce and use vast amounts of information. This information is
provided by a range of sources including Computer-Aided Software Engineering tools,
software documents, software metrics, and integrated project support environments.
Individuals performing tasks associated with software projects need information to help
describe, and hence make visible, characteristics of the evolving software products, the
processes used for their development, and project resources. However, the available
information often fails to provide effective descriptions which are appropriate to the user or the
task being performed. Many problems need to be addressed. These include information
overload, inappropriate use of representation and presentation techniques, problems of
integration and assimilation, consistency, and the high costs of producing and using
information. This thesis analyses these problems, defines a range of issues that need to be
addressed, and proposes abstract approaches together with practical solutions for more
effectively providing and using the required information.

A model-based integrated visualisation approach is proposed as a means of addressing

many of these problems and issues. This approach is based on the use of computer-based
visualisation techniques for providing descriptive information which can be readily integrated,
customised, and adapted to individual user needs. A product-oriented approach, based on
the concept of a Software Product Model, allows various types of software engineering
information to be provided in context and in a consistent manner.

This thesis argues that approaches of this type must form an integral part of software
engineering practice if they are to be used effectively. A conceptual process-based
framework is proposed which includes description processes, products, and related models.
The framework is based on a contextual model. This model considers the software
engineering processes and the project information resources which can be used to support
task-based information needs. This framework is used to aid discussion and understanding, to
support the transitioning of the integrated visualisation approaches into practice, and to
provide a basis for experimentation.

This thesis demonstrates that the proposed theory and approaches can form the basis of
practical implementations. The concept of an Integrated Visualisation Environment for
Software (IVES) is presented as a way of accessing and filtering information, performing
xii
model-based integration of core information, and providing Integrated Visual Descriptions
which support representational and presentational integration of various types of information.
This type of environment supports the process-based framework by providing scripting
facilities which allow the automatic preparation of customised descriptions. An interactive
direct manipulation interface allows users to mediate with the environment and so adapt
information to their specific needs. A practical implementation which embodies many of these
concepts has been developed and applied. An instrumented version of this environment serves
as a key apparatus for this research. It has been used to test and further refine the concepts
presented in this thesis.

Various forms of experimentation have been conducted to explore a range of key issues and to
test the proposed theory and models. A major element of this experimentation is a series of
industry case studies. This thesis discusses the results of the first of these studies. The
results to date suggest that the proposed approaches can provide technical managers with
improved oversight of product and project characteristics and can support many important
tasks that were difficult, if not impossible, to accomplish with existing methods. The proposed
concepts and approaches presented in this thesis have the potential for making more effective
use of project information resources for a wide variety of software engineering tasks and for
reducing the reliance on traditional paper-based documentation.

xiii
 Declaration

I declare that this thesis does not incorporate without acknowledgment any material previously
submitted for a degree or diploma in any university; and that to the best of knowledge it does
not contain any materials previously published or written by another person except where due
reference is made in the text.

(R.J. Vernik)

xiv
 Acknowledgments

It is with deep gratitude that I acknowledge the encouragement, friendship, and support
provided by my supervisors, Dr Tony Sobey (University of South Australia) and Dr Martin
Burke (DSTO).

The review process for this thesis went through several iterations. In addition to my
supervisors, I would also like to thank Gina Kingston, Stefan Landherr, and Dr Andre
Yakovleff for the many hours that they spent reviewing drafts and highlighting the many
weaknesses and problems. I would also like to thank Professor Ray Offen for reviewing the
thesis design and for providing stimulating conversations on software engineering research and
scientific practices.

The work discussed in this thesis was conducted as part of a research programme sponsored by
the Australian Defence Organisation. I would like to thank my employer, the Defence Science
and Technology Organisation (DSTO), for encouraging and supporting my PhD studies.
Particular thanks goes to Stefan Landherr (the Head of Software Engineering Group) who has
been instrumental in providing the management support which has made this work possible.

I would also like to thank the members of Software Engineering Group (past and present) who
participated in the implementation of SEE-Ada and the other components of the experimental
setup. I am sure that there have been many times when they have thought to themselves, "Oh
no, not yet another idea". Their efforts have been instrumental in showing that the concepts
presented in this thesis are implementable and can be used in practice.

The staff at the AWA Defence Industries Salisbury facility also deserve special mention. They
provided the focus for the Industry Case Study. I would particularly like to thank Chris King
for supporting this research and for the many hours of illuminating discussion that we had on
various aspects of the work.

Finally, I would like to thank the three most important people in my life. To my wife Sue, and
our children Michael and Maree, thank you for your patience, encouragement and support
during this often trying experience. This has been an education for all of us.

xv
 1. INTRODUCTION

1.1 General
This thesis is submitted for the degree of Doctor of Philosophy in Computer and Information
Science at the University of South Australia (UniSA). The original ideas presented in this
thesis were developed by the author while employed as a researcher at the Information
Technology Division of the Defence Science and Technology Organisation (DSTO) and while
studying as an external student at UniSA.

The research was conducted as part of a research programme sponsored by the First Assistant
Secretary Defence Materiel of the Australian Defence Organisation. The main objective of the
research is to address problems and difficulties being experienced in the development and
maintenance of large software systems due to :

• poor project and product visibility; and

• the ineffectiveness of, and high costs associated with, the provision and use of many forms
of project and product information (eg documentation, metrics).

This work is fundamentally about the effective provision and use of information in Software
Engineering (SE). It aims to understand the problems and issues related to the human use of
SE information and to propose solutions which will help improve the quality of software
products, reduce project risks, and provide a more cost-effective basis for software
development and support.

1.2 Context
SE is an emerging discipline that focuses on the creation, development, operation and
maintenance of cost-effective, reliably correct, and high quality solutions to software problems
(Berry 1992). As an engineering discipline, it involves the application of systematic,
disciplined, and quantifiable approaches. SE recognises technical, managerial (Rook 1986),
and socio-technical issues (Offen and Xia 1995). Although based on computer science, it is
multi-disciplinary including elements of mathematics, management theory, engineering science,
and the social sciences. SE is particularly important for large complex projects. These

1
projects often have complex requirements, strict cost and schedule constraints, and involve
significant interactions by teams of individuals. They make use of complex environments to
support the development and maintenance of large and involved software products. The work
presented in this thesis focuses on the SE perspective of large software projects.

Although SE draws from the more established scientific and engineering disciplines, it has
characteristics which are unique and must be addressed as such. A major difference between
SE and some other more established disciplines is the nature of the products that are developed
and supported. SE deals with non-physical, intangible products. Software is a synthetic
product which is not subject to natural laws or other physical constraints. In essence, software
is an information product with its own set of unique characteristics and difficulties. These are
well recorded in the literature and include issues related to visibility, changeability,
conformance, scalability, and correctness (Brooks 1987; Lehman 1991).

The underlying characteristics and difficulties of software have a major impact on how it is
produced and managed. Production of software focuses on development rather than
traditional manufacturing processes and has a significant reliance on the individual (Rook
1986; Basili 1992). Difficulties associated with controlling the acquisition, development, and
maintenance of software have resulted in a major emphasis on process definition and
improvement. The new standard on Software Lifecycle Processes (ISO/IEC_12207-1995)
provides a standard and representative set of software lifecycle processes that need to be
considered for software projects and is used in this thesis to help define the context for
discussions.

This thesis defines and focuses on the notion of a SE process (the concept is further discussed
in Appendix A and Chapter 4) which includes the development process, a set of organisational
processes (such as the management and improvement processes), and a set of supporting
lifecycle processes such as verification and validation, quality assurance, improvement,
configuration management, review, and documentation. The SE process transforms
requirements into software products through the interaction of these processes.

Information is both a prime product and a major resource of the SE process. Indeed, due to
these special characteristics, Software Engineering could be defined as an information-based
discipline. As will be discussed in Chapter 4, many different classes of information products
are produced and used during software projects by a variety of sources including people,
Computer Aided Software Engineering (CASE) tools, management tools, and integrated

2
environments. Various classes of information products provide a myriad of types of
information including requirements, plans, design, code, test, and review details. These
information products take on a variety of representations including models, diagrams, numeric
lists, text, and code. Information product classes are often combined into higher-level product
classes such as standard documents or software development files. They can be provided in a
variety of ways including paper, and various electronic forms (eg binary or ASCII files).

Some classes of information products are predominantly used by tools (eg object code,
compilation directives, design models), while other information is meant for humans. The
human-readable information is commonly used to describe, and hence provide indirect
visibility of, the characteristics of various classes of SE entities including products, processes,
and resources. This descriptive information (or descriptions in short) is used to support
individuals engaged in a range of SE tasks including analysis, planning, evaluation, monitoring,
and predicting. The effective provision and use of descriptive information is the main focus of
this thesis.

1.3 Motivation
As will be discussed in Chapter 2, many different types of information-related problems are
being experienced by those engaged in the various tasks associated with the SE processes.
These problems include information overload, the unavailability of required information, the
inappropriate use of representation and presentation techniques, and the high cost of provision
and use of information. These problems can create particular difficulties when a high-level
view of product and project characteristics is required. Individuals engaged in activities such
as those associated with the management and supporting processes, often require information
from a number of sources to be combined and presented in a composite, customised form.
This type of information often needs to be viewed from different perspectives for different
tasks, individuals, and roles. These issues are discussed further in Chapter 2.

Although significant work has been done to identify techniques for generating and managing
SE information, much less has been done to understand and support the information needs of
those involved in SE practice. This work aims to define approaches that can be used to more
effectively provide SE information for those engaged in tasks associated with technical
management and supporting processes. A key motivation of this work is to look at how these
approaches can be used and evaluated in practice.

3
New information technologies, based on the use of graphics-based workstations, are having a
major impact on how information is provided and used in a wide range of disciplines. Many
novel visualisation approaches are being used to more effectively present information to
individuals engaged in tasks ranging from criminal investigations (Davidson 1993) to
diagnostic radiology (Rogers 1995). There has also been significant interest in the use of
computer-based visualisation approaches for describing software (Price et al. 1993). These
techniques offer many potential benefits. For example, they are particularly useful for viewing
large amounts of complex, diverse, and changing information. They may also provide an
effective and flexible means of providing information that can be integrated, customised, and
adapted to individual needs.

1.4 Main Proposals and Ideas

The main proposals and ideas of this thesis are:

• Major problems and difficulties exist in SE because of ineffective provision and use of
information. Since SE deals largely with conceptual entities, individuals require
information to describe, and hence gain indirect visibility of, various product and project
characteristics. This descriptive information is important for a wide variety of SE tasks,
particularly those that are associated with the technical management and supporting
processes.

• The use of a model-based visualisation approach can help address many of these problems
by providing information which can be readily accessed, integrated, customised, and
adapted to the specific needs of users for the task being undertaken. The proposed
approach uses a product-oriented technique based on the concept of a Software Product
Model which allows various types of SE information to be provided, in context, in a
consistent manner.

• To be effective, these visualisation approaches must form an integral part of SE practice.

A conceptual framework which includes description products, processes, and models is
proposed as a means of providing the necessary basis for the provision, use, and evaluation
of this form of information. This framework is based on a contextual model which
considers the SE processes and the project information resources that can be used to
support task-based information needs.

4
• Practical tools can be built, applied, and evaluated using these concepts and models. These
tools can be used to help understand the problems and issues of information use in SE and
to more effectively support individuals involved in a range of SE tasks.

1.5 Aims of Thesis

The aims of this thesis are to:

• Highlight and discuss a major weakness in SE practice; namely, the ineffective provision
and use of descriptive information for those engaged in SE management and support tasks.

• Define an approach to researching and addressing these problems, discuss the results of
work to date, and to define future directions.

• Propose theoretical foundations for supporting the definition, development, transitioning,

and evaluation of new forms of descriptive information.

• Present concepts and the implementation of an integrated model-based visualisation

approach.

• Through experimentation and discussion, critically evaluate the concepts, models, and
environment that form the basis of this research.

• Discuss the potential implications of this work to the SE discipline and industry.

1.6 Readership
The concepts and approaches discussed in this thesis draw from a broad range of research
areas including software visualisation, management theory, cognitive science, and software
measurement. The thesis does not assume that the reader has significant knowledge in all or
any of these areas. Chapter 2 provides much of the background material necessary for
understanding. The thesis does assume that the reader has some understanding of the nature,
theory, practice, and problems related to the development of large software systems.

5
1.7 Presentation
This section is provided as an aid to the reader. It provides a brief overview of the structure
and contents of the thesis. Details of the main threads and the flow of argument are provided.
This section also highlights the layout practices and notations that have been used.

1.7.1 Structure and Contents

The thesis is structured as follows. Chapter 1 defines the general setting and aims of the thesis.
Chapter 2 provides background information on the nature of the problems being addressed. It
provides details of preliminary studies that have been undertaken as part of this research in
order to gain an understanding of the problems and issues. This chapter also provides an
extensive overview of related literature, and defines a set of issues and conjectures to be
addressed. Chapter 3 provides details of the research approach used to explore the problems,
ideas, and proposals that form the basis of this thesis. Chapter 4 provides the underlying
theory that supports this work, including concepts and models. Chapter 5 describes and
discusses the integrated visualisation approach and provides a conceptual model of an
Integrated Visualisation Environment for Software (IVES) which encompasses the previously
defined concepts, ideas, and models. This chapter also provides details of a practical
implementation of an IVES (called SEE-Ada) that has been built and extended to help explore
the concepts, problems, and issues related to the application of these techniques. Chapter 6
deals with experimentation and provides a summary of the various forms of experimentation
that have been undertaken to explore many aspects of the work. The main focus of Chapter 6
is an industry-based case study which is being undertaken to more fully explore and test the
main conjectures of this thesis. Chapter 7 is entitled “Discussion”. Its purpose is to provide a
balanced argument concerning the findings, benefits, and limitations of this work. Proposed
future work is also discussed. Chapter 8 sums up and highlights the achievements, limitations,
and potential impact of the work.

1.7.2 Key Sections and Threads

Figure 1-1 describes the structure and key features of the thesis. The flow of argument is
basically sequential with two main threads. (1) The problems, issues, and conjectures identified
in Section 2.3 provide the main focus for this thesis. They define the basis for the research
objectives and approach, the concepts and models, the use of the integrated visualisation
approach, and the experimental design of the industry case study. (2) Since this thesis deals

6
with providing effective information for people, cognitive science issues play an important part
in the discussions. Section 2.2.4 provides an overview of relevant research in this area. This
work is referred to at various points throughout the thesis to support conjectures and
arguments.

In addition to the cognitive science aspects, Chapter 2 provides an extensive overview and
discussion of other work that is related to the concepts and approaches proposed in this thesis.
These include discussions on information logistics, software visibility, sources of SE
information, and visualisation approaches. This background material provides the context for
presenting the main contributions of this thesis and is referred to throughout the remainder of
the thesis.

The results of the various forms of experimentation that have been conducted as part of this
research are discussed at appropriate places in the thesis. For example, the results of the
preliminary studies are provided in Chapter 2 since they help support the definition of the key
problems, issues, and conjectures. Chapter 6 provides a summary of these earlier forms of
experimentation as well as providing details of an Industry Case Study which is being
conducted to more rigorously test and evaluate the concepts, models, and approaches.

1.7.3 Layout, Terminology and Notations

Each chapter begins with a summary which introduces the material to be presented. Where
applicable, this summary also identifies original content and potential contributions attributable
to the author. A summary of original content and contributions is provided in Section 7.7

Standard SE terminology is used wherever possible. Unless otherwise specified, general SE

terminology is as defined in the IEEE Standard Glossary of Software Engineering Terminology
(IEEE_Std_610.12-1990). Terminology related to software lifecycle processes is from
ISO/IEC_12207-1995.

Abbreviations replace composite terms that are commonly used in particular sections of the
thesis. If an abbreviated term has not been used for some time, the expanded form will be used
to remind the reader of the meaning of the abbreviation. All abbreviations used in this thesis
are provided in the Table of Abbreviations.

Double quotations marks are used for actual quotations or for references to a particular word
in a table or figure. Single quotation marks are used to highlight unusual terms or terms that

7
are commonly used (and sometime abused) by the SE community but have a meaning different
to that which is generally understood.

Chapter1 Chapter 8
INTRODUCTION SUMMING UP

Chapter 7
DISCUSSION
Chapter 2 • Research Evaluation
BACKGROUND • Findings
• Potential Benefits and
• Preliminary Studies
Implicatons
• Related Work
• Original Content and
• Cognitive Science Issues Contributions
• Problems and Issues
• Conjectures

Chapter 6
EXPERIMENTATION

Chapter 3
• Summary
RESEARCH APPROACH • Industry Case Study

Chapter 4 Chapter 5
CONCEPTS AND MODELS INTEGRATED VISUALISATION
OF SOFTWARE SYSTEMS
• Contextual Model
• Project Information Resource
• Description Concepts • Integrated Visualisation Approach
• Description Process • Integrated Visual Descriptions
• DSE Framework • IVES
• Model Based Descriptions • SEE-Ada
• Product Oriented Description

Appendix A Appendix B Appendix C Appendix D Appendix E Appendices

Perspectives on Definitions of Fire Control Case Study F.. I
Software Projects Preliminary SEE-Ada Details System Project Goals and Case Study
and Processes Effectiveness Questions Results
Criteria

Figure 1-1 Structural Description of the Thesis

8
 2. BACKGROUND

This chapter explores the problems and difficulties associated with the provision and use of
information in SE. It presents the results of preliminary studies that were undertaken to gain a
better understanding of the key issues and problems. This is followed by a review and
discussion of related work. The final section of the chapter contrasts and compares the results
of the preliminary studies, together with related work, to help define the problems, issues,
criteria, and potential solutions that form the basis of this thesis.

Original Content and Contributions. These include new insights into, and a better
definition of, the problems and issues related to the provision and use of information in
Software Engineering; the definition of a preliminary set of effectiveness criteria for descriptive
information; and conjectured approaches for addressing the problems and issues.

2.1 Preliminary Studies

This section reports on the results of early studies that were undertaken to help gain insights
into the types of problems that were being experienced by those responsible for managing the
procurement and development of large software systems. The work was undertaken by the
author just prior to commencing his PhD candidature. The results of these studies are reported
here since they helped provide the necessary background knowledge and experience which
ultimately resulted in the definition of the issues, conjectures, and approaches that form the
basis of this thesis. Much of the discussion in this thesis is based on these early experiences.

Initial discussions with procurers and developers of software systems highlighted a range of
problems related to software product visibility. Procurers were particularly interested in
gaining visibility of a range of high-level quality characteristics of software products as they
were being developed. Of particular interest were those characteristics which could impact the
operation or through-life support of the product such as reliability, maintainability, and
portability (ISO/IEC_9126-1991 1991). Developers indicated that they were experiencing
difficulties in undertaking product evaluations and demonstrating these characteristics to
customers.

There was significant anecdotal evidence in the literature and within the Software Engineering
community to suggest that product metrics could provide the basis of the required analysis and

9
evaluation tasks (McCall et al. 1977; McCabe 1987; Murine and Carpenter 1983). Early
investigations showed that this was not the case and that effective evaluation of software
would require access to a range of information, not just metrics. The aim of the preliminary
studies was to investigate how various forms of information could be used to support
individuals engaged in these types of SE tasks.

2.1.1 Approach

The approach focused on the use of three main classes of information: metrics, technical
documents, and output from the reverse engineering component of a CASE tool. The
AdaMAT tool (Keller and Perkins 1985) was the prime source of product metric information.
A commercially available CASE tool called AdaGEN was used to help recover and describe
the code structure. Technical documents which were being developed as part of the standard
development approach (DOD-STD-2167 1985) were also made available for use in the
evaluations. The studies made use of the Software Requirements Specifications (SRS), the
Software Top Level Design Documents, the Software Detailed Design Documents, and the
source code listings.

Two software systems that were being acquired by Defence agencies were evaluated. These
were real-time embedded products of medium size and were developed using the Ada language
and defence development standards. The products were analysed and evaluated during the
integration phase of development (ie after the majority of the source code had been
developed).

2.1.2 Summary of Results

Many of the results of the preliminary studies were published in (Vernik and Turner 1992).
This section summarises some of the key findings in relation to the provision and use of project
information. Some findings relate to particular sources of information while others reflect the
underlying problems and difficulties.

2.1.2.1 Problems Related to Particular Sources of Information

The preliminary studies highlighted a number of problems that were directly related to the
classes of information used for the analysis and evaluation tasks.

• Use of Product Metrics. Several problems were experienced in the use of metric
information, including: (1) The sheer quantity of information that was produced by the
10
 metric tool was overwhelming. For example, one of the systems comprised 739 source
code files for which the metric tool produced some 162,580 measures. This amounted to
some 2217 pages of metric information. (2) Many of the metrics did not correspond with
the observations made of the code and so were not considered meaningful. This was
particularly so for the high level metrics, such as portability, which were computed using a
simple weighted-sum model based on counts of source code features. Many concerns were
raised over the validity of the metrics. (3) The metrics were not useful without additional
knowledge and experience. For example, there were few guidelines to help define
acceptable metric thresholds. Moreover, even when thresholds were defined, the question
remained: What action would need to be taken for the unit to become 'acceptable'? (4) The
metrics proved to be useful indicators but needed further interpretation. For example, it
was found that a low comment count for a unit was not necessarily detrimental to code
descriptiveness. In some instances the naming conventions used were sufficient for
understanding and little additional documentation was required. There were also cases
where metric values indicated poor commenting practices, but further analysis showed that
the code had been automatically generated and hence was not meant to be read or
modified. This in turn had a major effect on higher-level metrics that were computed using
these values. Analysis also showed that high comment counts did not always indicate good
practices. For example, on one system, the developers simply commented out residual
code that was no longer required. Although this practice was seen as detrimental to
maintainability, the maintainability metric was enhanced by the high comment counts.
There were also cases where, although there was a high ratio of comments to code, the
comments were no more descriptive than the code.

• Use of Technical Documentation. The technical documentation provided for the

evaluations was found to be inaccurate and inconsistent. For example, the design did not
necessarily reflect the structure of the implementation as described in the source code and
revealed by the output of the reverse engineering process. In one case, code analysis
identified a complete subsystem that was not documented in the design of one of the
product baselines. There were many other problems including changed names, missing and
additional units, and modified design dependencies. Major inconsistencies were found to
exist between the top level and detailed designs which were described in separate
documents. Inconsistencies also existed between the design documents and the
requirements document. The timeliness of the information was also a major problem. For

11
 example, by the time a design went through the release chain, it was not unusual for it to
have been two or more months out of date.

• Use of Reverse Engineering Information. The CASE tool provided accurate

descriptions of code structure. This information proved useful but major problems were
experienced due to the immaturity of the tool and its inappropriate use of representation
and presentation techniques. For example, the directed graph representations used did not
scale well and so it was necessary to partition the system into small sets of entities that
could be displayed in a legible manner (eg 10 to 15 units at a time). Another major
problem experienced was the time taken to produce these structural descriptions. The tool
could take several hours to produce representations of some sections of the code.

The information provided by the tool was also used in presentations to the customer.
These activities highlighted the fact that the choice of representation can communicate the
wrong message. For example, the multitude of lines on the display immediately suggested
poor structure and complexity. Analysis showed that this was not necessarily the case.
These experiences demonstrated that the type of the representation technique and layout
can have a major effect on how the information is perceived.

2.1.2.2 Underlying Issues and Problems

In addition to the problems related to particular sources and types of information, the
following underlying issues and problems were identified:

• Importance of Context. Experiences showed that many tasks require information to be

viewed in context. For example, the metric values were useful for showing general trends
and outliers. However, for some tasks it was necessary to consider both the metric values
and the structural representation of the system. The following example illustrates this
point. One of the evaluation tasks involved assessing whether mutual exclusion problems
could impact reliability. Metric information could be used to identify global variables in
Ada units. However, to consider the impact that each instance might have on reliability, it
was important to relate this information to the structural description of the system. For
example, if a global variable was found in a unit that was accessed by units containing
concurrent threads of execution, there was a higher probability that timing (and hence
reliability) problems could result.

12
• Integration of Various Types of Information. The preliminary studies highlighted the
need for integrating various types of information. For example, AdaMAT provided
measures that could help describe reliability attributes. However, this information alone
was insufficient to make an appropriate assessment. In addition to measures of product
attributes, other process-based attributes (eg degree of testing, degree of change, defects)
and resource information (eg the author of particular sections of code) needed to be
considered to gain appropriate insights into these high-level characteristics. However, this
information was not available for the studies and hence represented a major limitation.
Even if this information had been available, there would have been major problems in using
these various types and forms of information to support the evaluation tasks. In addition
to gaining access to the information from a range of sources, the user would need to
understand the relationships between the information items and provide a means of
assimilating and integrating the information.

• Consistency of Information. Problems were experienced because the available

information reflected different stages of product evolution. The metrics were produced on
baselined releases of the source code. This information often needed to be considered in
relation to structural descriptions in the design documents which were often significantly
out of date. For example, there were many cases where the names of components in the
design did not reflect actual units names found in the CASE tool descriptions or metric
information. There were also problems of consistency between the various representations
of the product. For example, it was difficult to relate diagrams produced by reverse
engineering to the design descriptions provided in the design documents because these
sources used different representational forms. These problems of representational
consistency can have a major impact on our ability to visualise and understand the overall
software product.

• Useability of Information. In many cases, the information was not provided in a form
that satisfied the needs of the individual for the task being undertaken. Information was
often provided in a single form with the intention of satisfying a range of tasks, individuals,
and roles. The technical documents were found to be the least flexible. This form of
information cannot generally be customised for particular users or needs. Accessibility is
also a problem. Documents of the type used in these studies did not allow efficient access
to information, particularly across several documents. Similar problems were found with
the metric information which was provided in tabular report form. This inflexibility of

13
 representation and presentation made it difficult for the user to access particular subsets of
information (eg those items which fail to reach a particular threshold level, or those related
to particular classes of entities). The metric tool did allow some flexibility in terms of
report layout. However, this could not be done interactively and the new reports took
some time to produce. The information provided by the AdaGEN tool was most flexible.
It allowed some degree of interactive customisation of the information provided.

Interactions with project personnel during these studies highlighted the fact that these types of
problems were faced in many other tasks, particularly those that required a high-level view of
products and processes (eg management and SE support tasks). These types of tasks often
require information which can describe large numbers of different types of entities, their
attributes, and relationships. Discussions with developers indicated that many important tasks
could not be undertaken if appropriate information was not available or if it was difficult to
use.

The results of these studies played a major part in the initial definition of the SEE-Ada
environment (as discussed in Chapter 6 and Appendix C) which implements many of the ideas
presented in this thesis. The SEE-Ada concept was proposed as a way of integrating various
types of project information and for presenting this information in a consistent customisable
way. The preliminary studies also helped provide the necessary background for identifying the
key issues and conjectures that form the basis of this thesis. These are provided in Section 2.3.

2.2 Overview and Discussion of Related Literature

This section provides an overview of related work. It aims to provide an appropriate context
for the concepts and ideas presented in this thesis and is used to support discussions and
arguments in the chapters which follow. The literature review which was conducted as part of
this research covered a range of sources including academic literature (eg journals, theses,
conference proceedings, books), commercial literature (eg product descriptions, user manuals),
and discipline-based literature (eg standards, company procedures and practices). A diverse
literature review was seen as an important part of this research since a significant amount of
work related to the SE discipline is not reported in the academic literature. This includes
information on new tools and methods developed by industry and the significant amounts of
work undertaken by the standards community. It can be argued that diverse literature reviews

14
of this type can provide researchers with better insights into problems and issues and can help
prevent us from 'reinventing the wheel'.

The literature review is presented in terms of four main areas related to the topic of this thesis:
namely, the provision and use of information in SE. These are:

• General Problems and Issues.

• Problems Related to Specific Sources of SE Information.

• Use of Visualisation Approaches.

• Cognitive Science Issues.

The section on general problems and issues provides a discussion of a range of underlying
issues such as software visibility, information logistics, and support for task-based information
needs. There has been considerable discussion in the literature on the problems and limitations
of particular sources and forms of information. Those discussed in this section include
technical documentation, software metrics, and tools and environments. Of particular interest
to the subject matter of this thesis is the section on "Use of Visualisation Approaches" which
presents a general overview of visualisation techniques and discusses various software
visualisation approaches. The benefits and limitations of these techniques are discussed. Since
this thesis deals with the human use of information, a range of cognitive science issues are also
discussed.

2.2.1 General Problems and Issues

Problems related to the provision and use of information pose a particular problem for SE
(Devanbu et al. 1991). However, these problems are not restricted to SE. For example, Love
(1993) discusses how information can be used to support a variety of enterprises such as
banking, transport, and retailing and shows how information can improve competitiveness.
However, Love argues that information itself is often the single largest barrier to success. He
cites several major obstacles that can be encountered including: extensive information
redundancy, unsynchronised copies of information, undefined relationships between items of
information, poor quality information (eg out-of-date, inaccurate, inconsistent), information
overload, untimely information, unavailable information, information volatility, and inadequate

15
information access. These types of problems were experienced during the preliminary studies
and are discussed in Section 2.2.2 in relation to particular sources of SE information.

Many of these problems can be addressed through the effective application of information
technologies. However, there are some important philosophical, cognitive, and procedural
questions that need to be considered in relation to SE. For example, SE deals largely with
conceptual entities, so how can information be most effectively used to help gain 'visibility' of
these entities? How is information used to support individuals engaged in specific SE tasks?
What information is required for particular tasks and when is it required? How should it be
provided? The following three sections discuss some of these general issues and problems.

2.2.1.1 Problems of Software Visibility.

In their interactions with the physical world, humans rely heavily on their senses, particularly
vision. We are able to determine many characteristics of physical artefacts through direct
observation (Norman 1993). It is not surprising that we have a major problem dealing with a
conceptual entity such as software which is not directly visible. Problems of software visibility
have been reported as a major difficulty for those engaged in the acquisition, development, and
support of software products (Brooks 1987; Simmons 1992; Youll 1990). The problems relate
directly to the underlying nature of software. As discussed in Chapter 4, software is a
conceptual entity comprising a set of information products. It has no physical form and hence
direct visibility is not possible. However, this thesis argues that we can gain indirect visibility
through description. This involves generating descriptive information to describe those
characteristics of interest. In so doing, we must consider problems of representation and a
host of other issues related to the effective provision and use of information. These are
discussed more fully in Section 2.3.

Brooks (1987) contends that "software is invisible and unvisualisable". He argues that
software has no geometric representation and so, when we try to diagram its structure, we find
that it contains several directed graphs superimposed upon each other. Brooks suggests that
this inability to effectively visualise software prevents individuals from using some of their
inherent conceptual tools and severely hinders communication. The experience gained during
the initial studies, which used a reverse engineering tool to provide structural descriptions of
the software, showed that these problems of representation and complexity can manifest
themselves in problems of understanding, even with moderate numbers of entities and simple

16
relationships. The problem is exacerbated if we consider integrating other types of information
as discussed in Section 2.1.

There is general agreement with Brook's contention that some aspects of software are not
directly visible (eg control flows, pattern of dependency). However, this thesis argues that
software is visualisable. Visualisation enables a form of indirect visibility by creating an image
which supports human cognitive processes. As such a visualisation can be referred to as a
"mind's eye" view of an entity. If we are to provide appropriate support for those engaged in
software-related tasks, we need to consider new ways of thinking about, and gaining indirect
visibility of, software products.

A variety of approaches have been proposed and used to help address these visibility problems
including relational code analysers, knowledge based approaches, metrics, and a host of
visualisation techniques. Devanbu et al. (1991) argue that the problem of software visibility is
caused by information barriers. They report on the use of knowledge-based hypertext
techniques which integrate architectural, conceptual, and code information into a knowledge
base for use by developers. Others have focused more on visual representations and the use of
computer-based visualisations of software. These are discussed in Section 2.2.3.

Discussions in the literature tend to focus on gaining visibility of structural and dynamic
characteristics of software code. However, as discussed in the results of the preliminary
studies, there are other characteristics of the product that need to be visualised. Many of these
are described in terms of the processes and resources used during software development. For
example, if we wanted to see how particular components of a product evolved over time, we
might describe this in terms of versioning attributes obtained from configuration management
data. Similarly, if we wanted to see how well the software has been tested, we might use
visualisations of the product which incorporate test results. These types of information need to
be considered within the overall context of the software product. This thesis contends that
even though software is not visible in the traditional sense, it can be visualised through the
appropriate provision and use of descriptive information.

2.2.1.2 Information Logistics

As discussed previously, the effective provision and use of information is vitally important in
SE. Information is needed to describe a range of "things" including products, processes, and

17
resources. Fernstrom et al. (1992) indicate that information logistics should be a major
consideration for SE environments (in their work, the term environment is a general concept
rather than a specific computer-based application). They suggest that “the logistics problem
is how to provide precisely the information that is currently needed and how to ensure that
the information is valid and consistent with the assigned user tasks”. Three perspectives on
information logistics are provided. (1) At the organisational level the environment must
manage access rights and enforce procedures according to roles and assignments. (2) At the
team level, the environment must provide change notification and mutability control to manage
updates of shared information. (3) At the individual level, the environment must reduce
information overload and provide focused views of relevant information. This thesis focuses
on issues related to the third perspective.

A significant amount of work has been done to address the first two issues. For example, there
has been considerable work in the areas of Integrated Software Engineering Environments,
tool integration, and supporting technologies such as object bases, repository standards, and
process languages. These approaches provide the infrastructure to support the logistics issues
and are discussed in Section 2.2.2.3. However, few studies have attempted to define, measure
and understand how, when, and which classes of information are produced on real projects.
There is also a lack of research available on how individuals use this information for particular
tasks. The Industry Case Study of Chapter 6 investigates these issues.

2.2.1.3 Supporting Task-Based Information Needs.

In an article about information use in large organisations, (Jones 1995) suggests that, although
there have been many articles about the 'information superhighway' and 'information
warehousing', they have contributed little to the fundamental question: "What kind of
information do managers and workers use?" A review of the literature suggests that this is
also true for Software Engineering.

Much of the work in SE has been directed towards understanding and supporting the needs of
those at the 'work face' (eg programmers, analysts, designers). This has resulted in the
development of a range of CASE tools and environments which support analysis, design,
coding, debugging, and testing tasks. Although many new technologies have been developed
and used in practice, much of this work has been criticised for its lack of appropriate validation
and evaluation (Fenton et al. 1994; Kitchenham et al. 1989). These approaches are often based

18
on perceived needs or commercial opportunities rather than on research that identifies actual
needs of individuals (eg through work study observations and task analysis).

Providing for the information needs of those involved in high-level SE tasks (eg those
associated with the technical management and supporting processes) has received far less
attention. Yet these areas are crucial to project success. Although there is wide belief that
issues related to the management of Software Engineering seem to count for much more than
technological issues (Fenton et al. 1994; Simmons 1992), there are few examples in the
research literature that have specifically studied the information needs of those engaged in
software management tasks. Moreover, although many approaches are meant to support the
information needs of managers (eg metrics, documentation approaches) they are rarely based
on sound research and there are few studies which look at the effectiveness of these
approaches in practice.

Duvall (1993) reports on one such study. The study was conducted to increase awareness and
understanding of the information needs of software managers and to provide an understanding
of the information environment surrounding software development. A total of 32 managers in
14 companies in the United States and Japan were observed and interviewed over a period of
nine months. These were middle-level managers who were two levels removed from the
programmer. The study showed that information was needed for a wide range of tasks
including planning, oversight, and technical support. Texts that discuss SE management
(Thayer 1992) tend to emphasise the traditional management tasks of planning, monitoring,
controlling, predicting, and reporting but neglect the importance of providing technical
support. Mid-level managers are often well placed to provide technical support because of
their high levels of developmental expertise. They are able to provide guidance to more junior
members of the team and this can have a positive impact on product quality and project risk.

The study also found that managers expressed information needs in terms of the objects of
interest. These included products, projects, and enabling technologies. The information needs
were found to be situation dependent and hence the same information was not necessarily
required for the same tasks. This has important implications for those attempting to define
information needs in terms of the tasks to be performed.

This study highlights a number of problems and issues that need to be considered for the
effective provision and use of information in SE. Some of the general problems that managers
reported included the inaccessibility of information, the inaccuracy of various forms of

19
information, and the unavailability of information. The quantity of information was also a
concern for managers: both too little and too much. Many of these problems and difficulties
were also observed during the preliminary studies discussed in Section 2.1.

Duvall summarises the results of her study by stressing that developers of systems and metrics
which are designed to support individuals involved in software development should consider
the multi-dimensional aspects of user information needs. This includes taking into account the
following issues: (1) the context of use; (2) the various applications for the information; (3)
presentation of information; and (4) the various social, political, and technical problems that
managers and developers may encounter in acquiring and using the information.

2.2.2 Issues Related to Specific Sources of SE Information

As discussed above, there are many general problems and issues related to the effective
provision and use of SE information. There are also many issues that relate to particular
sources of information. This section provides an overview and discussion of problems and
issues related to information provided by technical documentation, software metrics, and SE
tools and environments.

2.2.2.1 Technical Documentation

The term documentation is used to describe a wide range of information products. These
documents are typically seen as the deliverable information products that are provided by the
project to the customer and can include user documentation and technical documentation. The
main focus of the discussions in this section is technical documentation. Documents of this
type provide a range of technical information including plans, requirements, product
descriptions (eg specifications, design, code, version details), and test results. They are often
developed to standards (eg the IEEE Standards 1994; and DOD-STD-2167A 1987) which
dictate what the document should contain and how this information should be presented. The
use of these standard approaches is intended to support multi-person development, review, and
quality assurance. Large software projects can produce encyclopedic volumes of these semi-
structured and inter-related descriptions. Scacchi (1989) suggests that the cost of generating
this information is often justified since the scale of the development implies that the
documentation will be the focal medium for coordinating the engineering tasks and information
flows in the project. The resulting documentation is meant to support the information needs of

20
a wide range of individuals including those engaged in acquisition, development, and
maintenance activities.

There has been considerable discussion in the literature about the problems and difficulties
associated with the provision and use of these types of information products (see Jazzar (1988)
for an extensive summary). The importance of providing useful information and the enormous
costs that are often involved, has raised many questions about the applicability and usefulness
of documentation approaches (Jones 1992; Parnas and Clements 1986). Some of the problems
that have been cited include cost (of provision and use), consistency, accuracy, redundancy,
timeliness, and useability (eg accessibility and understandability). As highlighted by Love
(1993) and discussed in Section 2.2.1.2, many of these problems also relate to other types of
information. An analysis of the problems and issues of documentation can help in the
definition of issues and effectiveness criteria that need to be considered for SE information in
general.

A major problem with these more traditional documentation approaches is the sheer quantity
of information that can be produced. For example, Meyer et al. (1989) report on the volume
of design documentation produced during the development of a medium-scale software
project. The descriptions of the 470 design components comprised some 1800 pages of
documentation. In addition, the data dictionary comprised some 700 pages, while 650 pages
of information were produced for the data base specification. A total of 15 copies of this
documentation was delivered for each of the three reviews. As such, the overall amount of
design documentation produced for the project comprised some 142,000 pages of information.
This situation not only raises questions in relation to production costs, but also the cost of not
doing other important tasks because of the preoccupation with producing documentation, and
the cost of keeping the information up to date. Jones (1992) reports that up to 50 percent of
development costs can be attributable to documentation for some projects and that excessive
'paperwork' can be a major source of project risk.

As well as the documentation that is produced as contract deliverables, most projects also
produce and use a wide range of informal technical documentation, commonly referred to as
Software Development Files (SDFs). SDFs include various types of information such as
design rationale, informal test results, review comments, and various reference material (eg
published algorithms or interfaces). They are often managed as a series of folders containing
hand written notes, diagrams, and output from CASE tools. On many projects, this informal
project information provides the basis for the development of deliverable documentation and is

21
used as a prime source of information by developers. Although an important source of project
information, there do not appear to be any studies that have looked at the diversity, quantity,
and use of this type of information. Chapter 6 presents the results of an Industry Case Study
that has in part looked at issues related to the provision and use of this type of information.

Traditional documentation provides a means of recording information but often fails to support
individual information needs. Individuals often have difficulty accessing specific items of
information that might be required for a particular task. New approaches for providing this
type of information, such as hypertext (Conklin 1987), attempt to address this problem but can
create other problems. Software Engineering hypertext environments have been proposed as a
viable information management medium for organising, structuring, retrieving, and processing
the complex networks of software descriptions that arise within large SE projects (Scacchi
1989). Several benefits have been identified including the ability to support collaborative
work, easier document maintenance, and improved access to information (Oinas-Kukkonen
1992). However, a major shortcoming of this approach is that disorientation and cognitive
overload can ensue when dealing with large bodies of information (Blum 1989). Although
hypertext can support nonlinear text presentation, the nonlinearity is prescribed by the author
or authoring system. This imposes a predefined structure on the information rather than
allowing it to be structured according to user needs (Berleant et al. 1994). Browsing in these
systems is also susceptible to the "lost in hyperspace" phenomenon where users can lose their
contextual bearings.

Garg and Scacchi (1994) propose a knowledge-based approach for providing SE hypertext
information. They claim that this approach can establish the required context by capturing and
using knowledge about the tasks being performed and the types of users involved. This
presupposes that information needs can be effectively specified through task analysis. Work
done in the cognitive sciences (and discussed in Section 2.2.4) suggests that this is a difficult
problem, particularly for complex cognitive tasks (Grant and Mayes 1991). Others have also
looked at using knowledge-based approaches for addressing documentation problems. For
example, Johnson (1994) argues that "the key to substantial improvement in documentation is
an on-line system that can construct presentations dynamically". Instead of having the user
browse for information, he proposes a system that presents a set of questions to the user.
These questions are based on an analysis of the task being performed and the user's level of
expertise. The user then obtains the required information by selecting the question that needs
to be answered. If the system is able to answer the question, it supplies it. However, when it

22
can't, the system selects potentially relevant material and lets the user browse it through
hypertext. Although tools have been built to demonstrate these approaches, no significant
studies have been reported which would help provide evidence as to their benefits and
limitations in SE practice. Issues that need to be considered include set up and maintenance
costs, interfaces to relevant sources of underlying information (eg other CASE tools), and
issues of scale (ie can the approaches support real projects involving a wide range of needs and
diverse sets of information).

The documentation problem is one of recording, storing, managing, and providing relevant
persistent information. In essence, it is an information logistics problem. There are many ways
in which this type of information can be provided. It would be a mistake to believe that a
single approach could form the sole source of information. This thesis argues that prior to
considering the approach to be used for providing the information, the information needs of the
potential users of the information and the context of use need to be considered. The
appropriate approach can then be selected based on the type of information to be provided and
the usage criteria.

2.2.2.2 Software Metrics

Software metrics are another important source of information for those involved in SE.
Simmons (1992) argues that a metric-based management approach can overcome many of the
problems currently being faced by those using a documentation-driven approach. Others (eg
Andres (1989), Grady and Caswell (1987), and Hetzel (1993)) have shown that metrics can be
a useful source of information for software projects. But what is a metric and how can it best
be used?

Fenton (1991) argues that software metrics is one of the most misunderstood areas of
Software Engineering. The term is used to describe a vast array of information that has some
numerical basis. Metrics are used to support a wide range of SE activities. They can be used
as management indicators (Andres 1989; Schultz 1988), for quality assurance (Basili et al.
1986b; Card and Glass 1990; Delaney and Summerill 1988; Keller and Laurence 1988), and
cost estimation (Conte et al. 1986; Albrecht and Gaffney 1983). Metrics can be provided in a
variety of numerical, graphical, tabular, and report forms.

A plethora of metrics have been proposed. For example, Zuse (1991) reports that there are in
excess of 100 measures of software product complexity alone. Some metrics are defined in
terms of simple counts of code features (eg lines of executable code, number of comments).
23
Others are defined in terms of functions of sets of these counts such as McCabe complexity
(McCabe 1976) and Software Science metrics (Halstead 1977). Metrics can also be defined in
composite terms such as 'Average Fixed Defects per Working Day' (Grady and Caswell 1987).
Although metrics have become an important source of information for software projects, much
of this work has been criticised for not being based on sound measurement principles (Fenton
1991). The term "measure" is often used to distinguish information that is generated through
the use of well-founded measurement principles and processes (Kaposi 1991).

Measures are those elements of information (typically numbers or symbols) that describe the
attributes of some entity. These entities include products, processes, and resources (Fenton
1991). Measures have properties that make them extremely effective and important forms of
information. They are concise and can be objective descriptions. A single number (or symbol)
can be used to describe something that can take many words to express. However,
measurement is much more than simply allocating numbers to 'things'. Measurement is a
formal process that includes empirical observation, modelling (eg of the target set), mappings
to formal representation systems, and validation. Much of modern measurement theory is
based on the representational approach of measurement. This approach originated from
Tarski's (Tarski 1954) work on relational systems and model theory. This theory considers
measurement in terms of mappings between the empirical "real world" domain and the
mathematical domain. It is based on a formal set of mathematical theories, axioms, and
principles (eg transitivity, irreflexibility, meaningfulness). Kaposi (1991) suggests that this
approach to measurement is more suitable for disciplines such as Software Engineering (which
deals with less tangible products and processes) than the more classical theories of
measurement (Campbell 1957; Helmholz 1930) which were developed to support the
measurement of physical properties. The use of measures as a form of descriptive information
and the relationship between measurement and description is further discussed in Chapter 4.

Adherence to the basic principles of measurement can help support the provision of valid and
useful descriptive information. For example, a key principle of measurement is
meaningfulness. Numerical values alone have no empirical meaning. They are only the
mathematical representation of the measure. The numerical results have to be translated back
into the empirical domain if they are to be meaningful. For example, many of the high-level
metrics used during the preliminary studies (eg maintainability) were meaningless since they did
not relate to observations made during the evaluation tasks. It is questionable as to whether

24
the developers of these measures performed an appropriate level of empirical validation to
ensure that the measures would be meaningful within their domain of use.

Much of the published work in the general area of metrics has focused on proposing new
metrics and measures. However, more recently, there has been a greater emphasis on how this
information can be provided and used in practice (Roche 1994). A major problem for
organisations contemplating a metrics programme is to define what should be measured and
captured. Logic suggests that there is a need to first define information needs and then
determine what information can most effectively support those needs. Yet many metrics
programmes have failed because the information is seen as an end in itself. In many of these
failed attempts, there is a tendency to try to measure everything or only those things that are
easy to measure (Fenton 1991).

Several goal-based approaches have been defined to address this problem by providing an
appropriate framework for guiding the capture and use of metric information (Rombach 1991).
The most popular of these is the Goal/Question/Metric approach (Basili and Rombach 1988)
which provides a framework for defining goals and refining them into quantifiable questions.
These questions then provide a specification for the metrics that are needed. This approach
has been extended by Jeffery and Offen (1993) to include a business model. This Model
Measure Manage Paradigm (M3P) allows direct feedback of measurement data into business-
related aspects of the empirical model. These approaches focus on the use of metric or
measurement data as the primary source of information. However, there may be cases where
metrics or measures may not be the most appropriate or only form of descriptive information,
particularly in cases where appropriate relationships between the empirical and formal domains
may not have been established, or where the particular attribute may not be measurable.
Moreover, for effective use, other forms of information are often required to support the use of
this type of information (eg to view the information in context).

There are a range of issues that need to be considered in relation to the effective provision and
use of metric information. These include issues related to the validity and meaningfulness of
measures and issues related to how this type of information can be used to support particular
goals and tasks. There are also a number of issues that need to be considered in relation to the
human use of this type of information. For example, how do people use this type of
information for particular tasks, and how should it best be represented and presented? There
are also issues related to how this type of information can used together with other
information. As with other forms of information, the context within which metric information

25
is used is an important consideration. For example, as McGarry (1995) points out,
measurement data will not, in most cases, provide exact or complete information. In addition
to the measurement data, users need to consider context, the defined goals, subjective
information, and the results of qualitative analysis. This coincides with experiences gained
during the preliminary studies where measures needed to be viewed in conjunction with other
information to provide a basis for use.

2.2.2.3 Tools and Environments

Computer Aided Software Engineering tools and environments are becoming an important
source of SE information for many projects. A variety of tools and environments are currently
in use to support a range of SE activities such as analysis, design, testing, code development,
configuration management, project management, measurement, and documentation (Hewett
and Durham 1989). These tools can produce and utilise vast amounts of information. The
types of information supported by these tools are diverse: some of the information is produced
to support tool needs and some is provided in the form of descriptions that support users of the
tools. Since the tools often operate independently of one another, there are many problems
that need to be considered such as consistency, redundancy, and the interchange of
information. Moreover, from a project perspective, it is often difficult to know what is actually
available for use. There is generally no single model of project information which can be
employed to help provide an understanding of what information is available at a particular
point in time, the form of the information, or how it can be accessed.

Problems related to the availability, accessibility, and consistency of information are well
recognised by the SE research community. To address these problems, considerable research
has been undertaken in the areas of integrated environments, integration frameworks, object
management, and tool integration standards (Brown and Penedo 1992). The lack of a central
store of project information has been seen as a major impediment for software projects.
Significant work has been undertaken to define and develop Integrated Project Support
Environments (IPSE) based on the use of central repositories of project information. These
include proprietary (AD/Cycle 1992) and standardised approaches (Wakeman and Jowett
1993). Research environments of this type have also been developed (Taylor et al. 1989). The
use of these types of environments has not gained widespread acceptance in practice. This is
largely due to the fact that many vendors prefer to provide closed proprietary solutions for
their individual products, the high costs involved in procuring and supporting complex

26
environments, and the many practical and technical difficulties associated with defining and
implementing a project-wide information store (Brown et al. 1992; Emmerich et al. 1992).

In parallel with the IPSE research there has been a proliferation of individual tools that support
particular tasks. In some cases, standards have emerged to address specific issues such as the
interchange of SE information between tools (eg the Common Data Interchange Format
(CDIF) (Parker 1992) and the Ada Semantic Interface Specification (ASIS) (Barnes 1993)).
However, as investigations by Rader et al. (1995) suggest, these approaches are not widely
used. Some larger vendors provide integrated environments with simple proprietary solutions
for tool integration. For example, the Rational Apex environment (see Appendix D) uses
ASCII files for control and data interchange between tools. This has allowed Rational to
establish strategic alliances with other smaller vendors to provide CASE coalition (Brown et al.
1992).

Although significant research has been carried out in the area of integrated environments for
Software Engineering, it appears that, in many of the projects that do use tool support, these
tools are at best loosely integrated, each with their own repository of information and tailored
to support particular tasks. This poses major problems for those wishing to use the
information for tasks other than those specifically supported by the tool. As such, many
important tasks cannot be effectively undertaken because valuable information is not directly
accessible or useable. In addition to researching the theoretical and technical issues of tool
integration and repository support, it is important to address issues related to how tools and
environments integrate with, and support, current SE practice. This thesis is particularly
concerned with how information from tools and environments can be used to support the
information needs of those engaged in a range of technical management and project support
tasks.

As discussed above, most tools are developed to support particular types of tasks and classes
of users (eg designer, analyst, programmer). For example, the representations used by many
design tools are tied to a particular design method. Moreover, the design of the tool often
presupposes how the tool will be used (eg to directly support the method). This makes it
difficult to use the tool for other tasks. For example, an analyst may wish to use a design tool
to support the review of a design. This may require the design model to be viewed from a
number of perspectives. However, since the tool has not been designed to support a range of
needs, the underlying information is not readily accessible or useable. If tools are to more
effectively support those engaged in SE practice, greater flexibility needs to be provided in

27
terms of the representations used and the manner in which the user interacts with the tool to
obtain the required information.

In addition to providing integration of tools at the data or control level, there is a need to
consider presentational integration (Budgen et al. 1993). The need for consistent multiple
views of SE information is widely understood and recorded (Blum 1989; Marlin 1990). These
approaches allow for different perspectives of an entity to be provided in a single display.
Moreover, they can allow for additional information to be provided to support a particular
viewpoint (Nuseibeh and Finkelstein 1992). As Meyers (1991) suggests, maintaining
consistency between views is a key concern for this type of integration.

This thesis discusses a form of integration that is rarely addressed in relation to SE

environments and tools; namely, representational integration. As discussed in Chapter 5,
representational integration refers to the integration of different classes of information into a
single composite representation. Individual tools provide users with representations based on
their underlying information. For example, a design tool can describe the architecture of the
product, a measurement tool can describe attributes of product entities, and a configuration
management tool can describe version details. Through presentational integration, the output
of each of these tools can be provided on a single display, possibly using multiple windows.
These views may even be integrated so that only the metric reports for particular units selected
in the architecture view can be provided. Representational integration takes this one step
further by allowing the information from several sources to be provided in the form of a single
composite representation. For example, instead of the metric information being provided in a
separate view, this information could be overlayed onto the architectural representation of the
product (perhaps by way of colours). This approach allows information to be provided in
context. As discussed in Section 2.2.3.4, new approaches to software visualisation are
beginning to address this issue.

2.2.3 Use of Visualisation Approaches

The definition of the word "visualise" in the Oxford English Dictionary(Sykes 1976) highlights
two aspects of visualisation. The first refers to the fact that when we visualise we "make
visible to the eye". The second refers to the human conceptual aspects of visualisation where
the definition indicates that to visualise is to "make visible esp. to the mind". This aspect of the
definition highlights the notion of indirect visibility. Visualisation refers to the notion of a
mental image and hence the human perceptual and cognitive aspects. As previously discussed,

28
the intangible nature of the products developed through Software Engineering make
visualisation an important consideration.

2.2.3.1 Computer-Based Visualisations

McCormick et al. (1987) define computer-based visualisation as the "study of mechanisms in

computers and in humans which allow them in concert to perceive, use, and communicate
visual information". Colourful computer-based images are becoming a routine way of
conveying information for a range of tasks in a variety of application areas. For example, they
are being used to help support a range of scientific research endeavours through scientific
visualisation (Brodlie et al. 1992; Earnshaw and Wiseman 1992), to solve crimes (Davidson
1993), and to support the diagnosis of medical conditions (Rogers 1995). They are even being
used to convey weather information on the nightly television news.

Keller and Keller (1993) provide examples of, and attempt to categorise, the wide range of
computer-based visualisations that are in use today. Instead of focusing on specific application
areas, they identify thirteen main categories of visualisations based on the type of information
that is provided (eg surface and slice may be used for showing coronal sections of the brain or
structural geology for oil exploration). They show the power of computer-based visualisation
for presenting a range of different types of information including techniques for visualising
volume, surface and slice, multi-form representations, models, motion, and process. Many of
these techniques could find application in the general domain of Software Engineering.

Computer-based visualisation techniques have shown their value where large amounts of
complex, diverse, and inter-related information need to be presented, analysed, and explored.
They have proved to be particularly useful in areas where expert knowledge about the
relationships and correlations between various classes of entities have not been well explored
or defined (eg Software Engineering). Interactive computer-based visualisations can be used
to help discover patterns, form and test hypotheses, and identify exceptions (Shneiderman
1983). They can be particularly useful for integrating different types of information into
composite representations (eg representational integration), when the subject matter is
changing rapidly, or when information needs to be visualised in more than one way at the same
time (ie presentational integration). Computer-based visualisations can also provide a good
basis for information customisation (Berleant et al. 1994). The goal of information
customisation is to tailor information artefacts to a user's specific needs at a given time.

29
In addition to the host of novel visualisations that are found in the scientific visualisation
literature, there are many statistical visualisation techniques that can be used to effectively
provide numerical information. For example, various statistical visualisation techniques can,
and have been, used to present software metric information. Many cost-effective tools are
available to support these types of visualisations including statistical packages and spreadsheet
applications. These approaches are suitable for many purposes, however they cannot be easily
customised or adapted to individual needs (Mackinlay 1987) and do not always adequately
support the integration of different types of information. For example, a scatter plot may be
useful for identifying trends and outliers. However, scatter plots are not generally appropriate
for directly relating metric information to particular entities of interest. As discussed in the
results of the preliminary studies, there are many tasks where metric information needs to be
viewed in context. These issues are discussed in more detail in Section 2.2.3.4.

2.2.3.2 Software Visualisation

Software visualisation refers to the use of computer graphics, together with modern human-
computer interaction technology, to facilitate human understanding and effective use of
computer software (Price et al. 1993; Stasko and Patterson 1992). The term is broader than
the definition of program visualisation (Baecker and Marcus 1990) which focuses on the
visualisation of computer programs. In addition to the visualisation of the programs
themselves, the term software visualisation refers to software composed of multiple programs
and includes aspects of algorithm visualisation (or animation), code visualisation, and data
visualisation1 (Price et al. 1993). Many examples of software visualisation approaches can be
found in the literature, particularly in the area of parallel processing (Zabala and Taylor 1993).

In an attempt to better classify and understand software visualisation approaches and tools,
Price et al. (1993) provide a principled taxonomy of software visualisation. They define the six
major dimensions of their classification scheme as:

1. Scope: describes the range of information (predominantly source code) that can be handled
by the visualisation system. This dimension includes aspects of generality (eg specific
languages accommodated, target hardware, or operating system dependencies) and
scalability.

1
Data visualisation in this context refers to the visualisation of data values related to the program code.
For example, the value of a variable at a particular point in time.
30
2. Content: refers to the subset of the information from Scope that the system actually uses
in constructing the visualisation. For example, a system may be restricted to providing
visualisations of control flow from the imported source code.

3. Form: describes the output of the system in terms of the medium used (eg workstation,
paper, film), presentation style, use of multiple views, and the use of features such as
elision.

4. Method: characterises the important elements of the visualisation specification. For

example, a user may need to specify attributes of a visualisation by use of menus or through
some specification language. This category also defines how the connection is made
between the entity being visualised and the visualisation system (eg software probe
insertion).

5. Interaction: refers to how the user interacts with, and controls, the visualisation system.

6. Effectiveness: provides a basis for recording information in terms of purpose,

appropriateness and clarity, the degree of empirical evaluation that has been undertaken,
and the extent of production use.

Price et al. (1993) use this taxonomy to study and categorise a range of visualisation systems.
They argue that one of the biggest impediments to the use of software visualisation is the issue
of scope. Based on their observations and studies, they conclude that most Software
Visualisation (SV) systems still deal primarily with toy programs and that many scalability
issues remain to be addressed. They suggest that the challenge is to implement and test SV
techniques on production-scale systems. As discussed in Section 2.2.3.3, there are also other
aspects of scope that need to be considered.

Another important dimension of the software visualisation taxonomy is evaluation. Price et al.
(1993) indicate that over 100 software visualisation prototypes have been built in the past 20
years, yet few have been evaluated to ascertain their effectiveness. Moreover, they suggest
that only a small number have seen any kind of production use. Perhaps this lack of evaluation
and validation also plays a part in why many of these techniques have not found their way into
CASE tools and environments. This is disappointing since visualisation techniques may
provide a basis for improving the flexibility required by many users. This lack of flexibility and
lack of customisability has been cited as one of the major impediments to effective CASE tool
adoption (Rader et al. 1995).

31
2.2.3.3 Interaction Issues

Human Computer Interaction (HCI) issues also need to be considered in relation to software
visualisation systems. This is particularly so if we are to take advantage of the flexibility of
visualisation approaches for the provision of customisable and adaptable descriptions. Many
novel approaches to user interaction have been explored, particularly in the area of direct
manipulation (Shneiderman 1983; Hutchins et al. 1986) and dynamic queries (Shneiderman
1994). Cognitive issues related to these HCI approaches are discussed in Section 2.2.4.4.

The ability to externally control a visualisation system is seen as an important feature of SV

systems. However, this aspect of interaction has not been used extensively to date (Price et al.
1993). Visualisation systems are typically exploratory in nature. They provide a high degree
of flexibility in terms of how the information can be provided. This can prove beneficial and
disadvantageous. For example, some tasks require information as a basis for comparisons.
For these types of tasks, the same representation techniques and level of detail should be used
to describe the particular characteristics of the entities being compared. As such, a visualisation
system, when used in its interactive mode, may not provide the consistent and stable
descriptions that are needed for these types of tasks.

This thesis argues that the required information can be provided by externally controlling the
environment through pre-defined scripts. This approach is also important from a productivity
point of view. For example, particular tasks may require a certain type of information. Scripts
could be developed to provide custom descriptions based on an analysis of task and
information needs. As will be discussed in Chapter 6, scripting can also be used to help enact
and support well-defined tasks such as software evaluations.

2.2.3.4 Extending the Scope of Software Visualisation

The software visualisation taxonomy discussed in Section 2.2.3.2 focuses primarily on the
code-related aspects of the software product. A key input to these types of systems is source
code. The taxonomy does not consider other types of information such as design,
requirements, or the range of attribute descriptions provided as metrics and measures. The
scope and content dimensions of the taxonomy need to be extended if software visualisation
systems, which can support the input of a broad range of SE information, are to be considered.
These types of systems are discussed in the following paragraphs and form the basis of the
approaches proposed in this thesis.

32
This thesis is particularly interested in the use of computer-based visualisations for describing
characteristics of the software products in terms of their structural relationships and key
attributes. Most software visualisation systems either provide descriptions of structural
relationships of the code implementation or dynamic behaviour. This thesis argues that there is
a need to provide software product visualisations based on a composite model which includes
design and code entities. These visualisations also need to be able to describe a range of
product, resource, and process attributes within the product context. Chapter 5 provides
details of a model-based integrated visualisation approach which supports these needs. The
SEE-Ada environment (Vernik and Burke 1993) has been developed and has evolved as part
of this research to explore these concepts. The following paragraphs discuss and compare
other related systems which allow attribute information to be integrated with the structural
representations of software systems.

The SeeSoft system (Eick et al. 1992) uses a reduced representation of source code files as a
basis for providing a range of code-related information such as code history, change details,
and defects. The underlying code representation is provided as a set of columns, each
representing a source code file written in the C language. Individual lines of code are
represented as thin lines in each column. The colour of each line is determined by a value
which describes an attribute of the code. For example, the attribute of interest could be the
date the line of code was created, when it was changed, or the number of changes that have
occurred. A direct manipulation interface allows the user to interact with the visualisation to
view a range of attributes in relation to the system being observed. A major limitation of
SeeSoft is that its underlying representation does not scale for large systems (Baker and Eick
1993). Moreover, it does not provide facilities for capturing design related information, or for
showing relationships between code entities as provided by the more conventional software
visualisation systems.

SeeSys (Baker and Eick 1993) extends the scope of the SeeSoft visualisations to include
subsystem and directory entities as well as source code files. The visualisation approach used
in SeeSys addresses some of the scalability problems of SeeSoft. The visualisation technique
used in SeeSys is based on the use of a hierarchy of encapsulated rectangles to represent the
system, the subsystems, directories, and source code files. The area of each rectangle is
proportional to a product attribute such as lines of code. SeeSys uses colour and fill to overlay
other numerical information. For example, the area of a rectangle might represent total lines of
code and these rectangles could be filled based on the proportion of new lines of code, number

33
of bugs, number of changes, or the number of programmers making modifications. The
authors report that the system has been used to analyse the source code database for AT&T's
5ESS local telephone switch. This software contains several million lines of code, is organised
hierarchically into approximately 50 subsystems, has several thousand directories, and
hundreds of thousands of files. The software has been written by thousands of programmers
over the past decade. Baker and Eick (1993) show that the system can help answer a range of
questions commonly asked by project managers such as: Which subsystems are the largest?
Where are the big files? Where is the current development activity? Which code is stable?
Which subsystems are unusually complex or error-prone? When were the major releases?
Which subsystems are growing at the fastest rates? Although they show examples which
illustrate the types of visual descriptions that can be provided, no evaluation or case study
results are available in the literature to show the effectiveness of the tool in practice.

Imagix 4D is a commercially available visualisation tool developed and marketed by Imagix

Corp. of Beaverton, Oregon. It provides three dimensional representations of software
systems written in the C or C++ languages. In addition to allowing users to view various
relationships between code entities (eg calls, inheritance), the tool can be used to view
information extracted from Unix 'tcov' and 'gprof' files. This allows coverage and profiling
information to be presented in context. The tool is an advance over SeeSoft and SeeSys in that
it can present both relationship and attribute information. However, the three dimensional
directed graph representation can become illegible even with moderate numbers of entities.
The lack of tailorability also causes problems. For example, the relationships shown in
particular views are fixed and this information is supplied even if it is not required by the user.
Another limitation of the tool concerns the viewing of attribute information. The structural
representations do not provide a compact and stable underlying representation of the product.
As such, it is difficult to gain appropriate insights into the impact of particular attributes on the
product. This is a strength of SeeSys which employs a compact spatial representation of the
software system onto which various attribute descriptions can be superimposed.

The SEE-Ada environment which has been developed as part of the research discussed in this
thesis (see Chapter 5 and Appendix C) implements these types of extended software
visualisation concepts. The visualisations that SEE-Ada provides are based on a consistent
underlying composite model of the software product. This Software Product Model comprises
a set of code entities as well as design entities. It includes the relationships between these
product entities as well as related attribute information. These attributes can be numeric (eg

34
measures, dates) or symbolic (textual descriptions such as labels, strings etc). The flexible
importing facilities of SEE-Ada allow information from a wide range of SE sources to be
accessed, filtered, and integrated into a composite model of the software product. This allows
various types of information (eg design, requirements, test, integration, product measures,
configuration, and effort) to be viewed in context within a consistent framework. An
important feature of the SEE-Ada environment is that it can also support the recording of
information and knowledge that resides with individuals and which may not be available from
other sources (eg the results of reviews, known problem areas of code, an important
relationship or feature). A set of integrated views and novel visualisations allow users to view
structural details of the design, the code implementation, and the relationship between design
and code. Other SE information can be overlayed on these views through the use of attribute
colour mappings, attribute lists, and the controlled integration of relationship information. The
environment is highly interactive and can also be controlled through a scripting feature.

2.2.3.5 Summary

Computer-based visualisation approaches provide a flexible means of providing information.

They have a number of important characteristics which support the customisation, tailoring,
and adaptation of information. However, there are a range of issues that need to be considered
if this form of information is to prove effective. For example, visualisations must be more than
just a set of pretty pictures. They must be meaningful. They must also be understandable and
useable by individuals engaged in particular tasks. For example, when utilising visualisation
approaches, users often find themselves asking: “what is this showing me?” As previously
discussed, visualisation approaches have proved useful for tasks that require some element of
exploration. However, there has been less emphasis on using this form of information in a
more descriptive mode. In this mode, customised descriptions need to be generated to
describe particular characteristics of interest.

Although many software visualisation approaches have been proposed, most have not found
their way into practice. This thesis argues that for this to occur, consideration needs to be
given to the context of use and to the processes that support effective provision, and use, of
this form of information, not just the techniques themselves. These issues are discussed in
Chapter 4.

35
2.2.4 Cognitive Science Issues

This section discusses a range of cognitive science issues related to the provision and use of
visual information. Cognitive science covers a broad range of issues related to human
intelligence. Cognition includes a range of human functions such as perception, problem
solving, reasoning, learning, and memory. In this thesis, cognition is discussed from a
computational perspective which stresses aspects of human information processing (Posner
1989). These issues are important for helping understand how people use information to
perform particular tasks and how information needs might best be accommodated. This
section is not meant to give a definitive account of this extensive area of work but to provide
the context for the work which follows.

2.2.4.1 Understanding and Defining Information Needs

The previous sections highlighted a range of problems and issues related to the provision and
use of information in Software Engineering. A major problem with much of the available
information is that it does not effectively meet the needs of individuals engaged in particular
tasks. Indeed, there is ample evidence to suggest that a large amount of the information that
is produced in Software Engineering is based on perceived needs. The question is: how do we
determine information needs? There is a significant amount of literature to suggest that
information needs can be determined through task analysis (Barnard 1987; Grant and Mayes
1991).

Grant and Mayes (1991) suggest that there is no established canonical method for task
analysis. They argue that the more complex the task, the more potential choice there is of
methods for performing and analysing the task. Methods of task analysis have been established
and successfully used in the study of clerical and manual tasks (Card et al. 1983). In these
tasks the information needs are generally easily defined through task analysis. However, as
Grant and Mayes (1991) suggest, much of the skill in more complex tasks (eg managerial
tasks) lies in information control and processing. Information needs for these types of tasks
are not easily specified. These are precisely the types of tasks that are of interest in this thesis.
As such, although we might be able to specify information needs for some tasks, there are
many tasks where information needs cannot be adequately specified. This raises a number of
issues for those attempting to provide information which is specifically tailored to particular
users and tasks (eg the knowledge-based documentation approaches discussed in Section
2.2.2.1.)

36
2.2.4.2 Models of Cognition

Models of cognition can be useful for reasoning about and understanding human information
processing issues and for supporting activities such as task analysis. Several architectural
models have been proposed to support the overall theory of cognition which includes memory,
symbols, operations, interpretation, and interaction with the external world (Newell et al.
1989). Examples of these types of models include Act* (Anderson 1983), Soar (Laird et al.
1983), and the Model Human Processor (Card et al. 1983). These models address different
perspectives of human cognition. Both Soar and Act* include the notion of production
memory which represents procedural knowledge in the form of condition-action pairs. These
models are in part designed to emulate human learning and are useful for considering the
interactions between memory and cognitive processes. The MPC is a model based on
memories and processors. Parameters are provided for various human characteristics such as
memory storage and decay time. This model is more suitable for considering simple, short-
term tasks (eg to determine how quickly things can be done such as pushing buttons), rather
than the problem-solving and decision-making tasks which are of prime concern in this thesis.

Problem
Solving
Process

Hypothesis Context High

Report Buffer Level Plan

Visual VISUAL Long

Context INTERACTION Term
PROCESS Memory

Attention Perceptual Percepts

Plan Buffer

Perceptual
Process

Figure 2-1 Cognitive Model of Visual Interaction

This thesis is particularly interested in how people use visual information to support their
cognitive processes. A model proposed by Rogers (1992) specifically addresses the use of
visual information in human cognition. This model of cognitive visual interaction (shown in
Figure 2-1) forms the basis for many of the discussions in this thesis. The model helps describe
how the perceptual and problem-solving processes interact. In particular, it addresses issues
related to how the perceptual process delivers information to the problem-solving process and

37
how the problem-solving process communicates directions to the perceptual process. The
model introduces the concept of a Visual Interaction Process which provides hypothesis
management and attention direction. The working memory structures that support these
activities are described in terms of two conceptual buffers and a visual context store. The
internal representations of this working memory must accommodate knowledge from the
perceptual and problem-solving points of view. For example, on the perceptual side, visual
information in the form of percepts is provided to describe findings in the visualisation. The
Context Buffer on the problem-solving side contains decision-related knowledge based on the
current state of the problem-solving process (eg which hypotheses are active and a high level
plan which identifies information needs).

This model can be used to help discuss and understand how visual information may support
individuals engaged in visual reasoning tasks. Moreover, it can help support the development
of systems that assist and enhance cognitive activities of humans involved in these types of
tasks. For example, the model has been used to support the development of a co-operative
system which aids radiologists in their analyses of chest X-ray images. The Visual Interaction
Assistant for Radiology (VIA-RAD) (Rogers 1995) employs a knowledge-based approach to
support the Visual Interaction Process. Rogers argues that humans are much better than
computers at perceiving information in images and in using this information to achieve
solutions. Instead of trying to reproduce these human capabilities, as is done in many expert
systems, the goal of VIA systems is to support users by providing contextual information, to
support hypothesis management, and to facilitate the use of the visual information by providing
attention direction. Attention direction facilitates the use of the images by highlighting areas
that need to be looked at and through image enhancement.

This work raises a number of issues that need to be considered in terms of the provision and
use of information in SE. As discussed in 2.2.4.1, there are many SE tasks where information
needs would be difficult, if not impossible, to specify. This has major implications for those
attempting to provide information that is customised to user's needs (eg using expert system
such as I-Doc (Johnson 1994)). An alternative might be to provide the information in a
flexible visual form (eg using computer-based visualisations) which exploits the power of the
human perceptual process. Support could then be provided to the human Visual Interaction
Process by allowing appropriate adaptation of the information, highlighting major points of
interest, and by providing appropriate contextual information. In the VIA-RAD system the
images are based on a fixed representation of the entity being described (ie chest X-Ray

38
images). By contrast, the description of software products allows greater freedom in terms of
the representation techniques that can be used. This gives greater flexibility in how the
approach can be applied.

2.2.4.3 Cognitive Issues Related to Representation.

The choice of representation can have a major impact on how effectively information is used.
For example, is it more efficient to represent a set of numbers using a graph or would a table of
values be more suitable? What are the benefits of spatial representations? Lohse et al. (1994)
begin to answer some of these questions by examining the cognitive structure of graphics and
in so doing provide a structural classification of visual representations. Through an
understanding of the taxonomic relationships among a broad range of graphics, they aim to
help designers decide how to represent various kinds of information and to understand the
limitations of different representations for conveying particular types of information. Their
work is based on a study of 60 items of visual information. They identify 11 main visual
representation categories which include graphs, tables, graphical tables, time charts, networks,
structure diagrams, process diagrams, maps, cartograms, icons, and pictures. These were
rated in terms of the type of knowledge conveyed (eg spatial, temporal, numeric, dynamic etc)
and other characteristics related to use (eg ease of use, the amount of information that can be
conveyed). The results of this type of work need to be considered when defining approaches
for providing SE information. For example, a key consideration is the notion of cognitive
efficiency which ensues when perceptual inferences replace arduous cognitive comparisons and
computations (Lohse et al. 1994).

Humans have a highly developed visual system and as such can grasp the content of a picture
much faster than they can scan and understand text. They can quickly recognise spatial
configurations of elements in a picture and notice relationships among elements. Designers of
systems that provide visual information to people can capitalise on this by shifting some of the
cognitive load of information retrieval to the perceptual system. Shneiderman (1994) argues
that “by appropriately coding properties by size, position, shape, and colour, we can greatly
reduce the need for explicit selection, sorting, and scanning operations.” The work by Lohse
et al. (1994) suggests that spatial representations can be more effective at providing
information than non-spatial representations such as network charts. They indicate that this
may be due to the way in which expert users 'chunk' symbols and suggest that there is a need
to further investigate how chunking can facilitate the use of spatial information. Although
some work has been done in addressing these types of cognitive issues in relation to the

39
understanding of source code (Curtis et al. 1984), there appears to be little research that has
been undertaken to address these issues for higher-level software representations. Research
being undertaken in areas of memory representation (eg through the use of mental models
(Ackermann and Tauber 1990; Johnson-Laird 1989)) may provide a basis for addressing these
issues.

2.2.4.4 Building Cognitive Bridges

In addition to considering aspects of human information processing, we need to consider the

process of interaction between a user and the source of information. This thesis focuses on the
use of computer-based visualisations as the information source. This section draws together
cognitive issues, issues related to human computer interfacing, and the use of computer-based
visualisation approaches.

Norman (1986) argues that there are seven stages of user activity: (1) establishing the goal, (2)
forming the intention, (3) specifying the action sequence, (4) executing the action, (5)
perceiving the system state, (6) interpreting the state, and (7) evaluating the system state with
respect to the goals and intentions. He proposes a model that highlights two "gulfs" that need
to be bridged between the physical and cognitive domains. These are gulfs between a user's
goals and knowledge and the level of description provided by the system (Hutchins et al.
1986). The user of the system starts with goals that need to be addressed. The system to be
manipulated is in the physical domain and is represented by a range of variables which indicate
its current state.

The first gulf to be bridged between the user and system is the Gulf of Execution. To do this
the user defines an intention based on a particular goal. For example, the goal might be to
improve the quality of the software. The first intention might be to identify poor coding
practices. The user then specifies an action sequence. This is a cognitive process which
determines the psychological representation of the actions that are to be executed by the user
on the mechanisms of the system. The user then performs the actions by interacting with the
system (eg to provide the desired descriptions).

The Gulf of Evaluation is the gap from system to user. This gap is bridged by the system
providing required information which is then perceived by the user (eg through perceptual
processing), interpreted, then evaluated with respect to the goals and intentions.

40
This is a useful model in that it considers the cognitive and physical activity that a user
undertakes in interacting with systems to perform particular tasks. It helps identify the
processes that are involved. However, not all processes would necessarily be invoked for all
tasks and tasks need not necessarily follow the exact sequence from goal to evaluation.

In providing effective computer-based descriptions, the aim is to bring the system closer to the
user by supporting the processes that bridge the gaps. For example, the system could support
the cognitive processes involved in identifying the action sequence by providing a
representative action sequence based on a prior analysis of task-based information needs. The
automation of these action sequences (eg through the use of scripts) could then reduce the
amount of physical interaction required by the user. The Gulf of Evaluation can be reduced by
providing information that can be easily understood by the user, is meaningful, and
communicative. Other support can be provided to help highlight key aspects of interest in the
description. This thesis argues that the flexibility of computer-based visualisation techniques,
the use of scripting facilities, and the use of direct manipulation interfaces can help bridge the
gap between the user domain and the information source.

The term "direct manipulation" was first coined by Shneiderman (1983) to describe interfaces
which provide a continuous representation of objects of interest, where users interact directly
with the system rather than by way of complex syntax (eg command languages), and where the
impact of operations is immediately visible to the user. This type of interface allows users to
see immediately if their actions are furthering their goals, and if not, they can quickly change
the direction of their activity. Interfaces of this type are becoming widespread and form the
basis of many computer-based visualisation approaches. They form the basis of the approaches
proposed in this thesis.

Hutchins et al. (1986) provide a cognitive account of direct manipulation interfaces in terms of
Norman's model. They suggest that the "feeling of directness results from the commitment of
fewer cognitive resources". This is due to a reduction in "distance" between the user's goals
(ie in the cognitive domain) and the way in which they are translated into physical actions.
Also, there is a notion of "direct engagement" where the user has the feeling of directly
manipulating the object of interest. This approach can reduce the cognitive effort in terms of
the Gulf of Evaluation, particularly if the information provided is based on a model world
metaphor (Hutchins et al. 1986). In these cases, the world of interest (eg software product

41
under evaluation) is explicitly represented and there is no intermediary between the user and
the world.

This thesis argues that the success of the direct manipulation approach is largely due to
adaptation by the user. It is conjectured that some tasks (eg more exploratory tasks) would
benefit from this approach. However, there are other tasks with more defined action
sequences that would benefit from a more automated (eg script-based) approach to interaction.
It is believed that a combination of these two approaches can provide the most appropriate
basis for building cognitive bridges between users and information sources.

2.3 Analysis, Perspectives and Conjectures

This section draws upon the preliminary studies and literature review to summarise and identify
a range of problems and issues to be addressed in this thesis. A set of initial effectiveness
criteria is proposed based on an analysis of the problems and difficulties discussed in the
previous sections. These criteria are used to support further discussion and provide a basis for
theory development. The final part of this section conjectures possible new approaches for
dealing with the problems and for addressing the issues.

2.3.1 Analysis of Problems and Issues

A major difficulty being faced by those involved in many SE tasks is that, due to the
conceptual nature of software products and processes, direct visibility of product and project
characteristics is not possible. This creates significant problems for individuals engaged in a
range of SE tasks, particularly those associated with the technical management and supporting
processes. Although direct visibility may not be achievable, this thesis argues that it is possible
to gain indirect visibility through description. This can be done by using information to
describe the aspects of interest. Descriptive information is needed to help describe particular
entities, attributes, and relationships of interest. This information must provide a meaningful
representation of the aspect being described. A key problem is that much of the available
information cannot be used effectively for this purpose.

The initial studies and literature review have identified a range of problems related to provision
and use of information in SE. These are summarised in Table 2-1. In addition to the problems
of description (P11), a range of other problems have been identified with the many sources of
information which are available to software projects. A major problem with most software
projects is that users are overwhelmed by the amount of persistent information available (P1).

42
This information is provided by traditional deliverable documents, informal developmental
documents, various metrics and measures, and by the information contained in the many tools
and environments that support the SE process. Users have problems in understanding what is
available at a particular point in time, let alone being able to access (P7) appropriate
information to support their needs. However, even if users knew what information was
available and how it could be accessed, it may not prove useable or useful for the intended
purpose.

There are a range of problems attributable to the information itself. For example, the use of
representation and presentation techniques can have a major impact on understanding (P2).
Also, problems related to the scalability of techniques can result in information that is illegible.
Problems in representation can also cause information to convey the wrong message (P5).
There are also problems with assimilating information of different types to gain a composite
description of some aspect of interest (P8). This chapter has discussed a range of other
problems such as redundancy (P4), inaccuracy (P6), and inconsistency (P10). These all have
an impact on the usefulness of information.

A major problem reported in the literature is the high cost of providing and maintaining
particular classes of information products (P7). There is also the cost of not doing important
tasks if needed information is not available. The cost of using information can also be a
problem. This includes the time that it takes a user to access required information, and the
cognitive effort that is needed for a user to assimilate and understand the information.

43
 ID PROBLEM

P1 User is overwhelmed by the quantity of information.

P2 Information cannot be understood due to inappropriate use of representation

and presentation techniques

P3 Information is not available at the appropriate time.

P4 Superfluous and redundant information is provided.

P5 Information is misleading. It does not communicate the appropriate message.

P6 Information is inaccurate.

P7 Information is costly to provide, use, and maintain.

P8 Information is difficult to access.

P9 Information from more than one source is difficult to assimilate.

P10 Information is not consistent over time or between representations.

P11 Information does not effectively describe the aspects of interest.

Table 2-1 Summary of Problems Related to the Use of Descriptive Information

A key question is : How do we most effectively provide information to individuals to support

them in their respective tasks? Analysis of the results of the initial studies and the literature
review has helped identify several key issues that need to be considered if we are to begin to
address this question and the problems currently being experienced. These issues are
summarised in Table 2-2. and discussed in the following paragraphs.

ID ISSUE

I1 Understanding and supporting user information needs.

I2 Availability of, and access to, information.

I3 Assimilation and integration of information.

I4 Use of appropriate representation and presentation techniques.

I5 Customisation and Adaptation of Information.

I6 Cost of provision and use.

Table 2-2 Summary of Key Issues to be Addressed

44
I1. Understanding and Supporting User Information Needs. In addressing this issue
cognitive issues as well as technical and process issues must be considered. The main
objective is to provide information which most effectively meets user needs. To do this,
we need to determine what information is needed to support individuals engaged in
particular SE tasks. How do we specify and support these needs? For example, can these
needs be specified in terms of what needs to be described? How should this information be
represented and presented with respect to the context of use (eg user's level of knowledge,
experience etc)?

I2. Availability and Access to Information. Software projects produce and use vast
amounts of information. Information is produced by metrics and documentation processes
and by the array of CASE tools that are used for project support. There are several
questions that need to be addressed in terms of the availability and accessibility of
information. For example, how do we determine what information is available to fulfil
particular needs? Where can we get the required information? What are the available
sources (tools, documents, people etc)? If there is more than one source which can
provide the required information, which should we choose? Is there information available
which will fit user needs? Is it directly useable or does it need to be customised? How
well does it meet the user’s needs and will it require significant cognitive effort to use?
Can we rely on its accuracy? How difficult is it to access (ie how much time and effort will
be required)? If the required information isn’t available, can it be readily generated from
existing information?

I3. Assimilation and Integration of Information. Users often need to assimilate information
to support complex information needs. Some tasks may require a user to assimilate
information from a range of sources and of different forms (eg diagrams, tables of values).
For example, to support the evaluation of software maintainabilty, a user might need to
assimilate test information, architecture diagrams, and configuration information. Some
tasks may need the core information to be provided with contextual information to support
the effective use of the information. For example, metric information often needs to be
used together with other information to support particular tasks. The assimilation of
information of different forms can require significant cognitive effort. Approaches are
needed to help reduce this cognitive load by integrating information and providing it in an
appropriate composite form. When integrating information, issues of consistency must be
considered.

45
I4. Use of Appropriate Representation and Presentation Techniques. When selecting
representation and presentation techniques, the cognitive as well as technical aspects need
to be considered. Issues related to both representation and presentation need to be
considered. Presentation refers to the form, composition, and media used for the
information (eg paper, multimedia, animation, computer-based graphical images).
Representation refers to the mappings between the entity of interest and the techniques
used to portray characteristics of this entity. Representation must be valid (ie meaningful
to user), must be understandable, and communicative (ie must communicate the
appropriate message). The combination of representation and presentation techniques
needs to be considered in relation to criteria such as legibility and scalability.

I5. Customisation and Adaptation of Information. Software engineering information is

often produced to support a wide variety of needs. However, the generally available
information is often not provided in a way that best suits individual information needs. The
overall level of effectiveness of this information may be low since the representation
techniques used may not be understood by a particular user, the required information may
be obscured by superfluous information, and the user may not be able to assimilate the
information with other supporting information. Hence, it is desirable to consider how
information can be customised for particular users. This may be achievable if we can
accurately define the information needs for specific tasks and users. As shown in Figure 2-
2, customisation allows information to be tuned to a user needs. However, if these needs
cannot be accurately specified, the resulting custom descriptions may be less effective than
the more general descriptions (see Point A). This thesis argues that dynamic, adaptive
systems which can adapt custom descriptions to individual information needs are required.
These systems could harmonise and tune information to users needs (Point B).

I6. Cost of Provision and Use. The cost of providing information is seen as a major issue for
Software Engineering. In addition to the cost of providing information, the cost of using
information must also be considered. For example, this includes costs associated with
accessing required information from a particular source and the cognitive effort required by
user to extract and assimilate needed information from provided information. The costs of
not having effective information available must be considered (ie cost of not doing tasks).

46
Effectiveness LEGEND

B General Description
Customised Description
Adapted Description

Specific Information
Need Needs

Figure 2-2 Adapting Information to User Needs

It would be a mistake to believe that these problems and issues could be addressed by simply
developing some new and novel technology. This thesis argues that we must not only consider
the means of providing information, but the processes involved in provision and use of
information. These processes include those that must interface with, and support, the SE
processes, as well as the cognitive processes of the individuals using the information.

2.3.2 Effectiveness Criteria

An analysis of the preceding discussions suggests several criteria that need to be considered in
relation to the effectiveness of descriptive information. An initial set of effectiveness criteria is
provided in Appendix B. These criteria are used in this thesis to aid discussion and
communication. Definition of an initial set of criteria is also seen as important for providing a
basis for theory development and evaluation. The criteria of Appendix B are a preliminary
attempt at defining effectiveness. They should not be seen as the definitive or complete set,
but provide a starting point from which a more complete set of criteria can emerge.

Some of the criteria refer to cognitive aspects of information use such as understandability,
meaningfulness, and communicability. Others refer to important underlying characteristics of
the description products (eg completeness, conciseness, accuracy, and precision). The third
set deals with criteria that need to be considered for effective provision of descriptive products.
These include availability, accessibility, compositionality, tailorability, customisability,
scalability, and cost.

47
The criteria are not necessarily orthogonal. There is a need to consider trade-offs between the
criteria when addressing overall effectiveness of description products and processes. For
example, to enhance understanding there may be a need to present additional information to
supplement a particular user's level of knowledge or to provide information in context. This
then reduces the conciseness of description.

2.3.3 Conjectures

This chapter has highlighted a range of problems and issues related to the provision and use of
information in Software Engineering. To address these problems and issues, this thesis
presents and discusses the following conjectures and approaches:

C1. The effective provision and use of descriptive information plays an important role in
Software Engineering. This thesis argues that many important SE tasks cannot be
performed adequately (or are neglected) due to ineffective provision and use of
descriptive information. Descriptive information is human useable information that can be
used to describe product and project characteristics and hence provide a basis for indirect
visibility.

C2. Integrated visualisation approaches can address many of the problems and issues
identified in this chapter and hence provide a more effective basis for the provision
and use of descriptive information. This thesis proposes a model-based approach
which uses computer-based visualisation techniques for providing descriptive information
which can be readily integrated, customised, and adapted to individual user needs. This
approach takes a product-oriented perspective, based on the concept of a Software
Product Model, to provide various types of SE information in context and in a consistent
manner.

C3. A conceptual process-based framework can be established and used to help

understand how descriptive information is generated, provided, and used for
various SE tasks. A process-based framework is proposed which includes description
processes, products, and related models (eg cognitive models). The framework is based
on a contextual model which considers the SE processes and the project information
resources which can be used to support task-based information needs. This thesis argues
that this type of framework can help support the introduction and evaluation of new
approaches into SE practice.

48
C4. Practical tools can be built, applied and evaluated using these approaches to better
understand the problems and issues related to the provision and use of information
in Software Engineering. The concept of an Integrated Visualisation Environment for
Software (IVES) is proposed as a way of implementing the integrated visualisation
approach and for addressing many of the problems and issues raised in Tables 2-1 and 2-2.
The IVES concept provides a means of accessing and filtering information from a variety
of SE sources, performing model-based integration of core information, and providing
Integrated Visual Descriptions which support both representational and presentational
integration. This approach supports the process-based framework by providing scripting
facilities which facilitate the automatic preparation of customised descriptions. The SEE-
Ada Version 3 environment implements many of these concepts and is used as a major
apparatus for testing this conjecture.

49
 3. RESEARCH APPROACH

This chapter provides an overview of the research approach that has been used to explore the
problems, issues, and conjectures that form the basis of this thesis. It provides the context
within which the results of this research are discussed. The chapter begins by summarising the
research objectives. It then provides perspectives on Software Engineering research.
Software Engineering is a relatively new and evolving discipline. As such, there has been
considerable discussion within the SE research community as to the problems and difficulties
of this type of research, the need for a scientific approach, and experimental methods. The
approach which has been adopted for this research is presented in terms of this discussion.

Original Content and Contributions. The research approach (including the research
strategy and research process) which has been adopted for the research discussed in this thesis
is the original work of the author.

3.1 Research Objectives

The primary objectives of this research are to:

1. Investigate the issues and problems related to the provision and use of descriptive
information in SE. In particular, this research focuses on the information needs of those
individuals engaged in tasks that require a high-level view of SE products and processes.

2. Define theory and approaches for more effectively providing and using descriptive
information for these types of tasks.

3. Critically evaluate and test the theory and approaches through appropriate experimentation.

4. Systematically enhance concepts, methods, and understanding.

3.2 Software Engineering Research

Software Engineering research has often been criticised for its lack of scientific basis and
inappropriate experimentation (Basili 1992; Browne and Shaw 1981; Curtis 1980). A major
criticism is that it has tended to focus more on definition and generation of new technologies

50
than on the testing and evaluative aspects of research (Card et al. 1987; Fenton et al. 1994;
Votta and Porter 1995). However, as Fenton et al. (1994) suggest, experimentation in SE can
be difficult. Many of these difficulties arise because valid results are often only available if
research is conducted in real situations and based on real artefacts. This section discusses the
problems and difficulties faced by those undertaking research in this area. It addresses the
need for using a scientific approach, and considers issues related to experimentation in SE.

3.2.1 Difficulties and Problems

SE research covers a broad range of areas. It typically aims to understand and improve the
practice of acquiring, developing and maintaining software products. Offen and Xia (1995)
suggest that this type of research can be intrinsically difficult due to the abstract nature of
software products and processes and because of the "human/social-centredness of the software
development process". They suggest that a combined socio-technical viewpoint must be
pursued since many of the conventional approaches found in the physical and social sciences
are not necessarily useful in the SE domain. This creates problems for those undertaking SE
research, since there are as yet few well established guidelines and approaches for conducting
research in this domain.

It has been argued that SE research should focus on 'real' projects and artefacts (Basili 1992;
Fenton et al. 1994). However, this can create major difficulties for researchers. Proprietary
considerations often make it difficult for researchers to gain access to 'real' situations. The
scale and complexity of these situations is such that considerable domain knowledge is needed
to properly design experiments that will effectively test the conjectures that form the basis of
the research. Moreover, the experimental apparatus used for this type of experimentation
needs to be robust and well supported. This can prove expensive. As Fenton et al. (1994)
suggest, it is not surprising that little of this type of research is being done.

Project timescales can also be a problem. For example, the software development schedule for
the medium-scale project which is the focus of the Industry Case Study discussed in Chapter 6,
covers a 24 month period. Other larger projects can take several years to complete. As such,
research results are not generally available for some time. These timescales are compounded if
replicated studies are to be undertaken. As discussed in this thesis, studies can be designed in
an incremental manner to provide for a timely release of results. However, the limitations of
these results must be considered when discussing the findings of the research.

51
3.2.2 A Scientific Approach

Fenton et al. (1994) argue that many of the published SE research findings can be classified as
“analytical advocacy research”. They suggest that this approach involves describing a new
concept in considerable detail, analytically deriving its potential benefits, and then
recommending its transfer to practice. They suggest that the results are subjective, non-
repeatable, and not believable. Several authors have suggested that SE research needs to
follow a more scientific approach (eg see Basili et al. (1986b), Browne and Shaw (1981), Card
et al. (1987) and Curtis (1980)). But what constitutes a scientific approach? Fenton et al.
(1994) argue that such an approach involves the formulation of an idea and its related
hypothesis. This is followed by evaluative research which investigates whether the hypothesis
is true or false. However, as Popper (1977) suggests, the scientific approach supports the
growth of scientific knowledge. Rather than trying to capture an absolute truth, Popper
suggests that the growth of scientific knowledge is an evolutionary problem-solving process
where:

1. New conjectures are proposed in relation to problems or limitations of previous theories.

2. Propositions deduced from the conjectures are tested either experimentally or by

observation of the world.

3. If the tests are unsuccessful the conjecture is refuted. However, if the tests are successful,
the conjecture can be tentatively accepted. The conjecture is not necessarily verified, it is
merely not falsified.

The research approach used for the work of this thesis is based on Popper's philosophy. The
complexity and nature of the subject area suggest that no absolute conclusions can be reached.
However, with appropriately designed experimentation, the work can support the growth of
scientific knowledge in this area of Software Engineering.

Glass (1994) proposes a model which embodies the scientific approach. He suggests that a
scientific approach begins with an Informational Phase. This phase involves gathering
aggregate information by "observing the world". This is followed by a Propositional Phase
where the conjectures are proposed. The Analytical Phase then analyses and explores the
proposal, "leading to a demonstration and/or the formulation of a principle or theory". The
fourth phase is the Evaluative Phase which evaluates the conjectures by means of
experimentation, "perhaps leading to a substantiated model, principle, or theory". Glass argues
that much of the published SE research fails to cover these four phases of research. The lack
52
of an informational phase is a major limitation of many research approaches. Votta and Porter
(1995) also highlight this problem. They suggest that "there are few studies reporting what
people actually do when they do software development; rather most papers dwell on what their
authors think people should be doing". Many 'so-called' solutions are proposed in SE without
a clear knowledge of existing methods or the 'real' problems that are being experienced in
practice. The lack of appropriate evaluation or testing of conjectures is also seen as a problem.
These issues are discussed below in Section 3.2.3.

Chapter 7 discusses the research that forms the basis of this thesis in terms of Glass's research
model. This thesis argues that this type of discussion can help define the main characteristics
of the research which has been conducted, and can highlight limitations in terms of its scientific
approach. These limitations need to be considered in relation to the findings of the research.

3.2.3 Experimentation in Software Engineering

Experimentation is an important aspect of research. It is seen as the key element of a scientific

approach (Preece 1994). But what is experimentation? The Concise Oxford Dictionary
(Sykes 1976) defines experimentation in terms of "the procedures adopted or operations
carried out to make discovery, observation, test". As this definition suggests, experimentation
can be conducted in a variety of ways for a variety of reasons. The small amount of literature
available which discusses experimentation in SE, tends to focus on the evaluation of tools and
methods to answer the 'which is better' questions (Kitchenham and Pickard 1995). Basili et al.
(1986b) suggest that experimentation is performed for a wide variety of purposes. It can be
used to predict, understand, control, and improve the software development process and
products. Basili (1992) suggests that the experimental paradigm is important for Software
Engineering. He argues that industry-based laboratories are required to aid the understanding
of the various processes, products, and related experiences and to support the building of
descriptive models to help understand the problems associated with building software. The
research discussed in this thesis focuses on developing and evaluating descriptive models for
the provision and use of information in SE.

Various types of experimentation can be undertaken in SE. The DESMET Project (Law
1992), which was sponsored by the United Kingdom's Department of Trade and Industry,
identifies three main types of experimentation for SE. These include the use of formal
experiments, case studies, and surveys. Based on this work, Kitchenham et al. (1994) provide
guidelines for selecting particular types of experimentation based on their potential benefits and

53
limitations. They suggest that formal experiments are designed to minimise the effects of
extraneous factors. This is done by blocking (eg through the use of experimental controls),
replication, and randomisation. Formal experiments divorce the phenomenon of interest from
its context (ie the context is typically controlled by the laboratory environment). Attention can
then be focused on a few variables. Several authors have suggested that the types of formal
experimental approaches used in the physical sciences may not be appropriate for many areas
of SE research (Offen and Xia 1995; Votta and Porter 1995). This is in part because
laboratory experiments do not take into account the many 'real-life' variables and causal effects
(Basili 1992). Votta and Porter (1995) suggest that a hybrid approach should be considered
which draws from the formal experimental approaches used in the physical sciences, together
with methods used in the social sciences. Kitchenham et al (1995) provide guidelines for
performing case studies in SE which go part way towards providing this type of approach.

Case studies are a form of experimentation which investigates a contemporary phenomenon (ie
as it is occurring) within its real-life context, where the boundaries between phenomenon and
context are not clearly evident, and in which multiple sources of evidence are used (Yin 1984).
Case studies can include, and even be limited to, quantitative evidence. They can be used to
help explain the causal links in real-live interventions that are too complex for surveys or
experimental strategies, describe the real-life context in which an intervention has occurred,
evaluate the benefit or otherwise (in a descriptive mode), and explore those situations in which
the intervention being evaluated has no clear single set of outcomes. This type of
experimentation is often criticised for having insufficient precision, objectivity, rigour; and an
insufficient basis for scientific generalisation (Yin 1984). However, as Kitchenham et al (1995)
suggest, well designed case studies which include the key elements of the scientific approach
may provide a basis for more effective experimentation where researchers need to study a
phenomenon in real situations.

In response to some of the criticisms of SE research and experimental approaches, Fenton,

Pfleeger, and Glass ( Fenton et al. 1994) have suggested 5 questions that should be asked
regarding any claim arising from SE research:

1. Is it based on empirical evaluation and data?

2. Was the experiment designed correctly?

3. Is it based on a 'toy' or real situation?

54
4. Were the measurements used appropriate to the goals of the experiment?

5. Was the experiment run for a long enough time?

Chapter 7 discusses these questions in relation to the research approach which forms the basis
of this thesis. The research approach includes the research strategy and the research process.
These are discussed in Sections 3.3 and 3.4.

3.3 Research Strategy

This research focuses on SE practice. The reason for this focus is that the issues being
considered relate to problems that are being experienced by those involved in software
projects. These can be understood by observing and studying real situations. Moreover, the
concepts and approaches proposed in this thesis are aimed at providing improved support for
those engaged in a range of SE tasks. The conjectures that refer to these proposals must
therefore be evaluated within a real project context. The use of a case study approach has
been adopted as the most appropriate form of experimentation for this research. As discussed
in Chapter 6, this approach uses a goal-based method which helps identify the information to
be captured and supports the analysis and reporting of results.

The SEE-Ada environment plays a large part in the overall research strategy. This
environment embodies and supports many of the concepts proposed in this thesis. SEE-Ada is
used to test whether the various concepts and approaches are in fact implementable. The
implementation of a robust and supportable environment also provides the basis for a range of
trials which have been conducted at several receptor sites. It also serves as a key apparatus for
more formal case study experimentation.

A feature of the research strategy has been its use of extensive and diverse review to elicit
feedback on the proposals and results. DSTO's relationship with many universities and
research establishments, both in Australia and overseas, provides ready access to other SE
researchers and hence supports the review of concepts and ideas. Moreover, since there is
significant interaction between DSTO and industry, this academic review process has been
enhanced by feedback from industry personnel. In addition to these aspects of the review
process, elements of the work have been presented at conferences and workshops in the USA,
France, and Australia. This has further strengthened the review process.

55
3.4 Research Process
Figure 3-1 provides an overview of the research which has been conducted . It shows the key
research phases and main outcomes of these phases. The research process has been iterative in
nature. The lessons learned from each phase of the research have been used to further define
and refine the concepts and approaches.

As discussed in Chapter 2, the initial phase of the research involved a series of preliminary
studies which helped provide a general understanding of the problems and issues of
information use in SE. These studies suggested that better approaches were needed for
accessing and integrating various types of project information.

The second phase of research focused on approaches for integrating product metric
information with structural representations of source code. This work focused on supporting
analysis and evaluation tasks as performed by those involved with the Independent Verification
and Validation (IV&V) process and resulted in the initial SEE-Ada concepts. It also
highlighted the fact that process issues were important for the effective provision and use of
descriptive information. The need to provide better conceptual foundations for the work was
highlighted. Feedback and experiences also showed that the basic SEE-Ada concepts could be
extended into a more general integrated visualisation approach which could be used to support
a wide range of SE information needs.

To address these aspects, three parallel and interacting threads of research were conducted.
Since there was no known body of work which provided the conceptual foundations for this
area of research, a set of concepts, models, definitions and approaches were defined. These
are discussed in Chapter 4. The basic SEE-Ada concepts were extended into a model-based
integrated visualisation approach as discussed in Chapter 5. In parallel with these, the SEE-
Ada environment was redesigned and extended to include support for the conceptual
foundations (including several of the process issues) and the model-based visualisation
concepts. This approach allowed testing to assess whether the concepts were implementable
and provided early feedback on their validity.

56
 Preliminary
Studies

Problems
Issues

Investigations
into Integrated
Software
Visualisation
Initial SEE-Ada
Conjectures

Development
SEE-Ada Development
of Integrated
Evolution of Conceptual
Visualisation
Foundations
Approach
Concepts
Taxonomies
Models
Approaches

Case
Study
Design

Goals
Method
Experimental Setup

Preparation
of
Experimental
Apparatus

Industry
Case
Studies
Theory Enhancement
Improved Approaches
Problems

Figure 3-1 Overview of Research Phases and Results

Preparations then began for the Industry Case Studies. A case study design was prepared.
This identified the goals and questions which needed to be answered. The information
requirements, defined in terms of the questions, helped specify the experimental setup. These
requirements also impacted the evolving SEE-Ada implementation in that it needed to be

57
instrumented to support case study goals. Other experimental apparatus also needed to be
designed and developed.

A series of Industry Case Studies is to be performed to obtain experimental results which can
be used to test the conjectures which form the basis of this research. The first of these is being
conducted at AWA Defence Industries. It is focusing on the needs of technical managers (ie
software team leaders). The case study design and the results of the first stage of the study are
discussed in Chapter 6. Other studies will be performed to help replicate the results and test
the conjectures for other roles and tasks. The results will be used to help refine and extend the
conceptual foundations and the integrated visualisation approach.

58
 4. CONCEPTS AND MODELS

This chapter introduces basic theory (including concepts, terminology, and models) which
supports the definition and evaluation of the approaches proposed in this thesis. The chapter
begins by defining a Contextual Model. This model includes those elements of a software
project that are of prime interest in this thesis (ie the SE processes and tasks, and SE
information). The Contextual Model provides the setting in which the problems and issues
discussed in Chapter 2 can be considered and addressed. A key component of the Contextual
Model is the Project Information Resource (PIR). This is a conceptual entity representing the
totality of persistent information available to a project. An approach is provided for modelling
and classifying the Information Products that form part of the PIR.

The concept of a Description Process is proposed as a means of more effectively meeting the
information needs of individuals engaged in specific SE tasks. This process addresses issues
related to the provision of descriptive information (eg access, filtering, integration,
representation, and presentation) as well as issues related to the human use of information.
The Description Process includes other supporting processes which facilitate the preparation of
custom descriptions and support the adaptation of descriptions to user's needs. The
Contextual Model, together with the description processes and products, combine into a
process-based framework which can be used to support research in this area and to help
transition the approaches proposed in this thesis into practice.

The final part of the chapter defines model-based concepts for the provision and use of
descriptive information. These concepts form the basis of a Product-Oriented Description
approach which is based on the use of a Software Product Model. Descriptions of this type
provide a means of integrating various types of project information and providing this
information from a product point of view.

Original Content and Contributions. All concepts, models, and approaches defined in this
chapter are the original work of the author. These include the Contextual Model and related
classification and modelling approaches, the description concepts, the Description Process, the
Description in Software Engineering Process Framework (DSEPF), and the model-based
description concepts. The model-based description approach is based on the formal systems
and modelling concepts defined by Kaposi (Kaposi and Pyle 1993, Kaposi and Meyers 1994).

59
4.1 Establishing the Context
This section introduces high-level concepts and models which help provide the context for the
concepts, models and discussions which follow. This work draws on and extends standard
definitions (eg ISO/IEC_12207-1995) to provide the basis for a conceptual framework. This
framework is used to explore the issues defined in Chapter 2 and to support a range of new
concepts which are proposed as a means of addressing these issues.

4.1.1 Contextual Model

Models are manageable simplifications of systems (real or conceptual) which allow us to focus
our attention on the key aspects of interest without being distracted by peripheral detail
(Kaposi and Meyers 1994). Figure 4.1 provides a Contextual Model which includes those
aspects of a software project which are of primary interest to the work discussed in this thesis.
These are the Software Engineering Process, the Project Information Resource, and SE
information. This model provides the context for considering the problems and issues
discussed in Chapter 2 and for defining the concepts and approaches proposed as a means of
addressing these problems and issues.

V&V
Quality
Improvement
Development Doc

Config Mgt
Review
Technical SE Process
Management

LEGEND
Project Information Resource Lifecycle Process
SE Information

Process Interactions

Figure 4-1 Contextual Model

4.1.2 Software Engineering Process

The concept of a Software Engineering Process is discussed in Appendix A. This process

includes those standard software lifecycle processes (ISO/IEC_12207-1995) that are required

60
to generate and deliver software products as specified by the customer’s requirement. The SE
Process comprises a system of interacting lifecycle processes including technical management,
development, and supporting processes such as configuration management, quality assurance,
review, and problem resolution. The focal point of the SE Process is the development process.
In this thesis, development can refer to initial product development, or to maintenance (ie
product evolution). Each of these processes can include encapsulated subprocesses. For
example, the development process may include analysis, design, and coding processes while
the technical management process might include planning, control, review, and technical
support processes.

The lifecycle processes provide an overview of the types of SE functions that are performed as
part of a software project. Individuals undertake particular tasks within this framework. Tasks
are defined as the fundamental work units (or lowest level processes) that are performed by
individuals and are subject to management accountability (IEEE_Std_1074-1991). For
example, the technical management process might include tasks that monitor product status to
identify potential schedule risks, inspection tasks that identify quality problems, and analysis
tasks that support problem resolution. This thesis contends that the information needs of
individuals engaged in various SE processes need to be considered in relation to the tasks
being performed. As such, the definition of SE tasks is seen as an important contextual
consideration. The Industry Case Study discussed in Chapter 6 has identified a range of tasks
associated with the technical management process.

4.1.3 Software Engineering Information

Information is a broad concept. The term has generally been associated with knowledge or
intelligence of some type and relates to the way in which data is used (Yovits 1993). The
terms data and information are often used interchangeably, even amongst professionals in the
computing field. In this thesis, the term data will be used to describe the basic facts that are
captured and stored (generally in electronic form). The term information will be used in a
broad sense to describe data which is provided in a defined context thus allowing it to be
interpreted and used, given appropriate knowledge. No distinction is made as to whether
information is human or machine useable. There are many cases where information serves both
sets of users.

The term Information Product (IP) is used in this thesis to describe an identifiable piece of
information. Information Products can be defined in terms of the type of information provided,

61
the representation techniques used, the form that the information takes, and the provider and
users of the information. These products are also characterised in terms of their availability.
Persistent products are those that are available for use over a period of time (eg a requirements
specification). Transient products are those that are produced to serve a specific need at a
particular point in time (eg a computer-based visualisation). Section 4.2 provides a detailed
discussion of Information Products. An approach for modelling and classifying Information
Products is also provided.

Information has a variety of uses and purposes in SE. For example, information such as
compilation policy, source code, and configuration status is used by compilation systems to
generate other types of information (eg object code and executables). Testing tools require
information to perform regression testing on parts of the software that have undergone change.
Some information is used as a basis of control (eg instructions to tools, directions to other
personnel).

Information is often used to describe things for individuals engaged in specific SE tasks.
Information of this type can include: design documents which, in part, describe product
architectures; measures which describe product, process, and, resource attributes; and test
results which can describe the degree of testing. This thesis is primarily concerned with this
type of information (ie descriptive information, or descriptions in short). This concept
discussed more fully in Section 4.3.1.

4.1.4 Project Information Resource

In this work, the Project Information Resource (PIR) is defined as the source of persistent
information for a project. It is a conceptual entity comprising a system of Information
Processors and persistent Information Products. Information Processors are those entities that
transform, manage, and provide Information Products. They typically comprise the set of tools
and environments used by the project. Since SE relies heavily on the individual (Humphrey
1995), the people involved with the project must also be considered as a major source of
information. In this case, the persistent information is based on the knowledge that the person
has about the product and project.

The PIR is a key resource for projects and provides a basis for determining how and when
various classes of project information are produced. It also provides a basis for understanding

62
how this persistent project information is used to support SE tasks. Issues pertaining to the
efficient and effective use of information available from the PIR plays a major part in this work.

4.1.5 Information Flows

The SE process produces and uses vast amounts of information. This thesis argues that the
dynamics of how and when information is produced and used is an important aspect of
Software Engineering that needs to be studied.

As Kraut and Streeter (1995) point out, a large proportion of the information generated and
used in Software Engineering results from informal communications, particularly amongst
individuals. This is typically transient information which is used as the basis for exchanging
knowledge and experiences, and for coordinating activities. These informal information flows
form part of the interactions between the SE processes as shown in Figure 4-1. This type of
information has been studied by various researchers (Walz, Elam and Curtis 1993; Saeki
1995). Transient control information also plays a large part in the inter-process interactions
and can be used to provide directives to tools or individuals.

The work discussed in this thesis is concerned primarily with the provision and use of
persistent information products that can be used as a basis for description. As discussed, the
PIR represents the source of persistent information for a project. In an ideal situation, all
project information would be available from a central repository (eg as provided by an
Integrated Project Support Environment (IPSE)). However, as discussed in Chapter 2, the
reality is that the PIR for most projects comprises a variety of poorly integrated Information
Processors (eg tools and environments) and a host of Information Products. These
Information Products can be provided as individual artefacts including deliverable documents,
informal project documentation (eg Software Development Files), and various reports (eg test
reports, metrics reports). They can also represent the underlying information stored by the
tools and environments. These Information Products are produced during the execution of the
various SE processes and can also be provided by a range of external sources (eg customer
requirements). The information is available directly from individual Information Products, or
by way of an Information Processor which provides access to underlying Information
Products.

Individuals undertaking tasks associated with the various SE processes require access to this
information. As discussed in Chapter 2, major problems can be experienced in simply knowing
what is available at a particular point in time, let alone knowing how to access and use the
63
information. The classification and modelling technique discussed in the next section provides
a basis for defining the nature of Information Products, and for understanding how these
products are produced and used during software projects.

4.2 Classification and Modelling of Information Products

As previously suggested, a variety of Information Products are produced and used by software
projects. Since there are no known approaches for classifying and modelling project
information resources, the following scheme is proposed as a preliminary attempt at providing
an approach for identifying and classifying the various classes of Information Products which
may be available for use at particular times during a project. Information products can be
classified in terms of the following six dimensions.

1. Type. The type dimension defines the category of information to which the Information
Product belongs (ie it defines 'what the Information Product is'). Many different types of
information are produced and used in SE including plans, code, design details,
requirements, and test results. Each of these may have subtypes. For example, there are
various types of code used in SE such as source code, object code, and executables.

2. Representation. This dimension defines the representation technique that is used for the
Information Product. Various representation techniques are used for SE information such
as tables, single numbers, diagrams, charts, text (using natural and formal language), lists,
and models. Representations can be simple or composite. For example, a simple
representation involves the use of a single technique for the Information Product. A source
code file is considered to have a simple representation since it is defined in terms of a single
textual representation. Composite representations use more than one representation
technique. For example, a design document is a composite product since it includes a
structured set of representations for describing the software design.

3. Form. The form of an Information Product defines how it is stored or presented. For
example, Information Products can have a variety of forms including electronic files, paper
documents, and computer-based presentations. Information could also be provided in other
forms (eg in an auditory form). Since this thesis focuses on the provision and use of visual
information, only those forms that can be used to provide this type of information are
considered.

64
4. Lifetime. The lifetime of an Information Product defines whether it is persistent or
transient. Transient products are those that are produced for specific needs at a particular
point in time. For example, the display of a design diagram on a computer workstation as
requested by a user of a CASE tool would be considered a transient Information Product.
Persistent Information Products are those that are available over a period of time.
Persistent products provide the underlying information resource for a project and are often
used in the generation of transient products.

5. Provider. This dimension defines the Information Processor which provides the
Information Product. For example, in Software Engineering, the providers of Information
Products can include people and tools.

6. Primary User. Information Products are generated for a purpose which is generally related
to the information needs of a particular user. The primary user dimension helps define the
focus of use of the information. For example, some information is generated for tool use
(eg binary files). Other information is produced for use by people and is often tailored to a
particular role. Although the information may have been generated for an intended user,
this information can often be customised for other users.

Table 4-1 provides examples of IP classes to help illustrate how the classification scheme can
be used. The examples show that a particular type of information can provide the basis for
several IP classes2. For example, classes 1, 2, and 3 all relate to source code. Class 1 relates
to the persistent electronic version of the code which can be provided by an integrated
development environment (in this case the Rational Apex environment). This class is available
for use by a variety of tools. However, it is not directly useable by humans. Class 2 is a
transient IP class. It is a simple textual representation of the source code and is provided as an
electronic display by a Language Sensitive Editor (LSE). The third source code class refers to
paper printouts of the source code. This IP class is also source code but in this case it is a
paper printout as provided by a Unix printing tool. Class 4 is another code class. In this case,
the information is a code library. It is represented as a DIANA model which is available in the
form of a Unix binary file. Class 5 refers to design rationale provided in a Software
Development File (SDF) format. Class 6 provides an example of a composite class. A

2
The 'dot' notation is used to define subsets of an item. For example, there are many types of code
entities. The different types are defined by extending the 'code' type (eg Code.Source and
Code.Library).

65
composite class comprises a set of other IP classes. Class 6 refers to DOD STD 2167A
Software Design Documents (SDD) which could comprise sets of design documents, design
traceability matrices, textual descriptions, etc. They are provided by an automated
documentation generation tool called SoDA. Class 7 relates to design diagrams. Instances of
this class are provided in paper form and represent class structure (eg as defined by Booch
(1991)). Instances of this class may form part of an instance of a composite class (eg a design
document).

IP Type Representation Form Lifetime Provider Primary

Class User

1 Code.Source Text.Ada File. ASCII Persistent Apex Tools

2 Code.Source Text.Ada Electronic Transient Apex LSE People

Display

3 Code.Source Text.Ada Paper Persistent Unix People

4 Code.Library Model.DIANA File.Binary Persistent Apex Tools

5 Design. SDF Paper.Sheet Persistent People People

Rationale

6 Design. 2167A.SDD Paper. Persistent SoDA People

Details Document

7 Design. Diagram. Paper Persistent Rose People

Diagram Class

Table 4-1 Examples of SE Information Products

In addition to defining the individual classes, there are a range of class relationships that may
need to be modelled. For example, the 'comprises' relationship is used to define composite
classes. In the examples shown, an instance of Class 6 may comprise instances of Class 7. The
'derived from' relationship highlights the relationships between Information Processors and
Information Products. For example, instances of Class 2 may be derived from instances of
Class 4 which defines the underlying compilation library.

This classification and modelling approach can be used to help define the context for
information use in Software Engineering. It can be used as a basis for measuring and for
understanding the diversity, quantity, and timing of information produced during a software
development project. This approach has been used in practice during the Industry Case Study
discussed in Chapter 6 to gain an understanding of how persistent information is produced and

66
used during a real software project. The approach also provides a basis for defining the access
and filtering requirements for the integrated visualisation approach discussed in Chapter 5.

4.3 Using Information for Description Purposes

This section deals with the human use of information. As discussed in Chapter 2, software is a
conceptual entity with no physical form. As such, direct visibility is not possible. This thesis
argues that indirect visibility is possible through description. Description involves using
information as a basis for describing characteristics of interest. This section defines the
concept of descriptive information, discusses the relationship between description and
measurement, and discusses how this type of information is used within a project and SE
context.

4.3.1 Descriptive Information

Descriptions are partial (or incomplete) definitions of one or more entities, attributes, and
relationships. Most information produced for human use is descriptive in nature. As defined in
this thesis, SE descriptions (or Description Products) include a range of Information Products
that have the following characteristics:

1. Must be human useable. Descriptions must use appropriate representation and

presentation techniques so that information can be processed by humans. For example, a
Unix binary file is not a human useable form hence cannot be used directly for description
purposes.

2. Must refer to something. SE descriptions are information that refers to particular project
entities. These entities can include products, processes, and resources. For example, a
measure of the number of lines of code might be used to describe the size of a software
module. The structure of a software design entity might be described using some
diagrammatic representation. The entities or 'things' being described can be real or
conceptual. For example, a design document is a real entity. The number of pages
contained in a design document can be used to describe its size. A logical design
component is a conceptual entity devised by a designer. It has no physical form, however it
could be described if appropriate representations are used.

3. Must be a valid representation. If information is to be used for description purposes,

then it must be a valid representation of the 'thing' that it is describing. As discussed in

67
 Chapter 2, measures are considered to be a concise form of description. However, if the
measure does not map to the observations in the 'real world' domain, it is considered
meaningless. The notion of meaningfulness forms an important part of measurement
practice. However, it is just as important for other types of representation. For example, a
textual description is not valid unless it is an accurate account of the 'thing' that is being
described. Similarly, a design diagram that does not accurately reflect the structure of the
implemented system cannot be used as a valid description of the software product. When
information is used for descriptive purposes it must observe 'representation conditions'
(Kaposi 1991). This means that the mapping between the 'real world' and representational
domains must be valid and accurate. These issues are discussed more fully in Section 4.3.2
which addresses the relationship between measurement and description.

4. Must be used with appropriate contextual knowledge. If the information cannot be

interpreted by the user, then it is not useful as a description. For example, formal
mathematical notations are often used as a basis for description in SE. If the user does not
understand the notation, the information cannot be used. Descriptions must be considered
in relation to the level of knowledge and experience of the proposed users (ie the context of
use). Additional contextual information often needs to be provided as part of a description
to ensure that it can be understood. For example, dimensions are generally provided with
numeric information to indicate the type of information (eg 20 degrees Celsius).

4.3.2 Measurement and Description

The preceding discussions suggest a relationship between measurement and description.

Finkelstein (1982) highlights the relationship by suggesting that "measurement is the
assignment of numbers to properties of objects and events. It is thus a description of
properties3 of objects and events, and not the objects and events". These measures can then
be used as a basis for describing the entity of interest. Descriptions can be measures or
combinations of measures and other descriptive information. For example, a software module
could be described by the following sentence: "the Ada Package named Printer_Driver has
200 SLOC, 10 procedures, and was written by Dave Brown". Here, the size of the module is
described in terms of a measure of the number of lines of code. This description plays a part in
the overall description of the module (ie the entity of interest). Other attributes are described

3 The term property is used in measurement literature to distinguish between an attribute that is described in terms of measurement and other attributes. This
' '
distinction is not made in this thesis. Where required, the term
measurable attribute is used instead of the term property.
' '

68
in other ways, such as the text strings which define the name of the module and the name of
the author.

Measurement can provide concise and meaningful descriptions. However, measurement is not
simply the allocation of numbers to things. Kaposi (1991) proposes a definition which
highlights the important characteristics of measurement: "Measurement is the process of
empirical, objective encoding of some property of a selected class of entities in a formal system
of symbols so as to describe them". The process of measurement includes modelling (of the
target set), empirical observation, mapping to formal systems, generating measures, and
validation. Clearly, the values resulting from the measurement process have limited use unless
they have some empirical meaning and have been validated. This is true of any description.

Traditional measurement theory (Campbell 1957; Helmholz 1930) focused on the use of
numerical representations for measurement. However, several researchers ( eg Kaposi 1991;
Finkelstein 1975) argue that other forms of representation can also be used. As Kaposi's
definition suggests, measurement can be extended to include any "formal system of symbols".
She concludes that measures may be chosen from a range of numbers, letters, colours, shapes
or even pictures (Kaposi and Meyers 1994). Although we can consider a wide range of
representations for providing descriptions through measurement, this thesis argues that not
everything is measurable and we may not find it possible or practical to base our descriptions
on measurement alone. This is particularly so for new disciplines such as Software
Engineering where, for many attributes, there may be insufficient understanding or underlying
theory and models to support valid measurement. Moreover, even if we consider measurement
to cover a wide range of representations, we are still only describing the attributes of entities.
Description is a broader concept which includes the description of attributes, but also considers
the integration of these descriptions into the descriptions of the entities themselves.

The preceding discussion considered the relationship between measurement and description. It
highlighted the fact that measurement is an important element of description in that it provides
a means for producing concise and meaningful descriptions of attributes. The discussions also
identified some important characteristics that need to be considered in relation to the broader
description concepts. Measurement involves a formal process which supports the mapping of
values (symbols or representations) to attributes of entities. The process involves empirical
validation to ensure that the resulting descriptions are valid and meaningful. Description
processes in general need to provide a basis for providing accurate mapping between a referent
(the thing being described) and the representation used. Measurement provides an important

69
basis for description and may provide important formal, philosophical, and methodological
foundations for this area of work.

4.3.3 Use of Descriptive Information

Descriptive information provides a basis for indirect visibility of project entities. In Chapter 2,
it was argued that the visibility problem is in fact related to problems and issues associated with
the effective provision and use of descriptive information. Individuals require descriptive
information to support them in a wide range of SE tasks. It is also required to support
communication and understanding. Although individuals engaged in some tasks are reasonably
well supported by current approaches (eg through the use of a particular CASE tool), there are
many tasks associated with the technical management and supporting processes that are
difficult, if not impossible, to undertake with existing methods. These tasks generally require
high-level composite descriptions of software products and processes. These types of
descriptions are discussed in relation to the model-based description approach defined in
Section 4.6 and are explored in the Industry Case Study of Chapter 6.

The problems and issues highlighted in Chapter 2 can be discussed in relation to the contextual
model of Figure 4-1. An information 'use' perspective of this model is shown in Figure 4-2.
An individual undertaking a task as part of the SE process has specific information needs. For
example, a software team leader may need to undertake a pre-release inspection of a set of
software components. This task may involve ensuring that the design and implementation are
consistent, the requirements have been allocated correctly, and the code complies with the
development standards. To accomplish this task, the team leader may need to gain access to
various types of descriptive information from a variety of Information Products and
Information Processors. For example, design descriptions might be obtained from a design
tool. Descriptions of the source code may be needed and might be obtained by using an editor
which accesses some underlying code representation or paper printouts (eg as provided in a
SDF). Requirements information might be accessed from a software requirements document.
The use of product metrics might be necessary to help identify potential problem areas. These
might be available as a metrics report.

70
 SE Process
LEGEND
Lifecycle Processes
Task
Information Processor
Information Product
Descriptive Information
Process Interaction

Project Information Resource

Figure 4-2 Conceptual Model of Information Use

The team leader would most probably have significant problems in using these independent
Information Products for the specified task. For a start, the team leader would need to know
what was available and to identify which information would best support the descriptive
information needs for the task. The appropriate information would then need to be accessed.
This might involve expert knowledge of a range of tools and environments. In using the
information, the team leader would need to assimilate the required information. This may
prove difficult since the information may not be consistent, superfluous information may be
provided with needed information, and the quantity of information may be overwhelming.

4.3.4 Effective Descriptions

The preceding discussions suggest that there are a range of criteria that need to be addressed if
effective descriptions are to be provided. Effectiveness relates to how well the information
meets an individual's information needs. Chapter 2 identified a set of preliminary effectiveness
criteria based on an analysis of the results of the preliminary studies and a literature review.
Definitions of these criteria are provided in Appendix B. Several of these criteria have already
been discussed in terms of the description concepts and the contextual model. Some relate to
the effectiveness of the descriptions themselves (eg meaningfulness, accuracy, and
conciseness). Others pertain to how descriptive information is provided and used. For
example, issues of adaptability, customisation, availability, and composability relate to the
processes and techniques that are used to provide these descriptions. The Description Process,

71
as discussed in the next section, provides a basis for understanding and addressing issues
related to the effective provision and use of information.

4.4 The Description Process

The concept of a Description Process can be introduced into the project context to address
many of the problems and issues identified in Chapter 2. This thesis argues that process issues
are an important consideration when considering the effective provision and use of descriptive
information. In addressing these process issues, we can begin to address many of the
information logistics problems being faced by individuals engaged in SE tasks.

SE Process

Description
Process

Project Information Resource

Figure 4-3 Including the Description Process

The Description Process provides a basis for accessing and filtering appropriate information
from the PIR. As shown in Figure 4-3, this process is also intended to support the
customisation, tailoring, and adaptation of information. The following paragraphs discuss the
general characteristics of the process and introduce the four main description subprocesses.

4.4.1 Provision and Use Processes

Figure 4.4(a) identifies two key description sub processes: the Provision Process and the Use
Process. The Provision Process provides descriptive information. This process is associated
with the PIR. It provides a basis for accessing required information and can support the
transformation of this information into Description Products. These products must comply
with the characteristics defined in Section 4.3.1. The Provision Process may comprise a range

72
of provision subprocesses including Access, Filtering, Representation and Presentation. The
model-based description approach proposed in this thesis also considers the inclusion of a
Modelling and Integration Process as part of the Provision Process.

SE Process
SE Process SE Process

Use Use
Use Information
Need

Facilitation
M ediation Description Description
Description

Description
Specification

Provision Provision Provision

Project Project Project

Information Information Information
Resource Resource Resource

(a) CASE A (b) CASE B (c) C A S E C

LEGEND
Description Process
Information Flow
Process Interaction

Figure 4-4 Description Processes

The Use Process considers the information needs of those who will be using the provided
descriptions. This process considers a range of cognitive and action processes. The cognitive
processes include the human perceptual process as well as other high-level processes such as
the problem solving processes. Various architectural models of human information processing
can be considered in relation to the Use Process. Several of these were identified in Section
2.2.4.2. For example, the Cognitive Model of Visual Interaction proposed by Rogers (1992)
includes the Visual Interaction Process which supports communication between the perceptual
and higher-level cognitive processes. Studies have shown (Rogers 1995) that support for this
process can help reduce the amount of cognitive effort required for tasks that use visual
information. The Use Process also considers action processes (Norman 1986). These
processes consider the interactions that a user might have with an Information Product or the
provider of information.

Many approaches that support the provision of information do not consider the Use Process.
For example, the Provision Process might involve producing metrics reports that describe
particular attributes of the software product. If the needs of the potential users of this

73
information are not understood, the resulting information may prove useless. There are many
cases in Software Engineering where information is provided without a clear understanding of
how it will be used and the tasks and users that it is required to support. For example, as
discussed in Chapter 2, this is a common problem with many forms of technical
documentation. This thesis considers the inclusion of Mediation and Facilitation processes to
address these issues.

4.4.2 The Mediation Process

As shown in Figure 4-4 (b), the Mediation Process provides a mechanism for direct feedback
and sharing of knowledge between the Provision Process and the Use Process. Interaction is
often important since it allows users to specifically request the information needed for
particular tasks. When the Provision Process is based on the use of a computer-based tool,
mediation can be supported through the use of a direct manipulation interface. Several
approaches which provide descriptive information also provide support for the Mediation
Process. For example, hypertext documentation systems allow users to interact with the
provided information in order to obtain the desired information. Various computer-based
visualisation techniques also support this aspect of mediation.

The Mediation Process might also be used to support the human Visual Information Process.
For example, mediation can support human processes such as goal setting and hypothesis
management. Rogers (1995) shows how Visual Interaction Assistants can be used to support
hypothesis management and attention direction in the area of diagnostic radiology.

The sharing of knowledge between the Provision and Use processes is a key function of the
Mediation Process. For example, mediation might be used to support individuals in their
interpretation of descriptions. This might be done by providing additional information on the
nature of the representations used in the descriptions.

4.4.3 The Facilitation Process

The Facilitation Process shown in Figure 4-4 (c) supports the provision of custom descriptions.
Facilitation is based on an analysis of information needs for particular SE tasks. This process
could support the preparation of specifications for Description Products. These specifications
could then be used as an input to the Provision Process. For example, the knowledge-based

74
approach of Garg and Scacchi (1994) provides custom descriptions based on knowledge of the
user's role and the task being performed.

The Facilitation Process supports a range of other processes such as task analysis and
contextual modelling. If custom descriptions are to be provided that effectively support a
user's needs, then the context of use needs to be understood. The Facilitation Process must
have knowledge of what information is available from the PIR and how it can be accessed to
support the generation of the required descriptions. This process not only provides a basis for
the generation of custom descriptions, but may also control the integration and modelling of
information. This aspect of the Facilitation Process is discussed in relation to the integrated
model-based approach presented in Chapter 5.

4.5 Description in Software Engineering Process Framework

The Description in Software Engineering Process Framework (DSEPF) as shown in Figure 4-5
combines the concepts and models discussed previously into a process-based conceptual
framework. This framework includes the contextual foci (ie Software Engineering Process,
Project Information Resource), description products, and description processes.

4.5.1 Model Dynamics

As discussed in Chapter 2, this thesis argues that adaptation is an important consideration for
the provision and use of descriptive information. The previous discussions focused on the
definition of each of the four subprocesses that comprise the Description Process. Each is
important in its own right. However, when considered together, they can provide an effective
basis for supporting user information needs.

75
 SE Process

Use
Information
Need

Facilitation
Mediation Description

Description
Specification

Provision

Project Information Resource

LEGEND
D e s c r i p t i o n Process
Information Flow
Process Interaction

Figure 4-5 Description in Software Engineering Process Framework

The Facilitation Process can be used to support the generation of custom descriptions based on
an analysis of task-based information needs. This process is important since it can provide a
means for building the cognitive bridges discussed in Section 2.2.4.4. For example, facilitation
can support the specification of action sequences and can provide consistent descriptions for
tasks which need to make comparisons (eg evaluation tasks). However, as discussed in
Section 2.3.1 (I5), there are many tasks where task analysis and contextual modelling may be
difficult to perform. As such, the resulting descriptions may not be tuned to individual
information needs. In these cases, the Mediation Process is important since it provides a
means of adapting the custom descriptions to actual user needs.

4.5.2 Use of the DSEPF

The DSEPF has been proposed to support a range of activities. It can be used to support
research into the issues related to the provision and use of descriptive information by providing
a basis for measurement, understanding, and discussion. The DSEPF was used in this way in
the Industry Case Study discussed in Chapter 6. The framework also provides a basis for
considering other complementary research (eg cognitive science models) in relation to the
problems and issues highlighted in this thesis.

As will be discussed in Chapter 5, the framework guides the definition of new description
approaches. The framework can be used to help identify important features of these
approaches in relation to the project context and human characteristics. For example, support

76
for the Mediation Process might help reduce the cognitive load on the user. The model-based
approach proposed in this thesis provides a basis for the integration of information to help
reduce the amount of cognitive effort involved in assimilating different types of SE
information.

The transitioning into SE practice and evaluation of new description techniques is also an
important area that can be supported by the DSEPF. Chapter 6 shows how the DSEPF is used
to support these activities.

4.6 Model-based Descriptions

This section introduces the concept of a model-based approach for supporting the provision
and use of descriptive information. This thesis argues that a model-based approach to
description can address issues of consistency, conciseness, customisability, and adaptation.
Based on the systems and modelling concepts defined by Kaposi (Kaposi and Pyle 1993;
Kaposi and Meyers 1994), this approach provides a formal and defined approach to providing
descriptive information. This section provides the underlying theory for the approach. Section
4.7 outlines how the approach can be used to support the concept of Product-Oriented
Descriptions. This is a descriptive approach based on the use of a Software Product Model.

4.6.1 Systems Concepts

The Concise Oxford Dictionary (Sykes 1976) defines a system as "Complex whole, set of
connected things or parts, organised body of material or immaterial things". Kaposi and
Meyers (1994) consider a system to be a generic notion which applies to any entity (concrete
or abstract, natural or synthetic, alive or inanimate). The system concept can be applied to all
classes of entities including those that perform actions (eg processes) or entities that can be
defined at a point in time (eg products). Most real systems are enormously complex in nature.
They can comprise an almost infinite number of attributes and may comprise an immense
number of parts, depending on the point of view.

A system is a fundamental concept which can be defined in a formal way. It is used in the
definition of other concepts. An arbitrary system S can be defined mathematically as the
ordered pair:

S = (E, RE) .................................................................................................Eq (4.1)

77
where E = {e1, e2, .., en} is a set of elements which may be finite or infinite,

RE = {rE1, rE2, .., rEm} is a set of relations of the elements of E.

This universal definition can be used recursively to define a system to any required level of
detail. For example, each element may in fact be another system consisting of another set of
elements and relations. If the elements of the system are considered to be the entities or parts,
the entire structure of the system could be defined in terms of a recursive application of Eq
(4.1).

The elements might also be considered to be a system of attributes which describe the
characteristics of an entity. For example, the system of attributes of an entity X could be
defined as:

S(X) = (ATTR, RATTR)

where ATTR is the set of attributes of X and RATTR is the set of relationships between these

attributes4. For example, the attributes of a book might include its weight, the number of
pages, number of words, thickness, type (eg software document, PhD Thesis), author, etc.
Relationships might exist between various attributes (eg number of pages and its weight).

Most 'things' of interest comprise enormously complex systems of inter-related entities and
attributes. This inherent complexity creates major difficulties for those wishing to describe and
understand software systems. The universal systems definition provides a basis for modelling,
and describing these systems.

4.6.2 Modelling

A model can be considered to be a finite system. It comprises only those elements that are of
interest to the modeller.

Software Engineering uses a variety of models. Examples include models of user

requirements, design models, and measurement models. These models may reside in databases
of CASE tools, in the heads of designers and analysts, or be the basis of some mathematical
formalism. This work is specifically interested in models that can be used as a basis for

4
Upper case is used when describing the formal system definition. Mixed case is used for the model
definitions.

78
description of various SE entities, in particular the Software Product. These models can be
classified as:

• Atomic Models. These models provide an atomic or "black box" view of entities. They
can be used to describe the system as a whole (or a unity). They are models of the system
of attributes and can be defined in a formal way as:

M(X)A = (Attr, RAttr) ..............................................................................Eq (4.2)

where M(X)A is the Atomic model of entity X. It comprises a finite set of attributes
Attr and a finite set of relationshipsRAttr over these attributes

• Structural Models. Structural or architectural models form the basis of describing the
relationships between entities. They have the form:

M(X)S = (Ent, REnt) ................................................................................Eq (4.3)

where M(X)S is the Structural model of X. It comprises of a set of entities Ent and a set of
relationships REnt over these entities. The relations in this case are the relationships

between the various entities. For example, the entities of interest might be the design
objects and the code modules that comprise a software system. The relationships between
these entities might include dependency relationships between the code modules. They
might also include an encapsulation relationship that defines the entities that provide the
implementation of a design object.

• Composite Models. There is often a need to combine Atomic and Structural models.
These Composite models allow attributes to be described within the context of relationships
that define the structure of the system. As will be discussed later, this is often an important
consideration when describing software systems. Moreover, Composite models provide a
basis for developing higher-level descriptions by combining and processing information
pertaining to the attributes of other (often subordinate) entities. As will be discussed in
Chapter 5, composite models provide a basis for representational integration.
Combining Equations 4.2 and 4.3 provides the formal definition of a Composite model:

M(X)C = ((Attr, RAttr), REnt) .....................................................................Eq (4.4)

79
 where each entity is described in terms of a set of attributes (which could be related) and the
entities have a set of relationships as specified in the structural model. Figure 4-6 provides
a representation of a simple two-level composite model where the entities have a simple
encapsulation relationship. In this model, the entities (A,B,C,D,E) are defined in terms of
Atomic models and are shown as black dots. The Structural model is represented in terms
of the lines which represent the structural relationships between the entities (in this case a
simple encapsulation relationship).

M(A)A
TYPE:.....................Subsystem
NAME:...................Display_Services
SIZE:......................14,920 SLOC
M(C)A
COMPLEXITY: ..High
M(B)A DESIGNER:.........F.Jones
EFFORT:..............112 Manhours
STATUS:..............Released
PURPOSE:...........This subsystem ....
M(D)A M(E)A

Figure 4-6 Representation of Simple Figure 4-7 Simple Description using

Composite Model Atomic Model

4.6.3 Using Models as the Basis of Descriptions

The process of description (in part) involves mapping of the elements and relations of a system
(quite often a model in SE) into a system of symbols, notations, colours, etc. The resulting
descriptions can be classified according to the base model used to support the representations
of the entity (eg atomic, structural, or composite).

Atomic models can be used as the basis of atomic or 'black box' descriptions. These are
descriptions of a system based solely on the attribute's values. Figure 4-7 is a description of
this type. The entity in this case is a Subsystem and is described in terms of a set of attributes.
In this case the attributes are represented in textual and numeric form. However, the values of
these attributes could be mapped to other representational forms. For example, instead of
using the textual representation of type, the shape of the surrounding box might indicate the
type of entity (eg a hexagon). The nominal measure of complexity could be mapped to a
colour. For example, the hexagon could be shaded in red to represent a 'High' complexity.

Atomic models are also important for generating descriptions based on a set of attributes. For
example, the release status of a software unit might be based on its evaluation status, testing
status, and configuration status. These aspects may be defined as attributes of the atomic
80
model of the unit. For example, a unit may be available for developmental release if its
evaluation status is 'Passed', testing status is 'Unit Tested', and configuration status is 'Checked
In'. In cases where these conditions were met, the value of the release status attribute would
be described as 'Developmental'. The atomic model of an entity can provide the basis for
integrating and describing many types of information. As the results of the preliminary studies
showed, this is particularly important for many tasks.

Structural models can also be used as the basis of descriptions. These types of descriptions are
often provided by CASE tools. For example, the calling relationships of a set of code entities
might be represented in terms of a 'calls' graph. As discussed in Chapter 2, the selection of the
representation approach can have a major impact on the usefulness of the resulting description.
For example, the use of a directed graph approach can often lead to layout and scalability
problems.

This thesis is particularly interested in the use of composite models as a basis of descriptions.
These composite descriptions use models of the form of Equation 4.4. In these models, each
entity is modelled in terms of the attributes of interest. The relationships between the entities
are modelled in terms of a structural model. Consider for example, the composite model of
Figure 4-6 which represents a software system 'A'. As such M(A)A is the atomic model of the
whole system. The software is comprised of three code units represented as the leaf nodes B,
D, and E. Units D and E combine to form subsystem C which combines with B to form the
total system A. The atomic models of each of the entities can be used to describe each
individual entity or the system as a whole. For example, one of the attributes of the models of
the code units might be test status. This value of this attribute could be mapped onto various
colours. For example, red could mean that the unit has not been tested. To describe the
system in terms of the degree of testing that has been performed, this attribute information
could be displayed in relation to system structure (ie by colouring those nodes where testing
has not been carried out).

An important feature of using composite models as the basis of description is that the attributes
of higher-level entities can be described in terms of the attributes of other lower-level entities.
For example, if the size attribute of the atomic models was defined in terms of the total Source
Lines of Code (SLOC), the structural relationships could be used as a basis of providing SLOC
values at the system and subsystem levels. For example, the size of subsystem C could be
described in terms of the sum of the values which describe the size of units D and E.

81
The model-based descriptions approach can be used to describe various types of SE entities.
The simple examples provided in this section highlighted its use for software entities.
However, the entities of interest could include processes, individuals, work groups, SE tools,
or combinations of these. This thesis focuses primarily on model-based descriptions of the
Software Product. These concepts are discussed in the following sections.

4.7 Product Oriented Descriptions

This section discusses the use of the model-based concepts for providing descriptions of the
Software Product. This approach allows various classes of SE information to be presented
from a product point of view. It focuses on the problem of providing information in context
and in a consistent manner. This section begins by discussing the concept of the Software
Product. It then introduces the concept of a Software Product Model. Issues related to
provision and use of Product-Oriented Descriptions are then discussed.

4.7.1 Concept of a Software Product

A 'Software Product' is a conceptual entity comprising a set of information products and

cognitive insights. There is no one physical entity that can be defined as 'the' product. As
Offen and Xia (1995) suggest, unlike conventional engineering disciplines where the "product
space" is constrained by the laws of physics, such constraints with software "are mediated only
by the collective mental perceptions of the product shared by project participants". As such,
software can be difficult to describe.

The notion of a Software Product means different things to different people. Also, the type of
software has some bearing on how different people consider the software. For example, the
user of an application package such as Microsoft Word may see the 'product' as the executable
code and perhaps the user documentation. Customers, developers, and maintainers of large 'E'
type or evolutionary software systems (Lehman and Belady 1985), may see the product in
terms of a set of Information Products including the technical documentation, user
documentation, source code, and executables. In many current acquisitions of large software
systems, customers are requiring the delivery of support environments populated with relevant
developmental information to support the evolution of the product. In these cases, in addition
to the documentation and code, the Software Product is seen as comprising the Information
Products that were produced and used during software development including the design
models, test drivers, and configuration management details.

82
As the Software Product has no defined form, it is difficult to manage and understand. There
is no single representation that can be used as a basis for describing the product. This causes
significant problems for individuals engaged in SE since there is no common definition of the
system that is being built or maintained. This in turn creates major problems for collaborative
work since there is no basis for effective communication (Kraut and Streeter 1995).
Moreover, individuals tasked with monitoring the status of software projects do not have a
good basis for establishing the degree to which the work is complete or for predicting when
delivery may occur. Measures based on the more concrete aspect of the Software Product are
typically used to describe development status. For example, the total Source Lines of Code is
often used for this purpose. This is often an ineffective description since consideration is not
given to other important attributes such as testing status, defect profiles, and integration status.

This thesis argues that, although software is a conceptual entity, it can be modelled in such a
way that it provides the basis for generating the necessary descriptions and hence can provide
for improved visibility. The next section discusses the concept of a Software Product Model
which supports this conjecture.

4.7.2 Software Product Model

Offen and Xia (1995) stress the importance of modelling to Software Engineering. They
suggest that models and modelling can give substance to the abstract intellectual artefacts
characteristic of software. This enables them to be subjected to shared comprehension and
scrutiny. Models are also important for successful software measurement and experimentation
(Browne and Shaw 1981).

The Software Product can be modelled as a system of entities and relationships between these
entities. For example, the structural model of Equation 4.3 provides a definition of this type of
model, known as the Software Product Model (SPM). The Software Product comprises a
wide range of entities (Ent) including requirements, design, and code entities. These include
conceptual entities (eg logical design entities as understood by a designer) and those entities
represented by physical artefacts (eg source code). Many relationships (REnt) exist between
these entities. For example, these can include encapsulation, inheritance, calls, and uses
relationships.

If the Software Product Model is to be used effectively for description purposes, then it must
also provide a basis for describing the attributes of the entities. To do this, the atomic models
of the entities need to be considered. The composite model of Equation 4.4 provides the
83
definition of the Software Product Model. This approach allows both structural relationships
and attribute information to be integrated and used in a consistent manner. The results of the
Industry Case Study of Chapter 6 give details of how this approach can be used in practice to
provide a basis for product and project visibility.

Software Development

t t t t t t
0 1 2 3 4 5

Project Inform a t i o n

LEGEND
Software Product Model
Attributed Model
Model Update Point

Figure 4-8 Evolution of the Software Product Model

As shown in Figure 4-8, the Software Product Model evolves throughout software
development. During the early stages of the project, the model reflects available information.
The entities of interest at this stage of development might be the requirements. As
development progresses, logical design entities may be defined by the designers. The
relationships between the requirement and design may be defined in the model as well as
particular attributes of interest. For example, information on who designed particular entities
and design status might be considered important. As the project progresses, the SPM evolves
by including those entities, relationships and attributes of interest. Attribute values are
obtained from various sources and integrated into the model. Values of other higher-level (or
value added) attributes are computed within the model, based on structural and attribute
relationships.

This thesis argues that controlled evolution of the Software Product Model can provide a basis
for improved visibility and access to needed information. As will be discussed in Chapter 5,
the Software Product Model also provides an effective basis for the integration of various
types of SE information. It provides support for the preparation of composite descriptions
which can be used to present various types of information in context. The use of a consistent
underlying representation of the Software Product can help provide a common definition which

84
supports collaboration and communication between those involved in the various project
lifecycle processes.

Issues related to how the model evolves and update intervals are important research questions.
These aspects need to be determined by undertaking an information needs analysis and
measuring aspects of SPM evolution and use during real projects. The industry case studies
are, in part, investigating these issues.

4.7.3 Provision and Use of Product Oriented Descriptions

The Software Product Model provides a basis for the preparation and use of Product Oriented
Descriptions (Vernik and Burke 1993). This approach provides project information from a
product perspective thereby allowing information to be provided in context. A product
perspective can be important for individuals engaged in a variety of tasks such as those
responsible for the technical management of the project, quality assurance personnel, and those
involved in IV&V. Chapter 5 discusses a practical implementation of these concepts.

Product Oriented Descriptions implement the extended notion of software visualisation as

discussed in Chapter 2. These descriptions allow users to view the relationships between
various product entities. They also allow attribute information to be viewed in context. These
types of composite descriptions allow users to investigate structural relationships, relationships
between attributes and entities, relationships between attributes and structural relationships,
and the relationships between the relationships themselves.

Product Oriented Descriptions allow a variety of information to be presented in relation to the

structural representations of the software product. For example, information on project
resources could be used to highlight those areas of the product which were consuming large
amounts of effort. Process information could show the status of various product entities.
These descriptions could be used to answer questions such as: Which modules have progressed
through the testing phase? Where were most defects found during integration testing?
Product Oriented Descriptions also allow various items of information to be considered in
relation to one another. For example, descriptions of product attributes could identify areas of
the product which do not comply with company development practices. For these areas,
resource information could be used to identify those responsible.

85
4.8 Summary
This chapter has presented a range of concepts, models, and ideas related to the provision and
use of information in SE. A Contextual Model was introduced to support a discussion of the
problems and issues defined in Chapter 2 and to define the context for the approaches
proposed in this thesis. The Contextual Model also helped define the scope of this work. This
thesis focuses on the use of persistent project information resources for those engaged in a
range of SE tasks. An approach was defined to support the modelling and characterisation of
the Project Information Resource in terms of its Information Products.

Although various types of information are used in Software Engineering, this thesis is
predominantly concerned with the provision and use of descriptive information. This type of
information is considered particularly important since it provides a basis for indirect visibility of
the characteristics of project entities. The concept of a Description Process was proposed as a
means of more effectively providing this type of information to those engaged in specific SE
tasks. In addition to considering aspects related to the provision of descriptive information,
the process also considers the use process. This thesis argues that the cognitive and action
processes involved in using this type of information are important considerations. The
Description Process also considers the mediation and facilitation process which provide a basis
for producing descriptions which can be customised and adapted to individual needs.

The concept of model-based description was defined as a means of integrating and

representing various forms of project information. The use of a composite model as the basis
of a Software Product Model is seen as a means of addressing many of the problems and issues
identified in Chapter 2. This model forms the basis of a Product Oriented Description
approach which allows a range of project information to be presented from a product
perspective.

The next chapter builds on these concepts and models to define practical approaches for more
effectively providing descriptive information.

86
 5. INTEGRATED VISUALISATION OF SOFTWARE
SYSTEMS

This chapter describes and discusses approaches for providing practical implementations of the
concepts and models defined in Chapter 4. A integrated computer-based visualisation
approach is proposed as a means of providing Product-Oriented Descriptions. This approach
is based on the use of a Software Product Model to support the integration and assimilation of
project information. The use of a model-based approach also supports the integration of
representations and views that comprise the resulting descriptions. These Integrated Visual
Descriptions (IVDs) have characteristics that allow information to be more readily customised
and adapted to user needs. The concept of an Integrated Visualisation Environment for
Software (IVES) is proposed as a means of providing this type of information. In addition to
providing mechanisms for the accessing, filtering, integration, and customisation of
information, an IVES must support the process considerations (as defined in the Description in
SE Process Framework) if it is to be effective in practice. These aspects are considered as part
of the overall IVES conceptual model. This chapter concludes by providing details of a
practical implementation, called SEE-Ada, which incorporates many of these concepts. The
SEE-Ada environment has been used throughout this research to support definition, evolution,
and evaluation of concepts and ideas. It serves as a key experimental apparatus for the
Industry Case Study discussed in Chapter 6.

Original Content and Contributions. The following concepts and approaches are the
original work of the author: the integrated visualisation approach; Integrated Visual
Description concepts; the IVES concepts; and the underlying SEE-Ada concepts. (The
original concept of a layers representation in SEE-Ada was developed together with Dr Phillip
Dart, Gina Kingston, and Stefan Landherr.)

5.1 Overview of Integrated Visualisation Approach

The integrated visualisation approach is proposed as a means of providing Product-Oriented
Descriptions. The approach is based on the use of a Software Product Model. The key
characteristics of the approach include:

87
• The use of Integrated Visual Descriptions (IVDs) as the primary form of information.

• The use of a computer-based environment to support the accessing, filtering, modelling, and
integration of information, and to support the provision and use of IVDs.

• Support for contextual and process considerations as defined in the Description in SE

Process Framework.

This approach addresses three main aspects of integration. These are:

1. Integration of project information from multiple sources into a consistent underlying model
of the Software Product.

2. Representational integration which supports the integration of various types of information

into composite visual representations.

3. Presentational integration which provides integration of multiple views of the product and
supporting information.

5.2 Integrated Visual Descriptions

Integrated Visual Descriptions (IVDs) are classes of Information Products which use an
integrated set of computer-based visualisations to describe systems of interest. They are
designed to support the consistent representation and presentation of descriptive information.
Moreover, they provide a flexible means of customising and adapting information to suit
individual needs. This section discusses the characteristics and main features of IVDs.
Practical examples of these concepts are discussed in relation to the SEE-Ada environment in
Section 5.4.

5.2.1 General Characteristics

Figure 5-1 illustrates the main features of an IVD. The IVD concept is based on the use an
integrated set of primary and peripheral views. This section discusses the general
characteristics of this approach. Details of the visualisation approach used for the primary
views are provided in Section 5.2.2.

88
 Tree

Chart
List

Primary Views

Graph Text

Figure 5-1 Integrated Visual Description

The primary views are based on compact, spatial representations of sets of product entities.
The arrangement of these entities is based on a set of rules which can be defined in terms of the
structural relationships captured in the Software Product Model. Primary views provide
central visualisations which can be readily adapted to describe those characteristics of the
product which are of importance to the user. For example, the set of entities shown in the
example of Figure 5-1 might represent various types of code units of interest. They may be
arranged on the basis of their calling relationships. The compact representation allows other
information to be integrated and viewed in context. For example, lines may be overlayed onto
these 'base representations' to describe important control flows. The shading or colouring of
the entities may describe the values of particular attributes of interest (eg those entities that
have most recently changed, or those that have complex internal structure). The primary views
provide a basis for representational integration. They have been designed to provide flexible
and scalable representations which can be readily customised and adapted to user needs.

The peripheral views support the primary views. They can provide alternative representations
of the information provided in the primary views. They also allow for additional information to
be provided. For example, the user may find it more understandable to view particular
relationships of a subset of the software entities shown in the primary view, as a tree or graph.
A chart may provide an effective means of describing the trends in a set of numeric attributes

89
for particular entities. The text view might provide a link to some underlying textual
representation of an entity (eg a section of the source code or a link to a requirement in a
requirements specification).

The information provided in the primary and peripheral views is consistent since all information
is provided by way of the Software Product Model. For example, even though an entity,
relationship, or attribute might be represented using different techniques in different views,
they are based on a single underlying definition.

The use of a Software Product Model also provides a basis for presentational integration. As
such, if a user performs an action on an entity in a particular view, the results of this action can
also be shown in another related view. For example, the user may want to identify those units
in a system that have low comment counts. To do this, the user might define a mapping of
measure values to colours which represent the characteristic of interest. The following
mapping might be used: units with no comment lines are shaded red, those with between 1 and
20 comments are shaded orange, and those between 21 and 50 are shaded blue. The results of
this mapping could be overlayed onto the base representation of the primary view. The user
may also require the actual values for those units that fall within these thresholds. This
information could be provided as a list in a peripheral view. The list would include only those
elements of interest and the items of the list could be coloured according to the colour
mappings. The resulting IVD would comprise the primary view which showed the measure
information in context (from an overall product perspective), and a list view which allowed the
user to analyse actual values in more detail. For example, the user may sort the list in terms of
the metric values to identify those units with the lowest comment counts. These aspects are
discussed further in relation to the SEE-Ada views in Section 5.4

5.2.2 Primary View Visualisation Principles

As discussed earlier, primary views provide the central visualisations for the IVD concept. The
visualisation approach provides a compact, spatial representation of software systems (or part
of a system). These visualisations are similar to cartogram representations (Lohse et al. 1994)
and so can be thought of as spatial maps of the software system. Since they allow other
information to be superimposed, primary views can be used as a type of chloropleth
(Monmonier 1991). This type of representation uses colour, grey scale, or texture to display

90
related information. They can also be used as isopleths which use lines to join particular points
(eg to show a relationship between two entities).

These types of representations provide a basis for displaying a wide variety of information in
relation to the entities of interest. Since they provide compact, customisable, and tailorable
representations, they can address many of the scalability problems attributable to other
representation approaches. For example, instead of providing all the information of a
particular type (and which a provider of information may believe a user might require), the
primary views can be customised and adapted to provide only those aspects of interest to a
user. Moreover, since the human perceptual process is well suited to identifying patterns and
relative positions (Norman 1993), the spatial characteristics of this approach can enhance
understanding and use.

Primary views are generated using the following basic principles:

• Entity Selection and Representation. The primary views can be customised to show only
those entities of interest to a user for a particular task. As shown in Figure 5-1, these
entities can be represented as individual symbols or as display groups which comprise a set
of entities (often based on defined relationships between entities). Display groups are
important since it is often necessary to consider an entity in terms of its constituent parts.
For example, an Ada package can be thought of as a single library unit or as a set of
compilation units comprising a specification, body, and a set of subunits. In an object
oriented design, a class category could be viewed as a single entity or as a set of
encapsulated classes (Booch 1991). This ability to customise the visualisation shown in the
primary views is seen as important since the software product typically comprises large
numbers of different types of entities, and hence it would be impractical to show all entities
in one representation.

• Arrangement. The entities (and entity groups) of interest are arranged in terms of a set of
arrangement rules. These rules typically reflect a set of one or more structural relationships
captured in the Software Product Model. For example, if the entities of interest were Ada
compilation units, the entities could be arranged based on a set of 'with' dependencies. If
the primary view was used to show logical design entities, a 'uses' or 'inheritance'
relationship may provide the basis of the arrangement rules. The resulting base
representation provides a compact representation of the entities of interest. This

91
 representation then provides a basis for integrating and viewing other information. The
spatial arrangement of entities reflects important underlying relationships between the
entities. As such, the user can gain important information on the relationship between units
based on their relative positioning. Moreover, the spatial arrangement of entities gives the
base representation a unique shape (or footprint) which is important for many tasks,
particularly those which need to monitor changes to product architecture.

• Tailoring. Tailoring is used to add or reduce detail relative to user needs. For example,
representations could be tailored to include or delete entity names. Zooming of views can
support tailoring needs. For example, when zooming out, the level of detail for the entire
view is reduced. This can be important for tasks where the overall 'shape' of a system needs
to be understood, or where the user is interested in viewing attribute values for a large set
of entities. There is also a need to be able to tailor the detail of specific entities to help
support attention direction (Rogers 1995). For example, the user may be primarily
interested in a particular set of entities such as those that combine to perform a particular
function. Although not of prime concern, other entities may be provided to help describe
the setting in which these units operate. Since it may be sufficient to know that these
entities exist, their level of detail may need to be reduced so that they do not distract the
user. Figure 5-6 provides an example of a SEE-Ada view where this type of tailoring has
been undertaken.

• Integrating Relationship Information. A key principle of the visualisation approach used

in the primary views is the ability to incrementally add needed information. For example,
instead of describing all instances of a particular relationship in one representation, the IVD
approach allows the user to display only those relationships of interest (eg a particular
calling sequence). These could be overlayed onto the base representation through a direct
manipulation interface. Alternatively, they could be the result of some previous analysis and
hence may provide the basis of a custom description.

• Integrating Attribute Information. The visualisation approach used for primary views
also allows attribute information to be incrementally overlayed onto the base
representations. For example, colour could be used for attribute representations. This
information could be overlayed onto the base representations to highlight important product
characteristics. For example, this technique could be used to show which elements of the
system have undergone the most change or may identify those sections of the

92
 implementation which have unused code. The user may also wish to observe characteristics
of the system based on attribute combinations. For example, in undertaking quality
inspection tasks, the user may wish to identify those units which are poorly commented. To
help identify sections of code which warrant further investigation, the user may wish to
identify those units which have large numbers of Source Lines of Code, high complexity,
and low comment counts.

An IVD can use a set of integrated primary views to describe various levels of product
abstraction. For example, an entity might encapsulate a number of other entities. These
encapsulated entities could be shown in another primary view directly linked to the parent
view. For example, an Ada package body might encapsulate a range of other code entities.
These encapsulated entities could be shown in another primary view which is linked to the
package body representation in the higher-level view.

This concept could be used to support the representation of the entire software product in
terms of a layered set of primary views. For example, at the highest level of abstraction, the
product could be represented in terms of the set of logical design components. The next level
of abstraction could provide the representation of the code modules. The entities that
comprise the code modules could be described in primary views at yet lower levels of
abstraction.

These principles form the basis of some of the SEE-Ada views. Bailes et al. (1995) have also
made use of these principles for describing abstract representations of Ada software systems to
support design recovery tasks. They provide a view of the type described above to represent a
set of standard abstractions of the system under investigation. This view is linked to a view of
the concrete code entities. Selecting an abstract entity on the abstract view can then highlight
the code components that implement this abstraction in the concrete view.

5.3 Conceptual Model of an IVES

A key element of the integrated visualisation approach discussed in this chapter is the use of a
computer-based environment to support the provision and use of IVDs. These types of
environments must provide support for the contextual and process aspects defined in Chapter
4. This section provides details of a conceptual model of an Integrated Visualisation

93
Environment for Software (IVES). This model defines the key features that need to be
considered when developing practical implementations of the concepts defined in this thesis.

Figure 5-2 shows the IVES Conceptual Model in relation to the Description in SE Process
Framework defined in Chapter 4. It supports the Provision Process by providing access to and
filtering of project information and support for the generation of IVDs. The IVDs provide
descriptive information which can be customised and adapted to the needs of individuals
engaged in specific SE tasks. The following sections discuss the main features of an IVES and
discuss how they support the effective provision and use of descriptive information in SE.

SE Process

IVDs
Mediation

Information
User Interface Need
Presentation

Representation C
O
N Facililitation
Modelling and Integration SPM T Process
R Description
Filtering O Specification
L
Access

PIRM Details

PIR Model

Project Information Resource

Figure 5-2 IVES Conceptual Model

5.3.1 Modelling and Integration

The Software Product Model (SPM) is a key component of an IVES. It supports the filtering,
access, and integration of information from external sources and provides the basis for
preparing required descriptions. The SPM is a composite, descriptive model which includes
those entities, relationships, and attributes that are used to describe the Software Product. The
model is generated and updated with information extracted from various Information Products
in the PIR. The SPM evolves in a controlled way. Evolution is guided by an analysis of user
needs, the availability of required information, and the status of the project. The evolution of

94
the SPM is discussed in Chapter 6 in relation to the AMD/FCS product which forms the basis
of the Industry Case Study.

The SPM provides a consistent underlying representation of the Software Product.

Consistency is checked during the generation and update of the model. For example, the
environment will not allow information to be appended or changed unless there is an
appropriate mapping between the entities referenced in the imported information and the
entities that comprise the model.

5.3.2 Access

Information is accessed from the PIR by either directly accessing a particular Information
Product or by gaining access to underlying Information Products by way of an Information
Processor. For example, some Information Products may be directly available as electronic
files. Other information may need to be extracted from a CASE tool, perhaps through a
programmatic interface. The IVES must also provide a means of accessing information from
the people who form part of the project. A Project Information Resource Model (PIRM),
based on the classification and modelling approach defined in Section 4.2, can support the
access of required information. The model helps identify instances of various classes of
information, the availability of the information, and the access methods. An example of an
actual PIRM is discussed in Chapter 6.

5.3.3 Filtering

Filtering involves extracting required information from accessed information. The filtered
information needs to be in a form suitable for integration into the Software Product Model. As
such, attribute and relationship information must be defined in terms of the entities that form
the basis of the model. Attributes can be defined in terms of actual values or as references to
some underlying information. For example, an attribute could be a numeric value, a date, or a
text string. An attribute could also be a pointer to a section of an existing Information
Product. For example, the attribute could be a link to a section of a source code file or the
electronic form of a requirements document. Access and filtering can be driven by the
immediate needs of the individual (eg to access an existing textual description) or by modelling
needs.

95
5.3.4 Representation and Presentation

Representation and presentation approaches support the generation of IVDs. Representation

involves mapping aspects of the information provided by the SPM to representations that are
suitable for users of the information. This can include the mapping of entity symbols,
generation of display groups, and attribute colour mappings. Support for representational
integration also needs to be considered so that various representations can be combined into a
composite form.

Presentation refers to the manner in which the representations are presented to the user. IVDs
comprise sets of integrated primary and peripheral views. Presentation functions provide a
basis for this integration and for controlling the views. Presentation considers aspects such as
the zooming and rotation of views. Other more novel aspects of presentation also need to be
considered. For example, Colby and Scholl (1991) discuss the advantages of using
transparency and blur to support the use of complex multi-layer map images for air flight
planning. These types of techniques should be considered for the presentation of descriptive
information in SE. The use of appropriate presentation techniques can also help make more
appropriate use of the workspace as provided by a computer display. The use of fish-eye
views and the types of techniques proposed by the Information Visualiser research (eg the
perspective wall (Robertson et al. 1993)) could help support the scalability characteristics of
primary view representations.

5.3.5 Control

The IVES can be controlled directly by the user through a direct manipulation user interface.
The environment can also be controlled remotely by way of a description specification (eg by
using scripts). The control component of the IVES model provides a means of controlling the
functions performed by a particular layer or by sequencing actions of several layers.

The direct manipulation interface supports the Mediation Process. The user is able to interact
with the environment to develop IVDs to support a particular need. Mediation allows the user
to interact with the environment in an exploratory manner. This mode of operation is typical
of many visualisation approaches.

The environment can also be controlled by way of the Facilitation Process. Facilitation
provides a means of providing custom descriptions based on an analysis of user information
needs. In addition to providing custom descriptions, the Facilitation Process provides
96
automated support for accessing, filtering, modelling, and, integration. As will be shown in
Chapter 6, this approach can be used to automatically evolve the SPM in line with project
progression and information needs.

5.3.6 IVES Usage

This thesis argues that flexible approaches are necessary to support the information needs of
individuals engaged in many SE tasks. As discussed in Chapter 2, Budgen et al. (1993) argue
that a major problem with many CASE tools is that the designers of the tools presuppose how
they will be used. However, these tools often need to support opportunistic methods (eg
investigations which are driven by exploration). In other cases, users may prefer to be guided
and supported through a well defined process. For example, a common task which is
conducted on software projects is the evaluation of the product against a fixed set of criteria,
often defined in development standards. These tasks have a well defined process which must
be undertaken for each release of a software component. Support by a Facilitation Process in
these cases could automate many of the steps that need to be taken, and hence reduce the
amount of effort required of the individual.

The integrated visualisation approach supports the exploratory and the defined processes.
Moreover, the approach supports a combination of the two. For example, in an evaluation
task, the user may be provided with a custom description which highlights a potential problem.
The user may need to investigate this problem by interacting with the IVES to obtain
additional information, such as a related section of source code or relationships between the
non-compliant entity and other parts of the system. The IVES can support this adaptive mode
of operation.

5.4 SEE-Ada: A Practical Implementation

The SEE-Ada environment is a practical implementation of many of the concepts presented in
this thesis. It has been used to support research objectives by providing a vehicle for
developing and testing concepts and approaches. The environment has evolved over several
iterations of the research process. The initial version focused on integrating metric information
with code representations. Version 2 provided the ability to integrate a wide variety of SE
information and has undergone significant field trials.

97
This thesis focuses on the Version 3 implementation of SEE-Ada. This version provides
enhanced methods for representing and presenting descriptive information. Version 3 also
provides support for the contextual and process aspects of the Description in SE Process
Framework. This version has been instrumented to support experimentation. It serves as the
main experimental apparatus for the Industry Case Study discussed in Chapter 6.

SEE-Ada is a large and complex product that has been engineered to commercial standards.
The characteristics of the product are provided in Appendix C. As discussed in Chapter 3, an
'industry-strength' implementation is considered necessary if it is to be used as an apparatus in
field studies. In addition to providing the required functionality, the implementation needs to
be reliable, supportable, and well documented. It must be able to handle systems that are
representative of the target domain (ie large-scale software products, not 'toy' examples).

GRAHICAL USER INTERFACE

View Information
Generators Overlay

CONTROL Script Mode

Interface

Information Base SPM

Information I/O
Structure Attribute
Parser I/O I/O

Filters and Utilities

PROJECT INFORMATION RESOURCE

Figure 5-3 SEE-Ada Environment

5.4.1 Overview

The SEE-Ada environment implements many of the IVES concepts discussed in the previous
section. The main features of the environment are shown in Figure 5-3 and include:

98
• Information Base. The Information Base is a key feature of the SEE-Ada environment.
The central element of the Information Base is the Software Product Model (SPM). The
SPM is a descriptive information model which comprises a set of product entities,
relationships between these entities, and a set of related attributes. This model and its
related information can be used to describe a range of software product characteristics.
The SEE-Ada SPM is defined as an Entity Relationship Attribute schema and implemented
as part of an underlying relational database.

• Information I/O. The Information I/O feature of the environment provides support for
access, filtering, and integration of information. The Structure I/O interface allows the
import and export of product structure information, including design and code entities and
their relationships. This interface allows a wide variety of design and code relationships to
be imported and integrated into the Software Product Model. These can include the more
typical structural relationships described by other tools (eg calls, encapsulation, 'with'
dependencies), as well as relationships generated by external analysis tools (eg type reach
and information flow), and user-defined relationships. Attribute information is imported
and exported through the Attribute I/O interface. These attributes are related to the
entities that form the basis of the Software Product Model. Attributes can be numeric or
symbolic (ie text strings, labels etc). The Information I/O component can also provide
direct access to information products in the PIR through link attributes. For example, a
link attribute could specify a particular source code file and line number. An internal parser
is also included as a component of Information I/O. It can be used to access and filter
structural information from Ada source code files. This parser is used if structural
information is not available from other sources within the PIR (eg through interfaces to the
development environment). Users can also manually input and update information through
the graphical user interface.

• Filters and Utilities. SEE-Ada uses a range of filters and utilities which can be used to
extract desired information from the PIR. For example, filters are provided to extract
structural information from the Rational Apex5 development environment (eg through the
standard ASIS interface (Barnes, 1993)) and the Rational Rose Object Oriented Analysis
and Design tool. Configuration management information can be extracted from the Source

5
The Rational Apex environment and associated development tools are discussed in Appendix D.

99
 Code Control System (SCCS), Revision Control System (RCS), and the Rational Apex
Configuration Management and Version Control (CMVC) systems. Test information can
be accessed through a programmatic interface to the TestMate tool. Filters are provided
for the Information Products produced by the AdaQuest and AdaMAT product
measurement tools. SEE-Ada also has utilities that access the SEE-Ada Information Base
to generate new higher-level attributes. Simple text filters based on Unix text processing
tools (eg grep and awk) are also used to extract information from ASCII Information
Products.

• View Generators. The View Generators provide the primary and peripheral views. They
generate the base representations for these views and provide support for customisation
and tailoring. The View Generators also support presentation and integration of views.
This characteristic of the SEE-Ada views is discussed in Section 5.4.2.4.

• Information Overlay. The Information Overlay feature supports representational

integration within views. For example, attribute information can be mapped to colour
representations and can be superimposed on primary and peripheral views. Relationships
between entities can be superimposed through the use of shading or various coloured lines.

• Control. SEE-Ada can be controlled through a direct manipulation interface or through

scripts. The Control component controls the import and integration of information. It also
controls the preparation and use of the views and representations that comprise the
resulting Integrated Visual Descriptions.

• Script Mode Interface. The Script Mode Interface allows SEE-Ada to be controlled
through the use of scripts. The scripting language is defined in terms of a simple syntax
which reflects the user functions that can be performed by SEE-Ada. The Script Mode
Interface provides a means of automating the setup of the environment and can be used to
support the provision of custom IVDs. This interface supports the Facilitation Process of
the DSE Process Framework. The use of the Script Mode Interface is discussed in Section
5.4.6.

100
5.4.2 SEE-Ada Views

Table C-2 (Appendix C) provides an overview of the various views available in SEE-Ada. The
following paragraphs discuss these views in terms of the primary and peripheral view concepts
discussed in Section 5.2.

5.4.2.1 Primary Views

The Layers View is one of the SEE-Ada primary views. It can be described in terms of the
visualisation principles discussed in Section 5.2.2. Figure 5-4 provides an example of a Layers
View.

The Layers View of Figure 5-4 has been customised to show the set of Ada compilation units
that comprise part of a system of interest. The symbol mappings for this view are defined in
Table C-2 of Appendix C. The display groups show combinations of compilation units. For
example, an Ada package is shown in terms of its specification, body, and any subunits.

The view has been arranged in terms of the compilation (or 'with') dependencies. The
compilation units are arranged from left to right such that a unit can only be dependent on units
to its right or in the same column. This allows a compact, spatial representation of the system.
The resulting representation has a defined shape based on the number of columns and the
number of entities in each column.

The current SEE-Ada implementation only allows two types of the Layers View arrangements.
Both of these are based on 'with' dependencies. This is seen as a limitation of the current
implementation. Several other arrangement schemes have been proposed including
arrangements based on information flow, calling relationships, and design encapsulation.

101
Figure 5-4 Describing Entity Relationships

Figure 5-5 SEE-Ada Integrated Views

The Contains View uses the primary view approach to describe the entities encapsulated by
Ada compilation units. Figure 5-5 provides an example of a Contains View. In this case, the
Contains View shows the procedures that are encapsulated in the package body of
"File_Manager"6.

5.4.2.2 Peripheral Views

A range of peripheral views are provided in SEE-Ada. Figure 5-4 shows a Graph View which
provides a directed graph representation of the units that are shown as 'selected'7 in the layers
view. In this case, the lines describe the 'Simple With' relationship between the compilation
units. The Graph View representation is useful for analysing relationships between a subset of
units that comprise part of the system being viewed. This example helps demonstrate the
problems of using this type of representation for describing the relationships of a total system.
The representation does not scale for large numbers of entities and relationships and hence
becomes illegible.

Figure 5-5 shows another peripheral view that is available in SEE-Ada. The Source View is a
textual view used to describe the source code of a particular entity. In this case the source
code for the "Rename" procedure, which has been selected in the Contains View, is displayed.

SEE-Ada makes use of List Views to support the various graphical representations. These
views can provide simple lists of information. They can also be used to display combinations
of information. For example, the "Show Values Window" in Figure 5-7 is a List View which
shows attribute values in relation to a set of entities identified as meeting the attribute
mappings defined in the "Attributes Tool". The List Views allow representational integration.
In this case, the colour mappings are combined with the representation of list items. List
Views can be arranged using a sorting feature. A 'find' feature allows users to identify those
items of a list that are of interest. A preference feature in the List View allows the
arrangement of the view to change between single and multiple columns.

6
Label names for entities in the primary views are truncated. The user is able to obtain the full name
by positioning the mouse pointer over an entity and pressing the right hand mouse button. This
information can also be obtained from a Details View which provides full details of the entity.
7
Selected entities are represented in black with white writing. Users select entities by positioning the
mouse pointer over them and pressing the left mouse button.

103
SEE-Ada does not provide Chart Views as discussed in the IVD concepts. These peripheral
views would be useful for viewing trends in attribute values of selected sets of entities.
Moreover, charts could help users understand the relationships between various types of
numeric attributes.

5.4.2.3 Subsystem View

The Subsystem View is a design-level view. Although considered a primary view, the
information is provided by way of a tree representation. This is a remnant of early versions of
SEE-Ada where this type of representation was considered suitable for showing design-level
information. Experiences have shown that the spatial visualisation approach used in the Layers
View would be far more suitable for providing this type of information. This issue is discussed
further in Chapter 7.

Figures 5-5 and 5-8 provides examples of Subsystem Views. The example in Figure 5-5 shows
the 'zoomed out' or detailed representation of the view. The Subsystem View is based on the
notion of Logical and Physical Subsystems. Logical Subsystems are represented by the single
rectangles, and the Physical Subsystems are represented by the double-line rectangles. Various
types of design entities can be described in terms of these conceptual entities. For example, in
the Industry Case Study discussed in Chapter 6, the class categories of the object oriented
design are mapped to Logical Subsystems and the physical design as defined by Rational
Subsystems is mapped to Physical Subsystems. In this way, the SEE-Ada subsystem view
provides a description of the logical and physical design entities and the relationships between
them.

The Physical Subsystems encapsulate Ada compilation units. As such, there is a direct
relationship between the Subsystem View and the Layers View. If a Physical Subsystem is
selected in the Subsystem View, the encapsulated units can be shown in a corresponding
Layers View. Figure 5-5 shows a situation where a user has selected a section of the overall
system design in the Subsystem View. The corresponding Layers View shows the Ada
compilation units that implement this part of the system.

5.4.2.4 Presentational Integration.

Figure 5-5 shows how integrated views can be used to describe a Software Product from
highest-level design concepts through to particular sections of source code. The user can

104
select the section of the logical design of interest in a Subsystem View. All other related
design entities are then also automatically selected, including other logical and physical design
entities. The Ada library or compilation units which implement this section of the product are
then shown in a Layers View. Code entities encapsulated in particular Ada units can then be
shown in a Contains View. The source code which relates to a selected code unit can then be
displayed in a Source View.

In addition to using the Subsystem View to describe design-level characteristics of an existing

design, the view can also be used to support design recovery tasks. For example, if design
information is not available for a particular product, some design structure can be recovered
through an analysis of the implementation. Since the views are integrated, an analyst can
analyse the implementation, identify design groupings, and generate subsystem entities from
the Layers View. These recovered design entities can then be incorporated in the overall
design within the Subsystem View.

5.4.3 Describing Relationships

Figure 5-4 shows how relationship information can be overlayed onto a Layers View. The
"Relationships" tool allows the user to control what (and how much) information is provided.
The user can select the relationship of interest from a relationships menu. The relationship
selected in Figure 5-4 is a "Simple_With". A range of relationships can be made available
including other dependency and calls relationships8. The relationships between entities can be
viewed by selecting an entity on the Layers View and selecting the ">" button. For example, in
Figure 5-4, selecting the body of "Test_Tools" and applying the "Simple With" relationship
would shade those units that are 'withed' by "Test_Tools". If the ">>" button was selected,
this would display a transitive closure of all relationships of the type requested. For example,
"Test_Tools might be dependent on a unit in the second column. It might in turn be dependent
on a unit in the fourth column and so on. The "<" and "<<" buttons provide the reverse
relationships. They would therefore provide the 'withed by' relationships.

8
SEE-Ada provides a flexible means of including relationships into the Software Product Model. As
such there are few restrictions on the relationships that can be added. These relationships can be
provided by development environments, analysis tools, or could be user defined. Possible relationships
could include unit dependencies, calls, information flow, process communications, and control flow.

105
Users can also trace particular relationships. For example, tracing can be used to identify
relationship paths through the system. In Figure 5-4, the red line indicates a 'with' dependency
path which refers to those entities that perform the source instrumentation functions. The
green line shows a 'withed by' relationship which identifies the package which uses the standard
"Direct_IO" library. These traces can be labelled and saved. They can then be recalled for
future tasks which require this information.

5.4.4 Customisation and Tailoring

Figure 5-6 shows a Layers View which has been customised and tailored for a particular
purpose. This view describes the calling relationships of the set of functions and procedures
which provide the source instrumentation function. The "Layers - Display Level" tool allows
the user to select the types of entities that will be displayed on a Layers View. In this case, the
entities of interest are the Ada library and compilation units. The "Customisation" function
allows the user to further customise the display to indicate precisely what types of entities
should be shown. For example, the user may specify that only encapsulated Ada tasks should
be displayed in relation to a compilation unit.

Individual display groups can also be customised. For example, since the user is interested in
'Calls' relationships, the main procedure ("Test_Tools") has been customised to show all
encapsulated components. The display group for this entity shows that it encapsulates 10
procedures and instantiates a generic package. A Graph View has been used to show the
encapsulation relationships within the body of "Test_Tools".

The user may be interested in describing a particular call path. To do this, the user initially
tailors the view so that the detail of all entities (except for the "Test Tools") is removed (ie
they are shown as dots). The user then uses the "Relationships" tool in its tracing mode to
describe the calling relationships of interest. Selecting the body of "Test Tools" and selecting
the ">" button automatically identifies all entities that are called by this procedure. In this
'expand by reference' mode only those subprocedures that are called are shown. Of those
called, the user is interested in the code that supports the instrumentation functions. The user
selects the body of the "Source_In~" compilation unit and a red trace line is added.

106
 Figure 5-6 Customisation and Tailoring of Information

Figure 5-7 Viewing Attribute and Relationship Information

The process is repeated to identify the next level of the calling sequence. This action shows
that the body of the "Source_Instrumenter" procedure calls a procedure in the
"User_Interface" package. This procedure in turn calls other procedures in the same package.
These procedures then use the standard "Text_IO" package. This call path can be labelled (in
this case it has been called "Instrumentation Instructions") and saved. This 'value added'
information can then be used for other tasks.

In addition to the calls path for the instrumentation function, the user has added other
contextual information to support the analysis task. For example, references to types or key
procedures in other packages are required to support this functionality. The details of these
packages are shown. Other trace information might also be useful. For example, the green
trace in the "Relationships" tool can show important "Simple_With" relationships. This
information can be displayed by selecting the entry in the trace list.

This example demonstrates how users can mediate with the environment to provide
descriptions that support their specific needs. This approach provides ready access to needed
information. It supports customisation and tailoring so that relevant information can be
provided in an understandable and useful manner. Contextual information can be provided to
further enhance understandability.

5.4.5 Mapping and Integrating Attribute Information

Figure 5-7 provides an example which includes the representational integration of both
relationship and attribute information into a Layers View. Attribute values can be mapped to
colour representations using the Attributes Tool. SEE-Ada imports and stores attribute
information in attribute groups. These groups identify the type of information that is available.
For example, the results of the Industry Case Study show that a variety of information needs to
be made available for a typical project. (Appendix I provides examples of the attributes used in
the AMD/FCS project). The example of Figure 5-7 shows that the user has selected the
"CODE.AUDIT" group.

Each attribute group can contain a set of related attributes. For example, the CODE.AUDIT
group contains a set of numeric attributes which can be used to assess quality aspects of the
product. These attributes can help identify poor coding practices. For example, the attribute
of interest in Figure 5-7 identifies the use of global variables. The user has defined a mapping
such that if an entity has one global variable, it will be shaded in blue. If it has between 2 and 5

108
global variables, it will be shaded orange. Entities that have greater than 6 global variables will
be shaded red. This mapping can be recorded and recalled for use in other tasks. The
attributes mapping is overlayed onto the layers view by selecting the entities of interest and
then selecting the "Go" button in the Attributes Tool. In addition to overlaying the attribute
information, the user has requested a peripheral list view which shows the actual values of the
units which fall within the mapping conditions.

The example of Figure 5-7 also shows the importance of providing composite descriptions
containing both attribute and relationship information. Although the attribute information
identified the presence of global variables, the user may wish to know what impact they have
on the system as a whole or the amount of change that may need to be undertaken to remove
references to these global variables. The 'with_by' relationship is used to identify those units
that depend on the unit which has the highest number of global variables. Further information
may be overlayed to indicate whether these units contain encapsulated tasks. If this was the
case, there could be a high probability of mutual exclusion problems which in turn could have a
major impact on the reliability of the software.

Figure 5-8 provides another example of using attributes in different views. In this case, the
user is interested in identifying sections of the product that may not be properly documented.
An attribute which describes the number of comment lines contained in a compilation unit has
been used for this purpose. The user first applies these attributes to the Subsystem View to
identify those sections of the system that need further investigation. The model-based
approach is used to combine lower level attributes into an attribute at the subsystem level.
Three subsystems are identified as having encapsulated units which have no comments. The
user then selects one of these (ie "Parser_Tables") and requests a Layers View to show the
actual code entities for this subsystem. The user has 'Zoomed Out" on the Layers View to
remove unwanted detail and to provide a more compact presentation of the information. The
result identifies those units that have no comments through to those which may be overly
descriptive. The user can further investigate these by selecting units and viewing relevant
sections of source code in the Source View.

109
Figure 5-8 Viewing Attributes Between Views

Figure 5-9 SEE-Ada Script Mode

The use of attribute combinations is also important. For example, the analyst may wish to
identify the person responsible for developing the units that have no comments. These units
could be selected and configuration management information could be used to identify the
authors of this set of entities.

5.4.6 Support for the Facilitation Process

SEE-Ada supports the Facilitation Process by providing a Script Mode Interface. This
interface can support the setup of the environment. It also supports the preparation of custom
descriptions for particular tasks.

Figure 5-9 provides an example of a script that can be used to support the evaluation of coding
practices. The user can either step through each script action or 'Run' the script in which case
the script will automatically execute each action until it reaches a 'stop' command. The script
begins by opening the system to be evaluated and then automatically displays the Subsystem
View called "CSCI_DESIGN". The user then interacts with the environment to select which
sections of the system will be evaluated. A Layers View showing the Ada Compilation units
for this section of the system is then displayed. The view is then tailored to provide a compact
representation onto which other information can be superimposed. The script then supports
various evaluation activities. The first phase of the evaluation (at Step 3) is to check for usage
of anonymous types. The script selects an attribute that is used to describe the usage of
anonymous types and overlays this information on the Layers View. The user gets an
immediate visual indication of whether the code complies with this criteria. The user can then
interact with the environment to conduct a further analysis. For example, the user may wish to
see how the feature is used in the actual source code or may wish to change the colour
mappings to highlight units with the highest proportion of non-compliances.

The user then moves on to the next stage of the evaluation by running or stepping through the
script. The script automatically 'cleans up' the views and then produces a visual description
which will support the next stage of the evaluation.

By using scripts in this manner, the set of actions that need to be undertaken for these types of
evaluation tasks can be recorded and enacted. Relevant descriptions are provided to support
the analyst. Although custom descriptions are produced, these can be adapted by users to help
support their specific information needs.

111
5.4.7 SEE-Ada Usage

The SEE-Ada environment has evolved in line with research objectives. A major part of the
research strategy has been to release versions of the environment to selected receptor sites in
order to elicit feedback on the validity and usefulness of the concepts. The environment has
also been used by DSTO to support the analysis and evaluation of software products being
produced for the Australian Defence Organisation. This section summarises the results of
these trials and activities. The discussions refer only to the Version 2 releases of the
environment. Version 3 has yet to be released on a trial basis.

SEE-Ada has been used by several organisations including:

• Naval Undersea Warfare Center (NUWC). NUWC is a United States Navy centre
responsible for the procurement of submarine systems. SEE-Ada has been used to support
the analysis and evaluation of software for the new SeaWolf submarine. This software
comprises some four million SLOC. SEE-Ada has been used to analyse the structural
characteristics of the software. It has also been used as a means of managing and using the
vast amounts of product metric information that has been produced. SEE-Ada has also
been used by NUWC to support research into domain specific reuse (Roodbeen 1995).

• Defence Research Agency (DRA). SEE-Ada has been used by DRA Portsmouth in the
United Kingdom to support research into software design approaches and for an analysis of
the Data Fusion Technology Demonstrator software. A version of this knowledge-based
application will ultimately see use on a new generation of Frigates.

• Air Warfare Systems Centre (AWSC). The AWSC is a Royal Australian Navy centre
responsible for maintaining the software for the SeaHawk helicopter. SEE-Ada has been
used to support maintenance activities and to help estimate maintenance and update costs.

• Rockwell Ship Systems Australia (RSSA). SEE-Ada was provided to RSSA to help
support the evaluation and analysis of software being delivered by subcontractors for the
Collins class submarines.

• University of Queensland (UQ). UQ has used SEE-Ada to support knowledge-based

software engineering and re-engineering research. The Reasoning Systems Software

112
 Refinery has been used to generate a range of attributes for SEE-Ada. These include
attributes which classify Ada packages, show type reach, and identify unused entities.

• DSTO. SEE-Ada has been used by DSTO to provide visibility of, and to evaluate, software
produced for the Nulka (Vernik and Landherr 1993), Secure Link-11 Communication
System (SLCS), and the Tropospheric Refractive Effects Prediction System (TREPS)
projects.

Direct observations by the author and feedback from the receptor sites identified a range of
problems and issues related to the use of the SEE-Ada Version 2 environment. These
included:

• Many of the problems were not technical in nature. For example, contractual restrictions
often prevented access to needed information.

• Product visibility is required throughout a project. Analysis and evaluation of products

during the latter stages of a project may not be effective since the entrenched problems are
difficult and costly to rectify. More effective support needs to be provided to technical
managers (eg team leaders) to help identify problems and risks. Technical managers are
generally the most technically competent and experienced members of software
development teams and so can have a major effect on software quality if appropriate
support can be provided for inspection and evaluation tasks.

• The transitioning of technologies into practice is an important consideration. SEE-Ada was

used most effectively when appropriate training and follow-on support was provided.
However, even in cases where this training and support was provided, users tended to
utilise only a subset of the available functionality. Well designed case studies are needed to
help understand the benefits and limitations of the integrated visualisation approach.
Moreover, the effect of these new approaches on existing processes needs to be studied.

• Contextual and process issues are important when considering the effective use of tools
such as SEE-Ada. For example, the user needs to identify particular goals and objectives to
be accomplished and to define the set of actions to be undertaken. The context of use is
also important. For example, the environment needs to be set up to support access to
needed information. This involves understanding what information is available and how it
can be accessed. This thesis argues that these considerations can be addressed in relation to

113
 the Description in Software Engineering Process Framework discussed in Chapter 4. The
use of this framework for the transitioning and use of the integrated visualisation approach
has been studied as part of the Industry Case Study discussed in Chapter 6. This case study
also makes use of the script mode interface which was not available for use in the initial
SEE-Ada trials. This interface is seen as a way of defining and enacting processes.

• Although many problems were identified with the software being evaluated, SEE-Ada did
not have an effective mechanism for recording results. Moreover, except for direct
observation, there was no way of recording how the environment was used for particular
purposes.

• The environment was used predominantly for viewing product structure and product
metrics. Although facilities were available for accessing other SE information, these were
not used because of a lack of knowledge of the underlying information resources available
and how this information could be used in SEE-Ada.

Although the information gained through these transitioning and collaborative activities has
helped provide an understanding of how the integrated visualisation approach can be used in
practice, much of the feedback related to issues specific to SEE-Ada (eg problems or the need
for some new feature). Many process and usage issues need to be explored in the various
domains of use (eg development, maintenance, IV&V). These need to be undertaken with
properly designed case studies together with appropriate instrumentation which can provide
quantitative evidence. Chapter 6 discusses the design and results of a case study that has been
undertaken to address some of these issues.

114
 6. EXPERIMENTATION

This chapter provides an overview of the experimentation that has been undertaken to help
develop and evaluate the concepts, models, and approaches discussed in this thesis. The
chapter begins by providing a summary of three early forms of experimentation that have been
undertaken as part of this research. This work has already been discussed at appropriate points
in previous chapters. It is summarised in this chapter to provide a consolidated account of the
experimentation that has been done to date and to provide a basis for justifying and defining
the current experimental focus: a series of Industry Case Studies.

This chapter provides details of the design of an initial case study which is being conducted at
the AWA Defence Industries (AWADI) Salisbury facility. The focus of the case study is the
development of real-time embedded software for a mission critical defence application. The
case study is primarily interested in how project information can be provided and used to
support technical managers. The results of the first stage of this initial case study are
presented.

Original Content and Contributions. The case study design, including the approach,
procedures, and the design of the experimental setup are the original work of the author.

6.1 Summary of Early Studies, Trials, and Experiences

Experimentation can take many forms. The Concise Oxford Dictionary (Sykes 1976) provides
a broad definition of experimentation. In this definition, experimentation refers to: "the
procedures adopted or operations carried out to make discovery, observation, test". Preece
(1994) focuses on the testing aspects of experimentation. He suggests that an experiment is
"anything done to test an idea, or theory, or to discover something unknown".
Experimentation can be undertaken in a variety of ways depending on the nature of the
research domain and the types of conjectures that are being explored and challenged. For
example, mathematicians test new theories in terms of sets of axioms and accepted theorems.
The social sciences often perform experiments using empirical methods where experience and
observation are major sources of evidence. Regardless of the approach, experimentation plays
a key role in the scientific process and the growth of scientific knowledge (Popper 1977).

115
Chapter 3 discussed issues related to experimentation in the SE domain. Experimentation in
SE can take a variety of forms including formal experiments, case studies, surveys, and quasi-
experiments. Due to the nature of software and SE, researchers need to consider a wide
variety of techniques for gaining knowledge and testing conjectures. The previous chapters
have discussed three main aspects of experimentation that have been conducted as part of this
research. These early forms of experimentation include:

1. Initial Studies. Section 2.1 discussed preliminary studies that were undertaken to help
understand and define problems and issues related to the effective provision and use of
descriptive information in Software Engineering. This work identified a range of problems
including those that were related to information overload, meaningfulness, consistency, and
accessibility. These first-hand experiences highlighted the need for integrating various types
of information and for providing information in context. The initial studies helped gain a
general understanding of the problems and issues and helped define the concepts,
approaches, and conjectures that form the basis of this research.

2. Implementation of Concepts and Ideas. The development and evolution of the SEE-Ada
environment helped test whether many of the concepts and ideas could be implemented.
This work looked at issues of scale and demonstrated that the concepts could be
implemented with current technology. This work also helped refine the IVES concepts. As
Fenton et al. (1994) point out, this is often the only form of experimentation undertaken in
many SE research projects.

3. SEE-Ada Trials. Section 5.4.7 discussed the results of a set of early SEE-Ada adoption
trials. The SEE-Ada Version 2 environment was transitioned to several receptor sites and
was used in-house for product evaluation tasks. This work provided valuable feedback and
insights into the use of the integrated visualisation approach for a range of tasks including
those associated with quality assurance, IV&V, and software maintenance. This work was
empirically based. It highlighted a range of important considerations including the need for
process support, limitations of the SEE-Ada implementation, and the need to consider the
project context. Observations and feedback from these trials helped evolve and refine the
concepts and approaches defined in this thesis.

Although not based on a formal experimental design, these early forms of experimentation
were an important part of the overall research process. The preliminary studies proved

116
effective in helping define problems, issues and conjectures. The implementation and evolution
of SEE-Ada helped establish that the integrated visualisation concepts could be implemented
and helped refine the approach. Early SEE-Ada trials provided some insights into use of SEE-
Ada and provided valuable feedback which helped in the definition of the description concepts
and models. Although important, these forms of experimentation do not provide quantitative
evidence which could be used to address specific objectives and goals. Much of feedback on
the use of SEE-Ada focused on tool features rather than how effective the approach was for
particular tasks. This highlighted the need for a more controlled and formal study with defined
goals and quantifiable results. The Industry Case Study discussed in the next section was
designed to support these needs.

6.2 Industry Case Study

This section presents the case study design and the early results of the Fire Control System
(FCS) case study being conducted at AWA Defence Industries. It begins by defining the
purpose, goals, scope, and setting of the case study. The strategy and approach is then
discussed, followed by an overview of the experimental setup. The FCS software is being
developed in three incremental builds over a period of 24 months. The results provided in this
section refer to the first build of the software and cover the period from 6 September 1995 to 7
March 1996.

6.2.1 Overview

This case study has been designed to help explore and test the key conjectures, concepts,
models, and approaches presented in this thesis. An important component of this case study
involves defining and understanding the context within which information is produced and used
in SE. This involves studying the tasks being performed and the information resources that are
available. This case study focuses primarily on the information needs of technical project
managers (eg software team leaders). The study investigates how the integrated visualisation
approaches, together with the underlying description concepts and models, can be used to
support team leaders engaged in a range of SE tasks. The enhanced SEE-Ada environment
(Version 3) serves as the main apparatus of the study. This initial case study is being
conducted in an industrial setting at the AWA Defence Industries Salisbury Facility. The focus
of the study is the development of the FCS software. This software is part of the Active

117
Missile Decoy (AMD) system which is being developed by AWADI for the Royal Australian
Navy.

6.2.2 Purpose and Goals

The purpose of this case study is to provide information which can be used to help test the
conjectures, concepts, models, and approaches of this thesis in an industrial setting. The
specific goals are to:

G1. Understand how and when information is produced during software development.
This goal addresses the contextual issues related to the Project Information Resource.
Aspects of interest include the classes and instances of Information Products that are
available at particular points in time. This goal has been defined to help provide a better
understanding of the problems and issues related to the diversity and quantity of project
information. It also aims to test the Project Information Resource modelling and
classification approach.

G2. Investigate how descriptive information is used by software team leaders during
software development. This goal addresses contextual issues related to the tasks which
are being undertaken as part of the SE process. It aims to help understand and define
information needs. This goal investigates and records difficulties and problems related to
direct use of PIR information.

G3. Investigate setup and support requirements for SEE-Ada. This goal investigates
issues related to access, filtering, modelling, and integration of information. It addresses
issues related to the setup and evolution of the SPM.

G4. Investigate how an IVES can be used to support team leader information needs.
This goal studies the IVES concepts. It uses the SEE-Ada implementation to study the
benefits and limitations of the integrated visualisation approach for supporting individuals
engaged in a range of team leader tasks. This goal studies the use of the integrated
visualisation approach in relation to the Description in Software Engineering Process
Framework (DSEPF).

The goals, and the related research questions which provide the operational definitions of the
goals, are provided at Appendix E.

118
6.2.3 Case Study Setting

The development of the Fire Control System (FCS) software for the Active Missile Decoy
(AMD) system provides the setting for this case study. Appendix D provides an overview of
this system and the FCS project. The AMD system is an automated fast response decoy
system designed to counter anti-ship missiles targeted against a host ship. It performs its
function by deploying hovering rockets which seduce incoming missiles and hence attracts
them away from their target. The FCS is that part of the AMD which interfaces with the
ship’s Combat Data System to control the set up and launch of the decoy rockets.

The AMD/FCS software is being developed using an incremental approach (McDermid and
Rook 1991; Rational Approach 1994). The software is being developed using Ada as the
primary language and DOD-STD-2167A (1987) as the development standard. The
development environment includes a wide range of CASE tools. A central component of this
environment is the Rational Apex environment which provides a basis for tool integration and
maintains a repository of project information.

The AMD/FCS project was selected as the basis for this case study for the following reasons:

• The project is representative of the state of the practice for this initial domain of interest:
the development of real-time mission critical software for systems such as aircraft avionics,
air traffic control, telecommunications, and defence. The project uses object-oriented
development methods, a variety of CASE tools, and is supported by a central development
environment.

• The project makes extensive use of SE tools and environments and as such can provide
ready access to a range of Information Products. The use of the Rational Apex
environment made this project particularly attractive. Rational Apex uses ASCII files to
support data and control integration. Information is therefore available in a form which can
be easily accessed and filtered. The Rational Apex environment has been selected for
several large-scale software developments in Australia. As such, the experimental setup
developed for this case study will support a range of future studies and development
activities.

• Since this is an initial case study, the project needs to be of manageable size and
complexity. Moreover, the development time scales need to be such that results can be
obtained within a reasonable time frame. The incremental development approach used by
119
 the project allows the study to be conducted in phases, each phase being associated with a
developmental increment. This then allows early access to results and allows procedures to
be refined during successive increments.

6.2.4 Scope

6.2.4.1 Team Leader Focus

The information needs of technical managers (eg the software team leaders in the case of the
FCS project) serve as the primary focus of this case study. This focus was chosen because of
the importance of providing more effective support to technical managers and because of the
diversity and complexity of their information needs. Technical managers play a crucial and
pivotal role in software projects. They have a direct impact on product development and as
such can have a major impact on the quality of products, schedules, and costs. Technical
managers are responsible for, and perform, a wide variety of tasks. These can include
planning, estimating, monitoring, reporting, demonstrating, product inspection, and product
evaluations. They require access to a variety of descriptive information to support them in
these tasks. Many tasks have complex information needs that are not well supported by
current sources of information. For example, technical managers often require information on
a range of entities (often from many different sources) to be accessed and assimilated. The
results gained through studying the information needs of technical managers may also shed
light on the needs of others involved in similar types of SE tasks (eg those involved in
Independent Verification and Validation (IV&V), quality assurance, and process improvement
tasks).

6.2.4.2 Potential Limitations

The results presented in this thesis refer to the first incremental build of the FCS software. As
such, the results do not cover all phases of development. Moreover, since this is first time the
team has used the development environment, including SEE-Ada, the results may not reflect
what can be achieved with more experienced teams or during the later stages of the project.

The FCS project has a distinct set of characteristics. For example, it involves the development
of a medium size product for a critical real-time embedded application. It also makes use of
Defence development standards and uses Ada as the primary programming language. As such,

120
the results may not generalise to other projects (eg larger projects, different application
domains, or projects using different developmental approaches).

The results presented in this thesis refer to the initial results of a pilot case study. As such, the
case study design may not be optimal. Future studies will benefit from refinements derived
from the results obtained during this case study. These issues are discussed in Chapter 7.

6.2.5 Strategy and Approach

6.2.5.1 Type of Case Study

Case studies can be designed to serve a variety of needs (Yin 1984). This case study has been
designed to be predominantly descriptive, exploratory and analytical in nature. However, some
aspects of the study are evaluative. Evidence that supports the “what” and “when” questions
in Appendix E are typically descriptive. For example, the “What classes of Information
Products are generated during software development?” question provides information that
helps describe what is actually happening. The “how” questions typically relate to the
exploratory aspect of the study. For example, the question “How does the Software Product
Model evolve over time?” requires some exploration and analysis. The “which” questions are
mainly evaluative in nature. These types of questions are best answered when we have a good
knowledge and understanding of the domain, variables, and criteria involved. Since this
research is at an early stage, it is considered premature to consider many of the evaluative
aspects (ie “which is better”). Further studies will be designed to focus on these evaluative
aspects.

6.2.5.2 Use of a Modified GQM approach

The approach used in this case study is based on a modified Goal/Question/Metric (GQM)
approach (Basili and Rombach 1988; Rombach 1991). The GQM approach provides a
framework for defining metrics (ie quantitative evidence) based on a set of goals and questions.
Goals are defined in terms of the object of interest (eg a product or process), the objective or
task (eg to evaluate, to investigate), the focus (eg maintainability), the perspective (eg
customer, maintainer), and the context (eg the organisation) .

This thesis argues that the GQM approach provides a useful framework for supporting case
study design. It provides a structured and defined method for specifying goals, related

121
questions, and information needs. As shown in Section 6.2.8, the GQM approach also
supports the reporting, interpretation, and analysis of results.

The GQM approach focuses primarily on the use of measurement as the basis of investigations.
Although quantitative evidence is important for case studies, measures are only one form of
information that is needed. As discussed in Chapter 4, this thesis argues that not everything is
measurable and there are cases where we may not wish to describe an attribute through
measurement alone. Moreover, if the attribute of interest is not well understood, effective
measurement may not be possible since the mapping between the empirical and formal domains
may not be valid. The resulting measures may therefore be misleading.

The approach taken in the AMD/FCS case study uses a goal/question approach to help guide
the definition of primary information needs and the reporting of results. However, since the
case study is largely descriptive in nature, the approach considers a broad range of information
rather than specifically focusing on the use of metric information. This case study focuses on
capturing primary and supporting information. The primary information is defined in terms of
the questions that need to be answered. Supporting information is also captured to support
understanding, interpretation, and analysis of the primary information. For example,
supporting information may provide details of follow-up interviews conducted to help
understand particular trends observed in the primary information. This primary information
may have been captured by a measurement process. In this case study, primary information is
typically captured by automated tools and stored in electronic databases. Secondary
information is generally recorded as notes within a Case Study Folder.

6.2.5.3 Method

The case study method is defined in terms of activities that need to be performed. The method
used for this case study comprises the following main activities:

1. Establish goals and related questions. The case study goals and related questions are
defined in Appendix E.

2. Identify primary information requirements for the questions. This involves specifying the
primary information that needs to be captured to help answer the questions which have been
defined in terms of the case study goals. The information requirements for this case study
are discussed in Section 6.2.6.

122
3. Define, prepare, and update the experimental setup. The experimental setup which includes
an instrumented version of SEE-Ada and a range of measurement and analysis tools is
discussed in Section 6.2.7.

4. Define a set of procedures that need to be undertaken to conduct the case study. These
include procedures which need to be undertaken to manage the experimental setup,
procedures for capturing the required information, and procedures for analysing and
reporting results.

5. Execute the case study procedures.

6. Evaluate and update the case study approach for each incremental application of the case
study method. This thesis only reports on the first phase of the case study.

6.2.6 Establishing Goal-based Information Requirements

Appendix F provides an overview of the primary information requirements for this FCS case
study. They are defined in terms of the questions that need to be answered. The "Type"
column defines the representation of the information. For example "IP Classes" are
represented as a composite of information defined in terms of the IP classification scheme of
Section 4.2. The "Source" column identifies where the information will be stored and
managed. The various recording and analysis tools used for this case study (eg PIRMAS,
UMAS) are discussed in Section 6.2.7.6. These information requirements guide the design of
the experimental setup which is used to capture, manage, and store the information.

6.2.7 Experimental Setup

6.2.7.1 Overview

The experimental setup for this case study is shown in Figure 6-1. The setup mirrors the
DSEPF proposed in Chapter 4. This framework helps define the context for the case study.
For example, the SE process of interest is shown as the Technical Management Process. This
case study is specifically interested in those tasks that are performed by software team leaders.
Another aspect of the case study context to be considered is the Project Information Resource.
123
One of the goals of the study is to characterise and measure the persistent Information
Products that comprise the PIR. The PIR in Figure 6-1 highlights some of the key sources of
information available to software team leaders for the FCS project. Appendix D provides an
overview of the Rational Apex environment and the CASE tools used for the development of
the FCS software.

Technical
Team Leader Management
Tasks Process

Mediation Needs
IVDs
Use Context
Usage
Usage
Monitor
Monitor
Script Facililitation
SPM Direct
Mode Process
Scripts Use
SEE-Ada

IVES
PIRM
Filters Details

2167A AdaQuest APEX Software

Documents Environment Rose Development
Files

People
SODA Change
TestMate Records

Project Information Resource

LEGEND
Information Flow
Direct Interaction

Figure 6-1 Setup for Industry Case Study

Team leaders can gain direct access to PIR information or can use the integrated visualisation
approach to view needed information. The team leader gains direct access to PIR information
by way of the Direct Use process. This process involves the team leader accessing individual
Information Products either directly (eg viewing a document) or by way of an Information
Processor (eg using a CASE tool to access some underlying Information Product). The Direct
Use process is included in the experimental setup to help understand how PIR information is
used to directly support individuals engaged in specific SE tasks. This provides a basis for

124
comparison with the integrated visualisation approach which uses an IVES to integrate various
classes of information.

The SEE-Ada environment and a range of SEE-Ada filter tools provide the IVES
implementation for the case study. The filters which are used for this case study are discussed
in Section 6.2.7.4. These filters access and extract required structural and attribute
information from the PIR. This information is integrated into a descriptive model of the
Software Product (ie the SPM). The user interacts with the environment to generate
Integrated Visual Descriptions (IVDs). These forms of descriptive information allow various
types of information to be integrated and provided in a way that supports the user's needs. In
addition to accessing information through the filter tools, SEE-Ada can be used to directly
access a particular Information Product (eg a link to a source code file). Moreover, the user
can produce Information Products using SEE-Ada. For example, the user may print
information that is then inserted into one of the Software Development Files or produce 'value
added' attribute and relationship information. The use of SEE-Ada for recording and reporting
is discussed in Section 6.2.8.4.

The Facilitation Process is a manual process which involves an analysis of user information
needs and the context of use. These aspects are considered in relation to the information
available in the PIR and result in the preparation of scripts that control the SEE-Ada
environment to: (1) support setup and evolution of the SPM, and (2) support the provision of
custom descriptions based on specific task-based needs.

6.2.7.2 SEE-Ada

The SEE-Ada Version 3 environment serves as the main experimental apparatus for this
research. This environment is a significantly enhanced version of the environment used for the
initial trials discussed in Chapter 5. It incorporates the majority of IVES concepts, including
the Script Mode feature that supports the Facilitation Process. This version includes a Usage
Monitor which is used in the case study to capture a variety of usage information.

6.2.7.3 Usage Monitor

The SEE-Ada Usage Monitor is shown in Figure 6-2. The Usage Monitor can be used to
record the following information:

125
• The task being performed including the key objective of the user.

• User details.

• The date and time of the usage session.

• The actions performed and the time that it takes to perform particular actions.

• The types of information used for particular tasks.

• The representations and views that are used for particular tasks.

• The degree of mediation and facilitation (eg degree of customisation and adaptation) that is
undertaken by the user.

• User feedback on particular description criteria (eg understandability, meaningfulness etc).

SEE-Ada can be executed in a usage monitoring mode by setting the "-u" command line
switch. In this mode, the user must provide details of the usage session, by way of the Usage
Monitor Setup window, prior to using SEE-Ada. This window is shown in Figure 6-2. The
user selects the task to be performed from the existing task list or, if none are suitable,
generates a new task definition using the "Modify task names" feature. The user also provides
a text string which outlines the objective of the task. The system automatically records the
"User" details from Unix login records. This information can be edited if the login details and
actual user differ. The "Continued from:" selection indicates whether the task to be performed
is: (1) a follow-on from a previous task (in which case SEE-Ada information would already be
available as shown in Figure 6-2), (2) started from 'scratch' where no previous information is
displayed by SEE-Ada, or (3) commenced following the invocation of a script. This
information is automatically provided by SEE-Ada.

126
 Figure 6-2 SEE-Ada Usage Monitor

Figure 6-3 Monitoring Script and User Interaction

The user provides the required details and selects "OK" to invoke the Usage Monitor and
begin a SEE-Ada session. The Usage Monitor main window remains visible in the top right
hand corner of the screen to remind the user that the Usage Monitor is active. This window
also provides a range of control functions which are discussed in the following paragraphs.

When active, the Usage Monitor has no effect on the operation of SEE-Ada except that it will
periodically prompt the user. The user must then indicate whether the specified task is still
current. If the user changes to a new task, the Usage Monitor setup must be changed. This is
done by selecting the "Setup" button on the main window. The "Usage Monitor Setup"
window will then be displayed as shown in Figure 6-2. The user can then select a new task and
input the new objective. If the same task is to be performed with a different objective, the
current task identifier numeral is incremented.

The Usage Monitor produces Usage Log Files which record usage details. These are ASCII
files that can be filtered and imported into other tools for analysis. A file is produced for each
task completed by a user. The usage log can be viewed by selecting 'View Log' from the 'Log'
menu on the main window. Figure 6-2 shows a usage log for an evaluation task. The log has
a header which provides details of the task, objective, user, and the start date/time. The header
also indicates if there was a continuation (eg from a previous task), whether any windows were
open when the task started, and whether there was user interaction between the stopping of
one task and the continuation of another. The usage details are recorded following the header.
All actions undertaken by the user are recorded. In the example shown, the user has opened a
system called "Example", displayed the "CSCI_DESIGN" subsystem view, selected the
"Source_Instrumenter" subsystem with the mouse, and displayed a Layers View of this part of
the system. The detail on the Layers View was then reduced by using the 'Zoom' feature.

The user can insert comments into the Usage Log by selecting "Comments" from the "Log"
menu on the main window. For example, in Figure 6-2 the user has inserted comments in the
log to annotate the process that has been followed and to record results. For example, the first
part of the task involved evaluating the use of global variables. Global variables were
described by overlaying attributes from the "Code.Audit" group onto the package
specifications displayed in the Layers View. A list of the numeric results were then printed.
The unit with the highest number of global variables was then investigated by viewing its
source code. Further investigation was performed by superimposing relationship information
to identify those units that depend on the unit containing global variables. The user has then

128
inserted a comment to indicate that some action needs to be undertaken to rectify the problem.
The user then moves on to the next stage of the evaluation. This involves identifying the use
of 'GOTO' statements.

The Usage Monitor also captures script actions. These are actions that are performed
automatically by way of the Script Mode interface. Scripts can be used to provide custom
descriptions based on previous analysis of task needs. The integrated visualisation approach
allows the user to interactively adapt these descriptions to specific needs. As shown in Figure
6-3, the Usage Monitor can record both the script actions and user interaction to provide
details of this process. This example shows a script which is being used to support the
"Evaluate.Code.Coding_Practices" task. The Usage Monitor logs script actions and identifies
them in the usage log by a "+" sign. For example, the Usage Monitor Log of Figure 6-3 shows
that the script automatically opened the system "PROG_MET_ALL" and displayed a
subsystem. The user selected the design entity "Source_Instrumenter" as the focus of the
evaluation. The script then facilitated the generation of an IVD that would identify the use of
anonymous types to show those units that do not comply with this aspect of the development
standards. The user interacts with the environment to perform an investigation of relevant
source code files. The results of this evaluation step are then recorded by way of a comment.
The script then facilitates the provision of descriptions for the next step of the evaluation.

In addition to capturing task and usage details, the usage monitor can capture user feedback on
a range of effectiveness criteria as defined in Appendix B. This facility can be customised for
different investigations. This feature has not yet been used in the FCS case study.

6.2.7.4 Access and Filtering

A range of filters have been developed to access and filter information from the PIR. These
include:

• Rose Filter. Rose is an object-oriented design tool which supports the Booch method
(Booch 1991). The Rose filter accesses the underlying Rose design models to extract
information on logical design structure and design attributes. This information describes
the logical design in the Software Product Model (SPM).

• Rational Subsystems Filter. The Rational Apex Environment provides support for the
physical design of software systems. This approach uses Rational Subsystems as physical

129
 design constructs. These Subsystems provide a level of abstraction above Ada packages.
They encapsulate Ada compilation units and support the definition and control of interfaces
between sets of compilation units. The FCS design maps low-level Booch Class
Categories to Rational Subsystems. However, this translation is not directly supported or
verified by the environment. The Rational Subsystem Filter identifies Subsystem entities
and captures relationships between Subsystems and between Subsystems and related Ada
units. This information defines the physical design in the Software Product Model.

• Rational ASIS Filter. The Ada Semantic Interface Specification (ASIS) (Barnes 1993) is
a standard programmatic interface to Ada compilation libraries. This interface allows
access to the underlying semantic and syntactic information of the Ada implementation.
The Rational ASIS Filter uses this interface to extract information about Ada code entities
of the system under development. This information includes a range of attributes (eg
name, type, source code reference) and various relationships between the code entities
including encapsulation, with dependencies, and calls. This information describes the Ada
implementation in the Software Product Model.

• Configuration Management Filter. This filter extracts configuration management

information from the Apex Configuration Management and Version Control (CMVC)
system. This information provides a range of attributes for the various Software Product
entities. Examples include author and users names, the date an entity was last changed, the
amount of change that the entity has undergone, the latest version, and configuration status
(eg Checked In, Checked Out), and release details.

• Product Metric Filters. These filters provide a range of product attributes including size,
degree of commenting, complexity, use of language features, and quality characteristics. A
primary source of this type of information for the FCS project is a commercially available
product measurement tool called AdaQuest (AdaQuest 1995). Product attributes are also
provided by filters that generate measures based on the structure of the software product as
represented in the SPM. The SEE-Ada Structure_Measures filter is used for providing
information about the code structure (eg degree of encapsulation, fan in/ fan out
measures), and language feature usage (eg task usage, use of “use” clauses). The
Subsystem_Allocation filter provides attributes which show design/code relationships.

130
• TestMate Filter. This filter allows access to test information generated by the TestMate
testing tool. A range of attributes are accessed including specific test results, test coverage
information, and details of test cases.

Other information can be accessed and imported into the SEE-Ada environment from the
sources of information that cannot be electronically accessed or where it is inefficient to do so
(eg from paper-based Software Development Files, deliverable documents, or people).
Features of SEE-Ada which allow the capture and updating of this information are:

• Create/Modify Attributes Feature. This is a SEE-Ada feature that allows attributes to

be generated in an interactive way through, and in relation to, the SEE-Ada views. This
feature can be used to capture additional knowledge that developers may have about the
product. For example, these could include indicators which identify sections of code that
are difficult to understand, knowledge about the reuse potential of particular code units, or
information about the effort and time taken to develop a particular section of code.

• Subsystem Edit Feature. This feature allows for manual entry and editing of design-level
information.

6.2.7.5 SPM Capture and Update Mechanism

As discussed in Appendix D, the FCS development process is based on an incremental release

approach. The physical design is partitioned into a series of Computer Software Units (CSUs)
which represent separately testable elements of code (DOD-STD-2167A 1987). CSUs
typically map to a single Ada library unit but in some cases can be mapped to a small set of
related library units. The development process specifies that a CSU will be 'released' after it
has been evaluated and tested. The evaluation is carried out by the team leader responsible for
the particular build. These released units provide the stable software components that are
integrated into the evolving product. This approach provides a basis for effectively managing
product evolution and monitoring progress.

The Rational Apex environment provides support for this incremental release strategy. The
Ada units are released from working views into a developmental release tower in the Rational
Apex environment. The release tower comprises a set of release views, one for each Rational
Subsystem (or physical design entity). A capture and update approach has been developed to
take a snapshot of the release tower. This information can be used to automatically generate a

131
SPM in SEE-Ada. This then provides a basis for describing the evolving product at a
particular point in time. A series of SPMs could be used to describe product evolution.

The update mechanism uses physical design information extracted using the Rational
Subsystems Filter and information on code structure obtained through the Rational ASIS
Filter. This information then forms the basis of the SPM for the system. The Script Mode
Interface is used to automatically generate the new system for the current snapshot. In
addition to the physical design and code structure, logical design structure (which can be
obtained from Rational Rose) is also added to the SPM during this modelling and integration
phase. The final part of the update process is to attribute the structural model with appropriate
information. Various filters are automatically invoked to access and filter required
information. Information that does not change between snapshots is translated to the updated
model.

Each snapshot is accessible as a SEE-Ada system. For example the snapshots are labelled with
the system name and date of the snapshot. The system named "FCS_951003" would be the
snapshot taken on 3 October 1995. Initial discussions with team leaders suggested that weekly
updates were considered appropriate, at least during the initial phases of the project. The
update interval and update criteria are research issues that need to be addressed.

6.2.7.6 Recording and Analysis Tools

The experimental setup includes the following recording and analysis tools which have been
developed using Microsoft Access:

• PIR Modelling and Analysis System (PIRMAS). This tool provides support for
classifying and modelling Information Products which are part of the Project Information
Resource. It also stores information about specific Information Product instances. This
information is captured by a series of tools which identify and measure the actual
information products. The information captured includes the size of products and the date
they were created. This information is stored in relation to the PIR model. It can be
viewed from a number of different perspectives. For example, the information can be
viewed in terms of the its type, representation, form, primary user, and provider. PIRMAS
information is used to help gain an understanding of the diversity and quantity of
information produced during software development. It also provides a means of defining

132
 those products that need to be accessed and filtered for the set up of the SEE-Ada
environment.

• Usage Monitoring and Analysis System (UMAS). This tool is used to store and manage
information captured with the SEE-Ada Usage Monitor. UMAS has features which allow
analysis and viewing of usage information.

• SPM Measurement and Analysis System (SMAS). The SMAS tool captures information
which can be used to monitor Software Product Model evolution. An interface to the SEE-
Ada database allows SMAS to capture information on the number and types of entities
which comprise the SPM at a particular point in time. Information on the relationships
between these entities and the types and numbers of attributes is also captured and stored.

6.2.8 Results
This section provides a summary of the results obtained for the first phase of the FCS case
study. The results are presented in terms of the goals and research questions defined in
Appendix E. Since the results only refer to the first incremental build of the FCS software,
some questions cannot be fully addressed. The case study has captured and recorded
considerable amounts of information. This section only presents those results that are required
to address the conjectures and issues defined in this thesis. Although this section provides
some discussion of the results, most of the discussion is reserved until Chapter 7 where the
case study results, together with other results, are discussed in terms of the issues defined
Chapter 2.

6.2.8.1 Goal 1: Availability of Project Information

6.2.8.1.1 Modelling of PIR

To help answer Q1.1, a model of the PIR for the FCS project was developed by identifying
classes of persistent Information Products. Some classes were identified by analysing the
Software Development Plan to identify deliverable and developmental Information Products.
Other classes were defined by analysing and modelling the tools and environments used by the
project (eg Rational Apex, Rose, TestMate). Interviews were conducted to identify additional
classes of information products (eg company specific products). The PIR model was updated

133
during the project to include classes that were not identified during the preliminary analysis and
modelling activities.

The classes of persistent Information Products identified for the FCS project are provided at
Appendix G. These are defined in terms of the classification and modelling scheme discussed
in Section 4.2. The modelling activity identified 161 different types of information, 67
representation approaches, and 30 forms. The types of information were in turn grouped into
the following 14 categories: code, design, development, integration, measure, monitoring,
plan, policy, product, requirement, review, safety_issues, test, and versioning. These
categories resulted from the class groupings which were identified during the modelling
process. Figure 6-4 shows a breakdown of IP classes by type category.

versioning 32

test 6

tasking 1

safety_issues 1

review 3
Class Categories

requirement 3

product 1

policy 2

plan 7

monitoring 3

development 6

design 15

config_mgt 3

code 19

0 5 10 15 20 25 30 35

Total

Figure 6-4 Total IP Classes by Class Category

The results of this PIR modelling activity show the diversity of information that is produced
and used by this software project. The classes of Information Products highlight some
interesting characteristics of the project. For example, 32 versioning classes were identified for
the FCS project. Versioning classes provide core or underlying information for a project. The
versioning classes for this project show that a wide range of information is kept under version
control including budgets, design models, plans, requirements, schedules, and source code. The

134
number of versioning classes shows that the project relies heavily on information in electronic
form.

Although categorisation by type may be useful for presenting the PIR model and associated
measurements, there are also other perspectives that may be important. For example, there
may be a need to show which IP classes relate to Software Development Files or which classes
constitute deliverable information. A perspective which shows the proportions of information
held in electronic and paper forms may be of interest for other analysis activities. PIRMAS
provides a flexible reporting mechanism which allows the PIR model to be viewed from a
number of perspectives.

120

100

80
Classes

60

40

20

0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Snapshot

Figure 6-5 Total IP Classes Over Time

6.2.8.1.2 PIR Measurements

To help answer Questions 1.2 and 1.3, instances of information products were captured at
weekly intervals (referred to as snapshots). The results provided in this thesis cover 24
snapshots which were taken over the period 6 September 1995 to 7 March 1996. Figure 6-5
shows the total number of IP classes identified at each snapshot. The early snapshots (1-3)
show a rapid increase in the number of classes. This was largely due to effects resulting from
the initiation of the measurement process. The classes of information available during these
early snapshots included design models, requirements specifications, design notes, and plans.
Snapshot 6 coincided with the completion of the Preliminary Design Review (PDR). Since the
project was preparing for the PDR, a number of new IP classes emerged as documents and
135
design models were baselined. Most new activity ceased in the week prior to and during the
week of the PDR (snapshots 5 and 6).

Each IP class can comprise many actual Information Products (or instances). Figure 6-6
shows the total number of instances identified at each snapshot. This graph shows that the
number of IP instances increased slowly until PDR. Immediately following PDR, activity
increased dramatically. For example, in the week between snapshot 6 and 7, twenty new IP
classes were identified. There was a corresponding rapid increase in the number of class
instances. For example, during this period the number of IP instances increased from just over
1000 to more than 1500. This period also saw the Rational Apex environment configured for
code development. Many new classes of Information Products became available, including
those associated with the subsystem structure, working views, code libraries, and compilation
control files. Detailed design activity produced a range of new design instances. These
included many new design entities and scenarios. The Rose code generation facility began to
produce source code following snapshot 7.

6000

5000

4000

3000

2000

1000

0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Snapshot

Figure 6-6 Total IP Instances Over Time

The number of IP instances continued to increase as development activity continued through to

snapshot 15. The number of new classes increased in line with new phases of activity. For
example, the period between snapshot 13 and 14 saw the inclusion of test routines that had

136
been developed separately. Snapshot 15 corresponded with the start of the Christmas period.
Few new IP instances were produced until snapshot 18. A large proportion of the new
instances between snapshot 18 and 20 were Information Products produced by the compilation
system (eg DIANA code). There was also a significant amount of review during this period.
The period between snapshots 20 and 21 saw a large drop in the number of IP instances. This
was due to a 'clean up' of compilation libraries and working views. The period between
snapshots 23 and 24 shows few new IP instances since the project was undergoing
consolidation and internal review.

PIR measurements are stored and managed by PIRMAS. This information can be presented
for analysis in a variety of graphical and tabular forms. For example, Figures 6-5 and 6-6 are
charts that were provided by PIRMAS. These charts are useful for gaining a general overview
of the diversity and quantity of information at different stages of development.

versioning 2,664

test 368

safety_issues 1

review 147

requirement 64
Class Categories

policy 74

plan 11

monitoring 1

integration 9

development 110

design 290

config_mgt 11

code 1,438

0 500 1,000 1,500 2,000 2,500 3,000

Total

Figure 6-7 Instances by Class Category for Snapshot 24

PIRMAS can also provide a breakdown of IP instances for a range of class categories of a
particular snapshot. Figure 6-7 shows the total number of instances for the various class
categories for snapshot 24. These can be further broken down to show specific classes for
particular categories. For example, the design instances are due to several classes including
Rose design model, design notes stored in paper form in Software Development Files, and the

137
drafts of the deliverable design document. The design notes relate to various levels of design
and comprise 182 instances and a total of 1752 pages of information.

The versioning information comprised by far the greatest number of Information Products (ie a
total of 2,664 instances). This represents the totality of information under version control.
The versioning category of Information Products includes the electronic versions of documents
(stored as Wordperfect files), source code, test code, requirements, schedules, and budget
information. A major feature of the FCS development environment is that a high proportion of
the information is in electronic form and is managed by way of the Apex Configuration
Management and Version Control (CMVC) system. The information recorded in PIRMAS has
proved valuable for monitoring change in various classes of information.

300

250
Release
200 Versioning
Instances

Working
150

100

50

0
7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

Snapshot

Figure 6-8 Product Source Code Instances

PIRMAS can also provide details of specific IP classes. For example, Figure 6-8 shows the
number of instances of product source code classes (ie the source code that is part of the
product, not including test or prototype code). The graph has been designed to show three
source code classes. It shows the number of editable source code instances in working views,
the number of source code files that are under version control, and the total in release views (if
any were present).

The working view instances include source files that have been 'checked out' of the
configuration management system for editing and files that may never have been put under
version control. This graph shows some interesting characteristics. For example, following
the PDR, Rational Rose was used to automatically generate code from the design models.

138
This code was initially placed in working views. Approximately 125 source code files were
generated within a week. A proportion of these were put under version control.

As time progressed, the number of files put under version control continued to increase (as
would be expected). However, the number of editable files in working views remained high.
This was cause for some concern since one would expect that all source files would be under
configuration management. The 'working' files would comprise only that small subset of files
which were actively being edited. Investigation highlighted the fact that some developers
considered the Rose generated code as templates only. They would not put this code under
configuration management until they felt that the code was sufficiently complete to be classed
as production code. In other cases, developers would forget to check code back into the
configuration management system.

The graph also shows that there are no source files in release views thereby indicating that the
incremental release process had not been enacted. This problem is discussed in Section
6.2.8.3.2. Although the PIR information was captured and analysed for research purposes, it
has also helped the FCS team leaders gain new insights into the project and identified potential
problems with the SE processes.

6.2.8.2 Goal 2: Use of Descriptive Information

6.2.8.2.1 Definition of Team Leader Tasks

Appendix H shows an initial classification and definition of team leader tasks. These were
defined through work practice observations, interviews with team leaders, and definitions
provided by ISO/IEC_12207-1995. Analysis showed that tasks could be specified in various
ways and at different levels of abstraction. For example, a review task may require the
individual to undertake a range of lower-level tasks including evaluation, analysis,
investigation, and demonstration. Moreover, there is often little consensus as to the definition
of specific tasks. For example, higher-level managers generally consider reviews in terms of
formal meetings between customers, suppliers, and developers. Team leaders often consider
reviews in terms of product inspections. This research has attempted to capture and verify
definitions of primitive task categories which could be used to help specify the types of tasks
that team leaders undertake.

139
The SEE-Ada Usage Monitor was used to help capture and verify task classifications. The
Usage Monitor provides the user with a set of tasks definitions and provides a means of
updating task lists. Users were asked to update task lists with new definitions if none of the
tasks shown were suitable. In the investigations to date, there have been few instances where
users have updated task lists. This was partially because team leaders were involved in
developing and reviewing the initial definitions. Users were also more interested in 'getting the
job done' rather than trying to accurately specify the tasks that they were performing. They
found it easier to select one of the pre-existing tasks than to provide a more accurate
definition. Issues of task definition and classification are discussed further in Chapter 7.

6.2.8.2.2 Use of Descriptive Information

Questions 2.3 to 2.5 refer to how team leaders use descriptive information directly from the
PIR. This section discusses the preliminary results of observing and interviewing FCS team
leaders in relation to a subset of tasks specified in Appendix H. The aim of this part of the case
study was to gain insights into how team leaders use various PIR Information Products when
undertaking particular types of tasks.

Team leaders are required to monitor product development as a basis for reporting project
status, control, and assessing developmental risks. This requires the team leader to use
information that describes the status of the evolving product. As discussed above, vast
amounts of information can be produced during a software project. Observations showed that
although considerable information was available, team leaders had difficulty knowing where
the information was and how to access the information. For example, to monitor the status of
product development, team leaders needed access to information such as the size and number
of source code files produced, size of design models, and the number of test cases produced.
However, to do this, the team leader needed to know where the information was and how to
access it. Even after accessing this information, there was no guarantee that it would be in a
form that would help answer specific questions.

The results of PIR modelling and measurement helped provide consolidated information (eg
information how as to the design models or source code files changed over time) but team
leaders found that this information may not accurately describe progress. For example, simply
knowing how much information is being produced over time or how much it changes does not
indicate that progress is being made. Team leaders expressed the need to have information

140
which would then let them assess whether requirements were being met, the degree of
completeness of design and code, and whether the units had undergone inspection, test etc.
The investigations showed that improved visibility was needed for these types of tasks.

In addition to the monitoring tasks discussed above, team leaders were responsible for a range
of inspection and evaluation tasks. For example, one such task involved ensuring that
requirements had been allocated to appropriate design components and that all requirements
had been addressed in the design. Information on the allocation of requirements is used to
support this inspection task. As part of the FCS design process, designers insert requirements
allocation information as text strings into the Rose design models for individual components.
This information is also printed and inserted into the design notes section of the Software
Development Files. The SDF information is then used by team leaders as the basis of the
requirements allocation inspection tasks. Observations showed that team leaders can access
each individual piece of required information, but find it difficult to access and assimilate
collections of information to answer questions such as: Which requirements are not allocated?
and, Which design entities do not have requirements allocated? Although particular allocations
can be considered for individual design entities, this information cannot be viewed from a total
product perspective.

Team leaders are also required to ensure that the software product complies with company
development standards. Tasks need to be performed to evaluate the software based on the
requirements defined in the standard. The source code files are the primary source of
information for these tasks. Team leaders evaluate each source code file in relation to the
standards. This is a very time consuming and error-prone practice. As part of this case study,
company development standards were analysed to determine information needs for evaluation
tasks. This analysis found that evaluations needed to be performed in the areas of architecture,
multi-processing, error handling, initialisation and shutdown, and a variety of coding practices
including naming, layout, commenting, and language feature usage. A total of 94 separate
criteria needed to be considered as part of an evaluation. Team leaders found it difficult to
remember what the criteria were, let alone considering each of them while reviewing each
source code file. Moreover, some criteria could only be considered for sets of files (eg
interfacing and control criteria). The use of source code files as the basis of these evaluations
does not help answer many important questions such as: How widespread is a particular

141
practice? Is it a general problem or attributable to a particular person? What is the potential
impact on other criteria?

Team leaders also perform a variety of 'ad hoc' analysis and inspection tasks. These are
typically exploratory and unstructured (ie no predefined criteria to evaluate). For these tasks,
team leaders need access to a wide range of information. Timeliness is also important since a
particular item of information can result in several new hypotheses and questions that need to
be answered. For example, when analysing a design description, the team leader might wish to
quickly see how a particular feature is implemented in a source code file. The team leader may
then wish to know the status of this element of the code (eg Has it been released? When was it
last changed?). To do this, the team leader needs to know what information is available, where
it is, and how it can be accessed. The direct use of PIR information did not effectively support
these types of tasks.

Although these investigations provided insights into how team leaders use information for
particular types of tasks, more work needs to be done. A major limitation of this part of the
case study relates to the difficulty of accurately defining tasks and then capturing what
information is actually used. Problems related to task analysis are discussed in Chapter 7. The
use of the PIR model and related Information Product measurements may prove useful for
more accurately capturing information use. The model would need to be extended to support
a range of transient IP classes to support this approach.

6.2.8.3 Goal 3: Setup and Support Requirements

6.2.8.3.1 Access to and Filtering of Information

The Software Product Model (SPM) is a composite model comprising structural aspects of the
system and related attributes. The structural aspects that needed to be provided for the FCS
product model included three main classes of entities and their relationships. These were the
logical design entities (or class categories) obtained from Rational Rose (IP Classes 38 - 41
and 45), the physical design entities (or Rational Subsystems) obtained from the Rational Apex
design facility (including IP classes 18, 19, 33, 34, 119), and the Ada code entities which were
accessed through the Apex ASIS interface (IP Classes 15,16).

The composite SPM is generated by adding attribute information to the structural model. The
types of attributes to be added are dependent on the types of tasks which SEE-Ada will

142
support. Appendix I provides a summary of the attribute information that was made available
for the FCS project during the first phase of this case study. Attributes are stored and
presented in terms of attribute groups. These groups are defined in terms of the types of
information provided (eg code, configuration management, design, requirements, and test).
Attributes groups can be numeric or symbolic. Symbolic attributes are represented as text
strings. Numeric groups can encapsulate a set of attributes. Each attribute has a set of values
associated with it. The table in Appendix I also shows the filter tool that is used to provide the
information for SEE-Ada and the Information Product classes that provide the underlying
information from the PIR. The PIR model captured in PIRMAS proved useful for identifying
Information Products to be accessed and their filtering requirements.

The attribute values for several of the "code" groups were provided by the AdaQuest product
measurement tool (AdaQuest 1995) and filtered by the "AW2SA" filter tool. For example, the
"Code.Audit" group contains 45 different attributes. These attributes describe the degree of
compliance with recognised good development practices as specified in the Ada Quality and
Style Guidelines (SPC_Guidelines 1991). The attributes of the "code.profile" group describe
code feature usage (eg use of unchecked deallocation, interfaces to other routines written in
other languages, etc). The "code.quality" group provides code quality attributes in terms of
the Rome Laboratory Software Quality Framework (Bowen et al. 1985). The "code.size"
group describes the size of code components in terms of the Software Engineering Institute's
size measurement framework (Park 1992).

Some of the attributes were generated by accessing and processing information contained in
the underlying SEE-Ada database. For example, the "Structure Measures" filter was used to
measure the underlying SPM to provide measures based on structural features and
relationships (eg fan in and fan out of library units). Many of the attribute values are extracted
directly from the Apex environment. For example, the "VCTRL2SA" tool extracts
configuration management information from the Apex Configuration Management and Version
Control system. Physical design attributes are extracted from Apex Information Products by
way of the "IMPEXP2SA" filter. A major benefit found in using the Apex environment as a
source of information was that the majority of its Information Products are stored as Unix
ASCII files. As such, access and filtering tools can be readily developed using many of the
Unix text processing tools (eg 'grep', 'awk') and text processing languages such as Perl (Wall
and Schwartz 1991).

143
Although most attribute values were captured automatically, there was a need for manual
input. For example, customer requirements were defined in the Software Requirement
Specification (SRS). However, the SRS did not contain all the information that was needed.
Unique requirements identifiers and allocation mappings were provided in the Software
Development Files. As such, manual assimilation of the information needed to be undertaken
to provide the consolidated attributes that users needed. The SEE-Ada "Create/Modify
Attributes" feature was used to support the capture of this information. The set of
"requirements" attribute groups shown in Appendix I reflect the requirements groupings in the
SRS. A set of derived requirements was also defined. These requirements are specified in the
"AWADI" group and identify a range of additional technical requirements that need to be met.
Although some tasks require access to particular sets of attributes, observations of SEE-Ada
usage also showed that there needed to be consolidated sets of attributes (eg the
"Requirements_MMI_All" group. These consolidated attribute groups are produced
automatically through the "mrg_attr_gps" tool.

A range of SEE-Ada systems was produced to support the needs of team leaders on the FCS
project. These included systems that described code that had previously been developed and
was being considered for reuse. A SEE-Ada system which described the code for a Man
Machine Interface prototype was produced and used to support analysis tasks. Various
systems based on working views were produced to support analysis and evaluation tasks. The
systems were set up to include those entities, relationships and attributes of interest for the
tasks being undertaken.

6.2.8.3.2 SPM Evolution and Information Update

One of the aims of the case study was to determine whether an evolving composite model of
the overall Software Product (as shown in Figure 4-8) could be used to provide a common
point of reference for the project by providing indirect visibility of product, process, and
resource characteristics as the project progressed. It was thought that this approach would be
useful for project monitoring tasks and for providing a common definition of the evolving
product for the project (eg to support communication between managers, customers, and
developers). This was to be tested by generating a set of SEE-Ada systems which
corresponded to snapshots of the evolving FCS software product.

144
As discussed in Section 6.2.7.5, the development process for the FCS project is based on the
incremental release of code components into a Rational developmental release tower. Product
snapshots would show those elements of the product that were complete and stable.
Unfortunately, during early code development, the incremental release process was not
enacted. This was because project personnel were still gaining experience with the Apex
environment and the new tools. The use of these tools also had a major impact on the
development process. For example, problems were being experienced in the integration of
tools such as Rose (which was being used for code generation) and the Apex environment
which provided the developmental infrastructure. The inclusion of new project personnel who
were not well versed in the release process compounded the problem. These problems have
now been largely addressed, however they have had a major impact on initial phase of the case
study. Much valuable data on early product evolution could not be captured.

Due to a lack of an incremental release process, project and product status could not be
effectively monitored. As Figure 6-6 shows, there was a rapid increase in the number of
information products produced following PDR. The PIRMAS information indicated that a
large number of source code files were being generated by Rose and placed in a tower of
working views called "Code". These code fragments were then further developed using the
Apex development tools. At the same time, the PIRMAS information showed that the design
models were undergoing considerable change. Since the available information did not provide
sufficient or useable information to support the monitoring and inspection of development
activity, the SEE-Ada capture and update mechanism was used to provide visibility of product
development based on the "Code" tower. These interim systems are being used until the
release process is properly enacted.

The use of the "Code" tower has helped test the SPM evolution process. Weekly snapshots of
the developing product were automatically generated. This involved capturing current design
and code structure and updating the attribute information. The result is a composite model
that includes the logical design entities, the physical design, and the Ada entities. The set of
attributes shown in Appendix I were provided. Where appropriate, filters automatically
generate up-to-date information. Other information that had not changed, was transferred to
the new model.

145
6.2.8.3.3 Consistency of Information

Consistency of project information can be a problem. For example, the Rational Rose and
Apex products are not well integrated. Rose provides the logical design in terms of class
categories while Apex implements the physical design as defined by Rational Subsystems. The
mapping between the two is largely human-based and not recorded in the Apex environment.
Naming conventions provide some insight into the mappings, however most of the information
resides with individual designers. To ensure accuracy and consistency of these interfaces, the
SPM relationship between logical and physical design entities needs to be provided by those
who have a deep knowledge of the product architecture (ie designers). The Subsystem Edit
feature of SEE-Ada has been used to capture this information. Once the relationship has been
established, this information is translated to future evolutions of the SPM. If new physical
design entities emerge, the need for this additional information is identified by SEE-Ada.

Consistency of other information is automatically checked as the model evolves. This is

achieved through the Information I/O element of the SEE-Ada environment. Error logs are
produced for those elements of information that are presented for integration but are not
consistent with unique entity identifiers.

6.2.8.4 Goal 4: Use of an IVES

The SEE-Ada Usage Monitor was used to capture various aspects of SEE-Ada usage. The
usage monitor provided Usage Logs for each session. These were processed and key
information was recorded in the UMAS database. UMAS allows usage information to be
presented in a variety of tables and graphs. It also allows direct access to an electronic copy of
the usage log. These usage measurements were also supplemented with information based on
observations made during usage sessions. These were recorded in the Case Study Folder.

Figure 6-9 provides a task usage profile for the usage sessions that were captured during this
first phase of the case study. The largest amount of recorded usage is related to the
"Inspect.Design_Structure" tasks which account for some 10 hours of total usage. There are
also inspection tasks that deal with requirements allocation and code. Evaluation tasks,
particularly those that relate to coding practices, also feature prominently in these early results.
SEE-Ada has also been used for recording design and attribute information. As previously
discussed, SEE-Ada provides features which allow users to manually update the SPM. The

146
"Record" sessions shown in Figure 6-9 updated design relationships and requirements
information. The "research" and "test" tasks are those used by DSTO.

Demonstrate
Analyse.Design.Dependencies
Evaluate.Code.Architecture
Inspect.Design
Familiarisation.Product.Structure
Research.Test
Analyse.Code.Dependencies
Evaluate.Design
Analyse.Design
Demonstration
Test.Usage Monitor
Record.Design.Changes
Recording.Attribute Info
Demonstrate.SEE_Ada.Usage
Task

Review.Requirements Allocat
Inspect.Code.Configuration
Research.Capture Info Req
Research.Experimental_Setup
Familiarisation.SEE_Ada.Usage
Familiarisation.Of Attribute Types
Familiarisation.Of SEE-Ada
Monitor
Evaluate.Code.Coding_Practices
Record.Design.Features
Inspect.Code
Monitor.Configuration_Status
Inspect.design.requirements_allocation
Inspect.Design.Structure

Relative Time

Figure 6-9 Usage by Tasks

Usage observations showed that accurate recording of usage by task is difficult, even when
task lists are provided. For example, although a user may have initially intended to perform a
monitoring task, other tasks might have also been performed without changes to the task
identifier or task objective in the usage monitor. Observations during one usage session
showed that the initial intent of the user was to monitor the configuration status of a section of
code. However, the task quickly changed to an inspection task when the user observed an
interesting abnormality. Although the usage monitor would alert the user periodically to ask
whether they were still performing the same task, users were more focused on performing the
task than worrying about accurately recording the task being performed. As such, the
'continue' button was generally the preferred option. This situation improved when the author
was present and could remind users to more accurately consider the tasks being performed. As
such, the task identifier information captured in the usage logs can only provide an
approximate indication of the types of tasks being performed.

147
UMAS is able to provide usage information for particular usage sessions. For example, Table
6-1 provides a usage profile for several sessions. The table shows the task performed, the user
details, the total time of the usage session, the number of different views used, the number of
attributes used (Attrs), the number of relationships used (Rels), the amount of user interaction
(U_Act), the number of user comments (Com), the number of screen prints requested (Prints),
the number of scripts used (Scripts), and the number of actions performed by the scripts
(S_Act).

The user and script interaction refers to the direct manipulation actions that are performed.
For example, these include selecting particular entities on the screen, customising display
groups, and tailoring the view (eg by zooming). The following paragraphs discuss the usage
profiles in relation to the types of tasks that were performed.

6.2.8.4.1 Monitoring Tasks

As previously discussed, SEE-Ada systems based on the "Code" tower were used to provide
visibility prior to the release process being invoked. This proved important to team leaders
since it provided an overall view of the status of the product (including design and code
entities). Several problems were identified and recorded. For example, one usage session
highlighted the fact that there was significant development activity in areas that were not part
of the first incremental build. The team leader was able to redirect effort into key areas of
interest. The monitoring sessions were also able to identify those areas where the
configuration management process was not being properly enacted. In these cases entities
were displayed yet were devoid of configuration management attributes.

148
Task ID Task Definition User Time Views Attrs Rels U _Act Com Prints Scripts S_Act
1 Monitor cking 00:27:35 4 0 4 41 5 2 0 0
2 Monitor.Configuration_Status cking 00:50:08 3 16 0 28 2 2 0 0
3 Monitor.Configuration_Status cking 00:27:40 2 3 3 0 0 0 0 0
4 Inspect.design.requirements_allocation rdolan 02:14:52 1 13 0 37 7 2 0 0
5 Inspect.Code cking 00:21:00 3 19 4 0 1 0 0 0
6 Inspect.Design.Structure cking 00:45:09 3 6 0 32 2 1 0 0
7 Inspect.Code cking 00:34:53 3 15 0 38 3 1 0 0
8 Inspect.Code cking 00:13:23 3 0 0 10 2 1 0 0
9 Inspect.Code.Configuration jparsons 00:48:03 4 0 0 0 0 0 0 0
10 Inspect.Code rdolan 00:26:38 3 0 0 11 0 1 0 0
11 Inspect.Code cking 00:26:51 3 6 0 0 0 0 1 6
12 Record.Design.Changes cking 00:22:33 2 0 0 15 0 0 0 0
13 Analyse.Design rdolan 00:22:04 2 0 0 3 1 1 0 0
14 Evaluate.Code.Coding_Practices cking 00:18:53 3 2 0 0 2 0 1 4
15 Evaluate.Code cking 01:35:59 3 23 0 27 6 1 1 17
16 Evaluate.Code.Coding_Practices cking 01:49:48 4 30 2 0 37 0 1 24

Table 6-1 Details of Usage Sessions

Three "monitor tasks" are shown in Table 6-1. In Task 1, the user was interested in
determining which Ada units were available in particular subsystems and the degree of
completeness of the source code. The session involved quickly moving between the
subsystems view, the layers view, and particular source code files. The high number of user
actions (U_Act = 41) is attributable to this highly interactive mode of operation. Part way
though this monitoring task, the team leader identified a potential structural problem in the
code implementation. The task moved into an 'analysis' mode where the user began to use
'with' relationships and a contains view to help understand how this section of the code was
structured.

The number of comments provided by the user for the various tasks shows an interesting trend.
For example, the user in Task 1 inserted 5 comment lines into the usage log. These comments
refer to the problems found. Although the Usage Monitor was developed to support research
objectives, the users found that it provided an effective way of recording the results of a
session. They expressed that a key benefit of using the Usage Monitor in this way was that the
problems and observations could be recorded in context (ie the problems were recorded in
relation to the information which was used to identify the problem, such as the attribute
threshold, and the entity being observed). The commenting feature was subsequently enhanced
so that comments could be classified in terms of whether they referred to an observation, a
question that needed to be answered, or a problem that needed to be rectified.

Task 2 refers to a monitoring activity which used the configuration management attributes to
identify developmental status and trends. This task also shows a significant amount of user
interaction with the visual representations. This interaction was mainly related to the selection
of entities from a layers view to inspect their source code descriptions.

6.2.8.4.2 Inspection Tasks

The IVES concept proved to be particularly useful for inspection tasks where users needed to
quickly gain access to information based on previous observations. These were largely
unstructured tasks where users were interested in looking at particular characteristics of the
product. The objectives for some of these tasks were defined as "snooping".

A particularly productive session was Task 4. This task reviewed the allocation of
requirements to logical design entities. SEE-Ada was used in two modes. Mode one involved
selecting particular requirements and having SEE-Ada highlight those entities that were to

150
implement the requirement. This identified several inconsistencies and resulted in intense
discussions regarding design principles and rationale. In the second mode of operation, an
entity (or small set of related entities) would be selected, and SEE-Ada would provide a list of
those requirements that were satisfied by these entities. This proved useful for assessing the
diversity of requirements which were to be implemented by particular entities. It also identified
entities that had no requirements assigned.

Task 6 looked at the relationship between the design and code. It identified consistency
problems between the logical and physical design. This would have been difficult to identify
without automated support of the type provided by SEE-Ada and the update facility. Apex
does not have a central model of the overall software product. The logical design is managed
by Rose and the physical design is managed within the Apex design facility. The update facility
captures the subsystem structure in Apex and imports it into SEE-Ada. Any physical design
entities that are not defined in the SPM are shown as 'unattached' in the Subsystem view. As
such, any inconsistencies can quickly be observed (eg subsystems not related to the logical
design, inconsistent naming). These inconsistencies can then be explored within SEE-Ada.

6.2.8.4.3 Evaluation Tasks

Evaluation tasks were identified as those tasks which determine the extent to which an entity
meets a specified criteria. There are a wide range of evaluation tasks that need to be
performed in software projects. A large number of these refer to the development standards
being used. Code evaluations are currently a key area of interest on the FCS project. As
discussed in Section 6.2.8.2.2, manual code evaluations can be extremely time consuming and
error prone.

An analysis of the AWADI development standards was undertaken to determine the criteria to
be evaluated for the project. These criteria were mapped to information needs. Further
analysis then identified those attributes available in SEE-Ada which could be used to evaluate
specific criteria.

These analysis activities showed that the attributes which were made available in SEE-Ada
could be used to help evaluate the majority of the criteria. It also identified those areas where
new attributes needed to be provided. In some cases, more than one attribute needed to be
used to evaluate the criterion. Observations showed that individuals who use SEE-Ada in its
fully interactive mode for these types of tasks need to have an expert knowledge of the

151
environment and the attributes provided. Moreover, a well defined approach needs to be used
to ensure completeness and accuracy of results. Recording is also an important issue since
these types of evaluations are auditable and are inspected by the customer quality assurance
representative. As discussed in the next section, these issues can be addressed through the
combined use of the Script Mode interface and the Usage Monitor

6.2.8.4.4 Provision and Adaptation of Custom Descriptions

The SEE-Ada Script Mode interface was used to help enact the evaluation process and to
provide custom descriptions. Table 6-1 provides examples of evaluation tasks that were
undertaken using scripts (ie Tasks 14, 15, and 16). Scripts were developed based on the
criteria and mappings defined during the analysis of the AWADI development standards. This
information was used by an experienced user to perform the evaluation using SEE-Ada. The
usage monitor captured the actions that were performed. These actions are expressed in the
same language that is used for SEE-Ada Script Mode. The usage logs were then converted
directly to scripts. These scripts were then run on a series of tasks. Observations and an
analysis of usage data then helped 'tune' the scripts to better support task-based information
needs.

To perform an evaluation, the user selects the script to run from the Script Mode window. As
discussed in Chapter 6, the script takes the user through each stage of the evaluation and
automatically provides custom descriptions of the aspect being evaluated. For example, Task
16 used the current version of the script which is used to evaluate coding practices. This script
presents the required information by using a total of 4 different views and 30 attributes. The
script also uses two relationships in the custom descriptions that are provided. The script
performs 24 actions on the base representations to customise and tailor them to user needs.

Users can interact with the SEE-Ada environment to adapt the information provided by the
scripts to their specific needs. For example, the user may wish to change the metric threshold
levels or view a related attribute. The user may also want to change the layout of the view or
remove unwanted detail. Users commonly need additional information such as the actual
attribute values or a section of a source code file. This is provided in the peripheral views. It
is interesting to note that in Task 16 there are no user actions. Task 15 involves the use of an
earlier version of the script. Analysis of user actions on this earlier version helped 'tune' the
script so that it would more effectively meet user needs. Additional usage data is required to

152
ascertain how much additional user interaction would be undertaken by other users of the
script. It is thought that measures of user interaction in relation to script usage can help
determine how effective the custom description is for particular types of tasks. This conjecture
remains to be explored.

These script-based evaluations used the Usage Monitor to record the results of the evaluation
session. As previously discussed, the user can input comments regarding problems,
observations, questions etc. The usage logs then provide a record of the evaluation.

6.2.8.4.5 Summary of Observed Limitations and Problems

The integrated visualisation approach proved useful for a range of tasks. However, as yet only
preliminary results are available. Case study data over a longer period of time and from
different projects is needed to more fully evaluate the IVES concepts. The results to date
highlighted several potential benefits. A range of limitations, problems, and issues have also
been identified. These are discussed more fully in Chapter 7. Some of the observed problems
were attributable to the SEE-Ada implementation and others were 'setting' related. These are
discussed in the following paragraphs.

Many limitations of the SEE-Ada implementation were observed. These limitations have been
recorded as Software Change Requests in the SEE-Ada configuration management database.
They will provide an important input into future phases of the research process. Examples of
the recorded limitations include:

• Layout and scalability problems in subsystem views caused problems in representing design
information. The Subsystem View is based on a tree representation. Although sufficient to
describe the main features of the logical design, layout became particularly difficult when
the physical design entities were added. The automated layout feature could not be invoked
because of the complexity of the design representations. Moreover, users expressed the
opinion that the layout needed to remain constant since they learned to recognise entities
based on their position on the screen. Automated layout would change these and hence
destroy their mental map of the system.

• The Subsystem View lacked facilities for displaying a wide range of design relationships.
The Subsystem View was used to describe design encapsulation relationships and the
'implemented_by' relationship between the logical and physical design. Other relationships

153
 were also important (eg 'uses' relationship between logical design entities). Since SEE-Ada
could not display these relationships graphically, this information was provided in terms of
entity attributes. Although useful for some tasks, a graphical representation of relationships
would in most cases be more meaningful. SEE-Ada needs to be extended so that it more
closely implements the integrated visualisation approach where primary views are
represented in terms of the spatial layering of entities. In so doing, many of these problems
would be addressed since design information would be provided in a manner similar to that
used in the SEE-Ada Layers View.

• The SEE-Ada workspace becomes cluttered with views and windows. Workspace
management is an important aspect that needs to be investigated.

• SEE-Ada cannot be used to provide cross-system information. For example, each snapshot
was produced as an individual SEE-Ada system. However, there were cases where users
wished to view information from previous snapshots in relation to the current snapshot of
the product (eg to observe how various characteristics change over time). This could be
done by processing underlying information through a database interface and then re-
importing the information into the system being analysed. However this approach is clumsy
and inflexible.

• SEE-Ada provides facilities for capturing and providing a wide range of attribute
information. However, more flexibility is needed for viewing combinations of attributes.
For example, a user may wish to identify those units that have more than 100 SLOC, a
McCabe complexity of greater than 10, and less than 20 comment lines. At present, the
user must interactively apply successive attribute tool mappings to obtain this information.
Moreover, to view actual values, the user must assimilate information from three separate
"Show Values" lists.

• The concept of using peripheral views to provide supporting information for entities in the
primary views proved useful. However, access to additional textual and chart information
would have proved beneficial to users. For example, a textual view is used to show sections
of source code files. This same approach could be used to provide access to other
underlying textual information such as the design rationale from the Software Development
Files and requirements text from a Software Requirements Specification. Charts would also
prove useful for observing trends in numeric attributes for sets of entities.

154
Although SEE-Ada could be enhanced to make it more capable, the experiences gained
through the FCS study suggest that there are a range of other issues that determine how
effectively the approach will be used in practice. These are defined as 'setting' issues and relate
to the project and use contexts. Examples of setting issues include:

• The approach must form an integral part of a well defined and enacted process. For
example, the full potential of the approach could not be determined during the early phases
of the FCS case study because the incremental release process was not being enacted.

• Effective training must be provided at the appropriate time. A one day training programme
was provided for FCS personnel. However, this was provided before any significant
development activity had commenced. As such, usage data shows that many of the features
of SEE-Ada are not being used. A follow-up training programme will be provided prior to
the next phase of the case study. It will be interesting to see how usage patterns change.

• Users need support in the application of the approach. This includes support for the setup
and use of the environment. Support is needed to help users understand what information is
available and how it can be used to answer specific questions. For example, although a
variety of attribute information was available, users were often unsure as to the nature of
the information described and how it could be used. Additional mediation support is
required to help transfer this knowledge between provider and user. These issues are
discussed in Chapter 7.

6.3 Summary
A range of experimentation has been conducted as part of the research discussed in this thesis.
Preliminary studies helped provide a better definition of, and gave new insights into, the
problems and issues related to the provision and use of information in SE. It was conjectured
that an integrated visualisation approach, together with appropriate process-based framework,
could address many of the difficulties being experienced. A practical implementation of these
concepts was built, extended, and applied. Various trials have shown that these ideas can be
used to support a range of SE tasks and users.

An Industry Case Study was designed to provide a basis for more focused and controlled
experimentation. This case study addresses the contextual aspects of information use as well
as providing usage data captured through an instrumented version of SEE-Ada. The

155
contextual modelling and measurement activities have helped describe the diversity and
quantity of project information that is available at different times through the project.
Attempts were also made to classify and define team leader tasks as a basis for specifying
information needs. Preliminary results suggest that the integrated visualisation approach can
prove beneficial for a range of SE tasks. However, many 'setting' issues must be considered if
the approach is to be used effectively. Chapter 7 provides a more detailed discussion of these
preliminary results and considers them in terms of the issues defined in Chapter 2.

This initial case study has shown that the experimental setup can be used to capture and
analyse required experimental information. Additional information is needed from the future
phases of the FCS Case Study, and other proposed case studies, to more fully understand and
evaluate the concepts and ideas proposed in this thesis.

156
 7. DISCUSSION

This chapter consolidates and discusses the results of the research to date. The chapter begins
by summarising the main concepts and conjectures. This is followed by a discussion of the
characteristics and limitations of the research approach. This discussion provides the context
within which the findings can be interpreted and discussed. The findings are then discussed in
two sections. The first section discusses findings related to the key issues defined in Section
2.3. The second discusses other findings that have resulted from the work to date. The
chapter concludes by discussing potential benefits and implications of the work, ongoing and
future work, and a summary of original content and potential contributions.

7.1 Summary of Concepts and Conjectures

The research discussed in this thesis has focused on the provision and use of descriptive
information in SE. Descriptive information (or descriptions) was defined as that type of SE
information which is human useable and which can be used to describe characteristics of
various classes of SE entities (products, processes, and resources). Descriptions are defined in
terms of the relationships and attributes of entities. This thesis has argued that effective
provision and use of descriptive information is particularly important for Software Engineering
due to the abstract nature of software products and processes. Descriptive information can be
used to provide indirect visibility of product and project characteristics for those engaged in a
range of SE tasks.

An analysis of the problems and issues resulted in the following conjectures and concepts:

C1. The provision and use of descriptive information plays an important role in
Software Engineering. This thesis has argued that ineffective provision and use of
descriptive information can result in poor project and product visibility. Section 2.3
identified several key issues and criteria that need to be addressed if this kind of
information is to be more effectively provided and used in SE.

C2. Integrated visualisation approaches can provide a more effective basis for the
provision and use of descriptive information. An integrated visualisation approach
was proposed as a means of providing and using model-based descriptions. This
approach allows for the provision of information which can be customised and tailored to

157
 individual needs. Three main forms of integration are supported: (1) the integration of
underlying project information based on the concept of a Software Product Model, (2)
representational integration, and (3) presentational integration. The concept of an
Integrated Visualisation Environment for Software (IVES) was presented as a way of
providing practical implementations of these concepts. This type of environment
provides a basis for accessing and filtering of project information, modelling and
integration, and the preparation of Integrated Visual Descriptions (IVDs). IVDs provide
information by way of a set of integrated primary and peripheral views.

C3. A conceptual process framework can be established and used to help understand
how descriptive information is generated, provided, and used for SE tasks. A
Description in Software Engineering Process Framework (DSEPF) has been proposed to
help address these needs. The framework combines a Contextual Model and a set of
description processes and products. The Contextual Model provides the project setting
within which the information is provided and used. This model focuses on the key
aspects of interest; namely, the SE process and the underlying project information
resources. Four primary description processes have been defined. The Provision
Process supports the preparation of description products. The Use Process addresses
how the descriptions are used for particular SE tasks, with a particular emphasis on
human information processing (or cognitive) issues. The Mediation Process looks at
interaction and exchange of knowledge between the user and provider of information.
The Facilitation Process provides a basis for supporting the provision of the descriptive
information. Facilitation activities can include analysis of task-based information needs,
preparation of description specifications, and directing the access and filtering of
information. This framework has been used to help understand how information is
produced and used within a project setting. It has also been used to help support the
transitioning of the integrated visualisation approach to SE practice.

C4. Practical tools can be built, applied, and evaluated using these approaches to
better understand the problems and issues related to the provision and use of
information in SE and to more effectively support individuals involved in a range
of SE tasks. The SEE-Ada environment has been used to help explore, extend, and
evaluate many of the concepts presented in this thesis. This environment is an 'industry-
strength' implementation that has been trialed in various settings. It serves as a key

158
 apparatus for a series of Industry Case Studies. The environment incorporates a usage
monitor which has been designed to support a range of experimental activities.

7.2 Synopsis of Research Approach

This section discusses the research approach. It critically evaluates the work that has been
done to date in terms of its scientific approach and considers the research results in relation to
the five questions that Fenton et al. (1994) suggest should be asked regarding any claim arising
from SE research. This discussion provides the context within which the findings should be
considered. It helps describe the characteristics of the research approach and highlights
limitations of the research results.

7.2.1 Use of a Scientific Approach

The research to date can be summarised in terms of Glass's proposed research model. This
model, which was introduced in Chapter 3, includes 4 main phases of research (Glass 1994;
Glass 1995). Since it includes the basic elements of the scientific approach (Popper 1977,
Kuhn, 1962), the model provides a basis for describing and evaluating the characteristics of a
research approach. If the research approach covers the main phases of the model, then it has
the key elements of a scientific approach.

The following paragraphs summarise the research that has been conducted in terms of this
model:

1. Informational Phase. In this research, the informational phase included preliminary

studies and a literature review. It resulted in the definition of the problems and issues
defined in Tables 2-1 and 2-2.

2. Propositional Phase. The propositional phase resulted in conjectures being defined

based on the set of issues identified in the informational phase.

3. Analytical Phase. The analytical phase helped define and refine concepts, models, and
approaches. This provided a theoretical basis for the work. The model-based integrated
visualisation approach was defined. The evolution of the SEE-Ada system supported the
development of abstract concepts.

159
4. Evaluative Phase. SEE-Ada was transitioned to several receptor sites. The first phase
of an industry-based case study has been undertaken to test and evaluate ideas and
conjectures.

Although the model is useful for describing the main features of a research approach, the actual
process of research is far more complex than can be defined in a simple model. For example,
as with many software development processes (McDermid and Rook 1991), research typically
takes an iterative approach with significant amounts of feedback. Phases of research are not
always sequential. For example, the evaluative processes may be undertaken at different times
during the research process, not just as part of an evaluative phase. For example, the research
discussed in this thesis used evaluative processes to help establish and refine concepts and to
test the experimental apparatus. Nevertheless, a sparse model of this type can be useful for
guiding research and as a rational approach for reporting results (Parnas and Clements 1986).

7.2.2 Characteristics and Limitations of Results

Fenton et al. (1994) define a series of questions that they believe should be asked regarding
any claim arising from SE research. As discussed in Chapter 3, these questions provide a basis
for discussing the characteristics and limitations of the research approach and related results.
The following paragraphs consider the research presented in this thesis in relation to these
questions.

1. Is the research based on empirical evaluation and data? This research is based on
both empirical evaluation and data. Significant amounts of data have been recorded
during the FCS case study. For example, data has been captured on SEE-Ada usage,
SPM characteristics, and project Information Products. This data has been recorded in a
set of database systems which provide a basis for managing and analysing the data.
Several other forms of supporting information have also been captured. For example, the
results of interviews and field observations have been recorded as notes in a Case Study
Folder. The information captured by the SEE-Ada environment also provides an
important source of experimental data.

2. Was the experiment designed correctly? Various forms of experimentation were

undertaken. The design of the Industry Case Study was based on guidance provided by
the work of Kitchenham and Pickard (1995), Kitchenham and Walker (1989), Yin (1984),
and Basili et al. (1986b). Unfortunately, there are as yet few well defined and tested
approaches for this type of experimentation in SE. It is therefore difficult to determine
160
 whether the 'experiment' was designed correctly since effectiveness criteria are not well
understood in the SE research community. The case study was defined in phases so that
the approach could be tuned as more knowledge and experience was gained.

3. Is it based on a toy or real situation? The experimentation is based on real situations.

The trials and Industry Case Study have focused on Software Engineering practice and
have been conducted on real projects and products. This type of research required the use
of an 'industry strength' apparatus. Apparatus of this type must be scalable, robust, well-
documented, and supportable.

4. Were the measurements used appropriate to the goals of the experiment? Measures
are an important source of evidence for experiments. However, measures are only one
class of information that is required for the types of experimentation conducted in this
research. As discussed in Chapter 6, a modified GQM approach was used as the basis of
the Industry Case Study. This approach defined specific goals for the case study. A set of
questions for each goal provided the operational definitions for the goals. The primary
information requirements for the case study were defined in terms of the information that
was needed to answer the specific questions. The experimental setup was designed to
capture and provide this information. Other secondary information was captured through
observation and interviews. This secondary information was considered necessary to
support the analysis of the primary information. In addition to ensuring that the
information was relevant to the goals of the case study, the goal-based approach provided
a basis for controlling the case study and for reporting results.

5. Was the experiment run for a long enough time? The Industry Case Study provides
the quantitative evidence for this research. The results of the first phase of an initial case
study are presented in this thesis. As discussed in Chapter 6, the case study needs to be
continued through to the completion the FCS project. Moreover, other case studies need
to be conducted on other similar projects. The results to date are sufficient to discuss
some findings. However, additional results are needed for others. The discussions in this
chapter will highlight those areas where the results to date are insufficient to provide
conclusive findings.

An important aspect that is not addressed by these questions is the generality of results. The
case study results discussed in this thesis deal with the development of a real-time embedded
application. Other specific characteristics that need to be considered in relation to the case

161
study results include the development environment, development approach, and the
programming language. Care needs to be used when considering the results of the results to
date in terms of other projects and application areas. Other replicated case studies are needed
to help make the results more generally applicable.

7.3 Findings Related to Key Issues

This section discusses the findings of the work to date in terms of the issues defined in Section
2.3. The findings should be considered in relation to the limitations of the research results
discussed in Section 7.2.

7.3.1 Understanding and Supporting User Information Needs

This issue focused on the general aspects of information use in SE. Of particular interest were
those types of information which could be used to describe characteristics of interest and hence
provide a basis for indirect visibility of SE entities. A range of description concepts and
models was proposed as a means of classifying, understanding, providing, and using this type
of information. This issue also stressed the need for defining and supporting task-based
information needs. The findings of this area of the research are discussed in the following
paragraphs.

7.3.1.1 Description Concepts and Models

This thesis has primarily been concerned with the effective provision and use of information for
those engaged in SE tasks. The term descriptive information (or descriptions) was defined to
identify those classes of information products that are human useable and can be used to
describe something (eg SE entities). This thesis has argued that this type of information is
extremely important since it provides indirect visibility of software products and projects.

The importance of gaining visibility of software product and project characteristics has been
highlighted in the literature and recorded in the results of the preliminary studies, SEE-Ada
trials, and the Industry Case Study. Descriptive information is important for understanding,
communication, and collaborative work (Kraut and Streeter 1995). It provides a basis for
undertaking a range of tasks including evaluation, analysis, monitoring, and inspection tasks.
The results of this research have highlighted the fact that the information needs of individuals
engaged in complex high-level tasks (eg team leaders) are not well supported by many current
sources of information (eg CASE tools, project documentation). These tasks require ready

162
access to various types of information. Moreover, the information often needs to be filtered,
assimilated, and customised if it is to describe the characteristics of interest.

Description is a broad concept. It can include a wide range of approaches for describing a
multitude of things. This thesis has focused mainly on the provision and use of Product-
Oriented Descriptions. The results to date have shown that various types of information can
be viewed from a product perspective. The product perspective is important in that the
Software Product is a primary focus for many SE tasks. Moreover, as this research has shown,
concrete and stable structural models of the Software Product can be produced using other
underlying models and information (eg as provided by CASE tools).

The product-oriented approach provides a basis for accessing and integrating various types of
information. This allows information to be viewed in relation to a common underlying
representation of the Software Product thereby improving understanding and use of the
information. Other perspectives may also be important. For example, Vernik and Burke
(1993) suggested that a process-oriented perspective may also be useful for many tasks. This
approach has not yet been explored in detail. It would rely on having a stable and well defined
model of the processes which were being undertaken. This may not be possible. Observations
during the FCS case study suggest that project dynamics are such that it may be difficult to
generate such a stable and valid underlying model based on software lifecycle processes.

The DSEPF was defined to provide a basis for considering contextual and process issues
related to the provision and use of descriptive information. It is based on a Contextual Model
which focuses on the project context. The key elements of this model are the SE Process and
the Project Information Resource. The SE Process provides the basis for identifying
information needs and the PIR considers the totality of persistent information that is available
to the project. The concept of a Description Process was introduced. This process addresses
how information is provided as well as how it is used. This process provides a basis for
considering a range of human information processing issues in relation to the tasks being
performed and various providers of information.

The DSEPF is a general framework which has proved useful for a range of activities. It has
been used to support the definition and evaluation of the integrated visualisation approach. It
has provided a good basis for communication with other researchers, reviewers, and case study
participants. It has also helped transition the integrated visualisation approach into practice.
During future phases of the research, it will be used for considering how automated

163
approaches such as knowledge-based assistants (Vernik and Kingston 1993) and Visual
Interactive Assistants (Rogers 1995) might best be used to support the various description
processes. This will involve considering tradeoffs between the cognitive and human action
processes and the support that can be provided by the provision, mediation, and facilitation
processes. Although this framework has proved useful for the research to date, it may need to
be modified and extended to support these new activities.

7.3.1.2 Understanding and Defining SE Tasks

This thesis has argued that if we are to begin to address the information logistics problem
(Fernstrom et al. 1992) of providing information which is valid and consistent with the
assigned user tasks, we need to be able to describe the types of tasks that are performed.
However, is it possible to accurately classify and define SE tasks? The following paragraphs
discuss this issue.

One of the activities of the FCS case study involved conducting interviews, work-place
observations, and the review of standards and other literature to help identify initial task
classifications. The results showed that it is difficult to get a consensus on the types of tasks
that are performed by various roles in SE. For example, during interviews which attempted to
define team leader tasks, it was found that people used the same term to describe very different
tasks. For example, the term 'review' was used to describe tasks ranging from code inspections
to formal meetings with management and customers. However, some tasks were easier to
define such as those identified in contractual requirements or company procedures (eg many
evaluation tasks). Since there was little consensus on task definitions, the standard definitions
(IEEE_Std_610.12-1990; IEEE_Std_1074-1991; ISO/IEC_12207-1995) were used to provide
a preliminary set of team leader tasks. These task definitions were reviewed and extended by
the FCS team leaders. Information provided by future case studies and surveys may help
provide a more complete and accurate set of task definitions.

The FCS case study has attempted to capture additional task definitions and to validate the
preliminary definitions by way of the SEE-Ada usage monitor. This has not proved successful
to date. As discussed in Chapter 6, individuals found it difficult to classify and describe many
of the tasks that they were performing. They were more motivated towards getting the work
done rather than attempting to accurately describe the task being performed. As such, the
results obtained to date provide, at best, a general indication of types of tasks undertaken.
Accurate definition of some tasks may in fact prove impossible. It may be best to describe

164
some tasks in terms of a usage profile. For example, many inspection tasks could be specified
in terms of the entities that are described and the types of actions performed. There is a
possibility that the information captured by the SEE-Ada Usage Monitor over a wide range of
usage sessions could help investigate this conjecture.

These issues of task analysis need further investigation. As discussed in Chapter 2, many
approaches for task analysis have been proposed and used. However, growing opinion
suggests that methods that do not take into account aspects of human cognition are not fully
adequate for the analysis of complex tasks (Grant and Mayes 1991; Barnard 1987). Since
cognition is based on information flow and usage, the field of Cognitive Task Analysis (CTA)
may be important for helping define the information needs of complex SE tasks.

There are several methods which may support CTA. For example, if the task is defined
logically in a simple way, a formal analysis (eg using Task Action Grammars (Payne and Green
1986)) help define information needs at that level. If the complexity of the task lies in the
degree of internal mental processing necessary, models of cognition such as those discussed in
Chapter 2 might help define information needs and flows. In cases where the tasks have a
hierarchical structure, hierarchical task analysis may reveal the structure and information needs.
However, as Grant and Mayes (1991) suggest, complex tasks may have many of these features
and hence effective analysis may not be possible. These areas need more study in relation to
SE tasks. The use of the SEE-Ada Usage Monitor may support these investigations.

7.3.1.3 Defining and Understanding Task-Based Information Needs

A key issue to be addressed is whether information needs can be effectively defined for
individuals engaged in specific SE tasks. As discussed above, it is often difficult to accurately
specify many tasks. However, there are some tasks where task-based information needs may
be more easily specified. For example, the FCS case study showed that scripts could be
developed to provide descriptive information for individuals engaged in evaluation tasks where
the evaluations were based on well defined criteria (eg development standards). More
evidence is needed to determine how well this approach works for a wider range of tasks and
whether the same script is effective for a cross section of users.

Since the integrated visualisation approach allows the user to adapt the descriptions to their
requirements, measures of user interaction in relation to script actions may help define how
closely provided information supports user needs. However, as suggested by Duvall (1993),
the need for information often arises after a person recognises that their knowledge is
165
inadequate and that information may play a role in the resolution of the problem. As such,
even though we may be able to produce custom descriptions that provide the key items of
information, adaptive approaches, such as those defined in this thesis, may provide the most
appropriate solutions.

7.3.2 Availability of and Access to Information

A key issue highlighted in Chapter 2 was the need to consider the availability of and
accessibility to project information. This issue raises many important questions. For example,
if we cannot accurately identify what information is available at a particular point in time, how
can we begin to address how information can be used to support the needs of individuals? A
Contextual Model was defined to help address these questions. A key element of this model is
the Project Information Resource (PIR). The PIR is a conceptual entity which represents the
underlying source of persistent information available to a project. A PIR classification and
modelling scheme was devised to support measurements of project information and for
defining the access and filtering requirements when applying the integrated visualisation
approach. The following paragraphs discuss findings in relation to PIR modelling and
measurement, the direct use of this underlying information, and how an IVES supports user
access to needed information.

7.3.2.1 Availability of Underlying Project Information

The Project Information Resource concept proved useful in helping understand how
information was produced and used in software projects. The PIR model and the resulting
model-based measurements helped provide quantitative data on the diversity and quantity of
information produced at different times though the project (ie to describe the availability of
information). This model also provided a basis for identifying sources of information that
could be accessed, filtered, and integrated into a Software Product Model and hence made
available through SEE-Ada.

Experience gained during the FCS cases study showed that the PIR classification and
modelling scheme was flexible enough to accommodate the modelling and measurement
requirements for this research. The modelling approach can support various levels of detail.
The PIR model captured in the FCS case study included a wide range of Information Products.
The scope of the modelling activity was determined by the case study goals. Since the study
was interested in identifying and measuring key Information Products, the "comprises" and

166
"derived from" relationships were not modelled. Moreover, only persistent IP classes were
considered since this initial case study focused on the availability and access to underlying
project information resources. As with any approach to modelling, the modeller must consider
how the model is to be used, and then determine the level of detail that is required. The
evolution of the model must also be considered. For example, since an aim of the FCS case
study was to capture the diversity of information provided and used in software projects, it was
important to continue the modelling activities throughout the FCS case study. This helped
identify new classes of information which were not defined during the initial modelling
exercise.

Automated support for the modelling activities, as provided by the PIRMAS tool, proved to be
extremely important. This tool helped capture and manage the model. It also provided
support for storing and managing the IP instance measurements. As discussed in Chapter 6,
the analysis facilities provided by PIRMAS allowed viewing of the model, and related
measurements, from different perspectives.

The results of the PIR modelling and measurement activities for the FCS case study highlighted
the diversity of information that can be made available in software projects. A total of 181
classes of information products have been identified. Snapshots were taken at weekly intervals
to show when instances of the IP classes became available and how these various classes of
information changed over time. This thesis reports on 24 snapshots taken at weekly intervals
and cover the period of the first incremental build of the FCS software. Snapshot 24 identified
5201 actual instances of these IP classes. The results can provide a detailed account of when
particular classes of Information Products are produced during a software project. Details of
the number and size of the IP instances for each class were also measured.

The PIR modelling and measurement activities provided some interesting results. For example,
many classes of information were identified that were unknown to the project team members.
The results also gave a good indication of how the SE tools stored and used information.
These results also highlighted many problems faced by those wishing to use this information to
satisfy specific needs. In addition to contending with the problems of information overload,
the potential users need to know what information was available at a particular point in time,
and how the information could be accessed and used. The PIR details, as provided by
PIRMAS, can help provide information on the availability and accessibility of information.

167
This information proved useful for guiding the setup of SEE-Ada. It may also be useful for
supporting the direct use of PIR information. This conjecture has not yet been explored.

The results of the FCS case study found that in addition to supporting research goals, the PIR
Model and measurement information proved useful to project personnel. As discussed in
Chapter 6, this information helped identified process problems. It was also useful for
describing the characteristics of the project. For example, the FCS project is largely dependent
on information stored in electronic form. The vast majority of this information is version
controlled. These characteristics have implications for particular individuals and tasks. For
example, those undertaking quality assurance or IV&V tasks must be knowledgeable in the
tools and techniques which can provide access to required information. There are also
implications regarding the amount of traditional documentation that is needed for different
projects. For example, since projects such as the FCS project are 'information rich', there may
be less of a need for many of the traditional deliverable documents if the underlying
information can be effectively used to describe project and product characteristics. This issue
is discussed in more detail in Section 7.4.3.

7.3.2.2 Direct Use of Underlying Information

The results to date suggest that even if users can identify the particular item of information that
is needed, knowing how to access it can prove a major problem. In most cases, the tool that
was used to produce the information is needed to access and view the information. Access can
be difficult for people like team leaders who do not use the tools on a daily basis. For
example, on the FCS project, the team leaders would need to have an extensive knowledge of
the Apex environment and the various other CASE tools if they were to directly support their
information needs. Access difficulties can present information barriers which can ultimately
prevent people from successfully completing important tasks.

Another problem is the availability of tools. For example, to view the current description of
the logical FCS design, the team leaders needed to access the Rose tool. This required the use
of one of the Rose licences. These licences were in heavy use by those involved in the design
activities and so were not necessarily available when required. Even when access was gained,
the tool would not effectively provide the required information. Since the tool was
predominantly developed to support design tasks, it did not provide the features that allowed
design information to be presented from an appropriate point of view for the team leaders.

168
Access to paper based information in Software Development files and deliverable documents
was also a problem. For example, only one copy of an SDF was produced. Although stored in
a central location, the folders were often in use and so were not necessarily available when
required by a team leader. Also, as the IP classes in Appendix G show, SDFs contain vast
amounts of different types of information and specific items of information are often difficult to
identify and access.

7.3.2.3 Availability of and Access to Information using SEE-Ada

To address the availability and accessibility issues, this thesis has proposed an approach which
allows users to gain access to project information by way of an integrated visualisation
environment. Information is filtered so that only the key items are made available.

As shown in the FCS case study, a variety of information can be made available through SEE-
Ada. This information is accessed by way of the Software Product Model. A major function
of the Facilitation Process is to determine how the Software Product Model will evolve to
support the needs of those engaged in various SE tasks. An initial analysis performed as part
of this process, identified a range of information which needed to be made available to support
the information needs of team leaders. Feedback from users during the case study identified
other information that was required. The Usage Monitor was used to determine information
usage profiles which could then be used to guide future setup and evolution activities (eg to
identify information that was never used and the most used information). The usage results to
date are not sufficient to provide accurate usage profiles. This is partly because the study has
not been conducted for a long enough period and partly because users were unfamiliar with
some of the information that was made available and how it could be used. Additional usage
results over several projects will be needed to accurately analyse trends in information use.

People need to have a good understanding of the information that is available through SEE-
Ada. This can be a problem for less experienced users or where the meaning of a particular
attribute is not apparent. The results of the FCS case study found that some attribute
information was not used because the user either did not realise that it was available or did not
understand what it described and how it could be used. There is a need for an additional form
of mediation between provider and user to help explain the nature of the information that is
available. This issue is discussed further in Section 7.6.

The SEE-Ada environment provided a flexible means of providing access to a range of

structural, numeric, and symbolic information. However, some classes of information could
169
not be accessed. For example, the SEE-Ada implementation has restrictions on providing
access to underlying textual information. The current implementation of SEE-Ada only allows
access to source code text. There are cases where users would like to gain access to other
underlying text such as the definition of a particular requirement, as provided in a Software
Requirement Specification, or the textual description of a design entity. The use of a more
flexible form of link attributes would support this need. This aspect is discussed in Section 7.6.
The viewing of design relationships was also a problem. As discussed in Chapter 6, although
design relationships could be captured in the SPM, the representation used for the Subsystem
View did not support the viewing of many design relationships. The use of primary view
visualisation similar to that provided in the SEE-Ada Layers View, would be more suitable for
describing this design-level information.

Much of the information in the FCS PIR was in Unix ASCII form and hence automated access
and filtering was relatively straightforward using Unix tools such as 'grep' and 'awk' and text
processing languages such as Perl. Some tools provided programmatic interfaces to underlying
data and hence filters could be effectively developed for these sources. This may not be the
case for other projects where the tools, environments, and Information Products may not be as
accessible. As such, the approach proposed in this thesis may only be applicable for projects
where a large proportion of the information is stored in a readily accessible electronic form.
Although manual importing of information is possible, this may not provide a cost-effective
solution.

7.3.3 Assimilation and Integration of Information

The results of this research have shown that many SE tasks require individuals to assimilate
various types of information. For example, the FCS case study showed that the requirements
allocation inspection tasks required team leaders to access information from a range of
Software Development Files and assimilate these with a mental model of design structure.
Similar difficulties were observed when using metric information to identify potential problem
areas in the source code.

The use of a Software Product Model to support the integration of information is a key feature
of the integrated visualisation approach. It provides a consistent underlying representation of
the Software Product. This model also provides a basis for representational and presentational
integration. This approach also supports model-based measurement. These aspects are
discussed in the following paragraphs.

170
Consistency is an important issue that needs to be addressed when using project information.
For example, in the FCS case study, several inconsistencies were found between the Logical
design (captured in the Rose tool) and the physical design defined in the Apex environment.
This was clearly evident when the models were imported into SEE-Ada. SEE-Ada identifies
any inconsistencies and will not allow information to be integrated into the SPM if the entity
identifiers do not match.

The use of a SPM supports representational integration. This approach allows different types
of information to be combined into a single representation and has proved particularly useful
for tasks that required information to be presented in context. For example, when SEE-Ada
was used in the requirements allocation inspections, many problems which had not been
identified during previous inspections of SDF information were identified. This approach is
also useful when using metric information. Rather than contending with large reports of
values, metric information can be mapped to colour representations and superimposed onto the
entities of interest thereby highlighting potential problem areas or other areas of interest.

The use of an underlying SPM helped support presentational integration. This type of
integration refers to the integration of views. Results to date suggest that the use of integrated
multiple views is important. The use of integrated primary and peripheral views allowed users
to view information from different perspectives and allowed rapid access to additional
information. Since the entities represented in the views were based on a single underlying
definition, this allowed the results of actions performed in one view to be presented in an
associated view. Although the various views proved important, observations during the FCS
case study showed that as a session progressed, the workspace would become cluttered with
views. There were also problems with the Motif interface where views that were iconised
would be hidden under other views. Workspace management is an important consideration that
needs to be considered for IVES implementations.

The use of a Software Product Model supported model-based measurements and the
generation of 'value-added' information. For example, several of the attributes provided in
SEE-Ada were generated by accessing the underlying SPM and providing counts of
relationships. For example, fan-in and fan-out measures were produced by counting the
number of 'with' relationships between Ada compilation units. As discussed in Chapter 4,
composite models of this type can also provide the basis for generating attribute values based
on the structural relationships of entities. For example, the total number of SLOC for a
Rational Subsystem can be generated by summing the values of SLOC attributes of entities

171
which have an "encapsulated by" relationship with the subsystem. Definition of new attributes
based on relationships between existing attributes are also possible. For example, a
descriptiveness metric could be computed based on comment counts, the size of code entities,
and their internal complexity. Used in this way, SEE-Ada could provide a test bed for defining
and validating new metrics. In addition to supporting the definition and generation of the new
metric values, SEE-Ada could provide ready access to the underlying values that form the new
metric. These metrics could then be viewed in relation to other information to support
validation activities. Validation trials could be performed in real situations with appropriate
feedback and usage data provided by the Usage Monitor.

As discussed, the use of an SPM to support information access, integration, representation,

and presentation has shown clear benefits. There are several issues that need to be considered
in relation to the updating of the model if it is used to describe product evolution during
software development. As discussed in Chapter 6, project snapshots of the release tower were
to provide instances of the SPM at weekly intervals. These automatically generated SPMs
were to comprise a structural model of the product (including logical design, physical design,
and code entities) and selected attributes. In addition to providing visibility of the evolving
product for team leaders, this activity was to provide research data which could be analysed to
address how the structure of products changed over time and trends in various attributes.
Although the automated update mechanism has been tested on the FCS project using 'working
view' towers, the incremental release process was not enacted during the early stages of the
project. As such, stable snapshots of product evolution have not been captured and the
research to date has not been able to assess whether the use of an evolving SPM can provide
an effective method for monitoring and describing project status. A study of SPM update
intervals and update criteria also needs to be undertaken.

7.3.4 Use of Appropriate Representation and Presentation Techniques

The appropriate use of representation and presentation techniques was identified as a key issue
in Chapter 2. Some of the major problems to be addressed dealt with the scalability,
understandability, meaningfulness, and flexibility of representation and presentation techniques.
The use of Integrated Visual Descriptions was proposed as a means of addressing many of
these problems. The IVD concept is based on the use of an integrated set of primary and
peripheral views. The primary views use a compact, spatial representation of entities, where
the relative positioning of these entities is determined by a set of arrangement rules. These
rules are typically based on the structural relationships of the Software Product Model.

172
Peripheral views support the primary views by providing alternative representations of
information provided in the primary views or supporting information (eg a textual description,
a list of values, etc). The SEE-Ada Layers View provides an example of a primary view.

Observations and feedback from users showed that the use of a primary view based on a layers
approach provided a useful representation for describing software characteristics. The spatial
characteristics of the representation proved important. For example, the shape of the
representation provided important information. Users were able to quickly identify changes to
the overall code architecture and were able to identify important features such as areas of high
coupling. Users also began to recognise particular entities based on their position on the view.
This approach also allowed for representational integration where required information was
superimposed onto the base representations. Since information can be added in a controlled
and incremental manner, this approach allowed information to be customised and adapted to
user needs. The flexible and interactive nature of the views improved scalability and
understandability since only information that was needed was displayed. Although the primary
view visualisation approach has been successful for describing code entities, additional research
needs to be undertaken to evaluate the approach for other types of SE entities (eg design
entities, requirements). Moreover, the use of presentation approaches that make more efficient
use of the workspace need to be considered (eg fish eye views or the Perspective Wall view
proposed by Robertson et al. (1993)).

The use of integrated peripheral views proved important for showing supporting information.
For example, the Source View and the view showing lists of attribute values were widely used
in the FCS case study. Peripheral views can also be used to provide different representations
of the features shown in the primary views. For example, the Graph View in SEE-Ada can
describe the relationships between a set of entities in a directed graph form. The results of the
FCS case study to date show that this view was not often used. Relationship information was
most commonly viewed on the Layers View. This may have been because the Layers View
provided the most effective way of viewing this information. It may also have been because
the users were not experienced in the use of this feature or that the types of tasks that were
being undertaken did not require information to be provided in this way. Additional usage
results are needed to observe trends in this area.

Other types of peripheral views need to be considered. For example, a chart view may be
useful for observing trends in metric information. However, the addition of new peripheral
views in SEE-Ada can take considerable development effort. The design of an IVES needs to

173
provide defined and flexible interfaces which will support more effective integration of
additional peripheral views.

7.3.5 Customisation and Adaptation of Information

A major criticism of many CASE tools is that they do not provide flexible interfaces which can
be used to support a range of users engaged in different tasks (Budgen et al 1993). The
integrated visualisation approach proposed in this thesis has focused on providing
representations that can be customised and tailored to individual needs.

The approach makes use of a direct manipulation interface which allows users to interact with
an IVES to define what needs to be described and how this information will be represented and
presented. The FCS case study showed that this exploratory mode of operation proved
particularly effective for unstructured tasks (eg some inspection and monitoring tasks). These
tasks require a highly interactive mode of operation where the user requires immediate access
to a range of information based on previous observations.

The results of the FCS case study showed that other tasks required specific descriptions and a
defined process. For example, evaluation tasks which were based on well defined criteria
benefited from this approach. Scripts were developed to support the evaluation of Ada code
entities in relation to the criteria generated from company development standards. These
scripts guided the evaluation process and provided custom descriptions to support the product
evaluations. The scripts were developed by initially having an experienced user perform an
evaluation. The usage monitor captured the user actions. Since the actions captured in the
usage log are recorded in the script language, the logs were automatically converted to scripts
and replayed in SEE-Ada. A major benefit of this approach is that less experienced users can
perform evaluation tasks even though they do not have an expert knowledge of SEE-Ada or
the information that is available for use. Moreover, since the process is recorded and enacted,
there is some confidence that the evaluation will address all specified criteria. This approach
also provides a good basis for process improvement since the scripts can be updated based on
the results of the evaluations (eg to better identify problems). Since the procedures are
enacted, the improvements to the script can improve the way in which all future evaluations are
conducted.

This thesis has argued that adaptation is important since it may not be possible to accurately
specify information needs for many tasks and users. This adaptive approach was demonstrated
in the results of the FCS case study and discussed in Chapter 6. It uses scripts to guide the
174
preparation of custom descriptions Users then interact with the descriptions to adapt the
information to their specific needs. Preliminary results obtained using evaluation scripts
suggest that this adaptive mode is important, particularly during the early deployment of a
script. Usage results suggest that as a script is 'tuned' to support information needs, the degree
of user interaction decreases. The tuning process involves analysing usage results and
conducting follow-up interviews with users. This area of work requires further study. Scripts
need to be developed to support other SE tasks and these scripts need to be used by a wider
range of users.

7.3.6 Cost of Provision and Use

The actual costs of providing information using the integrated visualisation approach have not
been measured. Several costs need to be considered including analysis costs, setup costs, and
support costs. Analysis costs relate to activities which help define information needs and the
mapping of these needs to available information. Setup costs are incurred in setting up the
IVES environment to support the needs of the project. Support costs relate to the cost of
updating the environment with appropriate information during the project, preparing and
tuning scripts, and producing new filters as needs change. The cost of inserting new
technologies also needs to be considered. These include training costs and the costs which
result from a possible initial reduction in performance when new approaches are introduced
(Kemerer 1992).

A major cost in setting up SEE-Ada for the FCS project was the development of filters and the
automated update facility. This involved gaining extensive knowledge of the SE tools, their
interfaces, and underlying information products. It was found that many of the filters had
common functions. As such, a significant amount of the code was reused in new filters. Many
of the filters developed for the FCS case study could be used for other projects that use the
Rational Apex and associated CASE tools. Also, filters that use standard interfaces, such as
the ASIS filter, could be used with other compilation systems that provide this interface.
However, for projects that use a different set of tools, the cost of filter development is an
important consideration when performing a cost/benefit analysis.

The cost of script development also needs to be considered. The major costs in script
development are analysis costs. For example, the development of the evaluation scripts for the
FCS project involved analysing development standards and establishing mappings between
criteria and information requirements. Script development was partly automated by using the

175
Usage Monitor to define script actions. The tuning of scripts is an ongoing cost that needs to
be considered.

Support costs are another consideration. In the FCS project, the automated update facility
automatically accessed, filtered, and integrated underlying project information into product
snapshots. As such the cost of updating information from electronic forms was minimal.
However, there were costs involved in manual entry of information. For example, since there
was no accurate electronic source of requirements allocation information, this information
needed to be input through the Create/Modify Attributes feature. The cost of maintaining this
information must also be considered. There may also be ongoing support costs involved in
producing and updating filter tools.

The costs and benefits of using the integrated visualisation approach also need to be
considered. For example, how long does it take for a user to access a particular item of
information? Given particular information, how long does it take for the user to make a
decision? Does a new metric support the decision process? Do scripts improve productivity?
These aspects are more easily measured. For example, the Usage Monitor captures the time
taken to perform actions. An action may involve accessing a particular item of information or
recording the results of an investigation. The usage monitor also records user interaction in
relation to script actions. The SEE-Ada usage sessions on the FCS project showed significant
benefits in the time it took to answer specific questions and in the time it took to access
required information. For example, SEE-Ada could provide answers to many questions in a
matter of seconds (eg Are there any areas of code that use global variables? Who wrote these
units? What impact would a change have on other parts of the system? Which requirements
are implemented by Unit X?). These types of questions were difficult, and often impossible, to
answer with the existing sources of project information.

Although these usage benefits can be measured and compared with other approaches, the
overall cost benefits of the approach are much more difficult to quantify. These include the
benefits of being able to perform tasks in less time and in a more effective manner and the
benefits of being able to perform tasks which could not have previously been undertaken.
Improved visibility is also a potential benefit but would be difficult to quantify in practice.

Future studies need to capture a range of costs and need to consider how benefits can be
measured. For example, the Usage Monitor has been extended to capture the results of
particular tasks. As such, studies could be conducted to compare the results (eg

176
problems/risks found) of performing a task with SEE-Ada against some other approach.
However, although more problems may have been found, it may be difficult to determine the
effect that these problems might have on overall project schedule or ongoing support costs.

7.4 Other Findings

7.4.1 Measurement and Metrics Programmes

Tate et al. (1992) suggest that the use of well instrumented CASE tools and environments
could provide the basis for effective metrics programmes. The FCS case study has shown that
this is in fact possible. The FCS project did not include a measurement and metrics process.
These processes are often seen as expensive and more applicable to projects that have higher
levels of process maturity (Humphrey 1989). However, the FCS case study has introduced an
extensive measurement programme into the project with little impact on, or resistance from,
developers. This measurement programme was supported by PIR measurement tools, the
SEE-Ada update and capture facility, and the usage monitor.

The PIR measurements were automatically performed to capture information on the various
types of Information Products which were available during particular phases of development
and can be used to describe various characteristics of the project and the products. As
discussed in Chapter 6, this information proved valuable for identifying possible process
problems. It can also be used to analyse how particular tools were used and how the various
Information Products change over time. For example, analysis of the changes to the various
Rose design models may provide the basis of a design stability metric.

The use of SEE-Ada played a major part in this measurement process. The various filters
captured detailed information about the design and code entities. This included a wide range
of product, process, and resource information. A major benefit of this approach was that all
information was readily accessible from one source and could be viewed in relation to an
underlying representation of the evolving product. Since snapshots are being automatically
captured on a weekly basis, the information provides a detailed account of how the product is
evolving and how the various processes and resources played a part in the development. In
addition to the automatic capture of information, the next phase of the case study will look at
how SEE-Ada might be used to capture information from users. For example, SEE-Ada could
be used to record the time and amount of effort that was taken to develop or modify a
particular component of the product. This information could be captured through the SEE-

177
Ada user interface using the Create/Modify attributes feature. Since this is information that a
developer may use for future tasks (eg to help estimate how much effort it would take to
develop a similar component), these individuals may be more inclined to accurately record the
information. The accurate recording of information is often a major problem of metrics
programmes since individuals tend to report on what they believe managers want to see rather
than what is actually happening. These types of 'human' issues are discussed by Fenton (1991).

The Usage Monitor captures a variety of information that can provide details of the types of
tasks being performed at particular points in time, the individuals undertaking these tasks, and
the types of information that is used. For example, analysis of this information can show which
units have undergone inspection and evaluation, the number of problems found during these
tasks, and who performed the task. This information may be useful for presentation at formal
reviews, and for process improvement. Moreover, there have been suggestions that this
information would prove a valuable additional source of attribute information in SEE-Ada.

The FCS case study has shown that a measurement process can be quietly introduced into a
project. When supported by appropriate analysis and visualisation tools this information can
be used to help manage, control and monitor software projects. Moreover, this approach
provides a detailed account of various aspects of software development which could support a
range of future SE research activities.

7.4.2 Reporting and Recording

Although the Usage Monitor was developed to support research objectives, team leaders saw
it as an important addition to an IVES environment. As discussed in Chapter 6, the Usage
Monitor was used by team leaders to record the results of usage sessions. For example, they
found that they could insert comments into the usage log to record problems, observations,
and actions to be taken. The comments were recorded in context. As such, team leaders
could later view the comments in relation to the types of actions that were being performed
and the information that was being used. They also found that the logs provided an audit trail
which could be used to demonstrate that the task had been carried out correctly. The use of
the Usage Monitor in this way was an unexpected (yet beneficial) outcome of the research.
Recording and reporting features are seen as important and should be considered as part of on
IVES implementation.

178
7.4.3 Impact on Documentation Needs

There have been suggestions that the information captured in CASE tools should become a
project deliverable. Gray (1994) reports that during the Association of Computing Machinery
(ACM) Software Development Standards and Ada Working Group (SDSAWG) reviews of
new documentation standards in 1993, developers argued that the information provided by
CASE tool files should constitute the key form of deliverable documentation for software
projects. This would then reduce the documentation overhead imposed by many development
standards. There were suggestions that the only other information that should be required are
engineering notes (eg Software Development Files) and software development plans.

The FCS case study has highlighted the difficulties associated with directly using CASE tool
information by people who need an overview of product and project characteristics. The
diversity and quantity of information make it difficult to know what is actually available for
use. Moreover, the information is often not available in a form that can serve the needs of the
individuals engaged in tasks other than those specifically supported by the tool. The FCS case
study also identified information that was used to support tool needs, yet was not available to
the user through the user interface. In an appropriate form, some of this information would
have been particularly useful for monitoring developmental status. The lack of support for the
integration and assimilation of information from more than one tool can also cause difficulties.

The developmental information residing in CASE tools is an important project deliverable.

However, consideration must be given to how the information will be used. The use of the
integrated visualisation and the PIR modelling and measurement approaches presented in this
thesis may provide a means of more effectively using this information for project monitoring
and software maintenance tasks, thereby reducing the reliance on traditional deliverable
documentation. As discussed in Chapter 2, this type of documentation is often out of date, is
expensive to produce and maintain, and the required information is often difficult to access.
The integrated visualisation approach would provide ready access to actual project information
and allow this information to be filtered, integrated, and customised to the user's needs.
However, for this approach to be successful, consideration also needs to be given to the
context of use including an analysis of user information needs and the establishment of
appropriate processes and procedures.

179
7.5 Potential Benefits and Implications
It is the author's opinion that the work presented in this thesis is significant in that it provides a
basis for understanding how people use information for a range of SE tasks. It provides an
approach for modelling and understanding the context within which descriptive information is
used. An integrated visualisation approach is proposed as a means of more effectively
supporting individuals engaged in a range of SE tasks. A practical implementation has been
developed and applied within a process-based framework to test the conjectures and
approaches in real settings.

Although a significant amount of work has been done to date, more work needs to be done to
further understand, refine, and test the concepts and approaches presented in this thesis.
Nevertheless, the results so far have identified a number of potential benefits:

• Improved product and project visibility.

• Improved accessibility to information.

• Ability to present information in context and hence improve understandability.

• Ability to customise and adapt information to users needs.

The work presented in this thesis has a number of potential implications for those involved in
software projects, CASE tool developers, and SE researchers. These are discussed in the
following paragraphs.

7.5.1 Implications for SE Practice

The possible implications for software practice include:

• A reduced need for conventional technical documentation produced during software

projects. This thesis has argued that if underlying project information could be more
effectively accessed and used, there would be a reduced need for traditional technical
documentation. As discussed in Section 7.4.3, the approaches presented in this thesis have
the potential for more effectively providing information for those engaged in a range of SE
management and supporting tasks. However, this approach appears to be more generally
applicable for those projects which make use of information in electronic form (eg those
supported by an integrated set of SE tools).

180
• The ability to undertake many important tasks which were previously neglected or
poorly conducted. The research to date has shown that there are many tasks which are
difficult to perform with the information that is currently available in software projects.
These tasks often require the user to assimilate information of various types. Efficient
access to a variety of information is also important. Experiences with SEE-Ada have shown
that the integrated visualisation approach can help answer many questions that were
difficult, if not impossible, to answer with existing sources of information. The approach
has shown benefits for many inspection, analysis, evaluation, and monitoring tasks.

• More effective use of metrics and measures. As highlighted in Chapter 2, metrics are an
important source of information for software projects. However, measurement programmes
can produce copious amounts of this information. In addition to contending with the
problems of information overload, users need to have the information presented in such a
way that it supports the task at hand. For example, a user may wish to use a measure as an
indicator which identifies potential problem areas within a software product. This often
requires the measure to be used in relation to other information. The approaches defined in
this thesis address aspects of integration and assimilation of information. This allows
information to be presented in context. Moreover, the use of a model-based approach
supports the generation of 'value added' metrics which can be produced through appropriate
combinations of measures or through measurements of the underlying Software Product
Model.

• An improved basis for software maintenance. Trials and case study results suggest that
the proposed approaches may have benefits for software maintenance. Since many projects
are requesting that the information generated by tools and environments during
development be provided to help support software maintenance, the integrated visualisation
approach may provide a more effective basis for using this information. As shown in the
FCS case study, vast amounts of information are produced by software projects. The use of
an IVES may provide a means of accessing, filtering, integrating, and customising this
information. Moreover, if an IVES is used during software development, it can be used to
capture other important information that often resides with the developers (eg rationale,
experiences). Further case studies are needed to help determine the potential benefits and
limitations of this approach for those engaged in software maintenance tasks.

181
7.5.2 Implications for CASE Tool Developers

The work presented in this thesis provides new insights into provision and use of descriptive
information. It highlights the need to consider the human issues related to the assimilation and
integration of information. This is particularly important for developers of integrated
environments such as Rational Apex which support the generation and management of a wide
variety of project information. This information needs to be made available by way of flexible
representation and presentation techniques which can support a wide range of users and tasks.

The use of a model-based visualisation approach may provide a means of more effectively
providing and using project information. This approach allows various types of information to
be integrated into consistent, composite representations based on an underlying Software
Product Model. This thesis has argued that process issues are also important. For example,
support needs to be provided for the generation of custom descriptions. When used together
with direct manipulation interfaces, these descriptions can be adapted to user needs. This is
important since it may not be possible to accurately specify information needs for many SE
tasks.

This thesis has demonstrated how a Software Engineering tool can be instrumented to capture
usage information. It contends that CASE tools and integrated environments should be
designed to include instrumentation features. This would allow for more effective evaluation
of CASE products in real situations. The FCS case study has shown that in addition to
supporting research goals, this type of instrumentation can be used by project personnel to
record the results of tasks and for providing evidence that particular tasks have been properly
undertaken. Moreover, information provided from instrumented CASE products can be used
to support metrics programmes.

7.5.3 Implications for SE Researchers

Christensen (1980) suggests that “scientific knowledge typically begins with description”. He
argues that it is the first objective of science to accurately describe the phenomenon of interest.
There is a need to portray an accurate picture of the phenomenon, identify variables that exist,
and determine the degree to which they exist. As such description is particularly important to
science, especially in areas where scientific knowledge is at an early stage of development (eg
Software Engineering). This thesis has presented an approach for describing Software Product
entities. Since software is a conceptual entity, a model-based approach was used to support
the description of software from multiple perspectives. In addition to providing indirect

182
visibility of the characteristics and nature of software, this approach provides a basis for
model-based measurement and the definition of valid SE measures and metrics.

Description plays an important part in measurement. For example, measurement involves

establishing mappings between the empirical domain and a formal domain. If the proposed
measures are to be meaningful and useful descriptions of the attributes of interest, this mapping
must be validated. This validation process involves making observations in the 'real world'
domain to ensure that the measures do in fact describe the attribute of interest. As discussed in
Section 7.3.3, the use of a model-based integrated visualisation approach could support the
definition and validation of measures since many aspects of the empirical domain are captured
and accessible through the descriptive model and can be made available by way of the
Integrated Visual Descriptions.

The use of SEE-Ada and the automated capture and update facility show that project
information can be captured to support research objectives. For example, the FCS case study
is capturing a variety of information about product evolution, the impact that various SE
processes have on the product (eg testing, integration, product evaluation), and information
use. Since this information is captured within a consistent framework, it provides a basis for
various research activities. For example, the information could be used to support metrics
research, investigations into the evolution of design models, and research into product
architectures.

7.6 Ongoing and Future Work

Although a significant amount of work has been done to date, it has only begun to address
problems and issues. Many unresolved problems and issues remain to be explored. These are
discussed in the following paragraphs.

A major area of current work is the FCS case study. This thesis reports on the first phase of
the case study. The case study design and experimental setup will be updated based on the
lessons learned from this initial phase. The next phase of this case study will also begin to
capture feedback from users on the preliminary effectiveness criteria. Other replicated studies
of this type are also proposed. Negotiations are underway to perform a case study on another
AWADI project. This project will develop a similar real-time embedded application using the
same type of development environment and development approach. Although a different
development team is being used, many of the experiences gained by the FCS project in the use

183
of Rational Apex and SEE-Ada should benefit the new project and the case study. Moreover,
the lessons learned and experiences gained from the FCS case study should benefit the way in
which the case study is conducted.

The FCS case study predominantly looked at the information needs of technical managers
employed on development tasks. Other case studies are needed to address the needs of
different classes of users. For example, important areas to be considered are software
maintenance, quality assurance, and IV&V tasks. Although SEE-Ada has been used to
support individuals in these types of tasks at various receptor sites, evidence from controlled
and well designed case studies is needed if benefits are to be quantified.

The Software Product Model is a key concept in this work. Additional research is required to
explore how this model should evolve in line with project time scales and user needs. For
example, issues related to update intervals and criteria need to be explored. Moreover, other
SPM implementation approaches need to be considered. The SPM in SEE-Ada is
implemented as an Entity Relationship Attribute schema in a relational database. Although this
proved satisfactory for the initial research, other approaches need to be considered to address
issues related to distributed SE environments, the effective implementation and use of 'link'
attributes, and storage efficiency. Other implementation approaches which might be
considered include conceptual graphs (Brachman and Schmolze 1985; Sowa 1991),
hypernodes (Levene and Offen 1992), and object oriented approaches (Dillon and Tan 1993).

A series of SPM snapshots could provide a basis for animating and viewing product evolution.
For example, it may be possible to show how the product architecture changes over time by
displaying the structural elements of the each SPM snapshot in a primary view. In addition to
showing how structural characteristics change over time, attribute information could be
overlayed onto these changing base representations to show the effect of various SE processes.
For example, test information could be used to show how testing was conducted.
Configuration management information could show where most change occurred during a
particular integration phase. These product-based animations could be used to 'replay' projects
to support process improvement activities, to support formal reviews, and to help train project
team leaders. They would also have value for SE educators in that students could gain a better
appreciation of how various processes impact software products on real projects. The current
version of SEE-Ada may support a rudimentary form of animation through the use of the

184
Script Mode interface. However, this is yet to be tested. The animation feature should be
considered for future IVES implementations and as part of any follow-on research.

Although SEE-Ada has proved to be effective for the research to date, a more general and
capable IVES is required to conduct a broader range of experimentation. SEE-Ada does not
fully implement the primary view concept of the integrated visualisation approach. Moreover,
the addition of secondary views requires considerable development effort. Although the
underlying design supports the modelling of a wide variety of entities, the representation and
presentation elements of SEE-Ada focus on the visualisation of Ada systems. A more generic
approach is needed so that a wide range of entities and languages can be supported. For
example, the description of large Commercial Off-The-Shelf (COTS) systems is of particular
interest.

There are also a range of issues that need to be investigated in terms of the facilitation and
mediation processes. For example, additional support for human cognitive aspects of the
Visual Interaction Process (Rogers 1995) need to be explored. This could involve the use of
knowledge based techniques to provide support for hypothesis management and attention
direction. Automated support for the Facilitation Process also needs to be explored. For
example, is it possible to automatically generate description specifications based on knowledge
of the task, user profile, and the context of use? Can the SPM be evolved automatically to
support particular user roles and tasks?

An important aspect of future work is the updating and extension of the theoretical
foundations. A range of new concepts, models, terminology, and criteria have been proposed.
These need to be further tested and extended based on experimental results. Moreover, this
area would benefit from a more formal treatment. The relationship between measurement and
description was discussed in Chapter 4. Additional research needs to be undertaken to
consider whether some of the foundations of measurement theory could be extended to
provide the basis of a more formal theory of description.

7.7 Original Content and Contributions

Original content has been summarised at beginning of each chapter. All concepts, models, and
approaches defined in this thesis are the original work of the author unless otherwise stated.

The work presented in this thesis has benefited from the efforts of numerous individuals.
Several individuals have been involved in the implementation of the SEE-Ada concepts and

185
other experimental apparatus under the direction of the author. Those involved in the
implementation of SEE-Ada have included Ian Turner, Richard Altmann, Neil Fisher, Matthew
Phillips, Peter Duffett, Craig Baker, and Grant Slade. Andrew Loja provided management
support to help schedule development activities, prepare documentation, and to provide
configuration management for the product. In terms of the other experimental apparatus,
Matthew Phillips developed the Project Information Resource Modelling and Analysis System
(PIRMAS) and the Usage Monitoring and Analysis System (UMAS). The SPM Measurement
and Analysis System was developed by Peter Duffett and Susan Bowron. Peter Duffett also
implemented the SPM capture and update mechanism. The efforts of all these people have had
a major impact on the work since the various trials and the FCS case study could not have
been undertaken without robust implementations. Moreover, this work has shown that the
concepts are in fact implementable and can be used in real situations.

The work has also benefited from feedback provided by many individuals. These include the
many reviewers of concepts and papers, participants of the Industry Case Study, and users at
the SEE-Ada receptor sites. Discussions held with Professor Agnes Kaposi, Dr Barbara
Kitchenham, and Professor David Budgen during a visit to the United Kingdom in November
1993 provided insights which helped direct the research approach and the definition of
underlying concepts. The many discussions held with Professor Ray Offen have been
instrumental in helping review the concepts and have helped establish my perspectives on
Software Engineering research.

186
 8. SUMMING UP

8.1 Summary
This thesis has identified, recorded, and discussed a range of problems associated with the
provision and use of information in Software Engineering (SE). These include problems
related to the unavailability and inaccessibility of information, information overload,
inappropriate use of representation and presentation techniques, inconsistency, and problems
related to the integration and assimilation of information. An analysis of these problems
resulted in the definition of a set of issues, preliminary effectiveness criteria, and conjectures.

Since there was no known body of work which provided the conceptual basis for this area of
research, a set of concepts, models, and definitions was defined and explored. The research
discussed in this thesis focused on the provision and use of descriptive information. This type
of information was defined in terms of the classes of information that could be used to
describe, and hence make visible, various software project entities (eg products, processes, and
resources). A conceptual process-based framework was defined as a basis for considering a
range of issues related to the provision and use of this type of information. The framework is
based on a Contextual Model. This model considers the SE process and the underlying project
information resources which can be used to support task-based information needs.

A Description Process was defined as a way of considering how descriptive information

products might most effectively be provided and used. This process comprises four main sub-
processes. The Provision Process addresses the generation of description products. The Use
Process looks at the cognitive and action processes of human users. Other processes which
form part of the Description Process are the Mediation Process which looks at interaction and
the exchange of knowledge between the user and provider of information, and the Facilitation
Process which supports the analysis of task-based information needs and directs the generation
of custom descriptions. When considered together, these processes can support the provision
of custom descriptions which can be adapted to user needs.

A model-based integrated visualisation approach was proposed as a means of integrating

various forms of project information by way of a Software Product Model and for supporting

187
the Description Processes. In addition to supporting the integration of underlying project
information, this model supports both representational and presentational integration. This
approach relies on the use of an integrated visualisation environment which provides a means
of accessing and filtering SE information, modelling and integration, and the generation of an
integrated set of primary and peripheral views. The use of computer-based visualisation
approaches provides a means of customising and adapting information to satisfy user needs.

The SEE-Ada environment is a practical implementation of these concepts and approaches. It

has been used to help explore, extend, and evaluate many of the concepts presented in this
thesis. This environment has been trialed in various settings and serves as the key apparatus
for a series of Industry Case Studies. The results of the first phase of an initial case study have
been presented in this thesis.

8.2 Achievements
An original approach has been conceived and analysed to address a range of issues related to
the effective provision and use of descriptive information in SE. Robust and supportable tools
have been developed to support the approach and to be used as a basis for experimentation.
The results to date show that there is good reason to believe that the approach has value for
those involved in SE practice, both now, and in the future when the concepts and tools have
been developed further.

The main achievements of this work include:

• The identification, recording, and analysis of a range of problems and issues related to the
provision and use of information in SE.

• The definition of new concepts and models, in a consistent conceptual framework, that can
be used to aid further research and provide a basis for new approaches which could
dramatically improve many areas of SE practice.

• The definition and use of a model-based integrated visualisation approach which addresses
many of the issues and problems related to the effective provision and use of descriptive
information.

• New insights into how information is produced and used in software engineering based on
case study experiences.

188
8.3 Limitations
The work presented in this thesis has several limitations. These have been discussed at
appropriate points in the thesis and include limitations of the theory and approach, limitations
of the implementation, and limitations related to the experimentation that has been conducted.

The theory presented in this thesis is an initial attempt at defining and demonstrating the
concepts. It needs to be further evaluated and systematically enhanced through future
iterations of the research process. In particular, the theory may benefit from a more formal
mathematical treatment. This would support reasoning and would help provide more rigorous
definitions of effectiveness. Since there is a clear relationship between measurement and
description, the mathematical and philosophical basis of measurement theory may support a
more formal theory of description.

Limitations of the integrated visualisation approach also need to be considered. This approach
is considered most applicable for those projects which have ready access to information in
electronic form. The approach may not be applicable in situations where tool vendors do not
provide access to underlying information or where the information products cannot be
effectively filtered.

Several limitations of the SEE-Ada implementation have been discussed. SEE-Ada does not
fully implement the integrated visualisation approach. It does not support the use of the
primary view visualisation approach for other than 'code' entities. Moreover, it is limited in the
types of peripheral views that can be provided. This environment focuses on the visualisation
and description of Ada software systems and hence is not suitable for systems developed in
other languages.

The discussions of the experimental results have highlighted limitations of the case study
approach and the fact that more evidence is required in many areas. This thesis presents the
results of the first phase of an initial case study. These results have supported the discussion of
some findings. However, there are many issues that can only be addressed with evidence
captured over a series of well designed case studies. For example, the case study focused on
the needs of software team leaders. Other case studies need to be conducted to study the
needs of those involved in maintenance, quality assurance, and IV&V tasks. Replication
studies are also required. The initial case study provides the pilot for conducting these future
studies.

189
8.4 Further Work
Many possible areas of follow-on research were discussed in Chapter 7. Phase 1 of the initial
case study has been completed. The following two phases of the case study will be conducted
to obtain results for the remainder of the FCS project. In addition to the information currently
being captured, Phase 2 of the case study will begin to capture user feedback on the
preliminary effectiveness criteria. Other case studies need to be undertaken to help refine the
concepts and to further investigate the conjectures and issues defined in this thesis.

There is a need to further consider the cognitive science aspects of this work. This would help
build better 'cognitive bridges' between humans and providers of descriptive information. The
approaches for facilitation and mediation presented in this thesis have been largely mechanical.
For example, the facilitation process was based on the use of a human facilitator to perform
analysis and to generate SEE-Ada scripts. There is a need to consider how knowledge-based
techniques could be used to support users.

The SEE-Ada implementation has been a useful tool for the research to date. However, it is
limited in the manner in which it can support the integrated visualisation approach.
Consideration needs to be given to different approaches for implementing the Software
Product Model. A more flexible environment is needed to help explore a range of issues
related to representation and presentation techniques. The use of Integrated Visual
Descriptions for animation purposes has been considered, but cannot be fully investigated due
to inherent limitations in SEE-Ada. A more general and fully implemented IVES needs to be
developed to support future phases of the research. This environment would allow research to
be conducted in different development domains (eg COTS-based development), for different
programming languages, and for a wider range of SE activities (eg re-engineering, reuse, risk
analysis).

8.5 Potential Impact

As a final word, this section speculates as to the potential short, medium, and long term impact
of this work.
Several immediate benefits of using the integrated visualisation approach have been
demonstrated. The approach can support a range of SE tasks that were difficult, if not
impossible, to conduct using current forms of information. In particular, the approach

190
provides an effective way of using metric information for analysis, evaluation, inspection and
monitoring tasks.

Since the approach is based on the use of a consistent underlying representation of the
Software Product, an IVES could provide a common basis for providing project and product
visibility from various perspectives (eg as a basis for communication between customers,
managers, and developers). Since up-to-date information can be made readily available and
can be integrated and customised to specific needs, this approach could reduce the need for
much of the project 'paperwork' currently being produced. This would have significant
implications for project costs and the productivity of development teams.

The approach proposed in this thesis may also provide a better basis for software maintenance.
Information captured in CASE environments could be more effectively used and supplemented
with knowledge gained during the development process. The cost of maintaining information
would be reduced since descriptions could be automatically generated from underlying project
information.

In the medium term, the visualisation and description concepts discussed in this thesis could be
introduced into new generations of CASE environments. This would allow the approach to be
more readily used by projects since the CASE developers would consider access and filtering
as part of their designs. The use of these enhanced CASE environments would allow for more
effective use of project information. Improved visibility could provide earlier identification of
project risks and provide a better basis for collaborative work.

In the long term, these types of approaches could dramatically change the way in which
software is procured. Since customers would have an appropriate, up-to-date view of the
evolving product (and its related process and resource characteristics) they may be more
inclined to accept evolutionary development and acquisition approaches.

191
 References

Ackermann, D., and Tauber, M. J. (1990). “Mental Models and Human-Computer

Interaction.” , North-Holland, Amsterdam, 387.

AD/Cycle. (1992). “AD/Cycle Information Model.” GC26-4843-03, IBM Corporation,

Department J90, San Jose.

AdaQuest. (1995). “Using AdaQuest.” ADCN124, General Research Corporation, 5383

Hollinster Av, Santa Barbara, CA.

Albrecht, A. J., and Gaffney, J. E. (1983). “Software Function, Source Lines of Code, and
Development Effort Prediction: a Software Science Validation.” IEEE Transactions on
Software Engineering, 9(6), 639-648.

Anderson, J. R. (1983). The Architecture of Cognition, Harvard University Press, Cambridge,

MA.

Andres, D. (1989). “Software Project Management Using Effective Process Metrics: The
CCPDS-R Experience.”TRW-TS-89-01.

Apex. (1993). “Using Rational Apex.” 4100-00285, Rational Software Corporation, Santa
Clara, CA.

Baecker, R. M., and Marcus, A. (1990). Human Factors and Typography for More Readable
Programs, ACM Press, Reading MA.

Bailes, P. A., Burnim, P., and Johnston, D. (1995). “Knowledge-Based Requirements Analysis
for Ada Design Recovery: Design Entity Identification and Representation.” Dept of Computer
Science Technical Report No 333 , University of Queensland, Brisbane, Australia.

Baker, M. J., and Eick, S. G. (1993). “Visualizing Software Systems.” Sixteenth International
Conference on Software Engineering , Baltimore MD, 59-67.

Barnard, P. J. (1987). “Cognitive Resources and the Learning of Human-Computer Dialogs.”

Interfacing Thought: Cognitive Aspects of Human Computer Interaction, J. M. Carroll, ed.,
MIT Press, Cambridge, MA., Ch 6.

Barnes, G. E. (1993). “Ada Semantic Interface Specification: Detailed Semantics and

Implementation.” Version 1.1.0, ASIS Working Group, Association of Computing Machinery,
McLean Va.

Basili, V. R. (1992). “The Experimental Paradigm in Software Engineering.” Experimental

Software Engineering Issues: Critical Assessment and Future Directions, H. D. Rombach, et
al., ed., Springer-Verlag, Berlin, 3-12.

Basili, V. R., Gannon, J. D., and Katz, E. E. (1986a). “Metrics for Ada Packages: An Initial
Study.” Communications of the ACM , 29(7), 616-623.

192 References
Basili, V. R., and Rombach, H. D. (1988). “The TAME Project: Towards Improvement-
Orientated Software Environments.” IEEE Transactions on Software Engineering, 14(6), 758-
773.

Basili, V. R., Selby, R. W., and Hutchins, D. H. (1986b). “Experimentation in Software

Engineering.”IEEE Transactions on Software Engineering
, 12(7), 733-743.

Basili, V. R., and Weiss, D. M. (1984). “A Methodology for Collecting Valid Software
Engineering Data.”IEEE Transactions on Software Engineering
.

Basili, V. R., Wu, L., and Reed, K. (1987). “A Structure Coverage Tool for Ada Software
Systems.” Joint Ada Conference 1987 .

Berleant, D., and Berghel, H. (1994). “Customizing information: Part 1. Getting what we
need, when we need it.”IEEE Computer, September, 96-98.

Berry, D. M. (1992). “Academic Legitimacy of the Software Engineering Discipline.”

CMU/SEI-92-TR-34, Software Engineering Institute.

Blum, B. I. (1989). “Documentation for Maintenance: A Hypertext Design.” IEEE Conference

on Software Maintenance , 23-31.

Booch, G. (1991). Object Oriented Design with Applications, Benjamin/Cummings, Menlo

Park, CA.

Bowen, T., Wigle, G., and Tsai, J. (1985). “Specification of Software Quality Attributes.”
RADC-TR-85-37, US Air Force Materiel Command, Rome Laboratory, Griffiss AFB, NY
13441.

Brachman, R. J., and Schmolze, J. G. (1985). “An Overview of the KL-ONE Knowledge
Representation System.”Cognitive Science, 9, 171-216.

Brodlie, K. W., Carpenter, L. A., Earnshaw, R. A., Gallop, J. R., Hubbold, R. J., Mubford, A.
M., Osland, C. A., and Quarendon, P. (1992). Scientific Visualization: Techniques and
Applications, Springer_Verlag, Berlin.

Brooks, F. P. (1987). “No Silver Bullet: Essence and Accidents of Software Engineering.”
IEEE Computer, Apr 1997, 10-19.

Brown, A., and Penedo, M. (1992). “An Annotated Bibliography on Software Engineering
Environment Integration.”ACM Software Engineering Notes
, 17, 47-55.

Brown, A. W., Deiler, P. H., and Wallnau, K. C. (1992). “Past and Future Models of CASE
Integration.” 5th International Workshop on Computer Aided Software Engineering
, 36-45.

Browne, J. C., and Shaw, M. (1981). “Towards a Scientific Basis for Software Evaluation.” in
Software Metrics (eds.Perlis et al., )MIT Press, Chap 1.

Budgen, D., Marashi, M., and Reeves, A. (1993). “Case Tools: Masters or Servants?”
Software Engineering Environments '93
, Reading, England.

Campbell, N. R. (1957).Foundations of Science, Dover Publications, New York.

193 References
Card, D. N., and Glass, R. L. (1990). Measuring Software Design Quality, Prentice-Hall,
Englewood Cliffs, NJ.

Card, D. N., McGarry, F. E., and Page, G. T. (1987). “Evaluating Software Engineering
Technologies.”IEEE Transactions on Software Engineering
, 13(7), 845851.

Card, S. K., Moran, T. P., and Newell, A. (1983). The Psychology of Human-Computer
Interaction, Lawrence Erlbaum Associates, Hillsdale, NJ.

Christensen, J. (1980).Experimental Methodology

, Allyn and Bacon Inc, Boston.

Colby, G., and Scholl, L. (1991). “Transparency and Blur as Selective Cues for Complex
Visual Information.”Image Handling and Reproduction Systems, San Jose, California.

Conklin, J. (1987). “Hypertext: An Introduction and Survey.”

IEEE Computer, 20(9), 17-41.

Conte, S. D., Dunsmore, H. E., and Shen, V. Y. (1986). Software Engineering Metrics and
Models, Benjamin Cummings, Menlo Park, CA.

Curtis, B. (1980). “Measurement and Experimentation in Software Engineering.” Proceedings

of the IEEE, 68(9), 1144-1157.

Curtis, B., Soloway, I., Ehrlich, K., and Brooks, R. (1984). “Cognitive Perspectives on
Software Science.”Information Processing Management , 28(1), 81-96.

Davidson, C. (1993). “What your Database Hides Away.”

New Scientist, Jan 1993.

Delaney, R. P., and Summerill, L. F. (1988). “Ada Software Metrics.” 6th National
Conference on Ada Technology 1988
.

Devanbu, P., Brachman, R. J., Selfridge, P. G., and Ballard, B. W. (1991). “LaSSIE: A
Knowledge-Based Software Information System.” Communications of the ACM , 34(5), 34-49.

Dillon, T. S., and Tan, P. L. (1993). Object-Oriented Conceptual Modeling, Prentice Hall,
Sydney.

DOD-STD-2167. (1985). “Defence System Software Development.” , U.S. Department of

Defence.

DOD-STD-2167A. (1987). “Defence System Software Development.” , U.S. Department of

Defence.

Duvall, L. M. (1993). “The Information Needs of Software Managers: A Problem Driven

Perspective.” Seventeenth Annual International Computer Software & Applications
Conference, Phoenix, Arizona.

Earnshaw, R. A., and Wiseman, N. (1992). An Introductory Guide to Scientific Visualization,

Springer-Verlag, Berlin.

Eick, S. G., Steffen, J. L., and Sumner, E. E. (1992). “SeeSoft: A Tool for Visualizing Line
Oriented Software Statistics.”IEEE Transactions on Software Engineering , 18(11), 957-967.

194 References
Emmerich, W., Schafer, W., and Welsh, J. (1992). “Databases for Software Engineering
Environments.”Memo Nr. 66, Software Technologie, University of Dortmund, Germany.

Fenton, N., Pfleeger, S. L., and Glass, R. L. (1994). “Science and Substance: A Challenge to
Software Engineers.”IEEE Software, July, 86-95.

Fenton, N. E. (1991). Software Metrics: A Rigorous Approach

, Chapman and Hall.

Fernstrom, C., Narfelt, K., and Ohlsson, L. (1992). “Software Factory Principles, Architecture,
and Experiments.”IEEE Software.

Finkelstein, L. (1975). “Representation by Symbol Systems as an Extension of the Concept of

Measurement.”Kybernetes, 4, 215-223.

Finkelstein, L. (1982). “Theory and Philosophy of Measurement.” In Handbook of

Measurement Science (ed. P.H. Sydenham), 36-49.

Garg, P. K., and Scacchi, W. (1994). “Ishys: Designing an Intelligent Software Hypertext
System.” IEEE Expert, Fall, 52-63.

Glass, R. L. (1994). “The Software Research Crisis.”

IEEE Software, November, 42-47.

Glass, R. L. (1995). “A Structure-Based Critique of Contemporary Computing Research.”

Journal of Systems and Software
, Jan.

Grady, R. B., and Caswell, D. L. (1987). Software Metrics: Establishing a Company-Wide

Program, Prentice-Hall.

Grant, S., and Mayes, T. (1991). “Cognitive Task Analysis?” Human-Computer Interaction
and Complex Systems, G. R. S. Weir and J. L. Alty, eds., Harcourt Brace Jovanovich, London,
147-167.

Gray, L. (1994). “How Should Military Ada Software be Documented?” Guidelines for
Successful Acquisition and Management of Software Intensive Systems, Software Technology
Support Center, United States Air Force, Salt Lake City, Utah.

Halstead, M. H. (1977).Elements of Software Science

, Elsevier, New York.

Helmholz, H. (1930). Counting and Measuring (translated by C.L. Bryan from original work
published in 1887), Van Nostrand, New York.

Hetzel, B. (1993). Making Software Measurement Work

, John Wiley &Sons, New York.

Hewett, J., and Durham, T. (1989). CASE: the Next Steps, Ovum Ltd, 7 Rathbone St, London
W1P 1AF.

Humphrey, W. S. (1989).Managing the Software Process

, Addison-Wesley, New York.

Humphrey, W. S. (1995). A Discipline for Software Engineering, Addison-Wesley, Reading,

MA.

195 References
Hutchins, E. L., Hollan, J. D., and Norman, D. A. (1986). “Direct Manipulation Interfaces.”
User Centered System Design, D. A. Norman and S. W. Draper, eds., Lawrence Erlbaum
Associates, Hillsdale NJ, 31-61.

IEEE_Standards. (1994). “IEEE Software Engineering Standards Collection.” , Institute of

Electrical and Electronics Engineers, 345 East 45th Street, New York, NY 10017-2394.

IEEE_Std_610.12-1990. (1990). “Standard Glossary of Software Engineering Terminology.” ,

Institute of Electrical and Electronics Engineers, 345 East 45th Street, New York, NY 10017-
2394.

IEEE_Std_1074-1991. (1991). “IEEE Standard for Developing Software Life Cycle

Processes.” , Institute of Electrical and Electronics Engineers, 345 East 45th Street, New
York, NY 10017-2394.

ISO/IEC_9126-1991. (1991). “Software Product Evaluation - Quality Characteristics and

Guidelines for their Use.” , International Standards Organisation.

ISO/IEC_12207-1995. (1995). “Information Technology - Software Life Cycle Processes.” ,

International Standards Organisation.

Jazzar, A. A. (1988). “Understanding the Production and Consumption of Software

Documentation: An Empirical Analysis and Model,” PhD dissertation, University of Southern
California.

Jeffery, D. R., and Offen, R. J. (1993). “Successful Metrics Initiatives: Key Elements for
Metrics Technology Adoption (Half Day Tutorial).” Seventh Australian Software Engineering
Conference,, Sydney, Australia.

Johnson, W. L. (1994). “Dynamic (Re) Generation of Software Documentation.” Fourth

System Re-engineering Technology Workshop
, John Hopkins University, 57-66.

Johnson-Laird, P. N. (1989). “Mental Models.” Foundations of Cognitive Science, M. I.

Posner, ed., The MIT Press, Cambridge, MA, 469-499.

Jones, C. (1992). “Risky Business: The Most Common Software Risks.” American
Programmer, 5(7), 19-29.

Jones, C. (1995). “What goes into an information warehouse?”

IEEE Computer(Aug.), 84-85.

Kaposi, A., and Pyle, I. (1993). “Systems are not only software.” Software Engineering
Journal, Jan 1993.

Kaposi, A. A. (1991). “Measurement Theory.” Software Engineers Reference Book, J. A.

McDermid, ed., Butterworth-Heinemann, Oxford, Chap 12.

Kaposi, A. A., and Meyers, M. (1994). Systems, Models and Measures, Springer Verlag,
London.

Keller, P. R., and Keller, M. M. (1993). Visual Cues, IEEE Computer Society Press, Los
Alamitos, CA.

196 References
Keller, S. E., and Laurence, G. K. (1988). “Specifying Software Quality Requirements with
Metrics.”, Dynamics Research Corporation, Andover, MA.

Keller, S. E., and Perkins, J. A. (1985). “An Ada Measurement and Analysis Tool.” Annual
National Conference on Ada Technology , 188-196.

Kemerer, C. F. (1992). “How the Learning Curve Affects CASE Tool Adoption.” IEEE
Software, May 1992, 23-28.

Kitchenham, B., and Pickard, L. (1995). “Using Case Studies to Evaluate Software
Engineering Methods and Tools.”IEEE Software, July.

Kitchenham, B. A., Linkman, S. G., and Law, D. T. (1994). “Critical Review of Quantitative
Assessment.”Software Engineering Journal , March, 43-53.

Kitchenham, B. A., Pickard, L. M., and Linkman, S. J. (1990). “An evaluation of some design
metrics.” Software Engineering Journal, 50-58.

Kitchenham, B. A., and Walker, J. G. (1989). “A Quantitative Approach to Monitoring

Software Development.”Software Engineering Journal
, 2-13.

Kraut, R. E., and Streeter, L. A. (1995). “Coordination in Software Development.”

Communications of the ACM , 38(3), 69-81.

Kuhn, T. S. (1962).The Structure of Scientific Revolutions

, University of Chicago.

Laird, J. E., Newell, A., and Rosenbloom, P. S. (1983). “Soar: An Architecture for General
Intelligence.”Artificial Intelligence, 33(1), 1-64.

Law, D. (1992). “DESMET Methodology: Overall Specification.” Project Deliverable 2.1,

National Computer Centre, Manchester, UK.

Lehman, M. M. (1991). “Software Engineering, the Software Process and their Support.”
Software Engineering Journal
, Sep 1991, 243-258.

Lehman, M. M., and Belady, L. A. (1985). Program Evolution: Processes of Software

Change, Academic Press, London.

Levene, M., and Offen, R. (1992). “A Unified Graph-Based Data and Process Model.”
RN/92/99, Department of Computer Science University College, London.

Lohse, G. L., Biolsi, K., Walker, N., and Rueter, H. H. (1994). “A Classification of Visual
Representations.”Communications of the ACM , 37(12), 36-49.

Love, B. (1993). Enterprise Information Technologies

, Van Nostrand Reinhold, New York.

Mackinlay, J. (1987). “Automating the Design of Graphical Presentations of Relational

Information.”ACM Transactions on Graphics
, 5(2), 110-141.

Marlin, C. D. (1990). “A Distributed Implementation of a Multiple View Integrated Software

Development Environment.” 5th Conference on Knowledge-Based Software Assistant, 388-
402.

197 References
McCabe, T. (1987). “Air Force Systems Command Software Quality Indicators.” AFSCP 800-
43.

McCabe, T. J. (1976). “A Complexity Measure.”

IEEE Trans Soft Eng, 2(4), 308-320.

McCall, J. A., Richards, P., and Walters, G. (1977). “Concepts and Definitions of Software
Quality Factors.” , NTIS AD-A049-015,015,055.

McCormick, B. H., DeFanti, T. A., and Brown, M. (1987). “Visualisation in Scientific

Computing.” Computer Graphics (AC-SIGGRAPH), 21(6), 1-15.

McDermid, J. A., and Rook, P. (1991). “Software Development Process Models.” Software
Engineers Reference Book, J. A. McDermid, ed., Butterworth-Heinemann, Oxford, Chap 15.

McGarry, F. (1995). “Experimental Software Engineering and the Role of Measurement

(Keynote Presentation).”ACOSM'95, Sydney Australia.

Meyer, C. A., Lindholm, S. C., and Jensen, J. L. (1989). “Experiences in Preparing a DOD-
STD-2167A Software Design Document for an Ada Project.” Proceeding of TRI-Ada 89,
ACM, 118-124.

Meyers, S. (1991). “Difficulties in Integrating Multiview Development Systems.” IEEE

Software, Jan 1991, 49-57.

Monmonier, M. (1991).How to lie with maps, University of Chicago Press, Chicago.

Murine, G., and Carpenter, C. (1983). Applying Software Quality Metrics, ASQC Quality
Congress Transactions, Boston.

Newell, A., Rosenbloom, P. S., and Laird, J. E. (1989). “Symbolic Architectures for
Cognition.” Foundations of Cognitive Science, M. I. Posner, ed., The MIT Press, Cambridge,
MA, 93-131.

Norman, D. A. (1986). “Cognitive Engineering.” User Centered System Design, D. A. Norman

and S. W. Draper, eds., Lawrence Erlbaum Associates, Hillsdale NJ, 31-61.

Norman, D. A. (1993).Things That Make Us Smart

, Addison-Wesley, Reading MA.

Nuseibeh, B., and Finkelstein, A. (1992). “ViewPoints: A Vehicle for Method and Tool
Integration.” Int. Workshop on CASE, Montreal, 50-60.

Offen, R., and Aronov, V. (1993). “A Model-Based Approach to Software Measurement.”

ACOSM'93.

Offen, R., and Xia, G. (1995). “Extending the Scope of Measurement and Experimentation:
Some Important Methodological Issues.” Second Australian Conference on Software Metrics,
Sydney, 78-82.

Offen, R. J. (1994). “The Hypernode Meta-Model as a Basis for Software Process Modelling.”
Seventeenth Australasian Computer Science Conference , Christchurch NZ.

198 References
Oinas-Kukkonen, H. (1992). “Empowering the Components and Routine Use of Case
Environments with Hypertext Functionality.” Golden West International Conference on
Intelligent Systems, Reno, Nevada, 201-204.

Park, R. E. (1992). “Software Size Meaurement: A Framework for Counting Source

Statements.” CMU/SEI-92-TR-20, Software Engineering Institute, Pittsburgh, PA.

Parker, B. (1992). “Introducing EIA-CDIF: The Case Data Interchange Format Standard.”
Second Symposium on Assessment of Quality Software Development Tools, New Orleans, 74-
82.

Parnas, D. L., and Clements, P. C. (1986). “A Rational Design Process: How and Why to Fake
It.” IEEE Transactions on Software Engineering , SE-12(2), 251-257.

Payne, S. J., and Green, T. R. G. (1986). “Task-Action Grammars: A Model of Mental

Representation of Task Languages.”Human-Computer Interaction
, 2, 93-133.

Popper, K. R. (1977). The Logic of Scientific Discovery

, Hutchinson, London.

Posner, M. I. (1989). Foundations of Cognitive Science

, The MIT Press, Cambridge, MA.

Preece, R. (1994). Starting Research, Printer Publishers, London.

Price, B. A., Baecker, R. M., and Small, I. S. (1993). “A Principled Taxonomy of Software
Visualisation.”Journal of Visual Languages and Computing , 4(3), 211-266.

Rada, R., Wang, W., Mili, H., Heger, J., and Scherr, W. (1992). “Sofware reuse: from text to
hypertext.” Software Engineering Journal (Sept), 311-321.

Rader, J., Brown, A. W., and Morris, E. J. (1995). “Operational Use of CASE Integration: An
Investigation of the State of the Practice.”Journal of Systems Software
, 28, 59-68.

Rational_Approach. (1994). “The Rational Approach.” TP-39, Rational Software Corporation,

2800 San Tomas Expressway, Santa Clara CA.

Robertson, G. G., Card, S. K., and Mackinlay, J. D. (1993). “Information Visualisation Using
3D Interactive Animation.”Communications of the ACM , 36(4), 57-71.

Roche, J. M. (1994). “Software Metrics and Measurement Principles.” Software Engineering

Notes, 19(1), 77-85.

Rogers, E. (1992). “Visual Interaction: a link between perception and problem solving,” PhD
Dissertation, Georgia Institute of Technology.

Rogers, E. (1995). “Cognitive Cooperation through Visual Interaction.” Knowledge-Based

Systems, 8(2), 117-125.

Rombach, H. D. (1991). “Practical Benefits of Goal-Oriented Measurement.” in Software

Reliability and Metrics (eds. Fenton N., Littlewood B.)
, Elsevier Applied Science, Chap 14.

Roodbeen, S. (1995). “A Navy Software Reuse Conducive Software Engineering

Environment.”Seventh Annual Software Technology Conference
, Salt Lake City, Utah.

199 References
Rook, P. (1986). “Controlling Software Projects.”Software Engineering Journal
, 1(1), 7-16.

Rose. (1994). “Getting Started with Rational Rose.” 4100-02073, Rational Software
Corporation, Santa Clara, CA.

Saeki, M. (1995). “Communication, Collaboration, and Cooperation in Software Development

- How should we support group work in software development?” Asia Pacific Software
Engineering Conference, Brisbane, Australia, 12-20.

Scacchi, W. (1989). “Engineering Large-Scale Software Systems: an Organizational

Knowledge Base Approach.”COMPCOM, 232-235.

Schneiderman, B. (1993). Sparks of Innovation in Human-Computer Ineraction, Ablex Publ.,

Norwood, NJ.

Schultz, H. P. (1988). “Software Management Metrics.” AFSCP 800-43, Mitre Corporation

(for U.S. Air Force) Electronics Systems Division, U.S. Air Force.

SEE-Ada_V3. (1995). SEE-Ada Version 3 User's Guide, Defence Science and Technology
Organisation, Adelaide, Australia.

Shneiderman, B. (1983). “Direct Manipulation: A step beyond programming languages.” IEEE

Computer, 16(8), 57-68.

Shneiderman, B. (1994). “Dynamic Queries for Visual Information Seeking.” IEEE

Software(November), 70-77.

Simmons, D. B. (1992). “A Win-Win Metric Based Software Management Approach.” IEEE

Transactions on Engineering Management
, 39(1).

SoDA. (1994). “Using SoDA 1.3.” 4100-01489, Rational Software Corporation, Santa Clara,
CA.

Sowa, J. F. (1991). “Principles of Semantic Networks.” , Morgan Kaufmann Publishers, Sam

Mateo, CA, 582.

SPC_Guidelines. (1991). Ada Quality and Style: Guidelines for Professional Programmers
,
Software Productivity Consortium, 2214 Rock Hill Road, Herndon, Va.

Stasko, T. J., and Patterson, C. (1992). “Understanding and Characterizing Software

Visualization Systems.”IEEE Workshop on Visual Languages
, 3-10.

Sykes, J. B. (1976). “The Concise Oxford Dictionary of Current English.” , Oxford University
Press, London.

Tarski, A. (1954). Contributions to Theory of Models, Indagattiones Mathematicae, 16, 572-

88.

Tate, G., Verner, J., and Jeffery, R. (1992). “CASE: A Testbed for Modeling, Measurement,
and Management.”Communications of the ACM , 35(4), 65-72.

200 References
Taylor, R. N., Belz, F. C., Clarke, L. A., Osterweil, L., Selby, R. W., Wileden, J. C., Wolf, A.
L., and Young, M. (1989). “Foundations for the Arcadia Environment Architecture.” Sigplan
Notices, 24(2).

TestMate. (1993). “Using Testmate.” 4100-011411, Rational Software Corporation, Santa

Clara, CA.

Thayer, R. H. (1992). “Software Engineering Project Management.” , IEEE Computer Society

Press, Los Alamitos, CA.

Thayer, R. H., and Theyer, M. C. (1993). “Glossary of Software Engineering Terms.”

Software Engineering: A European Perspective, R. H. Thayer and A. D. McGettrick, eds.,
IEEE Computer Society Press, 587-647.

Vernik, R. J., and Burke, M. M. (1993). “Perspective-Oriented Description: Integrating and

Tailoring Information for Software Engineering.” 6th International Conference on Software
Engineering and its Applications
, Paris, France.

Vernik, R. J., and Kingston, G. I. (1993). “Description-Based Software Quality Evaluations.”

Proceedings of the Seventh Australian Software Engineering Conference , Sydney, Australia.

Vernik, R. J., and Landherr, S. F. (1993). “Lessons Learned from Quality Evaluations of Nulka
Software.” ERL-0761-RE, DSTO Australia.

Vernik, R. J., and Turner, I. (1992). “Techniques and Tools for Analysing Software Products.”
Australian Computer Journal , 24(3), 98-105.

Vernik, R. J., and Turner, I. (1993). “Software Reuse: Can it Deliver?” Proceedings of the
Seventh Australian Software Engineering Conference , Sydney, Australia.

Votta, L. G., and Porter, A. (1995). “Experimental Software Engineering, A Report on the
State of the Art.” 17th International Conference on Software Engineering, Seattle, U.S.A.,
277-279.

Wakeman, L., and Jowett, J. (1993). PCTE: The Standard for Open Repositories, Prentice
Hall, London.

Wall, L., and Schwartz, R. L. (1991). Programming Perl, O'Reilly & Associates, Sebatopol
U.S.A.

Walz, D., Elam, J., and Curtis, B. (1993). “Inside a Software Design Team: Knowledge
Acquisition, Sharing, and Integration.”Comm. of ACM, 36(10), 67-77.

White, I. (1994). Using the Booch Method: A Rational Approach, Benjamin Cummins,
Redwook City, CA.

Yin, R. K. (1984). Case Study Research Design and Methods, Sage Publications, Newberry
Park, CA.

Youll, D. P. (1990). Making Software Development Visible: Effective Project Control

, John
Wiley & Sons, Chichester, England.

201 References
Yovits, M. C. (1993). “Information and Data.” Encyclopedia of Computer Science, A. Ralston
and E. D. Reilly, eds., Van Nostrand Rienhold, New York, 653-655.

Zabala, E., and Taylor, R. (1993). “PowerTools: New Generation Data Presentation Tools.”
People and Computers VIII, J. L. Alty, ed., Cambridge University Press, Cambridge UK, 125-
142.

Zuse, H. (1991). Software Complexity Measures and Methods

, Walter de Gruyter, Berlin.

202 References
Appendix A: Perspectives on Software Projects and
Processes

This appendix provides background terminology and concepts related to software projects, and
processes. These definitions and concepts are used to support discussions in this thesis.

A1. Software Project

The term software project is often used but rarely defined. For example, this term is not even
defined in IEEE_Std_610.12-1990 which is seen as a key source of Software Engineering
terminology. In this thesis, a software project is defined as a system of processes, products,
and resources that combine to support the acquisition, development, and maintenance of
software. The processes are those entities that transform inputs into outputs. These software
lifecycle processes are discussed in Section A2. The products are typically information
products that are generated and transformed by the various processes. These can be
deliverable products and developmental products. The set of deliverable products is often
termed the Software Product and includes the complete set of computer programs,
procedures, and associated documentation and data (IEEE_Std_610.12-1990). The
developmental products are those underlying products used by the project in the development
or enhancement of the Software Product. These products can be considered as a project
resource. Other resources used by the project include people and tools.

A software project cannot be considered in isolation. The software project should be viewed
within a broader project context where the Software Products form part of some composite
product (eg a system containing several hardware and software products). In addition to
interactions with other parts of the overall project, consideration must also be given to the
interactions between software projects and the outside world (eg procurement agencies and the
operational domain).

The scope of a software project depends on the point of view. At a customer level, the project
is often seen as the total period from definition of initial needs through to disposal. This period
is often termed the software lifecycle. At the developer level, the project is generally
considered in terms of product development. At this level, developers see the project as
complete when a version of the product meets specified customer requirements. However, in
most cases, product development does not cease when the initial version of the product is

203 Appendix A
delivered to a customer. The product evolves through a process of enhancement and
modification to support changing needs or to rectify problems. During this maintenance phase,
maintainers often organise their work in terms of a series of small projects which result in
incremental product releases.

Projects can be defined in terms of the types of products being produced. For example, the
project for a large evolutionary (or E-type) product (Lehman and Belady 1985) can have
significantly different characteristics from projects that develop small, unsupported Personal
Computer applications. These large projects use a variety of lifecycle processes to support
activities such as quality assurance, Independent Verification and Validation, and joint review.
Moreover, the organisational processes such as management and improvement become vitally
important for these types of projects. This thesis is primarily concerned with the development
of these large 'E-type' software systems.

A2. Software Lifecycle Processes and Interactions

The term software process is often used in the SE literature. But what is a process?
ISO/IEC_12207-1995 defines a process as "a set of interrelated activities which transform
inputs to outputs". This standard defines processes in terms of sets of activities and tasks
which need to be performed. Tasks are well-defined work assignments and are defined as the
smallest unit of work subject to management accountability (IEEE_Std_1074-1991).
Humphrey (1995) extends these definitions by suggesting that a software process sets out the
technical and management framework for applying methods, tools, and people to the software
task.

ISO/IEC_12207-1995 defines a standard set of software lifecycle processes for software

projects. These are:

Primary Processes. The Primary Processes are those that serve primary parties during the
software lifecycle. These primary parties are the acquirer, the supplier, the developer, the
operator, and the maintainer of software products. Five processes are defined to support
the activities of these parties. These are the acquisition, supply, development, operation,
and maintenance processes. The development process is defined in terms of requirements
analysis, design, coding, integration, testing, installation, and acceptance activities.

• Supporting Processes. Supporting Processes are employed and executed, as needed, by

other processes. These processes are defined as the documentation, configuration

204 Appendix A
 management, quality assurance, verification, validation, joint review, audit, and problem
resolution processes.

• Organisational Processes. The Organisational Processes are employed by an organisation

to manage the activities being performed in other processes, provide and support the
project infrastructure, improve the way in which activities are performed, and provide
necessary training. The Organisational Processes are defined as the management,
infrastructure, improvement, and training processes.

A software project can be defined in terms of its primary, supporting and organisational
processes. Various supporting and organisational processes are defined for each of the
primary processes. For example, a management process may be defined for each primary
process that forms part of the project context.

A3. Software Engineering Process

This thesis defines the concept of a Software Engineering Process (shown in Figure A-1)
which includes those lifecycle processes required to provide cost-effective, reliably correct, and
high quality solutions to particular client needs. Central to the SE process is the Development
Process which supports activities such as requirements analysis, design, coding, etc. This
process could be substituted by the Maintenance Process for those projects that are involved in
supporting and evolving an existing product. Of particular interest in this thesis is the
Management Process which provides the technical management support for SE.

The SE Process exists within a project context. This is an important consideration since
project and external factors have a large impact on the way in which SE is undertaken. In
particular, there are a range of socio-technical aspects that need to be considered (Offen and
Xia 1995).

V&V
Quality
Improvement
Development Doc

Config Mgt Review

Management

SE Process Software
Needs Products

Project Context

205 Appendix A
Figure A-1 Software Engineering Process

206 Appendix A
 Appendix B: Preliminary Effectiveness Criteria

CRITERIA DEFINITION

Understandability The degree to which provided information uses techniques known to the
user. Information often needs to be presented in context to enhance
understanding. Other related criteria include legibility and
meaningfulness.

Meaningfulness The degree to which the choice of representation and presentation

techniques provides a valid description of the aspect of interest.

Communicability The degree to which information provides the correct message. Even
though information might be understandable and meaningful it might
send the wrong message.

Suitability The degree to which information is suitable for the intended purpose (ie
meets users needs). Other related criteria include understandability,
meaningfulness, communicability, legibility, and flexibility.

Legibility The degree to which users are able to perceive the required information.
The choice of representation and presentation techniques plays a large
part in the legibility of the resulting information.

Flexibility The ability to represent and present information in a variety of different

ways

Conciseness The degree to which information is free from superfluous or redundant

detail (ie provides only what is necessary).

Completeness The ability to describe all entities, attributes, and relationships that are
required to serve a particular need.

Accuracy The degree to which information is free from errors.

Precision The ability to distinguish between different representations.

Generality The ability of an information product to support more than one need.

Adaptability The ability of information to change to support changing needs.

Consistency The degree to which different classes of information can be used together
to provide a uniform description of an aspect of interest. Can be
considered in relation to the consistency of representations, between
multiple perspectives, and over time.

Availability The degree to which information can be provided when needed.

Accessibility The efficiency of access to appropriate information.

207 Appendix B
Composability The degree to which information can be integrated to form composite
descriptions (perhaps of different types, and from different sources).

Scalability The ability of representation and presentation approaches to effectively

describe larger numbers of entities, relationships or attributes.

Customisability The degree to which representations can be changed to serve a particular

need and user.

Tailorability The degree to which the amount of information can be modified to suit
particular needs.

Cost The cost of providing and using information Cost can be specified in
terms of the time taken to perform tasks, or money. The cost of not
performing tasks must also considered.

208 Appendix B
 Appendix C: SEE-Ada Details

This appendix provides details of the SEE-Ada Version 3 environment (SEE-Ada_V3 1995).
It supports the discussions in Chapters 5 and 6. This environment implements many of the
concepts defined in this thesis and provides the key apparatus for the Industry Case Studies.

C1. Overview
SEE-Ada is an integrated visualisation environment which allows information from a range of
project sources to be accessed, filtered, imported, and integrated. A set of integrated views
allows various types of information to be combined and displayed in a consistent and
customisable manner.

C2. Implementation Characteristics

The SEE-Ada environment has been developed to run on Sun Sparcstations under the Solaris
and SunOS operating systems. Figure C-1 provides an atomic description of a set of SEE-Ada
product attributes. The environment has been written predominantly in the Ada language. The
development approach is based on a component and design reuse strategy (Vernik and Turner
1993). It makes use of a range of general purpose components (eg Booch (1987)
components) and domain components (eg list display components, draw components etc). The
Oracle Relational Database System has been used as a Commercial-OFF-The-Shelf (COTS)
component to support database requirements.

No. LINES OF CODE: ................. 116,703

No. ADA LINES: ...........................70,281
No. COMMENT LINES: .............. 46,422
No. SOURCE FILES: .................... 679
No. COMPILATION UNITS: ..... 968
No. PROCEDURAL UNITS: ........ 5695

Figure C-1 SEE-Ada Version 3 Product Attributes

209 Appendix C
C3. Views
The main views provided by SEE-Ada are defined in the following table.

View Description

Subsystems The Subsystem View is a design level view. This view describes two
classes of entities: Logical Subsystems and Physical Subsystems.
Physical Subsystems encapsulate sets of Ada Units. Logical
Subsystems can encapsulate other Logical Subsystems or Physical
Subsystems. The symbols used in the Subsystems view are shown in
Table C-3

Layers The Layers View uses the primary view visualisation principles to
describe Ada entities. The view can be arranged based on 'with'
dependencies and can be customised to show individual Ada entities or
combinations of entities in terms of composite display groups.

Worksheet The Worksheet View is similar to the Layers View, however it allows
the user to organise the arrangement of entities. The view uses the
primary view visualisation principles and can show all classes of Ada
entities.

Contains The Contains View uses the primary view visualisation principles to
show those entities encapsulated in Ada library or compilation units.

Source The Source View provides a textual representation of sections of the

source code.

Graph The Graph View provides a directed graph representation of subsets of

units selected in the Layers or Worksheet Views. The lines between
entities are defined in terms of the relationship specified in the
Relationships Tool.

Details The Details View provides details of entities in a textual list form.

Show Values The Show Values View presents attribute information in terms textual
lists.

Table C-2 SEE-Ada Version 3 Product Attributes

210 Appendix C
C4. Symbol Mappings
The entities described in the SEE-Ada views have the following symbol mappings.

Symbol Description

Logical Subsystem

Physical Subsystem node

Package Specification

Package Body

Subprogram (body or specification)

Task Specification

Task Body

Null Object. Represents entities which are reference but have not been
compiled (eg Standard or vendor supplied libraries). May also represent
code that has not been delivered.

Generic Unit. These units are shown as dashed representations of the type of
Ada unit which is provided in generic form (eg the example shown is a
generic package specification)

Units can be elided to remove detail. This symbol represents the lowest level
of elision.

Table C-2 Symbol Mappings

211 Appendix C
 Appendix D: Fire Control System Project

This appendix provides details of the Fire Control System (FCS) Project which is the focus for
the Industry Case Study discussed in this thesis. The FCS is part of an Active Missile Decoy
(AMD) system which is being developed for the Royal Australian Navy. This system provides
self-protection for naval ships against modern anti-ship missiles. It allows for automatic or
operator designation of a missile threat. Upon designation of a particular threat, the system
will rapidly respond by launching an autonomous airborne decoy. Prior to decoy launch, the
system calculates the optimum decoy flight trajectory and programs this data into the decoy's
flight control unit. The hovering rocket decoy is then fired and positions itself to provide a
more attractive target to the anti-ship missiles. The decoy payload provides the means of
seducing a range of current and new generation anti-ship missiles.

The system design of the AMD is shown in Figure D-1. The FCS comprises three main
components. The software for the Remote Interface Unit (RIM) and Fire Control Panel (FCP)
are the focus of the Industry Case Study discussed in this thesis. The RIM provides interfaces
with the ship's combat system and the Electronic Support Measures (ESM) to gain information
on the threat type and direction. The Fire Control Panel (FCP) is the heart of the system. It
uses the information provided by the RIM to calculate the optimum launch time and trajectory
of the hovering rocket decoy. It also provides an operator interface, programs the decoys, and
initiates decoy launching.

D1. FCS Software Product Characteristics

The FCS software is a real-time embedded product. The target hardware comprises four
Motorola 68040 processors. Three of these processors support the FCP functionality and one
supports the RIM interface. The Ada software which is being developed is distributed across
these processors which provide support for Man Machine Interfacing (MMI), Input/Output,
and Flight Solutions. A major difficulty with this type of software is that it must conform to a
range of human and ship system interfaces. Timing is a critical aspect that needs to be
considered.

A key characteristic of the software is that it implements safety critical functions. This imposes
restrictions on how the product can be developed (eg the use of some language features is

212 Appendix D
restricted), and has an impact on the processes used for its development. For example,
extensive testing and formal verification play and important part of the development process.
Rigorous product inspections and evaluations are also considered to be integral to the process.

FIRE CONTROL SYSTEM

ESM
REMOTE INTERFACE
MODULE [RIM]

COMMAND
INTERFACE
SHIP
SHIP'S MODULE
DATA
COMBAT [CIM]
[WIND, GYRO FIRE CONTROL
SYSTEM
LOG] PANEL
[FCP]

SHIP SYSTEM

(DECOYS (DECOYS
X 4) X 4)

LAUNCHER (1) LAUNCHER (N)

AMD SYSTEM

Figure D-1 AMD System

D2. FCS Software Project Details

D2.1 Resources
The project is staffed and structured as shown in Figure D-2. The FCS case study is focusing
on the needs of the four software team leaders. The project has a senior team leader
responsible for the total software development project. Three other team leaders are
responsible for particular aspects of product development.

AMD FCS
Project Manager

Hardware Software Configuration

Team Leader Team Leader Manager

Team Leader Team Leader Team Leader

Systems MGT Threat MGT Infrastructure

Programmers

Figure D-2 FCS Staffing

D2.2 Use of IV&V

Due to the criticality of the software, Independent Verification and Validation (IV&V) of the
software is being performed by an independent contractor. This process is using
213 Appendix D
mathematically formal methods to support the verification of the source code. SEE-Ada is
also being used by the IV&V contractor to gain visibility of the product as it is being
developed.

D3. Tools and Environments

The main tools and environments used for this project include:

• Rational Apex. Rational Apex (Apex 1993) is an integrated programming support

environment. It uses an underlying persistent program representation to support an
interactive editor-based development model and tool integration. This representation is
based on the DIANA standard representation of Ada programs and can be accessed
through a standard Ada Semantic Interface Specification (ASIS). The DIANA
representation has been extended to manage a range of details about the software product
including requirements and design. Apex information management is supported by the
Configuration Management and Version Control (CMVC) system. CMVC can manage all
project information and supports multiple developers and teams. Apex also provides
design support by way of Rational Subsystems. These physical design entities can be used
to specify and manage systems of Ada compilation units.

• RCI for VADS cross compiler. The Apex environment can be used as a developmental
host. The Remote Compiler Interface (RCI) interface allows for seamless integration with
other compilation systems. For example, the FCS project uses a Verdix Ada Development
System for developing target code. RCI provides the interface to this cross compiler.

• Rational Rose. Rational Rose (Rose 1994) supports object-oriented development by

automating the use of the Booch notation and by generating Ada code from design models.
It provides support for both Object Oriented Analysis and Object Oriented Design. Rose
can be loosely integrated with the Apex CMVC.

• TestMate. TestMate (1993) is a set of tools that automates the development,

management, performance, and evaluation of functional and structural software tests. It
allows the development of suites of test cases that can be executed and evaluated
automatically. TestMate captures data about each test case, the pass/fail status, CPU and
elapsed execution time, and block or decision coverage.

• SoDA. The Software Documentation Automation product (SoDA 1994) helps automate
the production of technical software documentation. It is able to interface with various
214 Appendix D
 CASE tools (eg Rose) to extract required information. This information can then be
presented in standard document form (ie as DOD-STD-2167A documents).

• AdaQuest. AdaQuest (1995) is a product measurement tool which uses the ASIS
interface of the Rational Apex system to perform a static analysis of the Ada
implementation. AdaQuest performs five basic forms of analyses including Audit, Size,
Complexity, Profile, and Quality. It can provide a range of metric reports related to these
areas.

• SEE-Ada. SEE-Ada_V3 (1995) is being used to provide visibility of the product as it is

being developed. It supports a range of tasks including product inspections, product
evaluations, and status monitoring. It also provides a basis for communication between
various project personnel.

4. Development Process
The development process is based on an incremental approach (McDermid and Rook 1991;
Rational_Approach 1994) with 3 main builds. The first build was developed over a period of
23 weeks. The second and third builds are scheduled to be completed in 12 week periods.
The project is scheduled for completion in December 1996. The approach is based on
providing demonstrated functionality at each build review. The first build involves providing
the architectural framework and infrastructure of the product. As such, it provides a range of
interfaces and communication functions. Future builds will add additional functionality in an
incremental and controlled way.

The software is being developed to the DOD-STD-2167A (1987) standard. This approach
dictates the types of deliverable information products to be produced and the requirements for
many of the SE processes such as testing and product evaluation. The Rational SoDA tool is
being used to support the development of some of the deliverable documentation (eg Software
Design Documents).

The Rational Apex environment plays a central role in the development. It provides the
configuration management and version control for a range of project Information Products.
The use of Rational Subsystems also provides a basis for the definition and implementation of
the physical design of the software. The class categories defined in the logical design, as
developed using the Rose tool, are implemented as Rational subsystems as defined in the Apex
environment. Interfaces between, and characteristics of, these subsystems provide the basis for

215 Appendix D
controlling code development. At the implementation level, the product comprises a system of
subsystems. Each of these subsystems comprises a set of working and release views. A tower
is defined as a set of views of the same type, where one view is defined for each subsystem.
For example, the developmental release tower comprises a developmental release view for
each subsystem.

The project uses an incremental release strategy to support product evolution. The strategy is
supported by the use of Rational views and towers. Individuals developing code units for each
of the subsystems do so in their own working views. As each unit (eg a small set of one or
more Ada packages) is completed, it is migrated to a developmental release view which
provides a stable and controlled version which is integrated into the system as it evolves.
These developmental release views are used by other developers as they are implementing their
code units and for integration testing. This approach also provides a basis for monitoring
product evolution. The developmental release gives an objective indication of what has been
completed rather than relying on subjective estimates.

216 Appendix D
 Appendix E: Case Study Goals and Questions

G1. Identify how and when information is produced during software development.
Q1.1 What classes of information products are generated during software development?
Q1.2 What amounts of information do software projects generate?
Q1.3 When are the particular information products available?

G2. Investigate how descriptive information is used by software team leaders during software
development.
Q2.1 What tasks do team leaders do? How are they classified?
Q2.2 What descriptive information is required by those engaged in specific team leader tasks?
Q2.3 Can tasks be performed effectively with the information that is currently available?
Q2.4 How is the required information accessed and assimilated?
Q2.5 What problems are faced in relation to information use for different tasks?

G3. Investigate setup and support requirements for SEE-Ada.

Q3.1 What types of information can SEE-Ada support?
Q3.2 How is information accessed and filtered for use by SEE-Ada?
Q3.3 How can Software Product Model evolution be managed and controlled?
Q3.4 How does the Software Product Model evolve over time?

G4. Investigate how an IVES can be used to support team leader information needs.
Q4.1 What information does the user access for particular tasks?
Q4.2 What views and representations are used for particular tasks?
Q4.3 How much interaction is required for a user to satisfy their information needs?
Q4.4 Which tasks are best supported by the interactive approach?
Q4.5 Can custom IVDs be specified based on an analysis of task information needs?
Q4.6 Do the resulting IVDs meet the user’s needs?
Q4.7 How are custom descriptions adapted?
Q4.8 What problems are faced by users of anVES?
I

217 Appendix E
Appendix F: Case Study Information Requirements

Question Primary Information Type Source

Q1.1 IP Class Composite PIRMAS
Q1.2 IP Instance Identifier PIRMAS
Logical Size Integer
Physical Size Integer
Q1.3 Snapshot Date PIRMAS
IP Instance Identifier
Q2.1 Task Definition Text Case Study Folder
List SEE-Ada Task Lists
Q2.2 Task Definition List Case Study Folder
Information Requirement Text
Q2.3, Q2.4 Task Definition List Case Study Folder
Q2.5 Observations Text
Q3.1 Attribute Groups List SMAS
Product Entities List SEE-Ada
Relationships List
Q3.2 IP Classes Composite PIRMAS
IP Instances Identifier
Snapshot Date
Q3.3, Q3.4 Software Product Models Composite PMAS
Snapshot Date SEE-Ada
Q4.1 Entity, Attribute, Lists UMAS
Relationship
Task Descriptor Identifier
User Identifier
Q4.2 Views Used List UMAS
Task Descriptor Identifier
Q4.3 User Actions List UMAS
Timestamp Date/Time
Q4.4 Observations Text Case Study Folder
Problems Found List UMAS

218 Appendix F
Q4.5 Task Descriptor Identifier UMAS
Script Text
Q4.6 Usage Comments Text UMAS
Observations Text
Q4.7 User Actions List UMAS
Usage Log Text
Q4.8 Observations Text Research Folder

219 Appendix F
 Appendix G: Classes of Persistent Software
Information Products

ID Class Type Class Rep Class Form Provider

29 code.compilation.code_attributes Code.APEX UNIX.File.Binary APEX
19 code.compilation.export_set Text.List UNIX.File.Binary APEX
34 code.compilation.exports Text.List UNIX.File.ASCII APEX
33 code.compilation.imports Text.List UNIX.File.ASCII APEX
18 code.compilation.imports.all Text.List UNIX.File.ASCII APEX
35 code.compilation.imports.export_ Text.List UNIX.File.ASCII APEX
set
17 code.compilation.keys_timestamp Text UNIX.File.ASCII APEX
31 code.compilation.linker.api_list Text.List UNIX.File.ASCII APEX
30 code.compilation.linker.object_list Text.List UNIX.File.ASCII APEX
26 code.compilation.properties Attributes_List UNIX.File.ASCII APEX
11 code.executable Code UNIX.File.Binary APEX.Com
15 code.library Code UNIX.File.Binary APEX
16 code.library.diana Code.DIANA UNIX.File.Binary APEX
14 code.object Code.UNIX UNIX.File.ASCII APEX
173 code.source.prototype.release Text.Ada UNIX.File.ASCII APEX.RCS
175 code.source.prototype.working Text.Ada UNIX.File.ASCII APEX.RCS
153 code.source.release Text.Ada Paper.Listing APEX
13 code.source.release Text.Ada UNIX.File.ASCII APEX.RCS
12 code.source.working Text.Ada UNIX.File.ASCII APEX
199 code.subsystem Directory_Structure.APEX UNIX.Directory APEX
205 code.versioning Directory_Structure.APEX UNIX.Directory APEX
21 code.view.exports Text.List UNIX.File.ASCII APEX
23 code.view.exports.timestamps Text UNIX.File.ASCII APEX
20 code.view.imports Text.List UNIX.File.ASCII APEX
24 code.view.imports.export_set Text.List UNIX.File.ASCII APEX
22 code.view.imports.timestamps Text UNIX.File.ASCII APEX
28 code.view.lock_control Text.List UNIX.File.ASCII APEX
27 code.view.model Text.List UNIX.File.ASCII APEX
200 code.view.release Directory_Structure.APEX UNIX.Directory APEX
25 code.view.versioning Text.List UNIX.File.ASCII APEX
3 code.view.working Directory_Structure.APEX UNIX.Directory APEX
1 config_mgt.subsystem Directory_Structure.APEX UNIX.Directory APEX
7 config_mgt.versioning Directory_Structure.APEX UNIX.Directory APEX
4 config_mgt.view.release Directory_Structure.APEX UNIX.Directory APEX
45 design.diagram Diagram.Class Paper.Sheet Rose
47 design.diagram Diagram.Interaction Paper.Sheet Rose
48 design.diagram Diagram.Module Paper.Sheet Rose
46 design.diagram Diagram.Object Paper.Sheet Rose
50 design.diagram Diagram.Process Paper.Sheet Rose
44 design.diagram Diagram.Rose UNIX.File.ASCII.dsn Rose
49 design.diagram Diagram.State Paper.Sheet Rose

220 Appendix G
179 design.model.release Model.Class_Category UNIX.File.ASCII Rose
181 design.model.release Model.Kernel UNIX.File.ASCII Rose
182 design.model.release Model.Process UNIX.File.ASCII Rose
184 design.model.release Model.Properties UNIX.File.ASCII Rose
109 design.model.release Model.Reverse_Eng UNIX.File.ASCII.ptl Rose
185 design.model.release Model.Subsystem UNIX.File.ASCII.sub Rose
188 design.model.release Text UNIX.File.ASCII Person
39 design.model.working Model.Class_Category UNIX.File.ASCII.cat Rose
38 design.model.working Model.Kernel UNIX.File.ASCII. Rose
mdl
40 design.model.working Model.Process UNIX.File.ASCII.prc Rose
41 design.model.working Model.Properties UNIX.File.ASCII.prp Rose
43 design.model.working Model.Reverse_Eng UNIX.File.ASCII.ptl Rose
42 design.model.working Model.Subsystem UNIX.File.ASCII.sub Rose
187 design.model.working Text UNIX.File.ASCII Person
66 design.note.csc AWADI.SDF.Section Paper.Folder.Section Person
61 design.note.csci AWADI.SDF.Section Paper.Folder.Section Person
67 design.note.csu AWADI.SDF.Section Paper.Folder.Section Person
64 design.note.interface AWADI.SDF.Section Paper.Folder.Section Person
197 design.overview 2167A.SDD Paper.Book SoDA
201 design.versioning Directory_Structure.APEX UNIX.Directory APEX
130 design.view.release Directory_Structure.APEX UNIX.Directory APEX
131 design.view.working Directory_Structure.APEX UNIX.Directory APEX
202 development.file.csc AWADI.SDF Paper.Folder Person
203 development.file.csu AWADI.SDF Paper.Folder Person
204 development.file.interface AWADI.SDF Paper.Folder Person
89 development.note.general.csc AWADI.SDF.Section Paper.Folder.Section Person
88 development.note.general.csci AWADI.SDF.Section Paper.Folder.Section Person
90 development.note.general.csu AWADI.SDF.Section Paper.Folder.Section Person
91 development.note.general. AWADI.SDF.Section Paper.Folder.Section Person
interface
85 development.supporting_info.csc AWADI.SDF.Section Paper.Folder.Section Person
84 development.supporting_info.csci AWADI.SDF.Section Paper.Folder.Section Person
86 development.supporting_info.csu AWADI.SDF.Section Paper.Folder.Section Person
87 development.supporting_info.interface AWADI.SDF.Section Paper.Folder.Section Person
73 integration.build.note.csc AWADI.SDF.Section Paper.Folder.Section Person
72 integration.build.note.csci AWADI.SDF.Section Paper.Folder.Section Person
74 integration.build.note.csu AWADI.SDF.Section Paper.Folder.Section Person
75 integration.build.note.interface AWADI.SDF.Section Paper.Folder.Section Person
162 measure.code.complexity Numeric.Report UNIX.File.ASCII AdaQuest
155 measure.code.complexity Record.Comma_del UNIX.File.ASCII AdaQuest
163 measure.code.language_usage Numeric.Report UNIX.File.ASCII AdaQuest
156 measure.code.language_usage Record.Comma_del UNIX.File.ASCII AdaQuest
165 measure.code.quality Numeric.Report UNIX.File.ASCII AdaQuest
157 measure.code.quality Record.Comma_del UNIX.File.ASCII AdaQuest
164 measure.code.size Numeric.Report UNIX.File.ASCII AdaQuest
159 measure.code.size Record.Comma_del UNIX.File.ASCII AdaQuest
167 measure.code.structure Numeric.Report UNIX.File.ASCII AdaQuest
161 measure.code.structure Record.Comma_del UNIX.File.ASCII AdaQuest
148 monitoring.budget Spreadsheet.Excel Paper.Sheet MS_Excel

221 Appendix G
 51 monitoring.budget Spreadsheet.Excel UNIX.File.Binary.xls MS_Excel
132 monitoring.perform.baseline AWADI.Cost_Acc_Mgr_Hb Paper.Folder.Section Person
196 monitoring.performance.status AWADI.Cost_Acc_Mgr_Hb Paper.Folder.Section Person
55 plan.development 2167A.SDP Paper.Book Wordperf
53 plan.development 2167A.SDP UNIX.File.Binary.wp Wordperf
54 plan.development.diagram diagram UNIX.File.Binary.fm Frame
150 plan.schedule Database.MSProj Paper.Sheet MS_Project
52 plan.schedule Database.MSProj UNIX.File.Binary.mpp MS_Project
134 plan.test 2167A.STD Paper.Book Person
186 plan.test 2167A.STP UNIX.File.Binary.wp Person
2 policy.compilation.subsystem Attributes_List UNIX.File.ASCII APEX
59 policy.work_instruction AWADI.Project_Work_ Paper.Folder.Section Wordperf
Instruction
193 product.problems AWADI.System_Incident_ Paper.Folder Person.
Report
138 product.specification 2167A.SPS Paper.Book Person
141 product.user.information 2167A.SUM Paper.Book Person
139 product.version_descr 2167A.VDD Paper.Book Person
215 product.model SPM Database.Oracle SEE-Ada
58 requirement.diagram diagram UNIX.File.Binary.fm Frame
57 requirement.spec 2167A.SRS Paper.Book Wordperf
56 requirement.spec 2167A.SRS UNIX.File.Binary.wp Wordperf
81 review.notes.csc AWADI.SDF.Section Paper.Folder.Section Person
80 review.notes.csci AWADI.SDF.Section Paper.Folder.Section Person
82 review.notes.csu AWADI.SDF.Section Paper.Folder.Section Person
83 review.notes.interface AWADI.SDF.Section Paper.Folder.Section Person
194 review.slides Presentation.Slide Paper.Folder Person
77 safety_issues.csc AWADI.SDF.Section Paper.Folder.Section Person
76 safety_issues.csci AWADI.SDF.Section Paper.Folder.Section Person
78 safety_issues.csu AWADI.SDF.Section Paper.Folder.Section Person
79 safety_issues.interface AWADI.SDF.Section Paper.Folder.Section Person
133 tasking AWADI.Task_Statement Paper.Folder Person
168 test.case Text UNIX.File.ASCII.tc Test_Mate
206 test.code.executable Code UNIX.File.Binary APEX.Com
207 test.code.library Code UNIX.File.Binary APEX
208 test.code.library.diana Code.Diana UNIX.File.Binary APEX
176 test.code.source.release Text.Ada UNIX.File.ASCII APEX.RCS
178 test.code.source.working Text.Ada UNIX.File.ASCII APEX.RCS
209 test.code.subsystem Directory_Structure.APEX UNIX.Directory APEX
210 test.code.view.release Directory_Structure.APEX UNIX.Directory APEX
211 test.code.view.working Directory_Structure.APEX UNIX.Directory APEX
169 test.log Text.List UNIX.File.ASCII.tl Test_Mate
70 test.note.csc AWADI.SDF.Section Paper.Folder.Section Person
68 test.note.csci AWADI.SDF.Section Paper.Folder.Section Person
69 test.note.csu AWADI.SDF.Section Paper.Folder.Section Person
71 test.note.interface AWADI.SDF.Section Paper.Folder.Section Person
170 test.output.actual Text UNIX.File.ASCII Test_Mate
171 test.output.expected Text UNIX.File.ASCII Test_Mate
136 test.results 2167A.STR Paper.Book Person
172 test.results Text UNIX.File.ASCII.tr Test_Mate

222 Appendix G
116 versioning.changes.budget Record.RCS.V.xls UNIX.File.Binary.xls MS_Excel
92 versioning.changes.design.model Record.RCS.V.cat UNIX.File.ASCII Rose
93 versioning.changes.design.model Record.RCS.V.mdl UNIX.File.ASCII Rose
189 versioning.changes.design.model Record.RCS.V.overview UNIX.File.ASCII APEX.RCS
95 versioning.changes.design.model Record.RCS.V.prc UNIX.File.ASCII Rose
94 versioning.changes.design.model Record.RCS.V.prp UNIX.File.ASCII Rose
96 versioning.changes.design.model Record.RCS.V.sub UNIX.File.ASCII Rose
120 versioning.changes.plan Record.RCS.V.wp UNIX.File.Binary Wordperf
123 versioning.changes.plan.diagram Record.RCS.V.fm UNIX.File.Binary.fm Frame
126 versioning.changes.requirements Record.RCS.V.wp UNIX.File.Binary.wp Wordperf
129 versioning.changes.requirements. Record.RCS.V.fm UNIX.File.Binary.fm Frame
diagram
110 versioning.changes.schedule Record.RCS.V.mpp UNIX.File.Binary MS_Project
10 versioning.changes.source Record.RCS.V.source.ada UNIX.File.ASCII APEX
212 versioning.changes.source.test Record.RCS.V.source.ada UNIX.File.ASCII APEX
190 versioning.changes.test_plan Record.RCS.V.wp UNIX.File.Binary Wordperf
115 versioning.comments.budget Record.RCS.notes UNIX.File.ASCII APEX.RCS
102 versioning.comments.design.model Record.RCS.notes UNIX.File.ASCII APEX.RCS
119 versioning.comments.plan Record.RCS.notes UNIX.File.ASCII APEX.RCS
122 versioning.comments.plan.diagram Record.RCS.notes UNIX.File.ASCII APEX.RCS
125 versioning.comments.requirements Record.RCS.notes UNIX.File.ASCII APEX.RCS
128 versioning.comments.requirements. Record.RCS.notes UNIX.File.ASCII APEX.RCS
diagram
112 versioning.comments.schedule Record.RCS.notes UNIX.File.ASCII APEX.RCS
8 versioning.comments.source Record.RCS.notes UNIX.File.ASCII APEX.RCS
213 versioning.comments.source.test Record.RCS.notes UNIX.File.ASCII APEX.RCS
192 versioning.comments.test_plan Record.RCS.notes UNIX.File.ASCII APEX.RCS
113 versioning.details.budget Record.RCS.control UNIX.File.ASCII APEX.RCS
97 versioning.details.design.model Record.RCS.control UNIX.File.ASCII APEX.RCS
118 versioning.details.plan Record.RCS.control UNIX.File.ASCII APEX.RCS
121 versioning.details.plan.diagram Record.RCS.control UNIX.File.ASCII APEX.RCS
124 versioning.details.requirements Record.RCS.control UNIX.File.ASCII APEX.RCS
127 versioning.details.requirements. Record.RCS.control UNIX.File.ASCII APEX.RCS
diagram
111 versioning.details.schedule Record.RCS.control UNIX.File.ASCII APEX.RCS
9 versioning.details.source Record.RCS.control UNIX.File.ASCII APEX.RCS
214 versioning.details.source.test Record.RCS.control UNIX.File.ASCII APEX.RCS
191 versioning.details.test_plan Record.RCS.control UNIX.File.ASCII APEX.RCS

223 Appendix G
 Appendix H: Team Leader Task Definitions

This appendix provides the initial task definitions that were specified for the Industry Case
Study. The selections that could be made from the SEE-Ada usage monitor are provided as
examples of instances of the various classes of tasks.

1) Evaluation: Systematic determination of the extent to which an entity meets its

specified criteria.
• Evaluate.Code
• Evaluate.Code.Ada_95_Compatibility
• Evaluate.Code.Architecture
• Evaluate.Code.Coding_Practices
• Evaluate.Code.Commenting
• Evaluate.Code.Error_Handling
• Evaluate.Code.Init_and_Shutdown
• Evaluate.Code.Layout
• Evaluate.Code.Multiprocessing
• Evaluate.Code.Naming
• Evaluate.Design

2) Analysis: Examination for the purpose of understanding. (eg may need to perform
analysis to gain sufficient understanding to specify a solution to a problem).
• Analyse.Code
• Analyse.Code.Characteristics
• Analyse.Code.Dependencies
• Analyse.Code.Features
• Analyse.Design
• Analyse.Design.Dependencies
• Analyse.Design.Features
• Analyse.Product
• Analyse.Product.Problems

3) Inspection: Examination to identify potential risks and product problems based on

user's knowledge and experience. Inspections differ from evaluations in that they do
not use specified criteria.

224 Appendix H
 • Inspect.Code
• Inspect.Code.Configuration
• Inspect.Code.Descriptiveness
• Inspect.Code.Maintainability
• Inspect.Code.Numerical_Methods
• Inspect.Code.Portability
• Inspect.Code.Reliability
• Inspect.Code.Requirements_Allocation
• Inspect.Code.Safety
• Inspect.Design
• Inspect.Design.Requirements_Allocation
• Inspect.Design.Structure

4) Monitoring: Examination to assess progress and status of activities.

• Monitor.Configuration_Status
• Monitor.Integration_Status
• Monitor.Release_Status

5) Recording: Capture or update of persistent information to support future needs.

• Record.Code
• Record.Code.Features
• Record.Code.Requirements_Allocation
• Record.Design
• Record.Design.Changes
• Record.Design.Features
• Record.Design.Requirements_Allocation
• Record.Evaluation_Actions
• Record.Inspection_Actions

6) Reporting: Preparation of persistent information to support the needs of a requestor.

• Report.Evaluation_Results
• Report.Inspection_Results
• Report.Product_Characteristics
• Report.Product_Features
• Report.Product_Structure
• Report.Requirements_Allocation

225 Appendix H
7) Demonstration: Interactive presentation of information (eg to customers,
management, auditors etc).
• Demonstrate.Config_Mgt
• Demonstrate.Config_Mgt.Status
• Demonstrate.Evaluation
• Demonstrate.Evaluation.Actions
• Demonstrate.Evaluation.Status
• Demonstrate.Integration
• Demonstrate.Integration.Status
• Demonstrate.SEE_Ada
• Demonstrate.SEE_Ada.Concepts
• Demonstrate.SEE_Ada.Usage
• Demonstrate.Test
• Demonstrate.Test.Coverage
• Demonstrate.Test.Results

8) Familiarisation: Observations to help gain general understanding of products,

resources, or processes.
• Familiarisation.Product
• Familiarisation.Product.Attributes
• Familiarisation.Product.Features
• Familiarisation.Product.Structure
• Familiarisation.SEE_Ada
• Familiarisation.SEE_Ada.Concepts
• Familiarisation.SEE_Ada.Features
• Familiarisation.SEE_Ada.Usage

226 Appendix H
 Appendix I: Examples of Attributes Accessible from
SEE-Ada

Attribute Group Repres. Attrs Filter IP Classes

code.audit numeric 45 AQ2SA 16

code.complexity numeric 1 AQ2SA 16
code.profile numeric 118 AQ2SA 16
code.quality numeric 36 AQ2SA 16
code.size numeric 13 AQ2SA 16
code.structure numeric 4 Structure_Measures 215
code.subsys.exports symbolic 1 IMPEXP2SA 20,21,24
config_mgt.author symbolic 1 VCTRL2SA 8,9,10,
config_mgt.change_details numeric 11 VCTRL2SA 8,10
config_mgt.release_details numeric 2 VCTRL2SA 8,9,10
config_mgt.status symbolic 1 VCTRL2SA 8,9,10
config_mgt.user_names symbolic 1 VCTRL2SA 8,9,10
config_mgt.version_details numeric 3 VCTRL2SA 8,9,10,
design.logical.class_allocation symbolic 1 rb_to_sa_attr 38,39,42,179,
184, 185
design.logical.stats numeric 1 rb_to_sa_attr 38,39,42,179,
184, 185
design.logical.uses symbolic 1 tb_to_sa_attr 38,39,42,179,
184, 185
design.physical.imports.direct symbolic 1 IMPEXP2SA 20,21,24
design.physical.imports.effective symbolic 1 IMPEXP2SA 20,21,24
design.physical.imports.mutual symbolic 1 IMPEXP2SA 20,21,24
design.subsys.alloc symbolic 1 Subsystem Allocation 215
design.interface_count numeric 1 Spintex 215
design.interface_and_subsystem symbolic 1 Spintex 215
Defects symbolic 1 DEF2SA

227 Appendix I
Requirements_Built_in_Table symbolic 1 SRS, SDF 57
Requirements_External_Interface symbolic 1 SRS, SDF 57
Requirements_General symbolic 1 SRS, SDF 57
Requirements_Initialisation symbolic 1 SRS, SDF 57
Requirements_Internal_ symbolic 1 SRS, SDF 57
Interactions
Requirements_Launch symbolic 1 SRS, SDF 57
Requirements_MMI_Input_ symbolic 1 SRS, SDF 57
Capability
Requirements_MMI_IO_ symbolic 1 SRS, SDF 57
Capability
Requirements_MMI_Output_ symbolic 1 SRS, SDF 57
Capability
Requirements_MMI_Reserved_Displ symbolic 1 SRS, SDF 57
ay_Regions
Requirements_MMI_Sub symbolic 1 SRS, SDF 57
Requirements_Ship_Motion_ symbolic 1 SRS, SDF 57
and_Wind_Data
Requirements_Simulated_ symbolic 1 SRS, SDF 57
Launch
Requirements_System_ symbolic 1 SRS, SDF 57
Management
Requirements_Threat_Mgt symbolic 1 SRS, SDF 57
Requirements_AWADI symbolic 1 SDF 57
Requirements_MMI_All symbolic 1 mrg_attr_gps 215
Requirements_SRS_All symbolic 1 mrg_attr_gps 215
Test.Coverage numeric 1 TM2SA 168,136,172
Test .Segments numeric 1 TM2SA 136,170
Test. Unit_Map symbolic 1 TM2SA 168,136,171

228 Appendix I