Professional Documents
Culture Documents
ACCORDING TO COBIT
How does the IT performance within one of
the largest investment banks in the world
compare to COBIT?
JOEL ETZLER
Master Thesis
Stockholm, Sweden
XR-EE-ICS 2007:14
ABSTRACT
To improve the governance of IT and comply with regulatory demands,
organizations are using best practice frameworks to facilitate the work. One of
these IT governance frameworks is COBIT (The Control Objectives for
Information and related Technology). COBIT provides guidance on what could be
done within an IT organization in terms of controls, activities, measuring and
documentation. This framework is however large and require specific knowledge in
order to enable full use of its potential. This project was initiated to use a
straightforward method of working with COBIT while assessing the maturity of an
organization. The method was developed by myself and my advisor at The Royal
Institute of Technology in Stockholm and describes one way of using COBIT. The
organization under evaluation is one of the largest, most well known investment
banks in the world, in this project referred to as The Firm.
A specific part of the IT organization within The Firm was evaluated with COBIT
as a starting point and the gap between the framework and the organization was
underlined. COBIT provides an incremental measurement scale, where the internal
processes are measured in terms of how defined and structured they are. The scale
expresses levels of maturity and The Firm reached a level 3.3 out of 5.
The strongest and weakest areas have been emphasized and improvements on the
weaker areas have been suggested. These improvement actions could enable
organizations to better govern IT and facilitate compliance to regulatory
requirements.
PREFACE
This is my Master Thesis and it constitutes the final part in my Master of Science
education in Electrical Engineering at the Royal Institute of Technology in
Stockholm. Conducting this project has been a great experience for me. I have met
many, very kind and helpful people and would like to express my gratitude to all
involved. Above all I would like thank, my advisor at ICS, Mrten Simonsson and
key stakeholders at The Firm; Moss, Nikki, Andrew and Trevor. Thank you!
Joel Etzler
Stockholm, 16th of May, 2007
TABLE OF CONTENTS
1
INTRODUCTION ................................................................................................................... 5
1.1
BACKGROUND ....................................................................................................................... 5
1.2
PROBLEM............................................................................................................................... 7
1.3
PURPOSE................................................................................................................................ 7
1.4
DELIMITATIONS ..................................................................................................................... 7
1.5
METHODOLOGY .................................................................................................................. 9
2.1
INITIATION ............................................................................................................................ 9
2.2
2.3
2.4
EVALUATION METHOD......................................................................................................... 11
THEORETICAL FRAMEWORK....................................................................................... 12
3.1
3.2
IT GOVERNANCE.................................................................................................................. 18
3.3
3.4
COBIT ................................................................................................................................ 22
3.5
ANALYTICAL FRAMEWORK.......................................................................................... 33
4.1
4.2
MODELING .......................................................................................................................... 37
4.3
ANALYSIS ............................................................................................................................ 38
PROCEDURE ......................................................................................................................... 39
5.2
5.3
5.4
RESULTS ............................................................................................................................... 43
6.1
6.2
DISCUSSION......................................................................................................................... 49
7.1
7.2
7.3
VALIDITY ............................................................................................................................ 53
7.4
RELIABILITY ........................................................................................................................ 53
CONCLUSION ...................................................................................................................... 54
LIST OF FIGURES
FIGURE 1 FRAMEWORK LINKING CORPORATE GOVERNANCE TO IT GOVERNANCE8 ............................ 13
FIGURE 2 POSITIONING OF IT GOVERNANCE AND IT MANAGEMENT. SOURCE: PETERSON, SEE
GREMBERGEN, 2004................................................................................................................................................ 19
FIGURE 3 COBIT, OVERLYING FRAMEWORK PRINCIPLES. SOURCE: IT GOVERNANCE INSTITUTE,
COBIT 4.0 ................................................................................................................................................................. 23
FIGURE 4 COBIT, STRUCTURE AND INTERRELATIONSHIP OF PROCESSES. SOURCE: IT GOVERNANCE
INSTITUTE, COBIT 4.0 ........................................................................................................................................... 24
FIGURE 5 COBIT, OVERALL FRAMEWORK. SOURCE: IT GOVERNANCE INSTITUTE, COBIT 4.0 ........ 25
FIGURE 6 METRICS. SOURCE: IT GOVERNANCE INSTITUTE, COBIT 4.0 ................................................. 28
FIGURE 7 RACI-CHART. SOURCE: IT GOVERNANCE INSTITUTE, COBIT 4.0 ......................................... 30
FIGURE 8 DOCUMENTS. SOURCE: IT GOVERNANCE INSTITUTE, COBIT 4.0 ......................................... 30
FIGURE 9 MAPPING TO PCAOB TO COBIT. SOURCE: ITGI (2006), IT CONTROL OBJECTIVES FOR
SARBANES-OXLEY, THE ROLE OF IT IN THE DESIGN AND IMPLEMENTATION OF INTERNAL CONTROL
OVER FINANCIAL REPORTING. ............................................................................................................................... 31
INTRODUCTION
This chapter gives the reader an introduction to the subject of matter. I present
background to the research, a problem description, the purpose of my thesis where
I display my research question, then delimitations of this thesis and finally, my
thesis disposition.
1.1
BACKGROUND
Companies growing and merging with other businesses demand great changes to
their infrastructure. The equities market space is constantly evolving and the
implications to the IT systems and processes within the organizations are
substantial. Companies today depend to a great extent on the information stored
and managed through IT and many would not be able to operate without a
functional IT structure. The increasing regulatory demands also put a pressure on
the accounting, documenting and reporting through IT. The systems are required
not only to support the operations of the companies, but to report and store
financial and organizational data to meet external demands. It is no longer enough
to look at talented individuals to manage IT projects, the projects regularly need to
be structured as sustainable processes, where documentation and measuring is
standardized. Many companies acknowledge this need and put more effort into
standardizing the IT structure, policies and procedures and focus on aligning them
to the business objectives. This practice is called IT governance and will be further
explained and discussed throughout this report.
To facilitate the governing of IT there are several frameworks available on the
market. One of the most frequently used and chosen in this work is called COBIT1,
the Control Objectives for Information and Related Technology, further described
in section 3.4. COBIT gives guidance from best practices derived from major
global IT-related standards, practices and frameworks on processes and its
constituents to aid in the work of governing IT. The framework defines a set of
processes, to which there is a number of activities, suggested documentation and
measuring. It provides a high level view of an IT organization and what could be
done within it. COBIT also associates a maturity model that can be used to
benchmark the performance and level of definition to each process in a
standardized manner. The scale, which is obtained from the Capability Maturity
Model (CMM), described in section 3.3.3 spans from 0 to 5, with 5 being the
highest.
To many organizations, the help of external best practices is a cost efficient and
effective alternative to creating own frameworks and standards. This thesis will
highlight the work with one of these frameworks, namely COBIT and look at the
possibilities to improve the governance on a specific IT organization through the
help of that framework. The project has been performed at one of the largest
investment banks in the world at a global division on the IT side. The project has
followed the organizations desire to externally assess their IT performance with
COBIT as a frame for benchmarking.
The organization is in this thesis referred to as The Firm and the specific part of
The Firm that the project is focused on is called The Markets division. This is
further described in section 5.2. My advisor at the department of Industrial
Information and Control Systems (ICS) at the Royal Institute of Technology is PhD
student Mrten Simonsson. My advisor at The Firm is the European Head of
Technology Business Development. Key stakeholders at The Firm are the
European Head of Technology Business Development, the Head of Development at
The Markets Division and the people responsible for the scope and implementation
phase of the COBIT initiative at The Firm. The Head of Development did
participate in interviews, but when referred to as key personnel, they do not
represent a respondents view.
1.2
PROBLEM
How should IT be governed and how could COBIT be used as guidance? In this
project, there are two key issues I have addressed.
The framework itself does not say how it should be used; it merely states
guidance on its defined processes.
1.3
PURPOSE
The purpose of the project was to do an assessment of The Markets division at The
Firm with COBIT serving as a starting point. The assessment could be resembled
by a gap analysis where the difference between the framework and the actual
organization is emphasized. Derived from that assessment is the information about
strengths and weaknesses within the IT organization, in comparison to COBIT. The
four strongest and weakest areas should be emphasized and suggestions on how to
improve the weaker areas should be presented. The question I tried to answer was:
How does the IT procedures and processes at The Markets division compare to
COBIT- how big are the gaps, what could be improved and how?
1.4
DELIMITATIONS
The project was decided to be a high level assessment and was limited to gathering
information on the COBIT processes from one person per process. The definition
of a process is described in section 3.4 COBIT.
This project covers what is being done in respect to COBIT, not processes outside
those borders. The project was also limited to The Markets division which is further
described in section 5.2.
1.5
THESIS DISPOSITION
1. Introduction
METHODOLOGY
This chapter provides the projects course of action and motivates why I have
chosen this approach to address the given problem. I describe the initiation, the
method of collecting data, required theoretical knowledge and finally how I
evaluated the data.
2.1
INITIATION
The reason why the project was initiated relates to the research of PhD student
Mrten Simonsson and the department of Industrial Information and Control
Systems at the Royal Institute of Technology, previously described in section 1.1.
The purpose, also prior described, is evaluating a part of an IT organization with
COBIT as a starting point. The first problem of the thesis project was to find a
sponsoring company that would be willing to participate in this project. During a
previous employment, I came in contact with The Firm and proposed my project.
The Firm felt as a suitable sponsor where my project could be of value. This is
further described in section 5.2. The project was also further limited to The Markets
division, also described in section 5.2 as that area seemed to be just the right size
for my study.
2.2
CASE STUDY
The case study is but one of several ways of doing social
science research. Other ways include experiments, surveys,
The way to fulfill the purpose of this project has mainly been through a case study.
A more quantitative method, like questionnaires would possibly have been
applicable to this project as well. According to Holme & Solvang3 the qualitative
and quantitative methods both have their advantages and disadvantages. As COBIT
was new to many of the participants in the study, explanations were in several
cases necessary.
In general, case studies are the preferred strategy when
how or why questions are being posed2
The study required the presence of someone with knowledge in COBIT to facilitate
the questions- and answering process. This is the reason why I chose to do
interviews. That way I could participate as an interviewer with specific knowledge
in the COBIT framework and easier get accurate answers from the respondents. I
used COBIT as a starting point and asked the respondent to evaluate the maturity
on each activity within one process. I also asked them to answer how many of the
suggested documents and metrics The Markets division was actually using. Finally
I asked how the role assignment suggested in the RACI-chart corresponded to the
structure at The Markets division. COBIT specifics can be found in section 3.4.
2.3
THEORETICAL STUDY
After determining the method of gathering information there were a few areas I
needed more theoretical knowledge in. This also constitute a part of the curriculum
of a master thesis and motivates chapter 3, Theoretical framework where the
research is presented as needed to understand the empirical study. The research is
partly about corporate governance and its constituents. This along with the
relationship to IT governance depicts the foundation for the thesis subject. The way
to govern IT is suggested with help and guidance from an assessment framework
and the currently available frameworks are presented briefly as a benchmark for
2
3
Yin, Robert K. (1994), Case study research, Design and methods, second edition.
Holme & Solvang (1997).
10
organization
ensure
alignment
between
use
of
2.4
EVALUATION METHOD
After collecting the data from the interviews I needed a way to aggregate them into
results. Discussions with my advisor from ICS lead to the evaluation method. We
decided to take all results from all parts of the data collection and add them
together. The mean value generated the maturity on each process, and the mean
value on all 34 COBIT processes gave the overall maturity level.
Ridley G. et al (2004), COBIT and its Utilization: A framework from the literature.
Proceedings of the 37th Hawaii International Conference on System Sciences, IEEE
11
THEORETICAL FRAMEWORK
This chapter provides the theoretical foundation of the thesis. Initially I will discuss
theory around corporate and IT governance and the regulatory demands in that
space. This leading up to the ways IT could be governed. Brief reviews of possible
IT governance frameworks are presented to facilitate the governing of IT and the
framework used in this study, COBIT, will be described closer.
3.1
CORPORATE GOVERNANCE
In order to understand the concept of IT governance one needs insight into the
principles of corporate governance and its constituents.
"Corporate Governance is concerned with holding the
balance between economic and social goals and between
individual and communal goals. The corporate governance
framework is there to encourage the efficient use of
resources and equally to require accountability for the
stewardship of those resources. The aim is to align as
nearly as possible the interests of individuals, corporations
and society"
Sir Adrian Cadbury (2000), in 'Global Corporate Governance Forum', World Bank.
12
Weill and Ross8 have created a framework for linking the corporate governance
and IT governance principles together, which can be seen in figure 1. The areas that
relates to IT governance are marked in grey.
There are several ways of looking at the connection between corporate governance
and IT governance. Another is described by Van Grembergen, De Raes and
6
13
Guldentops8. They use Shleifer, A. & Vishnys9 work and mention three key
questions that they say the management team should address to display the
connectivity between corporate governance and IT governance.
- IT Governance Questions:
- How does management get their CIO and
IT organization to return some business
value to them?
- How does top management make sure that
their CIO and IT organization does not steal
the capital they supply or invest in bad
projects?
- How does top management control their
CIO and IT organizations?
3.1.1
These regulatory requirements constitute a large portion of the need for structure
within organizations and the implications on IT are substantial. In coordination
with various financial and regulatory requirements, a new era of high level
corporate and IT thinking has emerged. A key driver for IT governance have the
last couple of years, been these external demands and the most significant one so
far has been the Sarbanes-Oxley act, described below. There are a few other
important regulations, like Basel II, the European 8th Directive and Mifid but they
will not be discussed in this study and their implications to IT will not be taken into
account.
Schleifer A. & Vishny (1997), A survey on corporate Governance. The Journal of Finance, 52(2)
Ernst &Young (2005), Global Information Security Survey
10
14
Title III and IV are the titles that are closest related to this work.
15
Section 404 demands each annual report to contain an internal control report which
shall
(1)
Even though the act is focused on accounting and financial reporting, the
importance of appropriate IT systems as an integral part in the reporting procedure
is evident. The systems ensure the validity of information and provide fundamental
structure to the reporting standards and assessments of financial data. Section 409
of the act expresses the real time accounting demands and is central to the IT
systems involved.
11
12
Dietrich, Robert (2004). Sarbanes-Oxley and the Need to Audit Your IT Processes, MKS
Sarbanes and Oxley act of 2002 Section 404. PUBLIC LAW 107204
16
The relationship between IT systems and section 409 is described by Rob Smith,
Co-Chair of Industry Solutions SOX Committee and Michael Kuhbock, CoChairman and Founder of the Integration Consortium.
The only way for issuers to be aware of real time
information and trends on operations or the physical
activities of their organization is for the issuers systems to
report on anomalies and trends in real time and on an
exception basis. As well, the integration of any new system
into an organization will have to pass SOX compliancy
before it is either selected or plugged in. Failure of control
process, due to a systems failure will strictly fall under the
409 clause regarding material change.14
This could very well be the most grueling challenges in the compliance work and
one of the reasons corporations struggle to find easily adopted, implemented and
administered frameworks to facilitate the process of compliance. A framework is
required by the act; however the choice of version is free. One such framework is
provided by COBIT and another by COSO, described in section 3.4 and 3.3.2
respectively.
13
14
Sarbanes and Oxley act of 2002 Section 409. PUBLIC LAW 107204
Smith R. Kuhbock M.. Sarbanes Oxley 404/409-Integration Organizations and SOX.
www.integrationconsortium.org
17
The auditing standards are set by the PCAOB, The Public Company Accounting
Oversight Board. The PCAOB is created by Sarbanes-Oxley and described in title I
of the act. The purpose is to supervise and regulate the work done by auditing
companies. It also sets the working principles for the auditing companies.
3.2
IT GOVERNANCE
IT Governance is the organisational capacity exercised by
the Board, executive management and IT management to
control the formulation and implementation of IT strategy
and in this way ensure the fusion of business and IT.17
organisations
IT
sustains
and
extends
the
15
16
17
Grembergen, (2002)
18
18
The difference between them could help provide a better view of what IT
governance is, as confusion easily occurs. Weill and Ross (2004), says that
governance determines who should make decisions and management is the process
of making and implementing the decisions.
19
19
3.3
IT GOVERNANCE FRAMEWORKS
3.3.1
ITIL
20
21
20
3.3.2
COSO
The five components of internal control that COSO identifies can be resembled by
the guidance COBIT provides for IT.24
23
24
Damianides, Marios (2005), SarbanesOxley and IT governance: New guidance on it control and
compliance http://www.infosectoday.com/SOX/Damianides.pdf
21
3.3.3
CMMI
Capability Maturity Model Integration (CMMI) is a process
improvement approach that provides organizations with the
essential elements of effective processes. It can be used to
guide process improvement across a project, a division, or
an entire organization.25
3.4
COBIT
COBIT is short for the Control Objectives for Information and Related Technology
and was developed by the Information Systems Audit and Control Foundation,
ISACF in 1996. ISACF, founded 1969 later became ISACA, Information Systems
Audit and Control Association. ISACA, is now a global organization with over 50
000 members in more than 140 countries. The founders, a group of IT auditors,
recognized the increasing need for control within IT organizations and decided to
create a network for information and guidance in the field. In 1998 ISACA
established the IT Governance Institute, ITGI, who is now responsible for COBIT.
During the fall of 2005, ITGI released a version 4.0 of COBIT which constitutes
the framework of reference in this thesis.
COBIT was originally developed as a tool to control IT and reduce risk within IT
organizations, primarily in the banking and e-business industries. It has evolved to
become more business oriented and now gives a high level image on what to
accomplish within an organization rather than how. It is designed to provide
fundamental guidance to management and process owners to in best way possible
allocate the assets of the organization. Figure 3 shows the overlying framework
principles.
25
22
The COBIT framework has the aspiration to be both responsive and practical in the
sense of the business needs, while at the same time being independent to the
technical and structural differences within various organizations.
COBIT uses ideas from all frameworks above and even more standards when
creating its definitions and controls.
For this COBIT update (COBIT 4.0), six of the major global ITrelated standards, frameworks and practices were focused
on as the major supporting references to ensure appropriate
coverage, consistency and alignment26
The standards, frameworks and practices mentioned in the quote above are:26
23
24
The processes apply at different levels of the IT organization and each domain
could help to provide an understanding of the purpose of the processes. The names
of all the COBIT processes are displayed in Figure 5.
The four COBIT domains; Plan and Organise, Acquire and Implement, Deliver and
Support and Monitor and Evaluate as shown in figure 5, are clarified below.
25
3.4.1
MATURITY MODEL
It is not easy to know how to benchmark an organization and to what grade of
accuracy the evaluation should be scaled. COBIT suggests an incremental
measurement scale of six maturity levels. Going from 0, Non-existent to 5,
Optimized, COBIT covers the entire spectrum of maturity in a process. The
structure and design of the scale is the same as the one used by Capability Maturity
Model, (CMM), described in section 3.3.3. These maturity levels are individually
explained for each of the 34 processes but the general structure could be seen in
table 2.
26
Non-Existent
Complete lack of any recognisable processes. The organisation has not even
recognised that there is an issue to be addressed.
Initial
There is evidence that the organisation has recognised that the issues exist
and need to be addressed. There are however no standardised processes but
instead there are ad hoc approaches that tend to be applied on an individual
or
Repeatable
Processes have developed to the stage where similar procedures are followed
by different people undertaking the same task. There is no formal training or
communication of standard procedures and responsibility is left to the
individual. There is a high degree of reliance on the knowledge of individuals
and therefore errors are likely.
Defined
Managed
Optimised
Processes have been refined to a level of best practice, based on the results
of continuous improvement and maturity modelling with other organisations.
IT is used in an integrated way to automate the workflow, providing tools to
improve quality and effectiveness, making the enterprise quick to adapt.
ACTIVITIES
The activities are a significant part of the suggested guidance COBIT describes for
each process. They say what should be done and they are also associated to the
roles, further described under Roles and Responsibilities. An example of
activities is shown in figure 7, RACI-chart. As previously mentioned; COBIT also
describe detailed control objectives. The detailed control objectives often
correspond to the activities and their purpose is the same. COBIT is not entirely
consistent about this but in many cases, the activities are just simplified detailed
control objectives.
METRICS
To improve the efficiency and effectiveness of the processes, COBIT suggest a set
of metrics to use as measurement to each process. The metrics are different for
each process but some of the outlines are similar. The metrics are in the version
used in this study, COBIT 4.0, Key Performance Indicators, Process Key Goal
27
Indicators and IT Key Goal Indicators. For the process, Manage the IT
investment the metrics are shown in figure 6.
Just to clarify what is shown in the image, one metric COBIT suggests could be to
measure the percentage of projects with benefit defined upfront. That metric can
be seen in the upper left corner of the Key Performance Indicators box in figure 6.
According to Guldentops27 the primary purpose of the guidelines is to enable
corporate management to:
The indicators are the key inputs in the benchmarking process. The Management
guidelines indicators are Key Goal Indicators (KGIs), Key Performance Indicators
(KPIs) and maturity models.
28
have been met for a specific process and are often defined as the target to achieve.
Business requirements are generally expressed in terms of information criteria:
Availability of information needed to support the business needs
Absence of integrity and confidentiality risks
Cost-efficiency of processes and operations
The Key Performance Indicators define measures to explain to what extent the
process is fulfilling its objectives, how well its performing. They are the most
important indicators in revealing whether or not a goal will be reached and are
often used to in an early stage tell if the KGIs will be difficult to achieve.
activity is performed while the function of Informed is merely one who should
know about the activity. Figure 7 shows the roles as functions and their relationship
to the activities of the process Manage the IT investment. The activities extend
the understanding of the process and its purpose. To each activity there is either a
Responsible or an Accountable role to see to that the activity is executed in a proper
manner.
DOCUMENTS
Relevant documentation renders repetition and effective feedback of the processes
possible. COBIT defines which documents should exist at the initiation stage and
which should be produced during the process. They are referred to as Inputs and
Outputs, shown in figure 8.
30
3.5
4.0
emphasizes
regulatory
compliance,
helps
28
www.Isaca.org
31
29
32
ANALYTICAL FRAMEWORK
In this chapter I explain the method of collecting data in detail, the analysis of the
collected data and the method I have chosen to derive my results.
4.1
DATA COLLECTION
There are no rules that govern the way to use COBIT and to what extent it is to be
implemented. Each organization may adopt the framework to meet their business
objectives in which way they see fit.
COBIT works as a helping hand, providing guidance to the management on how,
according to best practice to use the assets and people within the organization.
However, the complexity of COBIT could make the usage difficult and time
consuming. Furthermore it leaves room for interpretation, which means that two
interviewers could obtain incomparable results on the same assessment. It is not a
given that for instace the COBIT-defined activities, are interpreted the same way
by two separate people. While the purpose of COBIT is to provide guidance on IT
governance, it does require a substantial amount of expertise with regards to the
framework. This has led to the creation of a tool through which COBIT can be used
in a more formalized and straightforward way. This improves the validity and
makes the framework more usable. It was created by PhD student Mrten
Simonsson at the department of Industrial Information and Control Systems (ICS)
at the Royal Institute of Technology. I will here describe how the data can be
collected, the modeling tool used and how to analyze the results.
33
As presented in section 2.4 the interviews will provide the input information to the
project. The vast majority of the respondents should be executives with
management functions as their knowledge is most likely to correspond to the kind
of strategic information COBIT deals with. The descriptions below explains the
steps to take when working with COBIT and conducting the interviews.
4. Evaluation of a process
The respondents should be asked about the activities within each process he/she is
either Accountable or Responsible to, according to the RACI-chart. The question is
on what level of maturity in terms of the maturity model the respondent places that
activity, section 3.4.1.
34
The respondent should also be asked about the documents associated to the process
and the measured KPIs and KGIs. This will be yes or no questions, adding up to a
total which later in the analysis is compared to the maximum number of metrics
defined by COBIT. In more detail the interviews can be done as follows.
MATURITY
ACTIVITY EXECUTION
LEVEL
LEVEL 0
SOME AWARENESS OF THE IMPORTANCE OF ISSUES RELATED TO THE ACTIVITY. NO MONITORING IS PERFORMED. NO
DOCUMENTATION EXISTS. NO ACTIVITY IMPROVEMENT ACTIONS TAKE PLACE.
LEVEL 2
INDIVIDUALS HAVE KNOWLEDGE ABOUT ISSUES RELATED TO THE ACTIVITY AND TAKE ACTIONS ACCORDINGLY. NO
MONITORING IS PERFORMED. NO DOCUMENTATION EXISTS. NO ACTIVITY IMPROVEMENT ACTIONS TAKE PLACE.
LEVEL 3
AFFECTED PERSONNEL ARE TRAINED IN THE MEANS AND GOALS OF THE ACTIVITY. NO MONITORING IS PERFORMED.
DOCUMENTATION IS PRESENT. NO ACTIVITY IMPROVEMENT ACTIONS TAKE PLACE.
LEVEL 4
AFFECTED PERSONNEL ARE TRAINED IN THE MEANS AND GOALS OF THE ACTIVITY. MONITORING IS PERFORMED.
DOCUMENTATION IS PRESENT. THE ACTIVITY IS UNDER CONSTANT IMPROVEMENT. AUTOMATED TOOLS ARE
EMPLOYED IN A LIMITED AND FRAGMENTED WAY
LEVEL 5
AFFECTED PERSONNEL ARE TRAINED IN THE MEANS AND GOALS OF THE ACTIVITY. MONITORING IS PERFORMED.
DOCUMENTATION IS PRESENT. AUTOMATED TOOLS ARE EMPLOYED IN AN INTEGRATED WAY, TO IMPROVE QUALITY
AND EFFECTIVENESS OF THE ACTIVITY
A mean value for all activities within a process, the average activity
maturity (AM), should then be calculated. The values are threshold values,
i.e. all criteria for level 3 have to be fulfilled in order to achieve level 3
maturity.
4. The same procedure for the metrics (Key Performance Indicators, Process
Key Goal Indicators, IT Key Goal Indicators) as with the documents. How
many of the suggested metrics they actually used as measurement should be
counted. This also shown in table 4.
MATURITY
DOCUMENTS IN PLACE
ASSIGNED RESPONSIBILITIES
LEVEL
METRICS
MONITORED
LEVEL 0
NO RELATIONS EXIST
0%
0%
LEVEL 1
LEVEL 2
20 %
40 %
20 %
40 %
LEVEL 3
60 %
60 %
LEVEL 4
80 %
80 %
LEVEL 5
100 %
100 %
The process maturity (PM) for the entire process is then calculated as the mean of
the average activity maturity (AM) , The assigned responsibilities maturity
(RM), the documents in place maturity (DM) , and the Metrics monitored
maturity (MM).
PM= (AM+RM+DM+MM)/4
The values are also threshold values, i.e. all criteria for level 3 have to be fulfilled
in order to achieve level 3 maturity. This means that it requires 100% usage of the
metrics suggested in COBIT in order to achieve level 5.
36
Regarding weights for separate metrics, the basic assumption is that all metrics
have the same weight. It is up to each organization to do their own weighting but a
guideline could be that activities should have the highest weight followed by the
metrics.
As an optional final step, the respondent should be asked to evaluate where he/she
thinks the entire organization or the suggested silo would land on the maturity
scale. This should not be used in the assessment but is interesting to collected for
future benchmarking and evaluation of the maturity assessment method.
4.2
MODELING
The modeling phase represents the aggregation of all the collected data and the
creation of a map showing all the COBIT processes and its relations to the
activities, metrics, roles and documents used by the organization. The reason for
creating an architectural map is to easier get an overview of the processes and their
relationships and to set definitions so that information about the model more easily
can be derived. The map in this case study was created with a modeling program
called Metis, a Troux technologies30 product. Metis is the software chosen by ICS,
which is why I used it for this study. User specific functionality in Metis is done
through an application Programming Interface (API) that supports Visual Basic and
Java script. At ICS an own Meta model that incorporates the definitions, rules and
restrictions of the model I used in this project has previously been created. That
Meta model describes what could be modeled, which processes, metrics,
documents and relations could be used in the model. It holds a reference model of
the complete COBIT framework to which the model of the organization under
evaluation could be compared. The gap between the reference model and the model
under evaluation generates the basis for the results and give the maturity to the
processes. The complete map can be seen in appendix 4, Model of The Firm. The
modeling in Metis is a method that is still under evaluation by ICS. It will be used
to a greater extent in future research as the benefit of using it increases the more
defined this method gets. One of the key beneficial aspects of the model, is that it
can be used to easier change relations to the processes.
30
37
4.3
ANALYSIS
The analysis is where the results are reviewed from the modeling and which
conclusions could be drawn from the work. As one of the goals in the thesis was to
find areas or processes with lower and higher maturity level and suggest
improvements, the conclusion of the modeling was crucial in this study. The
processes with more and less mature nature have been examined in detail. This is
further described in chapter 6, Results. From the interviews I have tried to figure
out which are the key gaps or specific strengths within those areas. To find out
more about the current state and the reason for the strong or weak procedures and
policies within those areas, key personnel from The Firm was involved and
questioned.
38
EMPIRICAL STUDY
This chapter portrays the data collection specific for the assessment at The Firm
and a description of the organization.
5.1
PROCEDURE
This project will initially be described with a short introduction of the company
where the study was done. After that follows in chronological order the phases of
the project with the Initiation followed by Project definition and Case study at The
Firm.
5.2
THE FIRM
For security reasons the name of the company where the study took place will not
be revealed, it will instead be given a fictitious name, The Firm. The company I
have chosen to call The Firm is one of the largest and most well known investment
banks in the world. It operates on a global basis and house more than 50 000
employees. The Firm has taken a silo like approach to enterprise structure, which
means that each division functions almost as a separate organization. Each silo has
got roles equivalent to what a normal company would have, like CIO (Chief
Information Officer) and CFO (Chief Financial Officer). As this thesis mainly is
about IT governance and the structure around IT processes, the following
description is focused on the IT organization at The Firm.
Many roles are clearly defined within each silo. Their responsibilities are most
often tied to the area they are stationed in but their superior officers
responsibilities could vary from central isolated groups to officers controlling
39
several silos. As many separate groups perform functions that are of use to all areas
at The Firm, those groups are in a way a part of all the silos. As will be described in
section 1.5 the purpose of this project is to do an assessment of a specific division
or silo at The Firm called The Markets division. The silo I, together with key
stakeholders from The Firm, chose for this project is not really a silo but a mixture
of three silos. The reason for choosing The Markets division was a result of several
discussions with people who later became key stakeholders in the project.
Because many external auditors and regulators use COBIT, The Firms internal
audit section has chosen to use it. Thereby they talk the same language. COBIT
is also the basis for the structure of their new global IT policy program31, which is
why I found this company to be a suitable sponsor of this project.
5.3
PROJECT DEFINITION
As the need for structure and definition of the project was evident, many
introductory interviews contributed to the project layout. These interviews along
with discussions with my advisor at The Firm lead to the definition of the project.
The assessment really had two different possible ways of being performed. One
being a very high-level with the role mapping on European executives level. The
COBIT roles, CEO, CIO, and CFO would correspond to the level of The Firms
European CEO, CIO, and CFO and so on. As The Firms IT organization keeps a
silo like structure, each silo functions as a small organization with between 2001000 employees within IT. A proper high-level assessment would require
interviews with respondents within each silo and from those with responsibilities
spanning the entire organization. My advisor at The Firm and I agreed that this
project was too large within the given timeframe so we turned to the second
alternative, to focus on one division within The Firm. Discussions throughout the
organization resulted in a desire to assess The Markets division. It seemed to
present a reasonably sized IT organization, 33 employees globally, where this
relatively small, and short project could find interesting results and still deal with
complex systems and structures, much like the other silos.
31
Information from a global IT policy conference at The Firm the 24th of April, 2007
40
5.4
As COBIT has a way of describing processes that was not familiar to all
respondents, explanations were often required. The problem occurred most
frequently when discussing the maturity on the activities. COBIT describes detailed
control objectives to each process that often corresponds to the activities. The
framework does not provide a consistent approach to this. Some of the activities
cannot be explained by a corresponding detailed control objective. Below is an
example of when an activity can be further explained by a detailed control
objective associated to the same process. It is taken from process PO5 - Manage
the IT investment.
Activity:
IT budgeting process
41
Some interviewees suggested ways to improve the COBIT framework with ideas
that made sense to the work they were doing at The Firm. One suggestion was to
include a Quality Assurance role to the RACI-chart. This was motivated by the fact
that in all the work done at The Firm there is interaction from a Quality Assurance
function that makes sure that the quality policies are followed. There were also
numerous suggestions on metrics and documents that could be added to improve
the framework. One example could be to add a document called space planning
to the process Procure IT resources. That document would describe the
available space within each area of company so that there was adequate space for
the manpower and hardware.
The results of this assessment will be described in the next chapter in the way they
have been weighted in this study. Together with the group responsible for the
initiation phase of the COBIT initiative at The Firm, I decided to give more weight
to the activities and metrics. The activities received weight 4 and the metrics
weight 2, the documents and role assignment stayed at weight 1. This meaning that
the activities were four times as important as the documents to the results.
42
RESULTS
In this chapter I reveal my results of the assessment beginning with general results.
I then explain the results for the stronger and weaker areas closer.
6.1
As described in chapter 1.5 and 5.2, the assessment was done at a specific division
within The Firm, called The Markets division. There were however difficulties
keeping the assessment to only The Markets division since many of the areas or
functions are centrally governed and managed. In those cases where one of the
COBIT processes was managed at a central level, the interview was conducted with
personnel working in that group, i.e. outside The Markets Division. Table 5 shows
where each process belongs.
Both
PO1
PO3
AI2
AI6
ME1
ME2
43
As shown in the table, almost half of the processes are managed on a central level
and operate across the board. Another relevant issue to consider, when revealing
the results, is the fact that The Markets division is a mix of three silos within The
Firm. That contributes to the rather high amount of centrally managed processes
which in some cases only stretches to the boundaries of these three silos and not the
entire company.
The complete results of this assessment can be seen in detail in appendix 4, where
the maturity level, (the result) is displayed and specified by activities, metrics,
documents and role assignment for each process. Since The Firm had desires to
weight the final results, the activities have weight 4, the metrics weight 2, the
documents and role assignment weight 1. The aggregated process maturity results
after weighting can be seen in figure 10. The average maturity across all processes
was 3.3 after weighting. The activity maturity was 3.1, metrics 2.9, documents 4.0
and role assignment 3.9. Since the activities and metrics were heavier weighted, the
result sank to 3.3, from an un-weighted result of 3.5.
Figure 11 shows the maturity on all the processes, with the top and bottom four
highlighted. Their definition according to COBIT can be seen in appendix 6.
44
These processes will be described further in the following sections to clarify how
big the gaps to COBIT are in these areas, which was a part of the purpose of this
project. The results and information are based on the interviews.
As seen in figure 12, the most mature processes based on the results of this case
study are Manage quality, Procure IT resources, Identify and allocate costs and
Manage the physical environment.
45
All of them have policies and procedures which are set from central groups, which
means, they cannot just be tracked back to the work within The Markets division.
Though some of the work is being done within The Markets division, the standards
and guidelines are set outside those borders.
The manage quality process has got strong procedures and a lot of work is being
done within that area. The Firm currently has various quality approaches and
systems for different groups and tasks. Methods like Six Sigma and Lean
Production is applied to improve processes by eliminating defects and waste within
them. According to the Head of Development at The Markets IT division, all
processes involved in their software development lifecycle interact with their
quality assurance function and align to the business objectives. All of those
processes are managed through a bug tracking tool called Jira32. Jira is an Atlassian
product that also supports measuring of the processes to improve the performance.
Jira can also be used for issue tracking and escalation procedures.
The identification and allocation of costs also follows a structured approach. Costs
of services provided are identified, verified, allocated and reported to management,
business process owners and users in a standardized manner. According to the
Business Manager at The Markets IT division there is a fair bit of documentation
and measuring being done as well. This work is primarily done by a group called
IT Finance, to which each group within IT reports. IT Finance holds the systems
that support the measuring and are responsible for optimizing the process
performance.
32
Jira - http://www.atlassian.com/software/jira/
33
46
According to responsible personnel within the security team the Management of the
physical environment (offices, datacenters and sites), is clearly defined and set on a
global basis. The procedures and policies are strong and all sites are managed
centrally. This meaning that the responsible group has taken the entire companys
sites into consideration when determining the strategy. They have developed a
framework for the standard of the security on the sites and a level where they
would like to be. In comparison to COBIT they do all the measuring and
documentation suggested, and more. There is a lot of focus on improving the
security on the sites, partly driven by terrorist attacks like 9/11 in New York City
and the bombings in the London underground.
6.2
The processes that showed to have the least defined procedures and the biggest gap
to COBIT, were Define and manage service levels, Define a strategic IT plan,
Manage the IT investment and Manage problems. The four processes with the
lowest maturity can be seen in figure 13.
47
The define and manage service levels process has got a structured approach when
dealing with service levels between vendors and IT, but the organization lacks an
IT service catalogue to agree service levels with the business. According to the
global head of ITIL34 this fact is recognized by involved personnel. One of the
goals for 2007 is to build an IT service catalogue and go towards a more defined
framework with Service Level Agreements (SLAs) towards the business. This is
partly done through the current ITIL initiative, which involves a big change process
to address this issue35.
The process called definition of a strategic IT plan seems to be more focused on the
tactical IT planning which allows the organization to adapt to the fast changing
industry and the policies and procedures in long term planning can more easily be
changed36. The interaction with the business and alignment to the business
objectives are not as developed as COBIT suggests. They would like the IT
sourcing and acquisition strategy to be more evolved. At the moment it is more
tactical than strategic. 34
Manage the IT investment is a process with relatively low maturity as well. The
allocation of responsibility for IT investment and financial planning is done on an
ad hoc basis and the project portfolio is inconsistently used in that area37.
34
Information from interview with the Global Head of ITIL at The Firm the 23rd of April, 2007.
35
Information from interview with Account Managers at The Firms IT department, the 14th of
March, 2007
36
Information from interview with key personnel at The Markets divisions IT department, the 13th
of March, 2007
37
Information from interview with the CFO at The Markets divisions IT department, the 19th of
March, 2007
48
DISCUSSION
This chapter will discuss the results of the assessment and highlight relevant and
interesting findings throughout the project.
7.1
In order to understand the maturity results and whether or not they are any good,
one needs to compare it to something. That benchmarking is crucial when drawing
the actual conclusions on comparative analysis. The average results of a 3.3,
average maturity can seem quite high, but how high are they really? Where would
other companies place on the scale? As this is one of the first studies made by ICS,
I really do not have any basis for benchmarking The Firm to other companies. My
results will however together with other assessments form the basis for
comparative benchmarking in future studies made by ICS.
The results of the assessment were initially un-weighted and the average maturity
was 3.5. The group responsible for the initiation phase of the COBIT initiative at
The Firm suggested putting a higher weight on activities and metrics. They also
considered the results to be very high.38 We agreed that a weight of 4 on activities
and 2 on metrics was adequate to form results that reasonably would reflect the
performance of the IT processes at The Markets division. The activities section is
the only input to the results where the respondent is able to grade the performance
on a measurable scale. That, in my opinion, makes the chosen weighting logical.
On metrics and documents it is either on or off. During the interviews the
discussions were slightly focused on the activities, which is another reason for
38
Information from discussion with key personnel for the initiation phase of the COBIT initiative at
49
them to have a more significant weight. For future reference, the weighting method
could be improved by further analysis to reach a suitable state.
The final results were discussed together with my advisor from ICS and key
stakeholders in the project at The Firm. We agreed that further analysis on the
processes with the highest and lowest maturity could be of interest. This due to the
fact that the least mature processes could possibly be improved and the most
mature processes could be reviewed to see if they are more defined than necessary.
By cutting down on the effort in those areas, the company could possibly achieve
cost savings. The results on these areas are described in section 6.2. These four
stronger and weaker areas actually gave one of the most notable acknowledgements
that I have received on my results. The processes I have highlighted as the least and
most mature seemed to correspond to the views of key personnel at The Firm. One
could imply that this increases the reliability of the results since the key personnel
did not have a subjective role in the assessment. Furthermore the results still
seemed accurate after aggregating the activities, metrics, documents and role
assignment, which is another sign that the results provide a true image.
50
As the goal of this project was to see how mature The Markets division at The Firm
was in respect to COBIT and suggest improvement actions to the least mature
areas, I will here give my suggestions and discuss the possible benefits of using
COBIT for improvement. The least mature processes were described in more detail
in the previous chapter.
7.2
What is important to notice is that a low maturity does not necessarily mean that
the company is performing badly. It could be a conscious choice to leave some
areas less defined, with less documentation and measuring in order to stay nimble,
agile and responsive to change. These suggestions below are more or less the gaps
on the four least mature processes to COBIT. If The Firm would like to use COBIT
as guidance, these suggestions could be useful. As previously mentioned, a few of
these suggestions have already been acknowledged and is something The Firm is
working on improving. What should be done within the process is suggested in the
top boxes in figure 14. The lower boxes show the suggested metrics.
51
FIGURE 14 SUGGESTED IMPROVEMENTS, CONTROLS AND METRICS
In order to work with these suggestions the company will need an action plan. It is
important to know where to start and evaluate what to focus on. Since there
currently is a large global IT policy program running at The Firm, it is important
that those procedures and standards are followed. In my opinion the first steps
would be to:
1. Make sure the above results are accurate by engaging more people in
interviews within the specific areas.
2. Focus on the processes with lower maturity, evaluate whether or not the gap
to COBIT is something that could really create value add to the
organization.
3. Figure out which are the most crucial processes to improve.
4. Begin by looking at the 10-50 most important activities or detailed control
objectives.
5. Look at the context and make an action plan for implementing those
controls.
6. After establishing procedures for the most important controls to fill the
gaps, look at what metrics and documentation are necessary to support the
work of those controls.
The goal of closing these gaps are to increase the maturity on the processes,
improve the IT governance and facilitate compliance to regulatory demands.
According to COBIT that is either to facilitate the management of IT risk or
resources, increase value delivery, align IT to business objectives or increase the
performance measurement of the IT processes.39 I would consider it important to
see if the benefit would justify the cost. Would it be economically viable to close
these gaps? In order to find that out, deeper financial analysis should be done to
selected areas.
39
52
7.3
VALIDITY
The validity assures that the assessed object was originally intended to be
assessed.40 The method is verified in this part to certify that the right measures
were chosen to assess this area. The area of investigation should be assessed with
COBIT as a basis for the method and as the benchmark for comparative analysis.
By using COBIT as benchmark in this study, which goal is to find the gaps
between an organization and COBIT, the validity is assured. This implies that what
is left to validate is the method through which COBIT has been used. The maturity
model associated to each process in COBIT provides a statement for every state on
the maturity scale. That eliminates some of the subjectivity since, the state is
already defined by COBIT and is not a measure for the respondent to single
handedly estimate.
7.4
RELIABILITY
The reliability of the answers given by the respondents is not as high as one would
wish. Each respondent has been chosen for their expertise in a specific area,
namely the area defined by that specific COBIT process. It has not been taken into
account that the respondent could have a partial opinion and that the maturity
derived from that interview could be overestimated. The method used in this
assessment has however as an objective focused on making the use of COBIT more
straightforward to deliver unbiased views. As opposed to asking the respondent to
evaluate the maturity on an entire process, the focus of the interviews in this study
has been to ask about smaller parts of the process. That way the respondent is
required to answer specific questions and even, in the documents and metrics case
answer yes-or-no questions. That way the generalization part of the answer is
eliminated and a great deal of the subjectivity as well. To improve the reliability of
the results, one could interview personnel from different parts of the process. One
suggestion could be to select respondents with both user and developer insight
to the process.
40
Yin, Robert K. (1994). Case study research, Design and methods, second edition
53
CONCLUSION
This chapter describes the conclusions that can be drawn from this assessment and
answers the question posed in the purpose section.
The IT procedures and processes at The Markets division reached a 3.3 maturity
level, out of 5. In order to get an average maturity of 5 in this assessment, an
organization would need to:
Perform each activity in an optimized manner, as described in
section 3.4.1.
Use all metrics suggested by COBIT.
Use all documents suggested by COBIT.
Have the same role assignment as the one suggested by the RACIchart for each process.
Due to the lack of comparative benchmarking data, the results cannot really be
compared to another organization, but these results will form the basis for future
studies at ICS and The Royal Institute of Technology. The results also show how
the 34 processes compare to each other. Those performance relations between the
processes seemed accurate to key stakeholders at The Firm. The areas that key
personnel considered as the strongest and weakest are the same as the areas that
have been highlighted in this study.
The four areas with the most defined structure and procedures were identified as;
Manage Quality, Procure IT resources, Identify and allocate costs and Manage the
Physical environment. These areas all have policies and procedures set from groups
operating on a central level at The Firm. The weaker areas are to a greater extent
managed on a local level, within The Markets division. This indicates some of the
54
The weaker areas are; Define and manage service levels, Define a strategic IT
plan, Manage the IT investment and Manage problems, figure 13. Suggestions on
how to improve these areas can be seen in figure 14. Implementing these
improvement actions could increase the maturity on the processes, improve the IT
governance and facilitate compliance to regulatory demands. Improvements must
however be evaluated and weighed against the cost of improvement. Finding that
balance is vital.
55
LIST OF REFERENCES
PAPERS AND BOOKS
IT governance institute (2005), Control objectives for Sarbanes-Oxley
Yin, Robert K. (1994), Case study research, Design and methods, second edition.
56
Sarbanes and Oxley act of 2002 Section 404. PUBLIC LAW 107204
Sarbanes and Oxley act of 2002 Section 409. PUBLIC LAW 107204
STATEMENTS
Sir Adrian Cadbury (2000), in 'Global Corporate Governance Forum', World Bank
Grembergen, (2002)
IT governance institute (2003)
The Ministry of International Trade and Industry (1999)
INTERVIEWS
Information from interview with the CFO at The Markets divisions IT department, the
19th of March, 2007
Information from discussion with key personnel for the initiation phase of the COBIT
initiative at The Firm, April 20th, 2007.
Information from a global IT policy conference at The Firm the 24th of April, 2007
Information from interview with key personnel in the IT procurement team
Information from interview with the Global Head of ITIL at The Firm the 23rd of April,
2007
Information from interview with Account Managers at The Firms IT department, the 14th
of March, 2007
57
Information from interview with key personnel at The Markets divisions IT department,
the 13th of March, 2007
INTERNET
www.Isaca.org.
Smith R. Kuhbock M.. Sarbanes Oxley 404/409-Integration Organizations and SOX.
www.integrationconsortium.org
Ernst&Young (2005), Global Information Security Survey http://www.ey.com/global/download.nsf/Sweden/GFISS_2005/$file/Global%20Informatio
n%20Security%20Survey%202005.pdf
58
59
60
61
62
63
Step 1.
Divide the role in the COBIT RACI-chart in to groups as follows.
Executives
The executives may not work directly with IT concerns, nor have a solid
understanding for its possibilities or limitations. They are however deeply involved
in the management of the entire enterprise and decides upon the overarching IT
strategy and the total IT budget to be distributed upon corporate IT functions and
projects.
COBIT roles: The board, Chief Executive Officer, Chief Financial Officer
Business
This role represents the need for IT systems and IT support functions in order to
conduct business effectively. If the enterprise is divided into several business
units, this role is then responsible for defining requirements for IT and financing
the IT needed.
COBIT roles: Business process owner, Business executive, Business senior
management
IT management
Given the requirements for IT to support business, the IT management role
formulates ITs own long-term goals, roadmaps and strategies. IT management
runs the portfolio of IT projects and assures that IT operations are executed
correctly. IT management is the link between IT and business and is typically
represented by CIO and a set of dedicated advisors or experts.
COBIT roles: Chief Information Officer, Chief Architect, Head Development,
Program Management Office
IT operations
64
IT operations represent the personnel that isnt just company overhead, but
actually operate and develop IT support systems. Several kinds of technical,
administrative, and support personnel reside in this group.
COBIT roles: Head operations, Deployment team, Head IT Administration,
Training
department,
Service
manager,
Service
desk/Incident
manager,
Step 2.
Choose the most frequently occurring assigned responsibility
in each group. If there is a doubt which one to choose, use this
as help. The goal is to find the average responsibility.
If for instance one group has different functions with
one R and the other A, select both as assigned.
If one group has got a few Cs and a few Is and a few
BLANKs choose the lower responsibility: I, sort of the
average.
If there are no or very few responsibilities, dont add
that as a role assignment.
Compare the role assignment of the organization to the responsibilities of the
groups. Calculate how large portion of the assigned responsibilities corresponds to
the groups responsibilities derived from the COBIT RACI-chart. Use table 4 in
chapter 4, Analytical framework to get the maturity contribution.
65
66