Transforming the company

Avoiding the Black Swans

Success Factors and
core beliefs in
Value Assurance
Mobily CIO Summit
Istanbul, April 2012
CONFIDENTIAL AND PROPRIETARY
Any use of this material without specific permission of
McKinsey & Company is strictly prohibited

BRU-AAA123-20100305-

Introducing Enrico Benni

Senior Partner Abu Dhabi

Managing Partner IT Consulting
Middle East previously
leading IT consulting in China
Leader of the global ERP/Value
Assurance McKinsey practice
Co-Leader of the global
McKinsey IT architecture
practice
Has served CIOs on large
transformations across
sectors, in particular in
Telecom, High Tech and
Logistics

McKinsey & Company

| 1

BRU-AAA123-20100305-

To strengthen its client service capabilities, McKinsey created the

business technology office

To enhance our value proposition to clients, McKinsey

launched the Business Technology Office (BTO) in
1997 to build knowledge and expertise in technology
and IT-related matters
Since its launch McKinsey BTO has gained extensive
experience in helping (multinational) businesses with
technology related top management issues. The BTO
has completed 3,600+ projects around the world with
over 920 clients
Over the last years, McKinsey BTO has built up a global
presence in 56 offices in 29 countries to serve
companies in an increasingly global economy

McKinsey & Company

| 2

BRU-AAA123-20100305-

Todays presentation

Collaboration of McKinsey and Oxford:

Research findings about success factors
for large-scale IT projects

2
Mastering technology
and content

Building team and

capabilities

1
Managing strategy
and stakeholders

Turning research into action:

Core beliefs in Value Assurance

4
Excelling in core
project management
practices

McKinsey & Company

| 3

BRU-AAA123-20100305-

Todays presentation

Collaboration of McKinsey and Oxford:

Research findings about success factors
for large-scale IT projects

2
Mastering technology
and content

Building team and

capabilities

1
Managing strategy
and stakeholders

Turning research into action:

Core beliefs in Value Assurance

4
Excelling in core
project management
practices

McKinsey & Company

| 4

BRU-AAA123-20100305-

IT projects are now so large and complex

they can bring whole companies down

Lord Chancellors
Department

Courts Computer
System

328% over budget

167m USD

Design finished in
2001 differences
in development
software
More than 12
months delay to
market; 26% drop
in share price
Resignation of
Nol Forgeard

After failing a
$1.4bn IT
modernisation

Kmart started a
$500mio SCM
project

The combination of
two failed projects
forced Kmart into
bankruptcy

Canadian fire arms

registry

37,000% cost
overrun,
IT system only
160% over budget

Total cost for the

tax payer 834m
USD and mounting

McKinsey & Company

| 5

BRU-AAA123-20100305-

Cost overruns in the public sector are not new

Pliny to Emperor Trajan (AD 110)

The citizens of Nicomedia, Sir,
have spent 3,318,000 sesterces,
on an aqueduct; which they
abandoned before it was finished
and finally demolished.
Then they made a grant of
200,000 sesterces towards
another one, but this too was
abandoned, so that even after
squandering such enormous
sums they must still spend more
money if they are to have a water
supply.

Trajan's response
Steps must be taken to provide
Nicomedia with a water supply,
and I am sure you will apply
yourself to the task in the right
way.
But for goodness sake apply
yourself no less to finding out
whose fault it is that Nicomedia
has wasted so much money up to
date.
It may be that people have
profited by this starting and
abandoning of aqueducts.
Let me know the result of your
enquiry.

McKinsey & Company

| 6

BRU-AAA123-20100305-

Key Research Question

Is this anecdotal
evidence representative for IT projects?

McKinsey & Company

| 7

BRU-AAA123-20100305-

This McKinsey-Oxford collaboration was chosen

No. 1 Idea To Watch" by HBR (09/11)

McKinsey & Company

| 8

BRU-AAA123-20100305-

We studied a total of 2,092 projects,

worth EUR 204 billion

Location

Sector

Project type

Private
Sector

Rest
2

Europe
35

40

65

US

Public
Sector

System type

Other
IT ArchiIntegration Office
tecture
Administration
Project
Information
IT InfraTransaction
Stanstructure
9 12
Customer
9 12
ERP
dard
2
8
7
facing
1
4
SW
28
20 Other
72
Custom
development

25
MIS

Actual Project Size (2010 EUR millions), percent

10
5
0.01

0.10

10

100

1,000 10,000

Largest academic database

2,092 projects and programmes
Average size of EUR 90m (median 1.8m)
Average duration 2.5 years (median 2 yrs)
Total portfolio value of EUR 204bn

McKinsey & Company

| 9

BRU-AAA123-20100305-

7 questions I always wanted to ask about projects

Are lines-of-code riskier than steel-and-concrete?
I

II

Are private and public sector different?

III Is standard software better than bespoke solutions?

IV Should we be afraid of big projects?

VI

Should we be afraid of long projects?

What is the value of experience?

VII
What is the value of benefits management?
McKinsey & Company

| 10

BRU-AAA123-20100305-

Hypothesis I Risk of ICT Projects

Engineering
projects are less
risky than ICT by
a magnitude

McKinsey & Company

| 11

BRU-AAA123-20100305-

Key finding Black Swans matter more than averages

in the world of megaprojects

Thin-tailed distribution
Infrastructure projects
IT projects

Frequency
Percent
50
45

40
35
30

25

20
15

If IT projects had thin tails all1 projects

would end up between
-30% and +48%
In reality 15% IT and 8% infrastructure
projects run out-of-control
IT projects 20 times more likely to run
out-of-control than expected: more than
2300% over-incidence of outliers!!

10
5
0
-100

-50

0
27%

50

100

150

200

250

300

Cost Overrun
Percent

1 Thin tails: not more than .7% projects are outliers outside these bounds
McKinsey & Company

| 12

BRU-AAA123-20100305-

Black swans have high cost, schedule and benefit

risks, which are hidden in the fat tails of ICT portfolios
Risk comparison, average percent

Outliers
cost overrun

Projects with
cost overrun

All projects

Cost
overrun

+197

+70

+27

Schedule
overrun

+68

+60

+55

Benefits
shortfall

-1

+49

Even if the average cost overrun is low, risk of cost overruns is high
Black Swans mean very high cost and schedule risks
And all the projects with a downside risk show significant risk

McKinsey & Company

| 13

BRU-AAA123-20100305-

Why are these risks so crucial?

The commercials

The calculus

EUR 10m fixed price

contract
18 months duration
Costs are equally
distributed over contract
length

10-30% profit margin

Best case
3m profit
7m cost
Monthly burn-rate
Profit: 170 thsd
Cost: 390 thsd

The impact
5 months delay wipe out annual profit
8 months delay wipes out all profits
In the worst case (10% margin) 2 months
eat up all profits
55% schedule risk = 18 + 10 months
Schedule risk requires vendors
to respond, e.g., cutting costs
McKinsey & Company

| 14

BRU-AAA123-20100305-

Hypothesis II Risk of ICT Projects

Public sector ICT

projects are more
risky than private
sector projects

McKinsey & Company

| 15

BRU-AAA123-20100305-

Black Swans are as common in the private sector as they

are in the public sector!
Cost escalation
Frequency (%)

Private sector

Public sector

25
20
15
10
5
0
-100

-50

50

100

150

200

250

300

350

400

450

500

Black Swan risk is the same!

Cost overruns and schedule overruns statistically not different
Public sector much larger risk of budget cuts to its projects!

Schedule escalation
Frequency (%)

2.0
1.5
1.0
0.5
0
-100

-50

50

100

150

200

250

300

350

400

450

McKinsey & Company

500
| 16

BRU-AAA123-20100305-

Hypothesis III Project Types differ in Risk

Don't do bespoke
software use tame
technology, ideally
COTS (commercialoff-the-shelf)!

McKinsey & Company

| 17

BRU-AAA123-20100305-

Different types of ICT projects have significant differences in performance

High Risk

Project type

Average
cost overrun

Standard Software

42%

Custom Development

Average
schedule overrun
26%

32%

IT Architecture

19%

45%

26%

0%

20%

IT Infrastructure

2%

18%

27%

75%
71%

Integration Project

Average

Average bene-fits
shortfall

43%

5%

50% ()

McKinsey & Company

| 18

BRU-AAA123-20100305-

Hypothesis IV The bigger the riskier

Bigger projects are

riskier than smaller
projects

McKinsey & Company

| 19

BRU-AAA123-20100305-

Surprisingly, smaller projects have higher cost risk variability

Cost overruns
Percent

ICT average
ICT projects

600

500
400

Very weak linear trend

Bigger budgets don't increase the relative cost overrun
Expected risk in monetary terms grows linearly not exponentially

300
200
100
0
-100
0

250

500

750

1,000 1,250 1,500 1,750 2,000 2,250

13,750

Project Budget
EUR millions

McKinsey & Company

| 20

BRU-AAA123-20100305-

Hypothesis V The longer the riskier

Longer projects are

riskier than shorter
projects

McKinsey & Company

| 21

BRU-AAA123-20100305-

The longer the project, the higher the expected cost overrun
ICT average
ICT projects

Cost overrun
Percent of initial budget
600

500

400

300

Every additional year increases the

expected cost overrun by 16.8% and
schedule overrun by 4.8%
As project duration lengthens,
the probability of becoming an outlier
increases, esp. for projects > 3 yrs

200
100
0
0
-100

12

24

36

48

60

72

84

96 108 120 132 144 156 168 180

Project duration
Months

McKinsey & Company

| 22

BRU-AAA123-20100305-

Black Swan risk highest for large projects, however expensive

is better than long
Probability of Cost Black Swans

L
Actual size
Cost Black
Swans
Duration
Cost Black
Swans

XL

XXL

10-30
million

30-350
million

>350
million

20%

29%

11%

1-2 years

2-3 years

4+ years

6%

31%

27%

Highest risk =
stuck in the middle
Short is better
than small
Expensive better
than long

McKinsey & Company

| 23

BRU-AAA123-20100305-

Black Swan risk highest for large projects, however expensive

is better than long
Probability of Cost Black Swans

L
Planned size
Cost Black
Swans
Duration
Cost Black
Swans

XL

5-20
million

XXL

20-200
million

>200
million

9%

14%

13%

1-2 years

2-3 years

4+ years

12%

28%

41%

Short and small is

best
Expensive better
than long

McKinsey & Company

| 24

BRU-AAA123-20100305-

Hypothesis VI We need Masterbuilder

More experienced
project managers are
a key to successfully
managing risks

McKinsey & Company

| 25

BRU-AAA123-20100305-

Is grey hair an achievement?

Risk profile

Initial hypothesis:
the more work
experience, the
better the project
performance

Work experience
<10

10-14

15+

Cost overrun

1%

16%

32%

Schedule
overrun

2%

15%

61%

Mind over
machine

Young brains
new methods

However, we find a
struggle between
"Master builder"
and "Fire fighter"

Benefits
shortfall
Black Swan
events

-49%

0%

0%

19%

size (m)

0.9

4.3

141

duration
(months)

11

32

McKinsey & Company

| 26

BRU-AAA123-20100305-

Is grey hair an achievement?

Risk profile adjusted for size and duration effects (ANCOVA)

Initial hypothesis:
the more work
experience, the
better the project
performance

Work experience
<10

10-14

15+

Cost overrun

28%

40%

23%

Schedule
overrun

4%

18%

63%

Mind over
machine

Young brains
new methods

However, we find a
struggle between
"Master builder"
and "Fire fighter"

Benefits
shortfall
Black Swan
events

-48%

8%

7%

17%

size (m)

0.9

4.3

141

duration
(months)

11

32

McKinsey & Company

| 27

BRU-AAA123-20100305-

Hypothesis VII Benefits management is key

Lack of benefits
management is the
single most important
deficiency in ICT
project performance
management

McKinsey & Company

| 28

BRU-AAA123-20100305-

A key challenge to projects is Benefits Management

Risk profile

Cost overrun
Benefits not
measured

85%

Benefits
measured

15%

Schedule overrun

32%

-7%

Black Swans

36%

17%

122%

7%

Benefits force projects to think about

output and outcomes

Cost-benefit-based performance
management seems to motivate
managers to trade off schedule for
cost/benefits

With benefits management black swan

risk decreases

McKinsey & Company

| 29

BRU-AAA123-20100305-

7 answers to re-think megaproject management

Are lines-of-code riskier than

steel-and-concrete?

II

Are private and public

sector different?

Not on average, but

more Black Swans!

Budget cuts are the

only unique issue!

III Is standard software better No one is late, the

than bespoke solutions? other more expensive!

The challenge is
black swans, more
than it is average
overruns
VI

VII

IV Should we be afraid
of big projects?

Should we be afraid
of long projects?

What is the value

of experience?

What is the value of benefits management?

Short and small are

best, long is worse
than big

Master builders matter

Knowing output and
outcomes reduce risks
McKinsey & Company

| 30

BRU-AAA123-20100305-

Todays presentation

Collaboration of McKinsey and Oxford:

Research findings about success factors
for large-scale IT projects

2
Mastering technology
and content

Building team and

capabilities

1
Managing strategy
and stakeholders

Turning research into action:

Core beliefs in Value Assurance

4
Excelling in core
project management
practices

McKinsey & Company

| 31

BRU-AAA123-20100305-

Root cause analysis identifies 4 key dimensions

that explain most project failures
Risk comparison

Rough cost overrun disaggregation (percent)

"Missing focus"
Unclear objectives
Lack of bus. focus
Projects
>EUR 10M with
cost overrun
Cost
overrun1
Schedule
overrun
Benefits
shortfall

+90%

90

"Content issues"
Shifting requirements
Technical complexity
"Skill issues"
Unaligned team
Lack of skills

1
21

+44%
-67%

2
23

3
11

"Execution issues"
Unrealistic schedule
No active project
planning

4
16

18

Unexplained
cause
1 Cost increase over regular cost
SOURCE: McKinsey Oxford Reference Class Forecasting for IT Projects Study

McKinsey & Company

| 32

BRU-AAA123-20100305-

Turning research into action: core beliefs in mega-project management

along 4 dimensions
Based on empirical findings,
successful mega-project delivery builds on 4 pillars

leading to a
"Value Assurance" framework
organized in 4 building blocks

Project focus is "on time, in budget, in scope" and not enough

1

"in value" (only 15% of projects measure value)

Value focus is essential in all project phases to ensure
alignment with business

Mastering and reducing complexity (e.g., by modularizing

2

Delivering
value

ambitious technology efforts) is key to success

Built on solid understanding of technology, masterful design,
focusing on functionality that "makes a difference", ensures
effective solutions for front-line use

Mastering
content
Building
team &
capabilities

Only "team of experts" bringing together best internal resources

3

across organizational boundaries and external specialists can

deliver a transformation
At same time, organizations seek to sustainably improve
internal delivery capabilities, reducing dependency on external
vendors
Research shows long projects are more likely to fail
requirements change, momentum suffers, people rotate over time
Squeezing time out of project schedules and adhering strictly
to schedules is a key to success

Excelling in
project delivery

McKinsey & Company

| 33

BRU-AAA123-20100305-

Achieving a step-change improvement in mega-project delivery

requires a much more holistic approach in all dimensions
From (traditional approach)

Delivering
1 value

Mastering
2
content

Building
team &
3 capabilities

Excelling

4 in project

delivery

Unspecified goals
Scope grows over time
Unclear business case
IT-driven project
IT goals ("output")
Project creates new legacy system
1000-pages+ design documents
Ivory-tower solutions
Technical migration

Project creates vendor dependency

Project manager is subject-matter
specialist with limited project management experience
Project team that's available
Users "accept"/Users are "trained"
Risks reported
Focus on cost/budget
Administration of project plan
Issues are surprises, long debated

Traditionally
neglected

To (holistic approach)

Realistic, concrete goals

Stable scope
Solid budget, clearly articulated value drivers
Business-driven project
Business goals ("outcome")
Solution contributes to IT strategy
Design excellence where it makes a
difference
Solutions with front-line impact
Full-scale organizational mobilization

Client remains master of own destiny

Project manager is empowered expert or
Master Builder

Project team that's capable

Users specify, design, test, coach, etc.

Risks managed
Focus on value delivery
Progress and critical path transparent
Issues identified early, problems solved fast

McKinsey & Company

| 34

BRU-AAA123-20100305-

The Value Assurance approach combines key capabilities

in the 4 dimensions
Objectives

Examples follow

Key capabilities

Continuous alignment with business strategy

Continuously ensure
Stakeholder mgmt.
aspired value delivery, Proactive risk identification and mitigation
1
involving all
Business case mgmt.
stakeholders
Vendor mgmt. (incl. selection, contract
2

negotiation)

Mastering
content 1

Building team &

capabilities
Delivering
1 value

Address all content

aspects involved in the
2 project, and ensure
they are addressed in
best possible way

4
Excelling in
project delivery

Address cultural
3 aspects through
"softer tools"

Execute project mgmt

4 processes with rigor
and professionalism

IT architecture, infrastructure1
Functionality design/optimization1
Quality assurance1
Migration and roll-out strategy (technology,
organization)1
Project scoping/scope control1

Team alignment (e.g., stakeholders, project)

End-to-end-change mgmt. (organization and
mindset)
Capability building (incl. Master Builders)

PMO and project set-up

Requirements and change request mgmt.
Masterplan mgmt. incl. critical path
Status reporting
Issues/risk mgmt.
Rollout control and quality gates

1 Content may depend slightly on project subject (e.g., ERP, Core system replacement, O&O, PMM)
McKinsey & Company

| 35

BRU-AAA123-20100305-

Proactive risk identification and mitigation

Starting point for proactive risk identification is framework

of 13 typical risk/success factors in 4 categories
Overarching goal

Categories
Missing/insufficient strategic
alignment?

VALUE ASSURANCE 360

Project risk factors

Clear objectives?
Well-defined business case?
Alignment of major stakeholders?
Minimized, stable project scope?
Robust vendor contracts with clear responsibilities?
Executive support?

Minimize project
risks

Unproven
technology?

Insufficient
capabilities?

Standardized, proven software technology?

User involvement to shape solution?
Experienced project manager?
Qualified and motivated project team?
Sustainable mix of internal and external resources?

Non-robust
project mgmt
practices?

Reliable estimates and plans, appropriate transparency

about project status?
Proven methodologies and tools?
McKinsey & Company

| 36

BRU-AAA123-20100305-

Proactive risk identification and mitigation

where good is defined by simple Golden Rules

Categories
Missing/insufficient strategic
alignment?

VALUE ASSURANCE 360

Golden Rules In a well-run project

the project motivation is 1 sentence, the project objectives fit on 1 page
the business case is still positive if the costs double and the benefits are halved
there are clear decision rules how to overcome a deadlock between
stakeholders (there is a final decision maker)
delivery is broken down in modules that do not last longer than a year
contracts assign end-to-end responsibilities
there are monthly one-on-ones between project manager and executive
sponsor(s)

Unproven
technology?

Insufficient
capabilities?

... the underlying technology is not "bleeding edge", but has been proven in
comparable references
frontline users are continuously involved
the project manager has done it before
everyone in the leadership team has a clear leadership role

leadership is not outsourced

Non-robust
project mgmt
practices?

there are not only green lights in the status reports, but they show a "healthy
level of pain"
every approach, every tool has been used before
McKinsey & Company

| 37

BRU-AAA123-20100305-

Business Case Management

Client example Stopping a too expensive and too long

healthcare system implementation
Client situation

Performance of IT Mega projects (1/2)

NOT EXHAUSTIVE

Across Industries
Total Project Spend
($ MM)
Number of Companies

300-400

400-500

500-600

12

600-700

15

12

Average spend ($ MM)

352

451

540

647

Spend on services and

development(%)

~80%

Average time to
completion (Months)

44

~80%
80-90
Performance
of IT
Mega projects80-90
(2/2)

Healthcare
56

49

33

Cost Overrun (%)

(1.8)

(1.4)

(3.5)

Average Overrun

(0.9)

25

(1.8)

75th percentile

200

200

60

Development type

Supply Chain
management
Operations
Management e.g. call
center, work force
management etc

(1.8)

Minimum

(0.9)

Average

26

Total Project Spend

16
($ MM)
Case management
Operations management Document management

72

system for managing

(e.g. fleet management,
systems
Average time
to management etc) 24Emergency management 34
healthcare costs
facility
(Months)
GIS systems completion
Content
management
systems

Cost over runs ( %)

US

US

Total Man hours* (MM)

3.7

4.8 Man hours (MM) *

US, Germany

N=20

Maximum
~470

48

25

270

.76

5.0

US

5.7

.17

6.8

Cost over runs are measured from the point when the project got Green light for execution

Examples of Projects
* Back calculated using spend on services, assuming average rate of $80 per hour

Geographies
represented

Federal Health Information Exchange of the Dept. of VA

| 1
McKinsey & Company
Integrated
Clinical Database
of the U.S. Airforce
HC Claims Management of Dept. of Health

US

Cost over runs are measured from the point when the project got Green light for execution

Question

Are these estimates

realistic?

Printed

Geographies

(1.8)

NOT EXHAUSTIVE

Working Draft - Last Modified 29.09.2011 02:31:58

25th percentile

11

Number of Projects

Printed

Revamp core Health

Care Mana-gement
System USD 500m
invest
Planned
implementation time
of 8 years
Project at green-light
decision

Working Draft - Last Modified 29.09.2011 02:31:58

What was the output?

* Back calculated using spend on services, assuming average rate of $80 per hour

McKinsey & Company | 2

Project compared to similar projects

Schedule too long for invest and invest
too high making it a high risk project
Comparable HMS projects invested max.
USD 250m and needed max. 3-4 years

Project stopped
at green-light
decision
Rightsizing of
project together
with McKinsey
team started
Aligning
duration and
cost of the
project with risks
of HCMS
projects

McKinsey & Company

| 38

BRU-AAA123-20100305-

5 questions to take away

What risk do I carry around in my IT project

portfolio?
All traffic lights green, on time, in budget but
how are my most critical projects REALLY doing?
Are my project managers empowered enough
are they allowed to refuse take-off?
As the stakeholder, am I reported to or am I
actively solving problems?
Who is calling the shots me, the customer,
or the vendor?

McKinsey & Company

| 39