You are on page 1of 14

,4

).
6

EFFORT ESTIMATION
METHODS
Estimation involves the steps and processes involved in
calculating the amount of effort required for the execution of a
project. This is absolutely critical from a commercial perspective
as miscalculations can overrun the financial viability of a project.
Its also important from the project management perspective in
determining the team size and schedule of the project.
This white paper explains different methods of effort estimation
and emphasizes on using multiple expert opinions, framework/
models and interval estimates. Another key aspect discussed is
the benefits of using a framework or a model. It also explains
how we can utilize residual uncertainty in the estimate to our
advantage - having estimates with different confidence levels
and make use of it strategically.

WHITE PAPER

AJIKUMAR TN

Effort Estimation Methods

Table of Contents
INTRODUCTION ..................................................................................................................................................................... 3
COLLECTIVE THOUGHT ......................................................................................................................................................... 4
DELPHI METHOD ................................................................................................................................................................... 4
FRAMEWORK METHOD ....................................................................................................................................................... 5
ESTIMATION STATEMENTS ................................................................................................................................................... 6
COLLECTIVE THOUGHT - ADVANCED ................................................................................................................................ 8
DELPHI- ADVANCED ............................................................................................................................................................ 9
FRAMEWORK - ADVANCED .............................................................................................................................................. 10
HYBRID METHOD ................................................................................................................................................................ 11
CONCLUSION ....................................................................................................................................................................... 12
REFERENCES ........................................................................................................................................................................ 13
ACKNOWLEDGEMENTS ...................................................................................................................................................... 13
ABOUT THE AUTHOR .......................................................................................................................................................... 13
ABOUT WIPRO TECHNOLOGIES ......................................................................................................................................... 14
WIPRO IN VERIFICATION & VALIDATION ...................................................................................................................... 14

Page: Table of Contents

Effort Estimation Methods

INTRODUCTION
Estimation process is used in project management to estimate the effort required for the execution of a project. This is required
to do costing of the project for commercial reasons and also in determining the team size and schedule of the project.
This article consolidates different schools of thought about estimation. The technical aspects of estimation are not discussed
here; rather, different methods of estimation and estimation-statements are discussed.
The methods of effort estimation can be classified according to 4 different criteria mentioned below.
1) Experts involved in estimation are working together or independently for estimation purpose.
2) Whether framework is used for estimation or not.
3) Estimators coming up with a single estimate or a triple estimate.
4) Whether confidence level is used or not in estimation.
Each of these 4 criteria has two choices each and we will thus have 16 different methods of estimation. It is not practical to
discuss all of them in this article but we will take up important ones. The different methods of estimation discussed in this article
are:
Collective thought
Delphi
Framework method
Collective thought - advanced
Delphi - advanced
Framework - advanced
Hybrid
Table 1 shows these 7 methods of estimation and how they relate to the criteria mentioned above. The terminologies given in
the criteria framework, single estimate, triple estimate and confidence level will be discussed as we proceed.

Experts are
together or
independent

Whether
framework is
used

Single estimate/
Triple estimate

Whether
confidence
level is used

Collective thought

Together

No

Single

No

Delphi

Independent

No

Single

No

Framework method

Together

Yes

Single

No

Collective Thought - advanced

Together

Optional

Triple

Yes

Delphi - advanced

Independent

Optional

Single

Yes

Framework - advanced

Together

Yes

Triple

Yes

Hybrid

Independent

Yes

Triple

Yes

Criteria
Estimation Methods

Table 1: Estimation methods

Wipro Technologies

Page: 03 of 14

Effort Estimation Methods

COLLECTIVE THOUGHT
In this method, the project team usually does the estimation. In most of the cases, the full team may not be available to do the
estimation at the time of estimation process. It is usually done by some of the senior members in the team. The characteristics
of this exercise are:
The estimation team meets in a discussion room so that they can interact with each other. Some people do join in over
teleconference too.
It is expected that all the team members attending the estimation effort meeting would be prepared for a good
discussion and would have studied the requirements of the project in advance. For the benefit of those who have been
unable to do so, the moderator would brief the team about all the requirements for the estimation before the process
is actually begun.
The team goes through each of the requirements, analyzes all the risks and arrives at an effort based on knowledge,
experience and instinct. If team members have different opinions, the moderator helps to reach a consensus on the
effort required.
The effort arrived through consensus is used as the estimated effort.
This method can be used under the following circumstances.
Senior team members are available to do estimation at the time of estimation process.
They can meet and interact with each other.
Team members are able to reach a consensus about the estimate.

DELPHI METHOD
Like the collective thought method, a moderator would be active in this scenario too, but there are two important differences:
Experts outside the project team could also be asked to participate in the estimation process.
Estimation is obtained from the experts in a confidential manner. There would not be any interaction between the
experts involved in the estimation process.
Experts use their experience and gut feel to arrive at the estimate after understanding the requirements. The moderator takes the
average of the estimates he gets from the different estimators and arrives at the project effort.
As an example,
Moderator approaches six independent experts and got the following estimates.
210, 215, 219, 215, 224,215
Average estimate AE = (210 + 215 + 219 + 215+ 224 + 215) / 6 = 216.3
Arithmetic mean need not be the only method to arrive at the estimated effort in Delphi. Median or mode could be used as well.
In the example mentioned here, median is 215 and mode is also 215. If you look at the estimates given by the experts above,
3 out of the 6 experts have given the estimate as 215. So it makes sense to go by the majority view.
This method is best used in the following circumstances.
There are enough experts available within the organization who can judge over the requirements in hand.
The project manager is able to find different experts inside and outside the project team to give estimates.

Wipro Technologies

Page: 04 of 14

Effort Estimation Methods

FRAMEWORK METHOD
The estimators use frameworks in order to reduce the variability in estimates between them. Usually estimators use their gut feel
and experience from their past projects to arrive at estimates. Frameworks capture data from past similar projects and thus
reduce the dependency on experts experience. There are two advantages here. The designer of the framework might have used
all the data available about similar past projects. Usually this information would be more than an expert can bring into an
estimate. Also, different estimators will have varied experiences, which may reflect in their estimates. This will bring wide
variance in their estimates. Frameworks help to reduce this variance.
This can be explained in another way as follows. There are four main parameters which contribute to overall variance in
estimation. They are:
1. Variance when the same expert does the estimation at different times.
2. Variance when two different experts do the estimation at the same time.
3. Variance when the same expert does estimation for two different requirements.
4. Variance when experts interpret risk parameters differently and give different ratings for the same risk.
The variance in estimation due to parameters (3) and (4) are unavoidable. Two different requirements will have two different
estimations anyway. Also, since the risks are something which would occur in future with a probability associated with it, it
would be very subjective. We can expect different estimations in this case too1 .
If an expert does estimation at two different times on the same requirement, nobody expects a variation between the two
estimates. Also, two experts are expected to give the same/close estimates for the same requirement. So, in both these cases
variance should be zero or minimum. This is where using frameworks would help us. It gives the same estimate at different times
for the same requirement. It also gives close estimates when two people are using it for doing estimation for the same
requirement. This leads to reduced overall variance.
Usually, 3 types of inputs are fed into frameworks:
Requirement attributes
Framework constants
Estimator attributes
Requirement attributes are inputs to the framework depending on the requirement in hand. Examples are no. of features,
complexity of the features etc.
Framework constants are usually built into the system based on past data, used at the time of designing the framework. It is true
that these constants would get fine-tuned as more data becomes available over time.
Estimator attributes are those inputs given by the estimator as per his gut feel about different factors, which might occur in the
future, such as risks, certain team profile etc. For example in a typical testing project, the time taken by a tester to execute a test
case is not known at the time of estimation. This is because the information about who will be in the team is not known at that
time. This will go into the framework as an estimator attribute.
Frameworks use equations to get estimates. Lets take a testing project as an example. The requirements say 10 new features
have to be tested and the guidelines for the complexity measure tell that on an average the complexity of these features is 2.5.
Now these are the inputs to the framework as Requirement attributes.
The data from the past projects reflect that on an average 55 test cases are needed for a feature with a complexity factor of 1.
The number 55 is a framework constant, which is already built into the equation in the framework.

1 Subjectivity in risk assessments can be reduced using any standard method of risk identification, rating and prioritization. Some guidelines
also can be used.

Wipro Technologies

Page: 05 of 14

Effort Estimation Methods

The execution time per test case is to be entered into the framework to get the estimate. But the execution time depends on
the experience level of the project team that would be executing the project. This will go into the framework as an estimator
attribute. If the estimator feels that the expected team profile suggests 20 minutes as the execution time for a test case, the total
time needed for the project can be calculated as follows.
No. of test cases = no. of features * complexity * test cases = 10 * 2.5 * 55
Execution time = 20 * 10 * 2.5 * 55 = 27500 person minutes = 458 person hours.
This method is used under the following circumstances:
There exist similar projects executed earlier in the organization and somebody analyzed the past data and has already
come up with a framework.
Senior team members are available to do estimation at the time of estimation process.
Breaking off from here, the next section describes different ways used for putting across effort estimates to a client or champion
of the project. This is required at the present stage as the remaining 4 methods of effort estimation integrate different types of
estimation statements in it. The discussion of estimation methods would resume after this section.

ESTIMATION STATEMENTS
There are 3 ways to state an estimate of the execution effort needed for a project.
Point estimate
Interval estimate
Upper limit estimate
Point estimate
This is a single value estimate. As an example, the estimate for Project A can be stated as follows. Project A may take 450 person
hours of effort. Usually point estimate is the average estimate arrived at through collective thought, Delphi or framework
method.
Interval estimate
This is a double value estimate defining an interval qualified by a confidence level. For example, one can say that Project A may
take 425- 475 person hours of effort with 95% confidence level.
Upper limit estimate
This is again a single value estimate; but it specifies an upper limit (rather than an average value) and is qualified by a confidence
level. As an example, an estimator could state that, Project A will take less than 460 person hours with a confidence level of
75%.
Interval estimate is important from the perspective of a project manager, as she/he is more comfortable with an interval rather
than a point estimate. Here, she/he will get a spread to bank on.
Confidence level
Confidence level is used in Interval and Upper Limit estimates. Confidence level is the probability that the actual effort will be
within the specified interval of the estimate. In the case of interval estimate, the two numbers in the estimate specify the interval.
In the case of upper limit estimate, 0 and the upper limit value specify the interval.

Wipro Technologies

Page: 06 of 14

Effort Estimation Methods

An upper limit estimate qualified by a confidence level is important from business perspective costing and bidding. There are
4 main criteria which determine the business criticality of a project. They are:
Existing client - yes/no
Term of association with the client long/medium/short/zero
Nature of competition strong/medium/weak
Nature of the project Fixed Price project (FPP) / Time & Material (T&M)
Combination of these parameters we have in hand determines the confidence level of the estimate that will be used for costing
or bidding. The following are the general thumb rules:
If the client is new, it is natural that we bid low; so we will go for a low confidence level estimate to get a new client for
us.
If we have already long term association with a client, we can go for a higher level of estimate.
If the competition is strong, we bid low.
If the project is FPP, the risk is high and we quote for a high confidence level estimate to accommodate for any risk
factors.
A tool has been developed which will help in determining which confidence level of estimate we can use to bid for a project,
the Excel version of which may be obtained from the author. This tool is developed based on thumb rules rather than any strict
formulas. Depending on the combination of 4 parameters mentioned before, the tool throws out any of the five confidence
levels - 5%, 25%, 50%, 75% or 95%. The effort estimation methods further discussed in the article would help a business
manager in arriving at an estimate corresponding to any of these confidence levels.

Business Criticality Vs
Estimation Confidence Level
Existing client

Yes

Term of Association

Medium

Nature of Competition

Medium

Nature of the project

FPP

Confidence Level to be used


(Upper Limit Estimation)

75%

Table 2: Confidence level selection tool for estimation

Wipro Technologies

Page: 07 of 14

Effort Estimation Methods

COLLECTIVE THOUGHT - ADVANCED


In this case, all the characteristics described for the collective thought method holds true, but the team would come up with 3
estimates (triple estimates). The three estimates are termed optimistic, most likely and pessimistic. The use of a framework is
optional and depends on availability of the same.
Optimistic estimate (OE) is defined as the effort wherein everything goes fine for the project. In terms of risks, it can be treated
as an estimate when most of the identified risks do not occur. Most likely estimate (ME) is defined as the effort project may take
most of the time as judged by prior experience. In terms of risks, it can be treated as an estimate after accommodating the risks
that occur frequently. A point to be noted here is that the most likely estimate is not an average estimate; it is the estimate of
effort the project may take most of the time. Pessimistic estimate (PE) is defined as the effort when everything goes against the
project. In terms of risks, it can be treated as an estimate when most or all of the identified risks do occur.
Now the average estimate and standard deviation are calculated using the following equations.
Average estimate AE = (OE + 4*ME + PE) / 6
Standard deviation SD = (PE OE) / 6
As an example, if the optimistic effort is 120 person months, most likely effort is 130 person months and pessimistic effort is
150 person months; the average estimate is:
AE = (120 + 4*130 + 150) / 6 = 131.7 person months
SD = (150 120) / 6= 5 person months
With this information we can arrive at the estimates with different confidence levels.
estimate: Project will take 131.7 person months.
Point estimate
Interval estimate
estimate: With 95% confidence level we can say that the project will take 121.9 to 141.5 person months.
Upper level estimate
estimate: With 75% confidence level we can say that the project will take less than 135.1 person months.
The tool generates the interval estimate for 75% and 95% confidence levels. It also gives upper limit estimates for 95%, 75%,
50%, 25% and 5% confidence levels. Table 2 shows a typical output from the tool for different confidence levels.
Collective thought-advanced method can be used under the following circumstances:
Senior team members are available to do estimation at the time of estimation process.
Team has enough insight into the project needs and is able to analyze the risk factors so that they can reach on the
optimistic, most likely and pessimistic estimates.
Even though the use of framework is optional, it should be used if one is available.
Project/business manager needs estimates for different confidence levels.

Wipro Technologies

Page: 08 of 14

Effort Estimation Methods

Effort Estimation Statement


Optimistic Effort

120

units

Most likely Effort

130

units

Pessimistic Effort

150

units

Point Estimate

Lower Limit

Project Effort

Upper Limit
131.7

Interval Estimate
Lower Limit

Upper Limit

75% Confidence Interval

125.9

137.4

95% Confidence Interval


Estimation by Upper Limit

121.9

141.5

Upper limit Effort


95% probability that effort will be less than

139.9

75% probability that effort will be less than

135.1

50% probability that effort will be less than

131.7

25% probability that effort will be less than

128.3

5% probability that effort will be less than

123.4

Table 3: Effort estimation statement (tool output)

DELPHI- ADVANCED
In this method of estimation, the moderator gets the individual estimates from different experts. Thereafter, the moderator
processes these estimates as follows and states his estimate using any of the 3 ways- point, interval or upper limit estimates. Use
of framework is optional depends on the availability of the same.
Point estimate: Moderator uses mean, median or mode. For example, let us assume that the individual estimates from experts
are 210, 215, 219, 215, 224 and 215. Then the Mean is 216.3.
Interval estimate: Moderator finds the mean and standard deviation from the individual estimates. In the above example,
Mean = 216.3
Standard deviation = 4.7
With this information, one can state with 95% confidence that the effort for the project will be between 207 and 225.
Upper limit estimate: From the above example, it could be stated with 75% confidence that the effort required for the project
will be less than 219.6 person hours.
Given estimations from experts, using Delphi techniques, the tool automatically calculates the mean estimate and gives effort
statements for different confidence levels. This tool can process inputs from a maximum of 20 experts.
Delphi-Advanced method is used in the following circumstances.
There are enough experts available within the organization who can judge over the requirements.
The project manager is able to identify different experts inside and outside the project team to provide estimates.
Even though the use of framework is optional, it should be used if one is available.
The project t different confidence levels for strategic or any other reasons.

Wipro Technologies

Page: 09 of 14

Effort Estimation Methods

FRAMEWORK - ADVANCED
In this method, a framework is used by an expert to arrive at an estimate. As discussed earlier, there is a category of inputs called
estimator attributes in a framework that adds variability to the estimates. In the Framework-Advanced method of estimation,
the expert is allowed to give 3 kinds of estimates - optimistic, most likely and pessimistic. The optimistic estimate is arrived at
with an input of the optimistic values for the estimator attributes in the framework. Similarly for most likely and pessimistic
values as well.
In the earlier example under Framework Method, if the test case execution time is 17 minutes optimistic, 21 minutes most
likely and 23 minutes pessimistic; the efforts will be:
Execution time optimistic = 17 * 10 * 2.5 * 55 = 23375 person minutes = 390 person hours
Execution time most likely = 21 * 10 * 2.5 * 55 = 28875 person minutes = 481 person hours
Execution time pessimistic = 23 * 10 * 2.5 * 55 = 31625 person minutes = 527 person hours
Mean = (390 + 4* 481 + 527) / 6 = 473.5 person hours
Standard deviation = (527 - 390) /6 = 22.8 person hours
Any one of the estimation statements could be used to state the estimate.
Point estimate
estimate: The effort required by the project is 473.5 person hours
Interval estimate: With 95% confidence the effort required by the project will be between 428 and 516 person hours
Upper limit estimate: With 75% confidence the effort required by the project will be less than 489 person hours
The tool introduced in the section of Collective Thought- Advanced method can be used here also to generate estimation
statements easily.
Framework-advanced method is used under the following circumstances:
Senior team members are available to do estimation at the time of estimation process.
There exist similar projects executed earlier within the organization and analysis has already been done on the past data
and a framework has been created.
Team has enough insight into the project needs and is able to analyze the risk factors so that they can reach on the
optimistic, most likely and pessimistic estimates.
The project manager needs estimates at different confidence levels.

Wipro Technologies

Page: 10 of 14

Effort Estimation Methods

HYBRID METHOD
In this method, participating experts in the Delphi method are asked to use the framework already available and provide 3
estimates - optimistic, most likely and pessimistic. This means that all the experts would give 3 estimates each. The following
tables illustrate how this information will be processed to arrive at overall mean and overall standard deviation of the estimates.
Optimistic
Effort

Most Likely Pessimistic


Effort
Effort

Mean Effort

Standard Deviation of
Effort

Expert 1

O1

M1

P1

A1= (O1+4*M1+P1)/6

S1= (P1-O1)/6

Expert 2

O2

M2

P2

A2= (O2+4*M2+P2)/6

S2= (P2-O2)/6

Expert 3

O3

M3

P3

A3= (O3+4*M3+P3)/6

S3= (P3-O3)/6

Overall Mean Effort

(A1 + A2 + A3)/3

Overall SD of Effort

Sqrt (S1^2+ S2^2 + S3^2)

Table 4: Processing of triple estimates from Delphi experts

An example is given below


Optimistic Most Likely Pessimistic
Effort
Effort
Effort

Mean Effort

Standard Deviation of
Effort

Expert 1

400

420

450

421.7

8.3

Expert 2

390

415

430

413.3

6.7

Expert 3

405

425

440

424.2

5.8

Overall Mean Effort

419.7

Overall SD of Effort

12.2

Table 5: Example of processing of triple estimates from Delphi experts

The overall mean and overall standard deviation are used to issue 3 types of estimation statements as explained in the previous
sections.
This tool gives interval estimates for 75% and 95% confidence levels. It also gives upper limit estimates for 95%, 75%, 50%,
25% and 5% confidence levels. This tool can process inputs from a maximum of 20 experts. A sample of the input data sheet
is given below

Wipro Technologies

Page: 11 of 14

Effort Estimation Methods

Inputs
Sl. No.

Optimistic Effort

Most Likely Effort

Pessimistic Effort

1
2
3
4
5
------20

420
390
425
435
420
-------

435
420
435
440
430
-------

440
430
445
445
435
-------

(Figure 1: Sample Input to the tool)


This method is used in the following circumstances:
There are enough experts available within the organization who can judge over the requirements.
The project manager is able to identify different experts inside and outside the project team to give estimates.
There are similar projects executed earlier within the organization and analysis has been performed on the past data to
yield a framework.
The project manager needs estimates at different confidence levels.

CONCLUSION
Meeting
Triple Estimate
Confidence level

Meeting

Independent Experts

Framework

Collective Thought

Collective Thought
Advanced

Delphi Method

Delphi Method
Advanced

Framework Method

Framework Method
Advanced

Framework
Triple Estimate
Confidence level

Independent Experts
Confidence level

Hybrid

Independent Experts
Framework
Triple Estimate
Confidence Level

Table 6: Estimation methods - Reference table


The project manager, business manager or the moderator is the person who has to decide which method of estimation would
ultimately be used. Table 1 and Table 5 can be used as references to take a decision on which method to use depends on the
situation in hand. The situation is defined by the 4 factors such as experts availability, framework availability, single estimate/
triple estimate and nature of project/commercial importance (confidence level).

Wipro Technologies

Page: 12 of 14

Effort Estimation Methods

REFERENCES
Levin, Richard I., Rubin, David S. (1999), Statistics for Managers, Prentice-Hall India, 7th Edition.
Mulcahy, Rita (2002), PMP Exam Prep: A course in a Book, RMC Publications Inc., 4th Edition

ACKNOWLEDGEMENTS
The author would like to thank Mahesh Venkataraman (Group Head, Wipro Technologies), Pramod Varma (SQA Manger, Wipro
Technologies) and Shyam Bhaskaran (Project Manager, Wipro Technologies) for their valuable inputs and support.

ABOUT THE AUTHOR


Ajikumar TN is currently a Technical Manager for the Verification & Validation division of Wipro Technologies. He holds a B Tech
and an MBA in Software Enterprises Management. Ajitkumar has got over 9 years of experience in the software industry and has
worked for enterprises like BPL Telecom. He may be reached at ajikumar.narayan@wipro.com.

Wipro Technologies

Page: 13 of 14

Effort Estimation Methods

ABOUT WIPRO TECHNOLOGIES


Wipro is the first PCMM Level 5 and SEI CMMi Level 5 certified IT services company globally. Wipro provides comprehensive IT
solutions and services (including systems integration, IS outsourcing, package implementation, software application development
and maintenance) and research & development services (hardware and software design, development and implementation) to
corporations globally.
Wipros unique value proposition is further delivered through our pioneering Offshore Outsourcing Model and stringent
quality processes of SEI and Six Sigma.

WIPRO IN VERIFICATION & VALIDATION


Wipros Verification & Validation (V&V) services help software achieve high quality. The V&V service portfolio comprises
software life cycle testing, test process consultancy, test automation, and specialized testing. The V&V practice combines years
of testing capability and a strong industry focus to offer a host of re-useable process frameworks and methodologies to crash
testing cycle times. With industry specific Centers of excellence, knowledge management practices, and above all a high
degree of quality orientation, partnering with Wipro assures a definite improvement in productivity & quality of testing.

For further information visit us at: http://www.wipro.com/verification&validation


For more whitepapers logon to: http://www.wipro.com/insights
Copyright 2004. Wipro Technologies. All rights reserved. No part of this document may be reproduced, stored in a retrieval system, transmitted in any form or by any means, electronic, mechanical,
photocopying, recording, or otherwise, without express written permission from Wipro Technologies. Specifications subject to change without notice. All other trademarks mentioned herein are the
property of their respective owners. Specifications subject to change without notice.

Worldwide HQ
Wipro Technologies,
Sarjapur Road,
Bangalore-560 035,
India.

U.S.A.
Wipro Technologies
1300 Crittenden Lane,
Mountain View, CA 94043.

U.K.
Wipro Technologies
137 Euston Road,
London, NW1 2 AA.

France
Wipro Technologies
91 Rue Du Faubourg,
Saint Honor, 75008 Paris.

Germany
Wipro Technologies
Horn Campus,
Kaistrasses 101,
Kiel 24114.

Japan
Wipro Technologies
# 911A, Landmark Tower,
2-1-1 Minatomirai 2-chome,
Nishi-ku, Yokohama 220 8109.

U.A.E.
Wipro Limited
Office No. 124,
Building 1, First Floor,
Dubai Internet City,
P.O. Box 500119, Dubai.

www.wipro.com
eMail: info@wipro.com

Wipro Technologies

Page: 14 of 14

You might also like