Performance Evaluation Methodology and Systems

Budget Performance Evaluation Methodology by Tarun Das
Budget Performance Evaluation (BPE)-

Methodology, Systems and Management
Professor Tarun Das1
The Lexicon of Performance
“Wherever there is government, there is government performance. On the basis of

such obvious statements, modest reputations are earned! However, what constitutes
performance within government is more complicated, pluralistic, value laden, and
controversial than is true with the performance of private firms, where performance
yardsticks are more limited and more universally accepted. In government,
performance is regarded as progress toward goals and objectives, but measurement
is complicated by the fact that the outcomes are often multiple, vague, shifting and
even conflicting; as they emerge out of wider political process and public debate.”
“In diverse countries such as Canada or Australia, individuals, groups, and regions
will often disagree strongly over what constitutes good performance. Also,
government performance is more subject, than corporate performance, to continuous
and critical scrutiny by opposition parties, interest groups and the media. So, for
governments, appearances matter as much or even more than the reality of
performance. Maintaining legitimacy and support for the government policies and
programs is also a legitimate performance goal. In short, performance is a many
splendoured thing, and good performance is partly in the eyes of the beholder.”
Source: Paul G. Thomas (2006) Performance reporting, obstacles and

accountability- Recent trenda and future directions, The Australian National
University, Canberra ACT 0200
Ministry of Finance
Government of Mongolia
ADB Capacity Building Project
On Governance Reforms
Ulaanbaatar, Mongolia
20 May 2008
_______________________________________________________
1
Glocoms USA Inc. Strategic Planning Expert. Formerly, Economic Adviser, Ministry
of Finance and Planning Commission, Government of India and Professor (Public
Policy), Institute for Integrated Learning in Management (IILM), New Delhi, India.
MOF, Govt. of Mongolia 1 Glocoms Inc. (USA)

Budget Performance Evaluation (BPE) -

Professor Tarun Das
Contents
1. Introduction, Scope and Objectives
1.1 The Budget and Public Sector Performance

1.2 Performance Based Budgeting
1.3 Integration of Strategic Plan with Program Budget
1.4 Performance Monitoring and Evaluation
2. Ideal Performance Evaluation Systems
2.1 Characteristics of an ideal performance evaluation system

2.2 Various Models of Performance Evaluation
2.3 Choice of Particular Performance Evaluation System
3. Budget Performance Indicators
3.1 Performance Measures versus Performance Indicators

3.2 Types of Performance Indicators
3.3 Characteristics of an Ideal Performance Indicator
3.4 Use of Performance Measures
4. An Operational Budget Performance Evaluation System for the

Government of Mongolia
4.1 Hierarchy of Activities, Functions and Output Classes

4.2 Types of Budget Performance Evaluation
4.2.1 Input/ resource use evaluation (compliance evaluation)
4.2.2 Output/ service evaluation (efficiency evaluation)
4.2.3 Clients’ satisfaction evaluation (efficiency evaluation)
4.2.4 Outcomes/ Community evaluation (effectiveness evaluation)
4.3 Benchmarks
5. Budget Performance Evaluation Methodology
5.1 Analysis of strategic plan and baseline profile

5.2 Diagnostic scan and SWOT analysis
5.3 Strategic and Operational Performance evaluation
5.4 Budget Compliance, Efficiency and Effectiveness Evaluation
5.5 Overall Assessment and Score

5.6 Major findings and recommendations
6. Budget Performance Evaluation Management
Selected References
Annex-1: Template for Review of Strategic Business Plan and Baseline

Profile of the Line Ministry/ Budgetary Agency
Annex-2: Template for Strategic Review and Operational Review
Annex-3: Template for Budget Compliance, Efficiency and Effectiveness
Evaluation
Annex-4: Methodology for calculation of composite score and grading the
Line Ministry/ Budgetary Agency
Glossary: Definition of Terms
List of Tables:
Table-1: Characteristics of an Effective Budget

Table-2: Characteristics That Adversely Impact Budget Performance
Table-3: Characteristics of an Ideal Performance Measurement System
Table-4: Budget Performance Indicators
Table-6: Broad Categories of Budget Performance Evaluation
Table-6: Scope of Performance Review and the Initial Profile
Table-7 Strategic Review and Operational Review Areas
Table-8A: Typical Best Practices for Strategic Review
Table-8B: Typical Best Practices for Operational Review
Table-9 SWOT Analysis for the Ministry of Finance
Table-10: Scores for Strategic and Performance Evaluation
Table-11: Rating of an Agency on the basis of Performance Scores
Table-12: Marks for Budget Compliance Evaluation
Table-13: Marks for Budget Efficiency Evaluation
Table-14: Marks for Budget Effectiveness Evaluation
Table-15: Weights for Various Types of Evaluation
Table-16: Rating of an Agency on the Basis of Overall Scores
Table-17: An Example of Performance Scores for three Agencies
Table-18: Estimation of Overall Rating for three Agencies
Table-19: Deadlines for various steps in Budget Performance Evaluation
List of Flow Charts
Flow-Chart-1: Strategic Plan and Performance Based Budgeting Cycles:

Flow Chart-2: Integration of Strategic Business Plan, Program Budget and
Performance Evaluation
Flow Chart-3: Pyramidic Structure of Performance Evaluation
Flow Chart-4: Deadlines for various steps in Budget Performance Evaluation

Budget Performance Evaluation (BPE) -

Professor Tarun Das
1. Introduction, Scope and Objectives
“Would you tell me please, which way I ought to go from here?” asked Alice.
“That depends a great deal on where you want to get to,” said the cat.
“I don’t much care where ….,” said Alice.
“Then it does not matter which way you go,” said the cat.
--- Extract from Alice in Wonderland, Lewis B. Carroll
Eventually Alice in Wonderland realized that it matters a great deal to know

“where to go” and “how to get there”. Similarly, in government budget it is
important to know the basic goals and objectives and the overall scope of
budgeting in terms of exact outputs and outcomes, and how to achieve these
goals, objectives, outputs and outcomes in time and with least cost.
Strategic Planning, Budget Formulation, and Performance Management Cycle

is an interactive on-going process to facilitate sound government business
and financial plan. Budget Performance Evaluation (BPE) makes a detailed
and critical analysis of the multi stakeholders’ requirements, develops action
plans and allocates resources by integrating budget and performance goals.
Government resources are not unlimited and there are constraints on public
resources and competing demands by various social sectors, public goods
and services. Consequently, an optimal resource planning and budgeting
requires that strategic planning, budget and performance management must
be integrated in a comprehensive and time bound plan.
1.1 The Budget and Public Sector Performance
Performance budgeting is primarily a management, transparency and

financial accountability tool rather than a mere budgetary instrument.
Effective and efficient delivery of public goods and services by budgetary
agencies requires performance management, under which an Agency
establishes its service objectives and monitors performance towards the
attainment of those objectives. Meyers (1996) has identified 11
characteristics of an effective budget which supports performance. These
characteristics are listed in Table-1

Table-1: Characteristics of an Effective Budget
Term Characteristics
1. Accountability Detailed controls should ensure that government

policy directives are carried out by managers,
contractors and all concerned parties.
2. Comprehensiveness Includes all uses of government’s financial

resources.
3. Constraint Contains limitations on the amount of money that

can be raised and spent by government.
4. Cooperation Budgeting should exist in harmony with other

decision processes and should not be dominant.
5. Honesty Contains unbiased projections and allocations

under different programs and heads of accounts.
6. Judgment Encourages participants to seek the most effective

ways to achieve the least costs.
7. Legitimacy The budget process should reserve important

decisions to legally appropriate authorities.
8. Perception Considers both near- and long-term plan.

9. Responsiveness The budget must adopt policies that match public
preferences.
10. Timeliness A budget process should complete regular tasks

within expected time period.
11. Transparency The budget and budget procedures should be

understood by participants and outside
stakeholders.
Source: Meyers, Roy T. (1996).
Some of these characteristics for an effective budget relate to the process of

budget prioritization (comprehensiveness, perception), while others relate to
operational efficiency (cooperation, timeliness). Major factors that might
affect adversely the budget performance are summarized in Table-2.
Table-2: Characteristics That Adversely Impact Budget Performance

Term Characteristic
1. Extra-budgetary Extra-budgetary funds lead to dual budgeting and

Funds do not allow tradeoffs between competing
priorities. Extra-budgetary funds generally
circumvent central controls, thereby violating the
comprehensiveness standard.
2. Earmarked Funds Earmarked funds are dedicated to particular uses
(Earmarking) or purposes. Agencies with earmarked funds may
have little or no incentive to economize.
Earmarking may impede the government’s ability
to be flexible in managing its budget or in
allocating resources.
3. Unpredictability Inaccurate budget estimates, arbitrary budget cuts,
and non-timely fund disbursements will create
substantial budget unpredictability.
4. Unreliable Inaccurate and unreliable information will conspire

Information against enhancing budget performance.
Source: Meyers, Roy T. (1996).
1.2 Performance Based Budgeting
A performance budget is an integrated annual performance plan that shows

the relationship between program funding levels and expected results. It
indicates that a goal or a set of goals should be achieved at a given level of
funding. An effective performance budget does more than act just as a
program budget with anticipated outcomes. It identifies and explains the
relationships between money spent and results. Such explanation is key to
manage a program effectively. Whenever there are variances between plans
and actual results, managers examine the resource inputs and how they
relate to outcomes to determine program effectiveness and efficiency.
A program performance budget defines all activities, direct and indirect,

required by a program, in addition to estimating activity costs. The following
figure outlines this process. By tracking the cost and number of units for
each activity, output, and outcome, unit cost information also may be
generated.
Money ⇒ Activity ⇒ Output ⇒ Outcome ⇒ Strategic Goals
Regardless of agency level, managers must know accurate and timely cost
and performance information to manage their resources most effectively.
This applies to both administrative officials and program managers. In certain

cases (e.g., computer services), managers require this information to

establish unit prices. In other cases, managers simply want to ensure that
every unit of money supporting a program is spent wisely.
Recently many countries have focused their attention to governance reforms

and the role of performance based budgeting. As in the case of many aspects
of Budget Modernization techniques, Australia and New Zealand have
developed over the years very extensive performance based budgeting
methodology and systems. The common principles in Australia and New
Zealand relate to governance reforms and the institutionalization of
performance standards in budgets:
(i) Budget transparency is the key issue. In Australia, for example, there
is a requirement for three-year forward estimates and for reporting of
budgetary versus actual budgetary allocations for the previous year. Under
the Public Sector Management and Finance Act (PSMFA, 27 June 2002)
government of Mongolia has also adopted the same requirement2.
(ii) Decentralization of operating functions is another important

development. There has been a substantial devolution of management
functions to operating line agencies to perform their tasks. Under the
system of portfolio budgeting, line agencies are required to prepare multi-
year Strategic Business Plans indicating vision, mission, objectives and
priorities, and identifying spending and revenue measures necessary to
meet aggregate fiscal targets set by the Head Quarter. PSMFA of Mongolia
also mandates the same requirements3.
The primary difference between the Australia and New Zealand performance
based budget frameworks is a tradeoff between a focus on outputs and a
focus on outcomes. For example, in New Zealand, the focus is on technical
efficiency in the delivery of outputs, whereas in Australia the focus is on
outcomes (i.e., the impact of outputs on beneficiaries). Mongolian PSMFA
puts emphasis on both technical efficiency and outcomes.
The main reason for the diverse paths is due to the underlying conditions
previous to the reforms. In New Zealand, the pre-reform state was
characterized by substantial budget overspending and pre-dominant role of
the public sector services. An important component of the reforms program
was the substantial privatization of these services. So in New Zealand,
performance is based on formal accountability mechanisms and sanctions.
On the other hand, Australia did not have an extended public sector. The
budget process evolves a system where political leaders set strategic
priorities, and operating line managers are given considerable discretion to
manage their agencies and programs to achieve the intended outcomes. In
2
Articles 26.1, 26.2 and 26.3 of the Public Sector Management and Finance Act (PSMFA, 27 June 2002).
3
Articles 26.1, 26.2 and 26.3 of the Public Sector Management and Finance Act (PSMFA, 27 June 2002).

the Australian system there are no formal output based contracts. Public
accountability is achieved through performance reporting and evaluations.
1.3 Integration of Strategic Plan with Program Budget
To accomplish its strategic objectives effectively, an Agency must link

outcomes, strategy, budget, and performance into an integrated
management system. This management system, based on a model of
continuous improvement, is shown in Flow Chart-1 below:
Strategic Business Plan

Broadly defines strategic goals, outcomes, outputs of an
Agency and the methods to achieve them.
Performance Program Budget and

Assessment and Performance Planning
Program Budgets- Funds allocated to specific
Improvement Plan programs to achieve desired goals, outputs,
Compares actual to target
and outcomes with least cost.
performance and
Performance - specifically designed results
benchmarks.
Value- achieving value for money
Determine changes that will
Establish long-term and annual targets for
produce the best value.
spending, performance and value.
Performance Monitoring
Track the progress, expenditure, and value for
money for achieving outcomes.
Flow-Chart-1: Strategic Plan and Performance Based Budgeting Cycles:

Source: John Mercer, regarded as the father of the USA GPRA, 1993.
The process begins with an understanding of important national priorities and

outcomes, which are then translated into broadly defined goals and intended
results for the Agency. These become the Agency’s strategic goals and
strategic objectives. Outcomes and Outputs related to these strategic goals
and objectives are then articulated. Programs are developed to achieve these
outcomes and outputs with least resources, and then performance measures
and indicators are identified to provide the means to assess the progress
made during the budget year and to suggest improvements for the next
year’s budget. Flow Chart-2 below explains the relationship between an

Agency’s Medium Term Strategic Business Plan and its Annual Program
Budget and performance evaluation.
Strategic Business Plan Next Year’s Budget
Vision, Mission, Strategy Improvement Plan

and Objectives
Strategic Goals Performance Monitoring and

Evaluation through PART
Performance Indicators
Performance Indicators
And Measures
And Measures
Strategic Outcomes Program Outcomes

And Outputs And Outputs
Activities and Processes Program Budgets
Inputs (Staff, Funds, Resources for

Goods, Services, ICT) The Budget Year
Flow Chart-2: Integration of Strategic Business Plan,

Program Budget and Performance Evaluation
1.4 Performance Monitoring and Evaluation

The latest stage of development as regards budgeting is the performance
(based) budgeting. Most of the OECD countries (e.g. Australia, Canada, New
Zealand, The Netherlands, Sweden, Finland, UK, and USA) have rewritten
their financial legislation or legal frameworks to allow for performance
(based) financial information systems, including budgets.
It is important to understand that performance-based budgeting is not simply
the use of program performance information in developing a budget. It does
more than just informing the resource allocations of a traditional type of
budget. In other words, it is not just "budgeting based on performance."
Instead, it is the process and integrated systems framework by which a
particular type of budget (such as output budget or program budget) is
developed. To design an effective system of performance-based budgeting, it

is vital to understand first what the end product should be, what it should
contain, and how efficiently it can be achieved. The pyramidic structure of
performance budgeting is illustrated in Flow Chart-3.
Flow Chart-3: Pyramidic Structure of Performance Evaluation
Source: John Mercer, an expert on Performance Budget and also regarded as the
father of the USA “Government Performance and Results Act (GPRA, 1993)”.
A true Performance Budget is not simply an output budget with some

program goals attached. It indicates how the budgeted resources are
expected to turn into results, by outlining a general chain of cause and effect.
The most effective Performance Budget shows day-to-day activities and
financing for each program area, and how these activities generate certain
outputs, and what outcomes should then be the result.
A Performance-based output Budget differs from an input-based budget in a

fundamental way. An input-based budget shows how budgeted fund is spent
on salaries, social benefits, goods, office supplies, travel, utilities, equipment,
etc. The Performance Budget shows what expenditure will accomplish:
establishment of a primary school, city hospital, building roads, houses etc.
and reviews a compliance activity. However, every output can also be
associated with input costs.
2. Ideal Performance Evaluation Systems

2.1 Characteristics of an Ideal Performance Evaluation System
Paul G. Thomas (2005) has mentioned the following properties of an ideal

perforance measurement system:
• It has clearly defined purposes and uses.

• It focuses on outcomes, not just on inputs and outputs.
• It employs a limited, cost effective set of measures.
• It uses measures which are valid, reliable, consistent,
comparable and controllable.
• It produces information which is relevant, meaningful,
balanced and valued by the leaders/ funders of the organisation.
• It is integrated with the planning and budgetary
processes.
• It is embedded in the organisation, is stable and is widely
understood and supported.
In somewhat less abstract terms, the Canadian Comprehensive Auditing

Foundation (CCAF) has developed nine principles 'to provide direction for
performance reporting in Canada. Box-1 presents these nine principles. The
first five principles provide guidance about what governments should report,
while the remaining four relate to how governments report. The principles
start with as ideals, the ‘ceiling’ that reporting aspires to reach, but over time
they become 'standards’, the floor below which reporting should not sink.
Taken as a set, the nine principles are meant to provide a framework for
performance reporting.
2.2 Various Approaches to Performance Measurement

Box-1 — Nine Principles of Better Performance Reporting
1. Focus on the few critical aspects of performance.

2. Look forward as well as back.
3. Explain key risk considerations.
4. Explain key capacity considerations.
5. Explain other factors critical to performance.
6. Integrate financial and non-financial information.
7. Provide comparative information.
8. Present credible information, fairly interpreted.
9. Disclose the basis for reporting.
Source: Canadian Comprehensive Auditing Foundation, Reporting Principles: Taking

Public Performance Reporting to a New Level. Ottawa, 2002.
Performance measurement has become so widespread that it is impossible to

know all what is taking place within public sector around the world. There is
no single, ‘one best’ approach to performance measurement. A government
needs to develop an approach that fits with its constitutional/ legal

requirements, arrangements, political ideology, its size, administrative and

institutional capabilities, and, not least important, what it can afford. The
general tendency for governments has been to apply a single and uniform
approach to all ministries, agencies and programs. This ‘across-the-board’
approach may have apparent virtues of consistency, comparability and
fairness, but it is not without problems.
It is interesting to know how the system evolved in Canada over the past
decade. The federal and provincial governments in Canada developed two
broad approaches to performance measurement. In Alberta, Nova Scotia,
Ontario and Quebec, governments reported on the performance of the entire
government in terms of the impacts of their programs on the society. This
‘social-indicator’ type of approach supports external and political
accountability. However, the selection of indicators included in ‘report
cards’ to citizens was inherently arbitrary (Thomas 2006).
Other provinces and the Government of Canada began their performance

measurement efforts by requiring individual departments to prepare business
plans and performance reports. The ‘business-line’ approach is more of a
managerial tool than something which is normally liked by politicians and the
public. However, these two broad approaches viz. the system-wide and the
business-line could be pursued simultaneously and complement one another.
This has been the evolution of the performance reporting system in the
Government of Canada. It began by publishing performance reports on a
departmental basis and now more than 80 such documents for departmental
and non-departmental bodies are tabled in Parliament on a annual basis.
Governments have developed a number of frameworks to identify successful

programs. Probably the most common framework involves the so-called
‘three big Es’: economy, efficiency and effectiveness as described below.
Economy Have inputs been acquired at least cost?
Efficiency Are the inputs (people, money, supplies) being combined to

produce the maximum volume of outputs (goods and
services)?
Effectiveness Are the goals of the organization/ program being met,
without undue unintended consequences?
These elements had been used by both the public and private sector
management over the past four or five decades. But, the framework misses
another important E i.e. equity, which deals with the distributional impacts of
performance. Omitting equity may have an adverse impact on another big E
in government i.e. electability.
In its earlier program evaluation scheme, the Government of Canada

considered a program to be well performing when it was:

Relevant Consistent with government and departmental priorities.

Successful Achieves the intended outputs and outcomes.
Cost-effective Involves the most efficient means to achieve goals.
This framework deals with the desirability of continuing a program, but does
not address the question: Does the organisation have the capability and
capacity to deliver the desired results?
Organisational report cards represent another type of performance

measurement and reporting. Box-2 presents one interpretation of the
requirements for such report cards.
Box-2: Organisational Report Cards — Criteria for Design
(a) Validity- satisfies legal requirements

(b) Comprehensiveness- covers all aspects of a budget
(c) Comprehensibility- easy to understand
(d) Relevance- appropriate for strategic objectives
(e) Reasonableness- can be achieved within time and at reasonable cost
(f) Functionality- operational and realistic
Source: William T. Gormley and David L. Weimer, Organisational Report Cards. Cambridge,
Mass.: Harvard University Press, 1999. pp. 36-37.
In 1987 the Canadian Comprehensive Audit Foundation (CCAF) published a

report on ‘organisational effectiveness’ which mentioned the following
attributes of an effective organisation management:
Box-3: Attributes of an effective organisation management

(a) Relevance
(b) Appropriateness
(c) Achievement of purpose
(d) Acceptance
(e) Secondary Impacts
(f) Costs and Productivity
(g) Responsiveness
(h) Financial Results
(i) Working Environment
(j) Monitoring and Reporting
Several Agencies have since applied this framework. But, there may be
conflict in practice among the attributes — e.g., cost efficiency may reduce
responsiveness. Besides, assigning weights to these attributes and
constructing a weighted overall index is a challenging job.
The Office of the Auditor General of Canada (OAG) has recommended

another performance framework with six components of performance:

1. Mission statements
2. Results statements
3. Performance indicators/measures
4. Performance expectations/targets/commitments
5. Strategies/activities
6. Performance accomplishments/achievements
This framework emphasises the desirability of integrating performance

planning, budgeting, monitoring and reporting, and also stresses external
accountability for results.
2.3 Choice of Particular Performance Evaluation System
Regardless of the approach adopted, a sound performance measurement and

evaluation system must have three qualities: it must be technically valid, it
must be functional, and it must be legitimate. Table-3 presents one generic
listing of the ‘ideal’ qualities of such a system.
Table-3: Characteristics of an Ideal Performance Measurment System
Characteristic Definition
Clarity Performance indices should be simple, well defined, and

easily understood.
Consistency Definitions of indicators should be consistent over time
and across agencies.
Comparability One should compare like with like.
Controllability A Mmanager’s performance should only be measured for
areas over which he/she has control.
Contingency Performance is not independent of the environment within
which decisions are made.
Comprehensive Covers all aspects of a budget.
Bounded Consider a limited number of performance indices which
provide the biggest pay-off.
Relevance Performance indicators are relevant to the special needs.
Feasibility Targets are based on realistic expectations.
Source: Peter M. Jackson, Measures for Success in the Public Sector.
3. Budget Performance Indicators
A budgetary agency has the responsibility to ensure that its programs meet
the stated objectives and are cost effective. By measuring the output,
outcome and efficiency of a program, performance indicators can guide an
agency for optimal resource allocation and communicate the results of public

programs to the stakeholders. It is necessary to review progress regularly

toward achieving outputs and outcomes, and continuously improve by
planning, executing, evaluating, and adjusting actions to achieve desired
results using an integrated performance evaluation system.
3.1 Performance Measures versus Performance Indicators
Most government manuals on performance measurement make a distinction

between performance measures and performance indicators. Ideally,
performance measures report unambiguously on the relationships between
program activities and the outputs and outcomes associated with them.
However, various economists have pointed out that most relationships
between programs and societal impacts are imperfectly understood and
subject to change over time. For example, if we want to measure the
relationship between safety regulation in the transportation field and the
reduction of fatalities and injuries, we must control the impact of other
influences within the program environment. Thus, performance measures
usually provide only a proxy indication of the performance of a program or
policy system.
While performance measures might be likened to numbers on a

gauge, performance indicators might be compared to alarm bells.
Like the bell on the proverbial cat in the famous fable, performance
indicators tied to specific programs can warn when unpleasant
surprises are on the way. Given the current state of our knowledge about
many programs, the distinction between true measures and approximate
indicators is somewhat artificial, and subject of public debate rather than of
automatic acceptance.
There is strong push within the performance measurement movement to

produce comparative evidence on performance. The comparisons can be
made with regards to other programs within a sector over time.
‘Benchmarking’ and the adoption of the ‘best practices’ of leading
organisations is a part of this trend. Here again there is no simple answer as
there are various techniques for benchmarking and best practices4.
For practical solutions, it would be wiser for agencies to pursue a

‘smart practice’ rather than a ‘best practice’ approach (Thomas
2006). Smart practice recognises that in statistical terms all
organisations cannot be ‘the best’ and an organisation cannot remain
best for all times. Organisations deemed to be the best at one point in time
4
See Tarun Das and E. Sandagdorj (2007e and 2007f) on measures for benchmarks
and best practices.

by one set of measures often lose that status in the near future. Pushing
organisations to strive for ‘the best in class’ status may ignore the practical
constraints in an agency, such as lack of able leadership and skilled
manpower, inadequate infrastructure, unsupportive organisational culture
etc. Developing reliable and consistent information for comparison purposes
is a worthy goal, but it has to be balanced by the costs involved and the
potential gains.
3.2 Types of Performance Indicators
Performance indicators measure what an Agency did in the fiscal year. There
are many kinds of performance indicators ranging from quantities or value of
goods and services produced in a given period (such as the number of crimes
or breaking of traffic rules detected by the police) to more complex indicators
such as efficiency and effectiveness of service delivery.
Nayyer-Stone (1999) has mentioned four primary types of performance

indicators: input, output, outcome and efficiency, which are described in
Table-4. In general, input indicators address the amount of resources used
in providing a service, whereas output indicators describe the activities
undertaken in providing a service. Outcome/ effectiveness indicators are
used to evaluate the quality and effectiveness of public services. Outcome/
effectiveness indicators are generally measured in terms of the number of
people served and the quality of the service delivered. For example, a
relevant outcome indicator for policing is the number of crimes committed
per capita. Outcome indicators provide an indication of how effectively
community services are provided. However, outcome/ effectiveness
indicators are sometimes difficult to use due to the inability to determine a
direct correlation between the service provided and the results measured,
and these are generally time consuming measurement techniques.
A final indicator is efficiency, which relates inputs to output or outcome- for

example, the cost per liter of water delivered to a household. When efficiency
indicators are used over time, they provide evidence of productivity trends.
These indicators and other indicators are explained in more details in
the next section on an operational performance evaluation system
for Mongolia.
Table-4: Budget Performance Indicators
Type of Indicator Definition Example

Input Indicator Measure of • Employees Required
Resources Employed • Goods and Services Used
• Equipment Needed

Output Indicator Quantity of Goods • Number of projects

and Services • Number of outputs
Provided • Number of people served
Effectiveness/ The degree to which • Increase in literacy rate
outcome Indicator the intended • Increase in employment
objectives of the • Decrease in crime rate
services are being • Reduction of poverty.
met. • Reduction of maternal and child
mortality rate
Efficiency Indicator Cost per unit of • Cost/ liter of water delivered.
output • Cost of garbage collected.
• Cost per student in schools

Source: Adapted from Hatry, Harry P. (1977).
3.3 Characteristics of ideal performance indicators
Like any statistical measure, performance indicators must satisfy a number

of criteria. In general, an ideal performance indicator should be S.M.A.R.T.
(i.e. simple, measurable, achievable, relevant and timely) and C.R.E.A.M.
(i.e. clear, relevant, economic, adequate and monitorable).
S.M.A.R.T. • Simple- easily defined

• Measurable- easily quantifiable
• Achievable – can be achieved, not a wish list
• Relevant- Appropriate for the strategic objectives
• Timely- can be achieved in time
C.R.E.A.M. • Clear- Precise, unambiguous, tangible and
quantifiable
• Realistic- achievable with reasonable cost and in
time
• Economic - Available at reasonable cost and in
time
• Adequate- Provides sufficient basis to access
performance
• Monitorable- Amenable to impartial/ objective
evaluation
3.4 Use of Performance Measures
Performance measures can be used in several ways, including the following:
a) Controlling costs – enabling agencies to identify costs which are much

higher or lower than average and determine why these differences exist.
b) Comparing processes – analyzing performance of services provided
with a particular technology, approach or procedure.
c) Maintaining standards – monitoring service performance against
established performance targets or benchmarks.

d) Comparing sectors – comparing costs of public sector delivery to costs

of private sector delivery of the same type of goods and services.
Box-4 — The Aims of Performance Measurement
• To help clarify organisation goals, directions and expectation.

• To help organisations learn how to accomplish goals more effectively.
• To communicate the priorities of the organisation.
• To support strategic/business line planning by linking broad statements of
direction to specific operational outputs and outcomes.
• To support budgetary planning and resource allocation processes.
• To monitor the operation of programs and to make continuous improvements.
• To motivate public servants and to restore pride within the public service that
it is making a positive contribution.
• To enable citizens to make better informed decisions in the use of public
programs.
• To restore public confidence that they are receiving value for money in public
spending.
• To assess whether the organisation is achieving its goals.
• To strengthen internal administrative and external political accountability.
Adapted from Thomas, Paul G. (2006) Performance Measurement, Reporting, Obstacles and
Accountability -Recent Trends and Future Directions, Research School of Social Sciences, The
Australian National University, Canberra ACT 0200.
4. An Operational Budget Performance Evaluation System

For the Government of Mongolia
Budget Performance Evaluation System provides an overview of what a

budgeting Agency has done and what it has accomplished with the budgeted
resources at its command. This involves describing the resources used, the
goods and services produced and the resulting benefits to the citizens and
the community at large. The evaluation process provides a coherent account
of the full range of goods and services provided by an agency. The resulting
report helps citizens understand how effectively and efficiently goods and
services are being provided by the agency.
4.1 Hierarchy of Activities, Functions and Output Classes
Performance evaluation classifies government services in terms of policy

areas, functions and output. To take into account variations in size and
complexity of agencies, performance evaluation may deal with sub-classified
areas of broad functions of an agency. For example, the broad functional
area of Law, Order and Public Safety of the Ministry of Justice and Internal
Affairs (MOJIA) could be subdivided into the provision of: police services;
court services; corrective services; and emergency services.

Broad Functional Area: Law, Order & Public Safety

Separate Functions: a) Police Services
b) Courts Services
c) Corrective Services
d) Emergency Services
Activities within each functional area are grouped together as “output

classes” based on a common characteristic or purpose. These groupings
describe how an agency fulfills its role. For example, the function of Police
Services could be sub-divided into sub-functions such as community policing,
crime investigation and traffic control.
Functional Area: a) Police Services

Sub-Functions: (i) Community policing
(ii) Crime Investigation
(iii) Traffic control
4.2 Types of Budget Performance Evaluation
Broadly, there are three types of Budget Performance Evaluation- Budget

Compliance, Budget Efficiency and Budget Effectiveness indicators. Budget
Compliance examines whether the actual expenditure for a program or
output is within the budgeted funds and whether there had been expenditure
over-runs. Budget Efficiency means that the actual outputs are at least equal
to the budgeted outputs, and Budget Effectiveness implies that the actual
outcomes are at least equal to the program outcomes.
Types of Performance Evaluation

Type of Performance Budgeted Actual
Evaluation
Compliance Budgeted finance → ← Actual expenditure
Efficiency Budgeted outputs → ← Actual outputs
Effectiveness Budgeted outcomes → ← Actual outcomes
As discussed in section-2 there are four types of performance indicators viz,

input indicators, output indicators, efficiency and effectiveness indicators.
Other types of indicators such as resource indicators, service indicators,
clients’ satisfaction indicators and community indicators have also been
suggested in the literature. For practical purpose, input indicators are the
same as resource indicators, output and service indicators are the same as
efficiency indicators, and outcomes and community indicators are the same
as effectiveness indicators. Depending on these performance indicators, we
can propose four types of budget performance evaluation as follow:
Table-5: Broad Categories of Budget Performance Evaluation

(i) Input/ Resource indicators (compliance evaluation) quantifying

levels of expenditure (or unit costs) on the delivery of goods and services,
the number of staff employed and the value of assets owned;
(ii) Output/ Service indicators (efficiency evaluation) measuring
the quantity, quality, accessibility and timeliness of services provided
within defined output classes. Such indicators include measures of the
outputs of services and their outcomes against an agency’s publicly
announced goals, targets or standards.
(iii) Clients’ Satisfaction indicators (efficiency evaluation)
measuring the personal assessment of services by clients and community
stakeholders based on their own expectations.
(iv) Outcomes/ Community indicators (effectiveness evaluation)
measuring broad social, economic and environmental standards achieved
in the community.
4.2.1 Input/ Resource indicators (compliance evaluation)
Resource indicators (or inputs) describe the quantity of resources the Agency
puts into each output, and where possible, the resources used on each
function. These indicators primarily measure inputs in terms of total
expenses, staff, goods and services. Unit costs of specific goods and services
can also be measured, if possible (e.g. average annual cost per child in a
kindergarten school or for educating a student in a primary, secondary and
high school etc.).
Expenses should take into account the full cost of services, funded not only
from the Government's Consolidated Fund, but also by agencies' own
revenues from service charges. Staff data should be presented as average
equivalent full-time (EFT) staffing for standardization during each year.
Ideally, the value of fixed assets, not fully reflected in recurrent expenditure
figures, particularly in the areas of investments on land, infrastructure,
buildings, plant and equipment may also be included. But, this may not be
feasible at present for Mongolia. We have recommended earlier that the
investment cost for public infrastructure such as rails, roads, airports,
bridges etc. may be taken as sunk cost. Even capital cost (comprising
depreciation and capital charges) may not be included as a part of output
cost and may be postponed for an indefinite period until all the assets have
been listed and evaluated5.
4.2.2 Output/ Service indicators (efficiency evaluation)
5
For detailed discussion on Output Costing, see Tarun Das and E. Sandagdorj (2007b).

Output indicators measure activities or processes in terms of the outputs

they produce. For example, an output indicator of teaching would be the
number of students taught in a year. Final outputs are the end products of a
chain of activities and are delivered to external clients. Other outputs may be
produced for internal clients or as steps towards providing services for
external clients. Such outputs are vital to determine the efficiency of supply
chains. For example, significant government agency activity may be devoted
to producing public policies, developing corporate or strategic plans,
providing advice to the Ministers, maintaining computer systems, managing
finances, and recruiting, training, managing and appraising staff. These
outputs may be costed for output budgeting but may not be delivered to the
people. Yet they are vital to the provision of government services. These
indirect costs may be apportioned among various outputs either on pro-rata
basis or according to some allocation principles.
(a) Costing outputs
Unit costs are important measures of efficiency, but several factors stand in
the way for calculating unit costs for the delivery of public services. In the
global budgeting system, many expenses (e.g. corporate overheads) are
notionally or pro-rata apportioned among various output classes. Sometimes,
cost estimates are based on crude assumptions, such as the cost of indirect
overheads could be distributed between different outputs according to the
direct cost of their production or their staffing numbers. It is necessary to
prepare detailed guidelines for agencies for a uniform costing methodology6.
4.2.3 Clients’ Satisfaction indicators (efficiency evaluation)
Client views are critical to evaluating services and it is necessary to collect

data on client satisfaction and feedback. Performance evaluation should place
importance on clients’ and stakeholders’ perceptions and should report client
feedback on the quantity, quality, accessibility and timeliness of services.
Other stakeholders might also provide valuable feedback about government

service provision. Various techniques such as conducting surveys and
stakeholders seminars may be used to assess stakeholders’ perceptions.
6
See Tarun Das and E. Sandagdorj (2007g).

4.2.4 Outcomes/ Community indicators (effectiveness evaluation)
Where possible, performance reports should identify and measure the

outcomes of public services i.e. the extent to which service outputs achieved
announced goals, targets or standards set by the agency. It is necessary to
adopt appropriate performance standards that define the quality of outputs
or which set targets for service delivery such as timeliness or accessibility.
However, it is often difficult to identify outcomes that are the direct results of
outputs produced. Due to the significant influences of external factors on the
output, it is a matter of judgments, as to which outcomes, agencies could be
held accountable for. For example, the number of students completing
vocational training who find employment reflects not only the quality of the
training but also the availability of jobs in the market. Thus, it is not fair to
judge vocational courses by the outcome of “percent of graduates who find
related employment”. Alternatively, vocational courses could be judged by
assessment of competencies acquired through such courses, although this
outcome might be influenced by other factors such as student motivation.
Community indicators measure aspects of community life relevant to the

government. These statistics are used for a variety of purposes, such as
assessing the need or demand for government intervention, and assessing
the impact of government services on the community. Community indicators
also influence public opinion on priorities to which the government might
respond with policy and additional resources. For instance, statistics on the
unemployment and poverty might induce government to introduce new
employment generation and poverty alleviation programs.
4.3 Benchmarks7
Benchmarks set the standard or point of comparison against which a

measure could be assessed. Often this standard is the average measure
achieved by comparable jurisdictions (such as government agencies in all
Line Ministries or Aimags) or the best result achieved over a period of time
by an agency. Benchmarking draws comparisons between like indicators, for
example, between Line Ministries or Aimags regarding the levels of resources
allocated for comparable services, the type and quantity of services provided
per 1,000 population or per 1,000 sq. kms or the level of client satisfaction
for a particular goods or services.
7
For comprehensive discussion on benchmarks setting, consult Tarun Das and E.
Sandagdorj (2007e and 2007f).

5. Budget Performance Evaluation Methodology
Budget Performance Evaluation involves five main steps as the following:
1. Summary of Baseline Scenario and Budget Profile

2. Diagnostic Scan and SWOT Analysis
3. Budget Compliance, Efficiency and Effectiveness Evaluation
4. Performance Evaluation
5. Major Findings and Recommendations
5.1 Review of Strategic Plan and Baseline Profile
In undertaking a performance evaluation, it is necessary to start with a

baseline and initial profile and to identify the key issues on which the
performance evaluation is to be focused.
Table-6: Scope of Performance Review and the Initial Profile
Scope of Performance Scope of Initial Profile (say for the Budget

Review: year 2008):
1. Strategic Business Plans 1. Scope, Governance, Vision, Mission and
objectives
2. Scope of review 2. Main functions, programs and activities
3. Review steps and key 3. Structure, staffing and time schedules
milestones
4. Preliminary assessment 4. Program-wise Budgeted funds
5. Focus of review 5. Output costs, benchmarks and performance
parameters
This assessment allows an agency to take a detailed look at their current

business activities and how they wanted to perform in the budget review
year. Agencies will be asked to provide a brief description 8 of their Strategic
Business Plan with vision, mission, objectives and goals. They will also be
asked to provide summary of their program budgets, which is being
reviewed, with budgeted resources, outputs and outcomes. Agencies will be
required to provide details of workforce size, their functions and skills,
workload volume and contributions to the strategic planning. Annex-1
provides the Template for Review of Strategic Plan and Baseline Profile.
8
Agencies are not supposed to hand over the original copies of the Strategic Business Plan (SBP) and
Budget document which they had earlier supplied to the MOF. They are required to make a summary of the
SBP and budgeted programs. Detailed Templates are provided in Annex-1 of this report.

5.2 Diagnostic Scan and SWOT Analysis
A diagnostic scan of an Agency is necessary before starting a performance

review, because actual performance is influenced by constraints on
resources, technical manpower and the ICT system. There are basically two
types of review- strategic review and operational review.
Strategic Review:
How well an Agency manages its external environment by delivering
relevant and effective services.
Operational Review:
How well an Agency manages its internal environment by using its
resources efficiently and prudently.
Both desktop and field scans are required to determine the following aspects:
whether best practice techniques were attempted;
whether the practice was documented; and
whether it was widely applied within the agency.
The desktop scan involves checking the existing material on strategic plans
and program/ output budgets already submitted by the Agency to the
Ministry of Finance, whereas field scans involve conducting surveys and
interviewing key stakeholders (clients, community groups, staff and
management), to obtain their views on how internal management tools are
working in practice.
There are 8 possible strategic review areas and 8 operational review areas as
indicated in Table-7.
Table-7 Strategic Review and Operational Review Areas
Strategic review areas Operational review areas

1. Strategies 9. Work Culture
2. Environment 10. Communications
3. Clients 11. Organisation structure
4. Other stakeholders 12. Reporting Lines
5. Regulation 13. Human resources
6. Policy regime 14. Processes and systems
7. Service delivery 15. Controls
8. Reviews 16. Cost and Asset management
For each of these 16 areas, it is necessary to test whether the agency has
applied any typical best practice management techniques. For example,

when examining “Clients”, agencies would be asked whether they have

applied any of the following types of management practices e.g. client needs
analysis, client segmentation, clients’ satisfaction surveys, grievances and
complaints handling and so on. When examining “Controls and Cost and
Asset Management”, agencies would be asked if they use the following
practices e.g. financial information system, management information system,
asset management plan and corporate overhead costs analysis, etc.
Table-8A: Typical Best Practices for Strategic Review

Strategic Review Areas Typical Best Management Practices
(number of sub-areas)
1. Strategies (2) Strategic Business Plan, Master Plan
2. Environment (2) Socio-Economic-Political Environment Analysis,
SWOT Analysis.
3. Clients (2) Clients Needs and Satisfaction Surveys,
Grievances and Complaints Handling.
4. Other Stakeholders (2) Stakeholders Consultation, Focus Groups
5.Regulation (2) Regulatory Review, Parliamentary Consultative
Committee Review
6. Policy (2) Ministerial Review, Donors Review
7. Service Delivery (2) Service Charter, Benchmarking
8. Review plan (2) Performance Agreements, External Audits
Table-8B: Typical Best Practices for Operational Review

Operational Review Areas Typical Best Management Practices
9. Work Culture (2) Code of Conduct, Regular Staff Meetings
10. Communications (2) Annual Report, Website for Public
11. Organisation Structure (2) Organisation Chart, Job Descriptions
12. Reporting Lines (2) Delegation of Powers, Chinese Walls
13. Human Resources (2) H/R Manual, Training and Development
Programs.
14. Process & Systems (2) Rules and Procedure Manuals, ICT development
Plans.
15. Controls (2) Financial Information System, Management
Information System
16. Expenditure and asset Asset Management Plan,
management (2) Agency Overheads Analysis

SWOT Analysis
After diagnostic scan, a SWOT analysis may be undertaken to determine the

strengths, weakness, opportunities and threats of the Agency. For example,
a SWOT analysis for the Ministry of Finance is illustrated in Table-8.
Table-9 SWOT Analysis for the Ministry of Finance
• Sustained high GDP growth averaging around 8% since 2004.

• Strong performance by minerals and services sectors.
Strengths • Manageable fiscal deficit, with fiscal balance generating
surpluses in 2005-2006.
• Declining and moderate inflation rates since 2005
• Declining interest rates of the central bank bills
• Declining ratios of public debt to GDP
• Declining ratios of domestic debt to GDP
Internal environment
• High dependence on mineral revenues

• Declining trend of the non-mineral revenues as percent of GDP
Weaknesses • Very high growth rate of money supply which could revive
inflationary pressures
• Low levels of domestic savings
• Large proportion (nearly 80%) of current expenditure in total
public spending leading to low levels of public investment
• High prevalence of the poverty ratio (32.6 percent in 2006)
• Economic growth remains vulnerable to unfavourable weather
shocks and recurrence of dzuds in future
• Herd size and composition may put constraints on the growth of
agriculture and allied sectors
• Degradation of environment due to over use of forestry
• Rise of government expenditures due to elections in 2008
Opportunities  Located between two large and fast growing neighbors (China
and Russia) with positive pulls and pushes on the Mongolian
economy
External environment
 Surplus in the external current account during 2004-2006

 Comfortable foreign exchange reserves (equivalent to 3.4
months of imports at the end of 2006)
 Declining ratios of external debt to GDP
 Very low external debt service ratios indicating sustainability
of external debt over time
Threats  Overall economic growth and government revenue remain

vulnerable to terms of trade shocks trigged by sharp declines in
international prices of copper and gold in future
 Balance of payments remains vulnerable to hardening of
global oil prices in future


5.3......䈘 Strategic and Operational Performance

Evaluation
Annex-2 provides a Template for Strategic & Operational Evaluation. An

agency’s performance can be assessed in relation to the 16 performance factors
listed in Tables-7A and 7B. Each factor can be given scores on a scale of 0 to 5 by
using an approach adapted from the Australian Quality Council:
Table-10: Scores for Strategic and Performance Evaluation
0 Approach had not been considered or attempted or does not exist.
1 Some evidence of individual initiative and unsystematic efforts.
2 Approach is planned and introduced in some areas in a limited way.
3 Systematic approach has been implemented in some areas and results are
under examination.
4 Approach has been implemented in some areas and results/outcomes have
been used to improve the planning and budgeting.
5 Approach used in most of the areas and results/outcomes have been used
to improve the planning and budgeting.
Then a “Borda Index” (i.e. sum of all ranks for all factors) can be estimated. This will
provide a composite index for rating performance of agencies. There are 32 (=16X2)
sub-areas. So a maximum 160 marks can be scored by an Agency. Total score can
be expressed as a percentage of 160 marks. Percentage can also be calculated
separately for strategic performance and operational performance. Then, total marks
for each category can be expressed as a percentage of 80 marks. It is most unlikely
that an Agency will be able to score 100% marks. On the basis of percentage of
marks, the strategic performance or operational performance, or the combined
strategic and operational performance of an Agency could be rated as follows:
Table-11: Rating of an Agency on the basis of Performance Scores
Range of Performance Scores

Rating of Agency
(in percentage)
(a) Effective (EF) 85 – 100
(b) Moderately Effective (ME) 70 – 84
(c) Adequate (AD) 50 – 69
(d) Ineffective (IN) 0 – 49
5.4 Budget Compliance, Efficiency and Effectiveness Evaluation
Under Budget Compliance Evaluation, program and sub-program wise budgeted

expenditure are compared with the actual expenditure, and the following marks are
assigned to each program:

Table-12: Marks for Budget Compliance Evaluation
0 If actual expenditure exceeds budgeted expenditure by more than 10%.

1 If actual expenditure exceeds budgeted expenditure by more than 7.5 per
cent but less than 10 per cent.
2 If actual expenditure exceeds budgeted expenditure by more than 5 per cent
but less than 7.5 per cent.
3 If actual expenditure exceeds budgeted expenditure by more than 2.5 per
cent but less than 5 per cent.
4 If actual expenditure exceeds budgeted expenditure by less than 2.5%.
5 If actual expenditure is within the budgeted expenditure.
Under efficiency evaluation, budgeted outputs are compared with actual outputs, and
the following marks are assigned to each program.
Table-13: Marks for Budget Efficiency Evaluation
0 If actual output falls short of budgeted output by more than 10 per cent.
1 If actual output falls short of budgeted output by more than 7.5 per cent but
less than 10 per cent.
2 If actual output falls short of budgeted output by more than 5 per cent but
less than 7.5 per cent.
3 If actual output falls short of budgeted output by more than 2.5 per cent but
less than 5 per cent.
4 If actual output falls short of budgeted output by less than 2.5 per cent.
5 If actual output is at least equal to the budgeted output.
Under effectiveness evaluation, budgeted outcomes are compared with actual

outcomes, and the following marks are assigned to each program. However, one has
to wait a number of years before the outcome results are available. Therefore, for
the next three years, effectiveness evaluation may not be feasible.
Table-14: Marks for Budget Effectiveness Evaluation
0 If actual outcome falls short of budgeted outcome by more than 10%.

1 If actual outcome falls short of budgeted outcome by more than 7.5 per cent
but less than 10 per cent.
2 If actual outcome falls short of budgeted outcome by more than 5 per cent but
less than 7.5 per cent.
3 If actual outcome falls short of budgeted outcome by more than 2.5 per cent
but less than 5 per cent.
4 If actual outcome falls short of budgeted outcome by less than 2.5%.
5 If actual outcome is at least equal to the budgeted outcome.
After assigning marks for all sub-programs, actual marks obtained for all programs of
an Agency will be expressed as a percentage of total possible marks. The following

Box indicates the calculation of marks for an agency for compliance and efficiency
evaluation.
Total number of sub-programs = N

Maximum possible marks for both compliance and efficiency= 10N
Total marks obtained for compliance = M1
Total marks obtained for efficiency = M2
Percentage of Marks assigned to the Agency for compliance evaluation

= 100 * M1 / 5N = P1
Percentage of Marks assigned to the Agency for efficiency evaluation
= 100 * M2 / 5N = P2
Percentage of marks assigned for both compliance and efficiency evaluation
= (P1+P2)/2 = 100 * (M1 + M2)/ 10N
For an example, Annex-3 provides the templates for the compliance and
efficiency evaluation for the program budget of the Ministry of Education, Culture
and science (MoECS) for the budget year 2008. However, it may be noted here
that MOECS Program budget has confused between outputs and outcomes9.
Immediate results of the Programs are called “outcomes”. But, these are, in
fact, “outputs” of the program, and not “outcomes”.
To reiterate the exact definitions, outputs are the immediate or end results of a
project, whereas outcomes are the long term impact of the project on the society
after the completion of the project. For example, literacy rate is an outcome, number
of students is an output and number of teachers is an input. So, I have replaced
outcomes by outputs in the Template. While preparing the next Program
Budget, the line ministries and the World Bank ECTAC Team may please
keep this distinction in mind.
9
For definitions of inputs, outputs and outcomes, consult Tarun Das and E.
Sandagdorj (2007a). Also consult the Budgets of the Australian Government for any
Portfolio.

5.5 Overall Assessment and Score
Thus, we have the following three broad evaluations:
(1) Strategic Plan and Baseline Profile Evaluation- Annex-1 provides 20 basic
questions each carrying 5 marks. Total marks obtained will be expresses as a
percentage which will be the score for strategic plan evaluation.
(2) Strategic and Operational Performance Evaluation – It has been explained

in section 4.3 above and the template is given in Annex-2. Total marks obtained will
be expresses as a percentage which will be the score for strategic plan evaluation.
(3) Budget Compliance and Effectiveness Evaluation – It has been explained in

section 4.4 above and the template is given in Annex-3. Total marks obtained will
be expresses as a percentage which will be the score for strategic plan evaluation.
For overall assessment a weight of 30 per cent will be given for strategic plan
and baseline evaluation (Annex-1), a weight of 20 percent will be given for
strategic and operational performance evaluation (Annex-2) and 50 per cent
will be given for budget compliance and effectiveness evaluation.
Table-15: Weights for Various Types of Evaluation

Type of Evaluation Weights
1-A Strategic Plan Evaluation Weight: 10%
1-B Systems Development Weight: 10%
1-C Human Resource Development Weight: 10%
2-A Strategic Performance Evaluation Weight: 10%
2-B Operational Performance Evaluation Weight: 10%
3-A Program Budget Compliance Weight: 25%
3-B Program Budget Effectiveness Weight: 25%
Total 100%
Translating Performance Scores into Ratings: Finally, the overall performance scores will be
converted into qualitative ratings using the scoring bands given in the following table:
Table-16: Rating of an Agency on the Basis of Overall Scores

Rating of Agency
(in percentage)
(e) Effective (EF) 85 – 100
(f) Moderately Effective (ME) 70 – 84
(g) Adequate (AD) 50 – 69
(h) Ineffective (IN) 0 - 49
There will be another category called “Results Not Demonstrated” when an

Agency does not have performance measures that have been agreed by MOF either
for baselines or for the assessment year.
An Example:

To provide an example, let us assuming that we are evaluating budget

performance for three Agencies- A, B and C. The results are given in the
following tables:
Table-17: An Example of Performance Scores for three Agencies
Performance Scores (%) Weighted Scores
Type of Evaluation Weight Agency Agency Agency Agency Agency Agency
-A -B -C -A -B -C
1-A Strategic Plan
Evaluation 0.1 50 60 85 5 6 8.5
1-B Systems Development 0.1 50 50 75 5 5 7.5
1-C Human Resource
Development 0.1 60 50 70 6 5 7
2-A Strategic Performance 0.1
Evaluation 50 50 85 5 5 8.5
2-B Operational 0.1
Performance Evaluation 60 70 70 6 7 7
3-A Program Budget
Compliance 0.25 40 70 70 10 17.5 17.5
3-B Program Budget
Effectiveness 0.25 40 70 70 10 17.5 17.5
Total 100% 47 63 73.5 47 63 73.5
On the basis of these scores, the Agencies would be graded as given in the
following table:
Table-18: Estimation of Overall Rating for three Agencies

Performance Scores (%) Rating of an Agency
Type of Evaluation Weight Agency Agency Agency Agency Agency Agency
-A -B -C -A -B -C
(1) (2) (3) (4) (5) (6) (7) (8)
1-A Strategic Plan
Evaluation 0.1 50 60 85 AD AD EF
1-B Systems Development 0.1 50 50 75 AD AD ME
1-C Human Resource
Development 0.1 60 50 70 AD AD ME
2-A Strategic Performance 0.1
Evaluation 50 50 85 AD AD EF
2-B Operational 0.1
Performance Evaluation 60 70 70 AD ME ME
3-A Program Budget
Compliance 0.25 40 70 70 IN ME ME
3-B Program Budget
Effectiveness 40
0.25 70 70 IN ME ME
Total 47
100% 63 73.5 IN AD ME
Note: AD stands for Adequate, EF for Effective, IN for Ineffective and ME for
Moderately Effective.
Finally, Ratings of the Agencies as regards different performance categories

may be put on government or MOF website for information to the general

public and all stakeholders. Actual performance scores (given in columns 3, 4

and 5) might not be published as these scores could suggest a false degree
of precision, only the ratings for each category (given in columns 6, 7, 8 in
the above table may be made available to the public. However, the detailed
methodology and parameters may be published for general information.
5.6 Major Findings and Recommendations
Major Performance findings can be drawn on strategic and operational issues

relating to the following questions:
Strategic 1. Policy Did policy address current and emerging

Appropriateness: needs?
2. Service Did agency services meet stated goals?
Effectiveness:
Operational 3. Cost Efficiency: Were the agency’s unit service costs
reasonable?
4. Operational Did the agency control its finances and
Prudence: risks?
On the basis of these findings, recommendations can be made to improve

performance in relation to the following broad areas:
Strategic issues; and

Operational issues.
6. Budget Performance Evaluation Management
The Department of Fiscal Policy and Coordination (DFP&C) of the Ministry of

Finance may be made nodal department for conducting Budget
Performance Evaluation in association with all line Ministries and
budgetary agencies. The management process and system along with time
schedules are described in Table-18 and Flow Chart-4. To start with
DFP&C of the MOF will prepare a detailed guidelines and manual for the
Budget Performance Evaluation and would circulate the document among all
agencies and line ministries after approval by the Cabinet Committee.

Table-19: Deadlines for various steps in BPE
Activities Deadline-
No later than
1. Department of Fiscal Policy and Co-ordination (DFP&C) of the 30th April
MOF prepares detailed guidelines and manual on Budget
Performance Evaluation (BPE), and circulates the Document
(along with Annexes) among line ministries/ Agencies after
approval by the Cabinet Committee
2. Pilot Line Ministries/ Agencies are selected by MOF for Budget 5th May
Performance Evaluation (BPE)
3. Pilot Line Ministries/ Agencies prepare replies and assign marks 1st July
to the Templates given in Annex-1 to annex-3 and submits the
draft reply to the DFP&C
4. MOF gives comments to the draft Budget Performance 15th July

Evaluation Report
5. Pilot Line Ministries / Agencies finalize the reply and resubmit 1st August
the Budget Performance Evaluation Report to MOF
6-A. MOF finalizes the Budget Performance Evaluation Report 15th August
(BPER) and sends copies to the Line Ministry Portfolio Minister and
to the the State Central Administrative Body responsible for
Finance and Budget.
6-B. Portfolio Minister also submits the Strategic Business Plan to 15th August
the State Central Administrative Body responsible for Finance and
Budget.
7. The State Central Administrative Body responsible for Finance 15th September
and Budget shall compile the Budget Performance Evaluation
Reports (BPERs) of pilot Line Ministries and submit these along
with the consolidated Portfolio Appropriation Estimates to the
Government.
8. The State Great Hural shall debate and approve the State 1st December
Budget.
9. Budget Performance Evaluation Report (BPER) published, 15th December
publicly released and put on government website.
10. MOF and line ministries/ Agencies conduct multi-stakeholders 31st January
feedback seminars on the Report.
11. DFP&C of the MOF revise the Manual and Guidelines for 31st March
Budget Performance Evaluation.

2. Pilot Line Ministries/ Agencies are

1. Department of Fiscal Policy and Co-
selected by MOF for BPE by the 5th May.
ordination (DFP&C) of the MOF prepares
detailed guidelines and manual on Budget
Performance Evaluation (BPE), and
circulates the Document (along with
3. Pilot Line Ministries/ Agencies prepare
Annexes) among line ministries/ Agencies
replies and assign marks to the Templates
after approval by the Cabinet Committee
given in Annex-1 to annex-3 and submits the
no later than the 30th April.
draft reply to the DFP&C (by 1st July 2008).
5. Pilot Line Ministries / Agencies

4. MOF gives comments to the draft Budget
finalize the reply and resubmit the
Performance Evaluation Report no later than
Budget Performance Evaluation Report
the 15th July.
to MOF no latter than 1st August.
6. MOF finalizes the Budget

Performance Evaluation Report 7. The State Central Administrative Body
(BPER) and sends copies to the Line responsible for Finance and Budget shall
Ministry Portfolio Minister and to the compile the Budget Performance
State Central Administrative Body Evaluation Reports of pilot Line Ministries
responsible for Finance and Budget no and submit these along with the
latter than 15 August. Portfolio Minister consolidated Portfolio Appropriation
also submits the Strategic Business Plan Estimates to the Government no later than
to the State Central Administrative Body the 15th of September.
responsible for Finance and Budget no
later than the 15th of August.
9. Budget Performance Evaluation

8. The State Great Hural shall debate and
Report (BPER) published, publicly
approve the State Budget no later than the
released and put on government website
no latter than 15 December. 1st of December.
10. MOF and line ministries/ Agencies 11. DFP&C of the MOF revise the
conduct multi-stakeholders feedback Manual and Guidelines for Budget
seminars on the Report by 31 January. Performance Evaluation no latter than
the 31st March.
Flow Chart-4: Performance Evaluation System and Monitoring

Selected References
Bergin, Jeffrey (2004) Performance Based Budgeting, Performance Management

Institute.
Canadian Comprehensive Auditing Foundation (2002) Reporting Principles:

Taking Public Performance Reporting to a New Leve, Ottawa, 2002.
Das, Tarun and E. Sandagdorj (2007a) “Preparation of Strategic Business Plans-

General Guidelines, Suggestions for Improvement, and Summary of
Recommendations”, Final Report, pp.1-74, 30 Sept 2007.
Das, Tarun and E. Sandagdorj (2007b) Output Costing and Output Budgeting-
Basic Concepts and Methodology, pp.1-51, October 2007.
Das, Tarun and E. Sandagdorj (2007c) Accrual Accounting and Accrual

Budgeting- Basic Concepts and Methodology, pp.1-43, November 2007.
Das, Tarun and E. Sandagdorj (2007d) Transition from Cash to Accrual

Accounting, pp.1-26, November 2007.
Das, Tarun and E. Sandagdorj (2007e) Benchmarks Setting and Best Practices
for Output Costing and Output Budgeting- Part-1: Basic Concepts, pp.1-31, Dec
2007.
Das, Tarun and E. Sandagdorj (2007f) Benchmarks Setting and Best Practices
for Output Costing and Output Budgeting- Part-2: Practical Applications for Mongolia,
pp.1-36, Dec 2007.
Das, Tarun and E. Sandagdorj (2007g) Terminal Report: Part-1, Major

Conclusions and Recommendations, pp.1-70 and Part-2 on Strategic Business Plans,
Output costing and Output Budgeting, Accrual Accounting and Accrual Budgeting,
and Benchmarks Setting, pp.71-157, ADB Capacity Building Project on Governance
Reforms, prepared by Tarun Das for detailed guidelines on Output Costing.
Government of Australia, Council on the Cost and Quality of Government

(2001) Annual Report 2001, November 2001.
Government of Mongolia (2002) Public Sector Management and Finance Act

(PSMFA, 27 June 2002).
Government of USA (2005) Performance Management Process: Strategic

Planning, Budget and Performance Management cycle, General Services
Administration (GSA), Office of the Chief financial Officer, 31 January 2005.
Hatry, Harry P. (1977) How Effective are your Community Services?, The Urban
Institute, Washington, D.C.
Kaplan, Robert S. and David P. Norton (1996) The Balanced Scorecard:

Translating Strategy into Action, Harvard Business School Press.

Mercer, John: See Website on GPRA and Performance Management:

www.governmentperformance.info
Mercer, John (2003) CASCADE Performance Budgeting- A Guide to an Effective

System of Integrating Budget and Performance Information and for Linking Long-
Term Goals for Day-to-Day Activities, USA, May 2003,
www.governmentperformance.info
Meyers, Roy T. (1996) Is There a Key to the Normative Budgeting Lock, The World
Bank, Washington, D.C.
Schick, Allen (1995) Federal Budget: Politics, Policy and Process, Brookings
Institution.
Steiner, George; Simon and Schuster (1997) Strategic Planning: What Every
Manager Must Know.
Thomas, Paul G. (2004) Performance Measurement, Reporting and Accountability:

Recent Trends and Future Directions, Saskatchewan Institute of Public Policy Paper
No 23, February 2004; http://www.uregina.ca/sipp/
Thomas, Paul G. (2005) Performance Management and Management in the Public

Sector, Optimum Online — The Journal of Public Sector Management, Vol 35, Issue
2, July 2005. http://www.optimumonline.ca/
Thomas, Paul G. (2006) Performance Measurement, Reporting, Obstacles and

Accountability -Recent Trends and Future Directions, Research School of Social
Sciences, The Australian National University, Canberra ACT 0200.
USA (1993) Government Performance and Results Act (GPRA) of 1993, Office of Management
and Budget (OMB).
United States of America, Office of Management and Budget (OMB) Homepage:

http://www.whitehouse.gov/omb/gils/gil-home.html

Annex-1
Template for Review of Strategic Business Plan (SBP)

And Baseline Profile of the Agency
Name of the Line Ministry ________________________________
Name of the Budgetary Agency _____________________________
Location _______________________________________________
Part-1: Strategic Business Plan (SBP) (Total Marks=50)
1. Do you have a Strategic Business Plan for your Agency? If Yes, answer the
following questions. Answers should be brief and to the point. It may be noted
that each question carries 5 marks and answers will be evaluated by the
Ministry of Finance.
Q1. Which is the period of your latest SBP? Does the SBP contain all the
components of a standard SBP as recommended by the MOF viz. Minister’s
Foreword, GM’s Statement? Vision, Mission, Values, Priorities, Clients, Strategic
Goals and Objectives, SWOT Analysis, Strategic Outcomes. Outputs, Processes and
Activities, Medium Term Budget estimate, Organisational Structure, System for
Monitoring and Evaluation? If some components are missing, indicate those
components and give reasons for not developing those components.
Q2. Does any of the Millennium Development Goals (MDGs) concern your Agency?
If yes, indicate the MDGs which relate to your Agency. Have you considered these
MDGs and suggested measures to achieve those goals in your Strategic Business
Plan? If yes, give details.
Q3. Does your Agency have a Master Plan? If yes, for which period? Have you
linked SBP to the Master Plan? Give a brief account of these linkages.
Q4. Indicate briefly Vision, Mission, Values, Priorities, Strategic Goals and
Objectives.
Q5. Reproduce the SWOT Table from the Agency’s SBP, if you have one.
Q6. Who are your clients? Is the Client base growing, shrinking or stable? Do they
need differentiated products, services, locations and quality? Are there significant
groups of potential clients who are not currently being reached? If yes, why? How
do you add value for clients and how can you serve them better?
Q7. Provide a current organizational chart of your agency?
Q8. Provide a list of the broad output groups indicated in your SBP. If the total
number exceeds 25, list the most important 25 outputs or output groups.

Q9. How well do your current information and communications technology (ICT)
systems assist in meeting your mission and long term goals? How is your Agency
developing and strengthening the ICT and the skills and capabilities of the
workforce to meet these needs? Are there any critical management issues which
also need to be addressed?
Q10. List and rank the highest priority strategic initiatives and operational priorities
required for the success of your Agency. Identify any new proposals, anticipated
outputs and link them to your strategic plans.
Part-2: Systems Evaluation (Total Marks=25)
As in Part-1 each question carries 5 marks and replies will be evaluated by

the MOF.
Q11. Does your Agency have a system for continual monitoring and annual
evaluation of performance parameters recognized in your SBP and Annual Budget?
Are the annual Performance Evaluation Reports used consistently to justify funding
requests, management actions and legislative proposals? If yes, describe the
evaluation system and illustrate your reply on the basis of latest budget.
Q12. Do your senior agency managers meet at least quarterly to examine reports on
financial and performance information for all outputs/ programs of the Agency?
(Evaluation: 5 marks if meetings held at least once in a quarter, 3 marks if
meetings held at least once in six months, 2 marks if meeting held at least once in
a year, 0 marks if no such review meeting is held in a year).
Q13. Does your Agency have well designed plans to improve program performance
and efficiency each year? If yes, provide evidence.
Q14. Does the latest Annual Budget and performance documents incorporate
performance measures identified in the SBP and Master Plan? If yes, provide
evidence.
Q15. Does your Agency have a proper costing methodology? Does the methodology
report the full cost of all outputs accurately in the budget and performance
documents? If yes, describe the methodology.
Part-3: Human Resource Development (Total Marks=25)
As in Part-1 and Part-2, each question carries 5 marks and replies will be
evaluated by the MOF.
Q16. Do you have Performance Evaluation Reports at least for the middle level and
senior level officers? Are these Evaluation Reports used for placements, promotions
and training for officers? Are these Reports also used to direct program
improvements and hold managers accountable for those improvements? Provide
evidence for your answer.

Q17. Has your Agency implemented a comprehensive Human Capital Development

and Utilization Plan that is fully integrated with the agency’s overall strategic plan
and annual performance goals? If yes, provide a brief account of the plan.
Q18. Has your Agency analyzed the existing organizational structure from service
and cost perspectives and is implementing a plan to effectively develop, train,
deploy, restructure, recruit and retain employees? If yes, provide a brief account of
the plan.
Q19. Does your Agency used competitive sourcing, E-Gov solutions, as necessary;
and has processes to address future changes in business needs?
Q20. Does your Agency have succession strategies, including structured leadership
and talent pool development programs, and is able to close leadership competency
gaps?
Summary of total marks and grading:
Questions Marks for Questions Marks for Questions Marks for

Part-1 Part-2 Part-3
Q1 M11 Q11 M21 Q16 M31
Q2 M12 Q12 M22 Q17 M32
Q3 M13 Q13 M23 Q18 M33
Q4 M14 Q14 M24 Q19 M34
Q5 M15 Q15 M25 Q20 M35
Q6 M16
Q7 M17
Q8 M18
Q9 M19
Q10 M110
Total Marks Sum of Sum of Sum of
obtained Marks Marks Marks
(M) (M1) (M2) (M3)
Express M P1 = 100 * P2 = 100 * P3 = 100 *
as % of M1/ 50 M2/ 25 M3/ 25
maximum
marks
Grade your G1 G2 G3
agency10
Overall Marks for Parts 1, 2 and 3 M = M1 + M2 + M3
Express M as % of maximum marks P = 100 * M / 100 = M
Grade your agency G
10
Grades are determined as per the following Table:
Rating of AgencyRange of Performance Scores
(in percentage)Effective (EF)85 – 100Moderately Effective (ME)70 – 84Adequate
(AD)50 – 69Ineffective (IN)0 - 49

Annex-2
Template for Strategic and Operational Review
On the basis of factual information, rank the following activities of your Agency on a
scale of 0 to 5. The rankings must be based on the following score card.
0 Approach had not been considered or attempted or does not exist.

1 Some evidence of individual initiative and unsystematic efforts.
2 Approach is planned and introduced in some areas in a limited way.
3 Systematic approach has been implemented in some areas and results are
under examination.
4 Approach has been implemented in some areas and results/outcomes have
been used to improve the planning and budgeting.
5 Approach used in most of the areas and results/outcomes have been used to
improve the planning and budgeting.
A. Strategic Review Areas- Best practices Ranking

(Marks)
1. Strategies and 1.1 Do you have a Strategic Business Plan?
reviews
1.2 Do you have a Master Plan of your
Agency?
2. Environment 2.1 Have you conducted recently any socio-
economic and political environment analysis
for your Agency?
2.2 Does your Strategic Business Plan contain
a SWOT analysis?
3. Clients 3.1 Have you conducted any clients’ needs
assessment and satisfaction surveys?
3.2 Do you have a permanent and
independent unit to deal with grievances and
complaints of your clients?
4. Other stakeholders 4.1 Do you conduct multi stake holders'
consultations/ workshops regularly?
4.2 Have your Agency formed focus groups
for the services of your Agency?
5. Regulation 5.1 Has there been any regulatory review of
the activities/ functions/ programs of your
Agency?
5.2 Has there been any review of the

activities/ functions/ programs of your
Agency by any Committee of the Parliament?
6. Policy regime 6.1 Is there any Ministerial Statement on the
vision, mission, objectives and scope of
functions of your agency?
6.2 Have the donors made any review of
functions of your Agency?

7. Service delivery 7.1 Have you published any Service Charter

indicating rights of your clients?
7.2 Have you conducted any benchmarking
analysis for the cost and quality of the goods
and services supplied by your Agency?
8. Review plan 8.1 Have you signed performance
agreements with the MOF?
8.2 Have the programs been audited by to
audit authority?
Grading of Agency for Strategic Performance Evaluation
Total Marks obtained Sum of

(M1) Marks
Express M1 as % of = 100 * M1/
maximum possible 80
marks (i.e. 80)
Grade your agency11 G1
B. Operational Review Areas- Best practices Ranking

(Marks)
9. Work culture 9.1 Is their written code of conduct for the
staff?
9.2 Do the senior officers hold review
meetings regularly with subordinate staff on
works plan?
10. Communication 10.1 Does the Agency produce an Annual
Report every year?
10.2 Does the Agency have a Website for
public information?
11. Organization 11.1 Does the Agency have an updated
structure organization chart?
11.2 Does the Agency have written work
charts and allocation of works for all staff
members?
12. Reporting lines 12.1 Does the Agency have written
delegation of powers?
12.2 Does the Agency follow the principle of
Chinese Wall12?
13. Human resources 13.1 Does the Agency have an updated
Manual on human resource development and
utilization?
13.2 Does the Agency have permanent

training program for skill development?
11
Grades are determined as per the following Table:
Rating of AgencyRange of Performance Scores
(in percentage)Effective (EF)85 – 100Moderately Effective (ME)70 – 84Adequate
(AD)50 – 69Ineffective (IN)0 - 49
12

14. Process and 14.1 Does the Agency have an updated

systems Manual on general rules and procedures for
the Agency?
14.2 Does the Agency have a definite plan for
ICT upgradation?
15. Controls and 15.1 Does the Agency have computerized
metrics government finance information system for
accounting and auditing?
15.2 Does the Agency have general
management information system (MIS)?
16. Expenditure and 16.1 Does the Agency have a definite plan
asset management and program for asset management?
16.2 Does the Agency conduct regular
analysis for management of agency
overheads?
Grading of Agency for Operational Performance Evaluation
Total Marks obtained Sum of

(M2) Marks
Express M2 as % of = 100 * M2/
maximum possible 80
marks (i.e. 80)
Grade your agency G2
(as given in footnote
11)
Overall grading:
Total Marks obtained M=

(M) M1 + M2
Express M as % of = 100 * M/
maximum possible 160
marks (i.e. 160)
Grade your agency G2
(as given in footnote
11)
17. If the grade is ineffective, analyze reasons for that and suggest
measures for improvement?

Annex-3
Template for Compliance and Efficiency Evaluation
Here we present an example from the Program Budget for the Ministry of
Education, Culture and Science for the budget year 2008. However, it may
be noted here that MOECS Program budget has confused between outputs
and outcomes13. Immediate results of the Programs are called “outcomes”.
But, these are, in fact, “outputs” of the program, and not “outcomes”.
To reiterate the exact definitions, outputs are the immediate or end results of a
project, whereas outcomes are the long term impact of the project on the society
after the completion of the project. For example, literacy rate is an outcome, number
of students is an output and number of teachers is an input. So, I have replaced
outcomes by outputs in this Template. While preparing the next Program
Budget, the line ministries and the World Bank ECTAC Team may please
keep this distinction in mind.
Program/ Output Budgeted Actual Marks

Performance
Pre-school education Total budget: 69.6 billion. MNT
1.Services for end-users Total budget: 64.3 billion. MNT
Budget share of Pre-school

education: 92.4%
Output 1.1 Total budget: 61.2 billion MNT

Kindergarten service % within the section: 95.0%
• Number of children to be
covered : 96901
24-hour kindergarten services % within the section: 1.0%
covered: 1239

Preschool education alternative % within the section: 3.0%
services
covered: 37682
2. Policy renovation activities Total budget: 5.13 billion. MNT
(strengthening institutions )
education: 7.4%
13
For definitions of inputs, outputs and outcomes, consult Tarun Das and E.
Sandagdorj (2007a).


Performance
Output 2.1 Total budget : 30 million MNT
Preschool human recourse % withinthe section: 0.3%
capacity development
• Number of teachers to be
covered by refresh training:
223
Improve coverage and learning % within the section: 99%
environment
What is performance parameter?
3. Administration and Planning Total budget: 156.6 million MNT

education: 0.2%
Output 3.1 Total budget : 81.4 mil.MNT.
Preschool education policy, planning % within the section: 47.0%
and coordination
Output 3.2 Total budget: 10.6 million MNT
Provide professional
methodological guidence % within the section: 11.8%
Output 3.3 Total budget :64.6 mil MNT

Information, monitoring and % within the section: 41.2%
evaluation
a. Kindergarten service Total budget : 61.2 billion MNT
b. 24-hour kindergarten Total budget: 1.0 billion MNT
c. Alternative preschool Total budget: 2.2 billion MNT

education services
d. Expand kindergarten capacity Total budget: 5.1 billion MNT
(increase no. of bed/chairs)
a. Indicators for Year 2008
Kindergarten Service
Quantity Number
(i) No. Of kindergartens 737
(ii) No. Of classes 3311
(iii) No. Of 96901

children
(iv) Expenditure BE (Bln. MNT)
• Wage expenditures 36.5
• Social insurance fee 7.9
• Other var.and fixed exp 19.6
• Food expenditure 8.2

• One time allowance 0.4
Total 72.6


Performance
b. Indicators for 24-hour Quantitaves in number /
kindergarten Service expence in million MNT
(i) No. of kindergartens 6
(ii) Number of classes 45
(iii) Number of 1239

children
(iv) Wage expenses 674.8
(v) Social insurance fee 146.1
(vi) Other variable costs 38.9
(vii) Food expense 97.9
Total 957.7
c. Alternative Per-School
Services
c-1. Indicators for Shift-classes Quantitaves in number /
expence in million MNT
(i) Number of children 14676
(ii) Number classroom days 42
(iii) Children/day planning 616.3
(viii) Total expense 861.3
c-2. Indicators for Mobile Quantitaves in number /

classes expence in million MNT
(i) Number of children 23006
(ii) Number classroom days 24
(iii) Children/day planning 483.1
(viii) Total expense 2226.7

General Education /GE / Total budget : 235.7 billion MNT

Variable exp: 208.9 billion MNT
Investment: 26.8 billion MNT
1. Services for end-users Total budget : 189.3 billion MNT
% of GE budget : 80.3%
Output 1.1 Total budget : 172.1 billion MNT
Primary, basic and high school % within the section: 91.0%
education services • Number of children to be
covered : 548005
Dormitory services % within the section: 6.1%
covered: 46864
Performance
Night and out-of-class education % within the section: 0.3%
services
• Number of students to enroll in
night classes : 1553

outside classroom program: 1936
Primary and basic education services % within the section: 0.8%
for children with disabilities
specialized schools : 1431

Support services for students from % within the section: 0.6%
vulnerable groups
• 8750 children be provided by 4
textbooks
• 66590 children be provided by

school supplies

Informal Education Serices % within the section: 1.2%
• 11690 students be covered by
consistent programs
• Literacy education services be

delivered to 342 soums
• Education channel will conduct

training for all
2. Policy renovation activities Total budget : 29.9 billion MNT

(strengthening institutions ) % of GE budget: 12.7%
Output 2.1 Total budget : 203.2 million MNT

Strengthening human resource % within the section: 0.7%
capacity of GE
• Number of teachers to be covered
by refresh training: 1682
Output 2.2 Total budget : 3 billion MNT

GE 12-year school system create % within the section: 10.1%
learning environment


Improve GE learning environment % within the section: 15.4%
Capital repair
Expand GE school coverage % within the section: 64.2%
Text boo supply services % within the section: 3.2%

English language advanced training % within the section: 6,4%
3. Admiinistration and Planning Total budget : 16.7 billion MNT
% of GE budget: 7.0%

Performance
GE Policy, Planning and % within the section: 2.6%
Coordination
Provide GE institutions with % within the section: 18.6%
methodological guidence
GE information, monitoring and % within the section: 4.6%
evaluation
Output 3.4 Total budget : billion MNT
% within the section: 74.2%
School lunch
I Primary, basic and general Schools under local

education services : administration
Indicators
(i) Number of schools 746
(ii) Number of children 548000
Expenditure Million MNT
(i) Wage costs 113,721.8
(ii) Social insurance fee 24,620.7
(iii) Other variable expenses 7,720.6
(iv) Classroom and practice 1,121.9

expenses to be charged from
city’s schools’ tuition fee for
their advanced training
programs
(v) Fixed costs 20,829.4
(vi) Olympics participation 423.0

funds
(vii) One time allowance 1,856.6

Total budget 172,117.4
Dormitory services :
Indicators Quantities
(i) Number of dormitories 500
(ii) Number of children in 46864

dormitories
Expenditure Million MNT
(i) Other expenses for 4965.0

goods and services
(ii) Food 6746.1
Total 11711.1

Performance
Higher education Total budget : 63.1 billion MNT
Variable cost: 60.6 billion MNT

investment: 2.5 billion MNT
1. Services for end-users Total budget: 60.3 billion MNT %

within higher education section :
96.0 %
% within the GBD’s total budget :
13.7%
Output 1.1 Total budget: 39.5 billion MNT %
Higher education services within the section : 65.5%
(bachelor, master, doctorate
degree)
Output 1.2 Total budget : 20.8 billion MNT %
Student scholarship services within the section: 34.5 %
• 6200 students will be covered

2. Policy renovation activities Total budget: 2.53 billion MNT %
(strengthening institutions ) within higher education section:
4.0%
Output 2.1 Total budget: 30 million MNT
Strengthen human resource % within the section: 1.2%
capacity
Strengtheining higher education % within the section: 98.8%
learning environment and
material basis
3. Administration and Planning Total budget: 248.9 million MNT
% within higher education section:
0.4%
Higher education policy and % within the section: 57.7%
planning

Output 3.2 Total budget : 64.3 million MNT

Higher education policy and % within the section: 25.4%
coordination
Higher education information, % within the section: 16.9%
monitoring and evaluation
Art and Culture Total budget : 33.8 billion MNT
Variable costs : 23.8 billion MNT
Investment: 10,0 billion MNT
1. Services for end-users Total budget: 23.4 billion MNT %
within the art and culturral section
budget share: 68,9%
Historic and art immovable % within the section: 15.4%
heritages and museum services
Services of stage and screen art % within the section: 38.2%
Library services % within the section: 11.6%
Soum Art center services % within the section: 33.5%


Performance
From this : Output 1.5 Total budget : 0.3 billion MNT
Art and cultural events % within the section õóâü: 1.3%
2Policy renovation activities Total budget: 10.13 billion MNT %
budget share: 29.9%
Strengthen human resource within the section: 1.5%
capacity of Art and culture
institutions
Output 2.2 Total budget: 1,6 billion MNT Á%
Improve Art and culture within the section: 25.8%
institutions’s condition
Strengthen material basis of Art % within the section: 16.4%
and culture institutions
Expand the Art and culture % within the section: 56.3%
service coverage
3. Administration and Plannng Total budget: 0.42 billion MNT %

budget share: 1.2%

Art and Culture development within the section: 33.3%
policy and planning
Art and culture policy within the section: 63.8%
implementation coordinaation
Art and culture section within the section: 2.9%
information, monitoring,
evaluation
Organization Number of new vacancies/

job postions
(i) Ensenmble of National 40
Folk and dance
(ii) State philharmonics 18
(iii) State cartoon theatre 8
Total 66
Aimag theatre and ensembles Number of increased position
(i) Bulgan 3
(ii) Umnugobi 2
(iii) Sukhbaatar 4
(iv) Darkhan 4
(v) Mandal soum theatre, 42

Selenge aimag /newly established

/
Total 55
Annex-4:
Methodology for calculation of composite score and grading the Line

Ministry/ Budgetary Agency
For overall assessment, a weight of 30 per cent will be given for strategic plan and
baseline evaluation (Annex-1), a weight of 20 percent will be given for strategic and
operational performance evaluation (Annex-2) and 50 per cent will be given for
budget compliance and effectiveness evaluation.
% of Marks Weighted Grades

Weights Marks in total percentage
Type of Evaluation
obtained marks in the
category
Strategic Plan 10% 10% * P1 G1
1-A M1 P1
Evaluation
Systems 10% M2 P2 10% * P2 G2
1-B
Development
Human Resource 10% M3 P3 10% * P3 G3
1-C
Development
Strategic 10% M4 P4 10% * P4 G4
2-A Performance
Evaluation
Operational 10% M5 P5 10% * P5 G5
2-B Performance
Evaluation
Program Budget M6 P6 25% * P6 G6
3-A 25%
Compliance
Program Budget M7 P7 25% * P7 G7
3-B 25%
Effectiveness
Total 100% M P P G
Translating Performance Scores into Ratings: Finally, the overall performance scores will be
converted into qualitative ratings using the scoring bands given in the following table:
Rating of Agency
(in percentage)
(i) Effective (EF) 85 – 100
(j) Moderately Effective (ME) 70 – 84
(k) Adequate (AD) 50 – 69
(l) Ineffective (IN) 0 - 49

There will be another category called “Results Not Demonstrated” when an

Agency does not have performance measures that have been agreed by MOF either
for baselines or for the assessment year.
Glossary
Definition of Terms
Activities are the tasks undertaken by the staff to transform inputs into outputs.
Annual Action Plan describes year-wise tactical actions to carry out a Strategy to
achieve a Long Term Outcome Goal.
Annual Performance Report summarizes an Agency’s performance progress relative

to its stated goals for a given year.
Annual Target sets a target level of performance for a specified fiscal year that is
expressed as a tangible and measurable objective, or quantifiable standard, value or
rate against which actual achievement can be compared.
Benchmark Analysis answers the question “how is X performing in compared to Y. A

benchmark is a quantifiable standard or norm against which actual performance can be
compared.
Budget Year is the Fiscal Year for which budget estimates are provided.
Business Action Plan is a document that summarizes the tactical actions required for
the implementation of the Strategic Business Plan.
Core Values are the ideals that guide behavior for all interactions internal and external
to the organization.
Current Year is the Fiscal Year immediately preceding the Budget Year.
Goals are long term wide-spread results of public programs and policies, like the
achievement of the Millennium Development Goals by 2015
Inputs are resources (Personnel, financial, goods and services) are the basic
requirements for any output and strategic planning.
Indicators are broad-based metrics which show whether outcomes are trending in the
desired direction.
Long Term is a period of 5 years or more.

Measures help determine the impact or influence of Department activities on outcomes.

They have targets associated with them and can be quantitative or qualitative in nature.
Medium Term is a duration of 2-4 years or the Budget Year + 3 years.
Mission is the stated purpose of the organization.
Outcomes are the medium term impact of public programs and policies on the economy
and user groups.
Outputs are deliverables (products and services produced) from inputs. Outputs are the
immediate or end results of activities.
Performance Goal sets a target level of performance for a multiple year period that is
expressed as a tangible, measurable objective, against which actual achievement can
be compared. It is expressed as a quantitative standard, value or rate. These goals must
be included in the Program Budget.
Performance Measure is a basis or standard of comparison used to monitor the

success in achieving the Performance Goal. A Performance measure is frequently
referred to as an Indicator.
PERT Chart stands for Program Evaluation Review Technique. It is project

management tool used to plan, schedule and co-ordinate plan. Sometimes, it is referred
to as the Critical Path Method (CPM).
Policy is a general statement designed to guide associates actions in recurring

situations.
Previous Year/ Prior Year is the Fiscal Year immediately preceding the Current year or
the last completed Fiscal Year.
Procedure is a sequence of steps describing how to carry out an activity.
Program Assessment Rating Tool (PART) is an assessment tool that rates agency
programs on effectiveness and ability to achieve desired results.
Short Term is a period of less than one year.
Strategy is a broad course of action or methodology designed to achieve a long term

outcome goals.
Strategic Assessment is the process of gathering and analyzing data that impacts an
organization’s success in achieving its long-term goals.
Strategic Goals are long-term outcomes that help the organization achieve its mission.
Strategic Planning is the process of developing and analyzing an organization’s

mission, long term goals, performance goals, business actions and allocating resources.

Strategic Objectives are broad based outcome statements for a group of value chains.
SWOT Analysis is a review of strength-weakness-opportunities-threats impacting an

organization. It is critical to the creation of a strategic plan in that it assesses the internal
an external environment, existing performance data and forecasts relative to the
thresholds an organization has established for itself.
Tactical Action Plan is the process of making detailed decisions about what to do, who
will do it, when it will be accomplished and how.
Tactical Plan is a document stating how an organization will implement its strategic
business plan.
Value Chains are programs grouped by a common purpose. These help employees
understand their shared objectives by identifying activities that cross organizational
boundaries in the achievement of the Department’s strategic goals.
Value Chain Outcomes describe the intended result from carrying out activities for a
group of programs with a common purpose.
Vision is a compelling picture of the organization that all can strive for in the future.
Although it may never be achieved, it is intrinsically motivating and drives the
organization toward a common purpose.

Performance Evaluation Methodology and Systems

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Performance Evaluation Methodology and Systems

Uploaded by

Copyright:

Available Formats

Budget Performance Evaluation Methodology by Tarun Das

Budget Performance Evaluation (BPE)-

Professor Tarun Das1

The Lexicon of Performance

“Wherever there is government, there is government performance. On the basis of

Source: Paul G. Thomas (2006) Performance reporting, obstacles and

MOF, Govt. of Mongolia 1 Glocoms Inc. (USA)

Budget Performance Evaluation (BPE) -

Professor Tarun Das

1. Introduction, Scope and Objectives

1.1 The Budget and Public Sector Performance

2. Ideal Performance Evaluation Systems

2.1 Characteristics of an ideal performance evaluation system

3. Budget Performance Indicators

3.1 Performance Measures versus Performance Indicators

4. An Operational Budget Performance Evaluation System for the

4.1 Hierarchy of Activities, Functions and Output Classes

5. Budget Performance Evaluation Methodology

5.1 Analysis of strategic plan and baseline profile

MOF, Govt. of Mongolia 2 Glocoms Inc. (USA)

5.6 Major findings and recommendations

6. Budget Performance Evaluation Management

Annex-1: Template for Review of Strategic Business Plan and Baseline

Glossary: Definition of Terms

Table-1: Characteristics of an Effective Budget

List of Flow Charts

Flow-Chart-1: Strategic Plan and Performance Based Budgeting Cycles:

MOF, Govt. of Mongolia 3 Glocoms Inc. (USA)

Budget Performance Evaluation (BPE) -

Professor Tarun Das

1. Introduction, Scope and Objectives

--- Extract from Alice in Wonderland, Lewis B. Carroll

Eventually Alice in Wonderland realized that it matters a great deal to know

Strategic Planning, Budget Formulation, and Performance Management Cycle

1.1 The Budget and Public Sector Performance

Performance budgeting is primarily a management, transparency and

MOF, Govt. of Mongolia 4 Glocoms Inc. (USA)

Table-1: Characteristics of an Effective Budget

1. Accountability Detailed controls should ensure that government

2. Comprehensiveness Includes all uses of government’s financial

3. Constraint Contains limitations on the amount of money that

4. Cooperation Budgeting should exist in harmony with other

5. Honesty Contains unbiased projections and allocations

6. Judgment Encourages participants to seek the most effective

7. Legitimacy The budget process should reserve important

8. Perception Considers both near- and long-term plan.

10. Timeliness A budget process should complete regular tasks

11. Transparency The budget and budget procedures should be

Source: Meyers, Roy T. (1996).

Some of these characteristics for an effective budget relate to the process of

Table-2: Characteristics That Adversely Impact Budget Performance

MOF, Govt. of Mongolia 5 Glocoms Inc. (USA)

1. Extra-budgetary Extra-budgetary funds lead to dual budgeting and

4. Unreliable Inaccurate and unreliable information will conspire

Source: Meyers, Roy T. (1996).

1.2 Performance Based Budgeting

A performance budget is an integrated annual performance plan that shows

A program performance budget defines all activities, direct and indirect,

Money ⇒ Activity ⇒ Output ⇒ Outcome ⇒ Strategic Goals

MOF, Govt. of Mongolia 6 Glocoms Inc. (USA)

cases (e.g., computer services), managers require this information to

Recently many countries have focused their attention to governance reforms

(ii) Decentralization of operating functions is another important