TERMS OF REFERENCE

INDEPENDENT PROGRESS REVIEW

of the
PNG DISASTER RISK MANAGEMENT PROGRAM (2010-2014)

These Terms of Reference (TORs) are to conduct a review and prepare an
independent report on the implementation of AusAIDs Disaster Risk Management
Program (2010-2014) in Papua New Guinea (PNG). In particular, the Review Team
will:

1. Assess and rate the progress of the PNG DRM Program against AusAIDs
evaluation criteria

2. Provide recommendations that allow AusAID to make strategic decisions on
the future direction of the Program itself while also providing
recommendations on the continuation of funding and nature of support for
specific activities


BACKGROUND:

AusAIDs Papua New Guinea (PNG) Disaster Risk Management (DRM) Program
(2010-2014) commenced in September 2010. Its goal is that of PNGs own draft
Disaster Risk Reduction and Disaster Management National Framework for Action
2005-2015: To ensure safe and resilient communities in high risk provinces of Papua
New Guinea

Its purpose is:

To strengthen the DRM sector in Papua New Guinea, at the national,
provincial and sub-provincial levels, to prepare for, mitigate and respond to
the effects of disasters

The anticipated outcomes of the DRM Program are:

1. Increased capacity of organisations, both government and non-government, to
manage disaster risks and respond effectively to disasters through improved
financial and human resources, planning and management systems

2. Increased capacity of selected high risk provinces to manage disaster risks and
respond effectively to disasters through improved financial and human
resources, planning and management systems

3. Strengthened disaster awareness, disaster preparedness and risk mitigation in
targeted, vulnerable communities
2

4. Increased capacity of AusAID Post to effectively respond to disasters in PNG


Through the DRM Program, the following activities have been supported by
AusAID:
1. Improved national and provincial disaster response capability: technical
support for the National Disaster Centre (NDC) and selected high-risk
Provinces (initially West New Britain) to build and improve disaster risk
management and emergency response capacity.
2. A better coordinated response capacity: funding to the United Nations
Office for the Coordination of Humanitarian Assistance (OCHA) to improve
coordination for disaster preparedness activities and during an emergency
response.
3. Improved monitoring of major risk factors: continued support for
Geoscience Australia, in partnership with PNG technical agencies, to
strengthen assessment, monitoring and early warning of natural hazards such
as volcanic eruptions, tsunamis and earthquakes.
4. Strengthened community-led disaster awareness and response capacity:
continued support to the PNG Red Cross and funding to community groups
and NGOs for disaster risk reduction activities.
5. Enhanced AusAID response: continue to train Emergency Response Team
members and build the capacity of AusAID PNG Post to respond to
emergencies.


PURPOSE OF THE REVIEW:

The Review Team will:

1. Assess and rate the progress of AusAIDs PNG DRM Program against the
following evaluation criteria (refer to Annex A):
a) Relevance: to assess whether the activity contributes to higher level
objectives of the aid program (outlined in country and thematic strategies).
b) Effectiveness: to assess whether the activity achieves clearly stated
objectives.
c) Efficiency: to assess whether the activity is managed to get value for
money from our inputs of funds, staff and other resources, and to
continually manage risks.
d) Impact (where feasible): to assess whether the activity produces positive
or negative changes (directly or indirectly, intended or unintended). The
degree to which the various aspects of impact can be assessed will vary
according to the nature and duration of the activity. Whether impact can be
assessed, or the way impact can be assessed will need to be determined by
the Review Team. Impact will not be rated.
3
e) Sustainability: to assess whether the activity appropriately addresses
sustainability so that the benefits of the activity will continue after funding
has ceased, with due account of partner government systems, stakeholder
ownership and the phase-out strategy.
f) Monitoring & Evaluation: to assess whether the monitoring and
evaluation framework effectively measures progress towards meeting
objectives.
g) Gender Equality: to assess whether the activity advances gender equality
and promotes women (considering the four dimensions of gender equality:
access, decision-making, womens rights, capacity-building).
h) Analysis & Learning: to assess whether the activity is based on sound
technical analysis and continuous learning.

The Review Team will be expected to draw out lessons from their assessment of each
of the evaluation criterion that may be relevant to ongoing implementation of the
DRM Program.

2. Provide recommendations that allow AusAID to make strategic decisions on the
future direction of the DRM Program itself while also providing recommendations
on the continuation of funding and nature of support for the following activities:
a) Rabaul Volcanological Observatory Twinning Program due to end in
September 2013
b) Natural Hazard Risk Assessment Mapping Program due to end in
September 2013
c) Support to OCHA due to end in December 2012
d) Community based DRR activities due to begin in 2012/13
e) Support to the National Disaster Centre and West New Britain Province
initial technical assistance support provided in early 2012
f) Red Cross Disaster Management Program due to end in January 2013


REFERENCE FRAMEWORKS:

The Review Team will base its analysis on the following Reference Frameworks:

1. AusAIDs Humanitarian Action Policy (2011)

2. OECD-DAC and AusAID criteria for evaluating development assistance

3. AusAID Disaster Risk Management Program (2010-2014) Performance
Management Framework

4. Other relevant AusAID policies, including the Multilateral Engagement
Strategy and Civil Society Engagement Framework


4
SCOPE OF THE REVIEW:

The Review Team will undertake the following:

Stage 1: Finalisation of TORs and Review Plan

1. Review these TORs once selected and if required, the Review Team will
revise and update the TORs (to be accepted by AusAID in writing)

2. Develop Review Plan as outlined on page 4, the Team Leader (in
consultation with other members of the Review Team) will submit a Review
Plan that outlines the methods the Review Team will use to meet the TOR
objectives and scope, including the evaluation questions

Approximate timeframe: 3 days


Stage 2: Document Review

1. Document review including AusAID DRM Program documentation (design
document, Program Logic/Capacity Building Diagnostic, Performance
Management Framework, Quality at Implementation reports, activity design
and milestone reports, etc)

Approximate timeframe: 4 days

Stage 3: Field Visits and Consultations

1. Field visits including Port Moresby, Kimbe (West New Britain) and
Kokopo/Rabaul (East New Britain)

2. Stakeholder meetings including with Government of PNG agencies
(National Disaster Centre, Department of Mineral Policy and Geohazard
Management, Rabaul Volcanological Observatory, National Weather Service,
Provincial representatives from West and East New Britain, etc), PNG and
Australian Red Cross, UN Office for the Coordination of Humanitarian
Affairs (OCHA), Geoscience Australia, NGOs, other donors and members of
AusAIDs Disaster Management and Emergency Response Teams.

Approximate timeframe: 10 days in-country

Stage 4: Reporting

1. Finalise the Independent Progress Review of the AusAID DRM Program,
ensuring that this report addresses the objectives outlined in this TOR

Approximate timeframe: 7 days



5
REPORTING REQUIREMENTS:

1. Review Plan which outlines the methods and timeframe the Review Team will
use to meet the TOR objectives and scope, including

a) an evaluation design that describes a logical model for assessing the DRM
Program and its activities;
b) a process for information collection and analysis (identify specific
evaluation questions and how these will be answered, including those
identified in Annex A); and
c) any substantive challenges to achieving the evaluation objectives that will
have to be addressed.

The Review Plan should also provide an outline of the roles and responsibilities of
team members and target dates for deliverables. The Review Plan will be no more
than three (3) pages and will be submitted to AusAID for approval two (2) weeks
before the Review Team undertakes field visits.

2. Aide Memoire presented (and electronically submitted) to AusAID before
departure from PNG. The Aide Memoire will briefly (no more than 5 pages)
discuss the evaluation criteria and recommendations for future implementation of
the DRM Program.

3. Draft Independent Progress Report due within three (3) weeks of completing
the field visit to PNG. The Team Leader will be responsible for coordinating
inputs from the Review Team and submitting this Report to AusAID. A template
will be provided to the Review Team by AusAID.

4. Final Independent Progress Report due within seven (7) days of receiving
AusAIDs written comments on the Draft Report. The Team Leader will be
responsible for coordinating inputs of the Review Team and submitting this
Report to AusAID.


TIMEFRAMES:
The Review Team will be available from October December 2012 to complete the
requirements of these Terms of Reference. Exact dates will be discussed directly with
the Review Team, once selected.


REVIEW TEAM:
The Review Team will be comprised of the following two (2) individuals:

M&E Specialist (Team Leader)
Discipline Category C, Job Level 3

Required Skill Set:

6
1. Demonstrated experience and understanding of contemporary M&E strategies
and systems, including in a developing country and/or Pacific region context.
Experience in PNG is highly desirable.

2. Previous experience in reviewing and evaluating other DRM/DRR activities,
including those supported by United Nations agencies, development agencies
or other donors, is highly desirable

3. Understanding of AusAIDs M&E policies

4. Extensive experience in the use of participatory and consultative evaluation
methods and techniques, including the collection and analysis of qualitative
and quantitative data

5. Proven ability in effectively managing small teams and utilising the expertise
of each team member in meeting Terms of Reference and contractual
obligations (including reporting)

6. Excellent English writing skills

Disaster Risk Management Specialist
Discipline Category B, Job Level 2

Required Skill Set:

1. Strong familiarity with best practice initiatives, current issues, developments
and trends in the humanitarian/disaster management sector (preferably disaster
preparedness and risk reduction), including in a developing country and/or
Pacific region context. Experience in PNG is highly desirable.

2. Understanding of AusAIDs disaster management/humanitarian policies and
the Hyogo Framework for Action

3. Strong understanding of national disaster management organisations, their
role, structure and any issues they face

4. A strong understanding of cross-cutting issues, particularly incorporating
gender perspectives and ensuring the equal participation of men and women in
humanitarian assistance, HIV/AIDS, environment/climate change, inclusion of
those with special needs, and how these relate to disaster risk reduction and
preparedness initiatives


Review Team members will also possess:

1. Strong liaison, communication, representation and negotiation skills

2. Ability to work effectively in a cross-cultural environment

7
AusAIDs Port Moresby based Disaster Management Team (Second Secretary and/or
Program Manager) will accompany the Review Team for the majority of in-country
field visits and consultations. It is expected that a Government of PNG representative
will also accompany the Review Team.


REMUNERATION:

The Adviser Remuneration Framework is a set of market-based, long and short term
remuneration rates applying to AusAID-funded advisers that are commercially
contracted by AusAID directly or through a managing contractor. Further information
on the Adviser Remuneration Framework can be found on AusAIDs website.

Considering the required skills and experience required to meet the obligations under
these Terms of Reference, the following remuneration for these positions are:

M&E Specialist (Team Leader)
Discipline Category C, Job Level 3

Disaster Risk Management Specialist
Discipline Category B, Job Level 2

As clearly outlined in the Adviser Remuneration Framework, AusAID will provide
other costs, such as:

Fixed daily travel rates (or per diems) are prescribed by AusAID for each
country, which are in line with equivalent rates paid to AusAID staff
undertaking short term travel. Per diems will only be available for travel in
PNG.

Adviser support costs are expenses incurred by the adviser that are directly
related to their assignment. These costs are reimbursed by AusAID at cost and
up to a prescribed limit set by AusAID in the contract. Support costs may
include, but not be limited to:
o Short-term accommodation
o Airfares and other work related transport costs
o Compulsory arrival and departure taxes
o International communication costs
o Medical insurance
o Security costs, where applicable

Accommodation (hotel) costs for short-term advisers are prescribed by
AusAID for each country, which are in line with equivalent rates paid to
AusAID staff undertaking short term travel. Accommodation will only be
reimbursed for hotels in PNG.

For individual advisers, management fees are limited to insurance costs that
are not solely linked to the current assignment. Refer to the Adviser
Remuneration Framework for further details.

8
For advisers engaged through a company, management fees must be clearly
documented and justified. Refer to the Adviser Remuneration Framework for
further details.
9
ANNEX A

These suggested standard evaluation questions for Independent Progress Reports have
been provided as a guide for evaluation teams in developing questions that get the
most value from the evaluation.

They are based on evaluation criteria that provide a comprehensive view of aid
effectiveness. The evaluation criteria are: relevance to who, effectiveness for who..,
efficiency, impact (if feasible) on who? Who benefitted? The aid activity must be
rated against these criteria.

The questions can be used as provided, or can be adapted to be more relevant to the
aid activity, country context and the size of the evaluation. The independent
evaluation team can be asked to adapt the evaluation questions when they develop the
methods design in the Evaluation Plan, including developing evaluation questions that
assess relevant cross-cutting issues.

Specific questions should be developed to assess compliance with the Paris
Declaration on Aid Effectiveness and the Accra Agenda for Action under the relevant
criteria that are relevant to the activity.
Questions for an Independent Progress Report
Relevance
Are the objectives relevant to Australian Government and partner government
priorities?
Are the objectives relevant to the context/needs of beneficiaries?
If not, what changes need to be made to the activity or its objectives to ensure
continued relevance?
Effectiveness
Are the objectives on track to being achieved? If not, what changes need to be
made to objectives to ensure they can be achieved?
To what extent has the activity contributed to achievement of objectives?
Efficiency
Has the implementation of the activity made effective use of time and resources to
achieve the outcomes?
Sub-questions:
Have there been any financial variations to the activity? If so, was value for
money considered in making these amendments?
Has management of the activity been responsive to changing needs? If not, why
not?
Has the activity suffered from delays in implementation? If so, why and what
was done about it?
10
Has the activity had sufficient and appropriate staffing resources?
Was a risk management approach applied to management of the activity (including
anti-corruption)?
What are the risks to achievement of objectives? Have the risks been managed
appropriately?
I mpact (if feasible)
Has the activity produced intended or unintended changes in the lives of
beneficiaries and their environment, directly or indirectly?
Have there been positive or negative impacts from external factors?
Sustainability
Do beneficiaries and/or partner country stakeholders have sufficient ownership,
capacity and resources to maintain the activity outcomes after Australian
Government funding has ceased?
Are there any actions that can be taken now that will increase the likelihood that
the activity will be sustainable? Are there any areas of the activity that are clearly
not sustainable? What actions should be taken to address this?
Gender Equality
Was the activity designed to provide equal participation and benefits for women
and men, boys and girls?
Is the activity promoting equal participation and benefits for women and men, boys
and girls?
Sub-questions:
Is the activity promoting more equal access by women and men to the benefits
of the activity, and more broadly to resources, services and skills?
Is the activity promoting equality of decision-making between women and
men?
Is the initiative helping to promote womens rights?
Is the initiative helping to develop capacity (donors, partner government, civil
society, etc) to understand and promote gender equality?
Monitoring and Evaluation
Does evidence exist to show that objectives are on track to being achieved?
Is the M&E system collecting the right information to allow judgement to be made
about meeting objectives and sustainability at the next evaluation point?
Is data gender-disaggregated to measure the outcomes of the activity on men,
women, boys and girls?
Is the M&E system collecting useful information on cross-cutting issues?
Analysis & Learning
How well was the design based on previous learning and analysis?
11
How well has learning from implementation and previous reviews (self-assessment
and independent) been integrated into the activity?
Lessons
What lessons from the activity can be applied to (select as appropriate: further
implementation/designing the next phase of the activity/applying thematic
practices [i.e. working in partner systems/environment/fragile stages] to the rest of
the program/designing future activities).

Rating scale:
Satisfactory Less that satisfactory
6 Very high quality 3 Less than adequate quality
5 Good quality 2 Poor quality
4 Adequate quality 1 Very poor quality