Result-based communications:

Some examples taken from EU

public policy communication
George Ellis Ruano, Director
Sophie Mestchersky, Consultant
Lugano, 10 February 2009
 ● The CHF10 question

How do you know if your

communication activities
are effective?

www.gellis.com
● The answer

www.gellis.com
 ● Measurement & evaluation: what’s in a word?

We measure an input, an output,

but…

We evaluate a result, an outcome, an impact.

www.gellis.com
 ● A CHF100 question

How do you know if your communication activities

can be evaluated?

www.gellis.com
 ● Requirement 1: Easy access to data and information

Example of impact - Vote, access to public services, etc.

Example of outcome - Level of knowledge about the policy

being promoted amongst target groups;
- Level of endorsement.

Example of result - Visits, downloads, comments left by

visitors
Example of output - A website (number of pages, number of
languages available, etc.)
Example of input - A webmaster, a budget line

www.gellis.com
● Requirement 2: A coherent and logical presentation

www.gellis.com
 ● Requirement 3: SMART objectives

• Example: a 15% increase of subscriptions to your

newsletter by the end of the programme.

• Example: Each year until the end of the

programme, secure at least two additional
sponsors for the annual conference

• Example: 50% awareness of your activities

amongst Decision Makers

www.gellis.com
 ● Example (1/5)

1. Measuring inputs

• Evolution of the dedicated budget Y0 / Y-1 / Y-2, etc.

• Comparison with the budget of a similar organisation

www.gellis.com
 ● Example (2/5)
2. Measuring outputs

Checklist:

Number of website updates;

Number of events and conferences;

Improvement of the corporate identity;

Number and quality of contacts made with
potential partners;

www.gellis.com
 ● Example (3/5)
3. Evaluating results

• Quantitative results: Number of visits, number of

downloads, number of comments, average duration
of a visit, etc.

• Qualitative results: nature of comments, visitors’

patterns, preferred language version, etc.

www.gellis.com
 ● Example (4/5)

4. Evaluating outcomes

• Quantitative outcomes: share of the target group

that is aware of the service being promoted, etc.

• Qualitative outcomes: target group’s opinion, etc.

www.gellis.com
 ● Example: (5/5)
5. Evaluating impact

• Evolution of the market share, of citizens’ support for

a specific policy, etc.

www.gellis.com
 ● Key success factors (1/3)
1. A broad skill set

• Cutting across different disciplines

– Communication
– Political science
– Organisational behaviour
– Sociology
– Psychology, etc.

www.gellis.com
 ● Key success factors (2/3)
2. A well-defined methodology
Qualitative tools Quantitative tools
Impact •Focus groups •Large-scale
•Expert panels surveys
Outcome •Stakeholder •Small-scale surveys
consultations
Result •Checklists, •Checklists
•Media analysis •Media analysis
Output •Checklists •Checklists
•Benchmarking •Benchmarking
Input •Checklists •Checklists
•Benchmarking •Benchmarking

www.gellis.com
 ● Key success factors (3/3)
3. A participatory approach

• To get new ideas

• To facilitate data collection
• To distribute tasks fairly
• To secure commitment
• Etc.

www.gellis.com
 ● Case studies (1/3)
Client: URBACT

Issue: Evaluation of the existing communications strategy

and design of a seven-year communication plan

Results:

−Evaluation of the URBACT website (including usability study, communication value study
and online user survey)
−Evaluation of the 2002-2006 communication strategy (desk research, stakeholder
interviews, benchmarking exercise, SWOT analysis)
−Design of the 2007-2013 communication plan (Identification of needs and opportunities,
formulation of SMART - Specific, Measurable, Attainable, Research based, Time-bound -
communication objectives, identification of target audiences, definition of strategy and
dissemination tactics, guidance on organisational structure, resources and appropriate
timing, drawing up of a 2008 action plan)

www.gellis.com
 ● Case studies (2/3)
Client: European Commission, DG ENV

Issue: Market analysis in view of developing a new approach

for the ‘Environment for Young Europe’ website

Results:
−Review of the website, assessment of its continued relevance and identification of its
strengths and weaknesses;
−Comprehensive market analysis through surveys conducted in 10 EU Member States
with a view to assemble the teachers’ and students’ ideas on how the European
Commission could make better use of the Internet to communicate environmental issues
to young audiences;
−Identification of the appropriate target groups for the website and design of a
strategy to turn them into multipliers;
−Strategic recommendations about the optimal structure, content, features and visual
and verbal identity for the development of a new website.

www.gellis.com
 ● Case studies (3/3)
Client: European Commission, DG Employment

Issue: Scoping study for communication and networking (CN)

activities under the PROGRESS programme

Results:

−Definition of new communication objectives;

−New Target Group Classification based on policy cycle;
−Need for more targeted messages;
−Need for a more ‘national approach’;
−Need to focus on most effective communication tools and channels;
−Need to rethink organisation and systems;
−Need for a Monitoring and Evaluation Plan

www.gellis.com
 ● Gellis’ key assets

• A multi-national, multi-disciplinary team of 13 highly

qualified professionals;
• Value-adding partnerships;
• A roster of experts in a wide range of policy areas;
• Sound understanding of the policy-making process;
• Professional memberships;
• Strong project management skills;
• Proximity to EU institutions;
• Stability due to long-term contracts with EC

www.gellis.com
 Thank you!

George Ellis Ruano, Director

george.ellis@gellis.com

Sophie Mestchersky, Consultant

sophie.mestchersky@gellis.com

www.gellis.com