Design, Implementation and Evaluation of EU agri-environmental policies

- key concepts and analytical approaches

Abstract This paper introduces key concepts and analytical approaches to be utilized for analysis of agricultural and environmental policy measures (AEM) in the Ph D Project: Design, Implementation and Evaluation of EU agri-environmental policies. The theoretical policy framework for the analysis is presented and discussed, including the term impact model. Further three key properties of the EU Agrienvironmental policy are presented: The bottom up-top down issue, integration between agricultural and environmental policy, and the role of evaluation.

Jens Peter Vesterager Research assistant Dept. of Urban and Landscape studies Centre for Forest, Landscape and Planning Royal Veterinary and Agricultural University Denmark

Introduction: The policy context and the object of the study During recent years, the Common Agricultural Policy (CAP) has moved from production support over hectare and headage premiums - to direct (income) support de-linked from agricultural production. Parallel to this, a Rural Development Programme based on goals related to sustainable development of landscapes, biodiversity and rural communities has evolved as the so-called second pillar of the CAP (Brouwer 2003). As a whole, the CAP reform is likely to be the most important driving force in the countryside for the next decade. Research in policy design and implementation and development of evaluation methods to support these processes are therefore of vital importance. The above mentioned need for policy development and evaluation methods is the subject of my coming Ph. D. project. In this paper I discuss some of the key concepts and analytical approaches to be utilized on agricultural and environmental policy measures (AEM) in the following just termed AEM. The overall aim is to get a research based understanding of the processes and effects of The Danish Rural Development Programme (DRDP) within the framework of the CAP and with the purpose to provide evidence for the improvements of the programme. Design as well as implementation processes are analysed with respect to vertical (EU, member states, regions) coherence and horizontal coordination of agricultural and environmental policies. It is a concrete goal to develop evaluations methods for continuous improvements of the so-called agri-environmental policies and other environmental and landscape schemes to be implemented under the Rural Development Programme 2007-2013. The object of the study is agri-environmental measures that have been implemented under the DRDP 2000-2006. The focus is on agri-environmental schemes implemented under art 22-24, reg. 1257/99 and demonstration projects implemented under art. 33, reg. 1257/99) The measures are examples of different kinds of policy design and use different means in the implementation process, involving landscape, biodiversity and rural districts in different ways. European experience is included through international research co operations concerning Agrienvironmental policies and other issues of relevance to agriculture and environment, i.e. cross compliance regulation (EC reg. 1782/2003) will be analysed and discussed and results from the EASY project (Whitfield 2006;Kleijn et al. 2006;Knop et al. 2006) and the AEFootprint project (AEFootprint consortium 2006) will be included. Policy concepts and processes A classical way of analysing the policy process is through the so called Stages Approach (deLeon 1999;Winter 1994), in which the different stages are often described as follows: Agenda setting Policy design Policy Implementation evaluation reformulation of the model

It is a classical circular description of policy design, implementation, and evaluation based on stages.

The Stages Approach will be used as conceptual policy framework in the study because it provides a straight forward way of looking at specific policies in space and time. The Stages Approach has the advantages of providing a rough overview of different agendas, stakes, policy alternatives and stages on an analytical level (deLeon 1999), and it seems to be a good alternative, when question of policy effects are dealt with. The approach is far from ideal and has been criticised for an artificial division of policy processes that may not be found in reality (Sabatier & Jenkins-Smith 1999). The arguments for this are: It does not identify a set of causal drivers that govern the process within and across stages The proposed sequence of stages is often descriptively inaccurate. The focus neglects interaction of implementation and evaluation in a policy domain, The approach often has a top-down bias The usual process of multiple interacting cycles is often neglected. Alternative theoretical frameworks for description of the policy framework are e.g.:Advocacy Coalition Framework, Ambiguity, Time and Multiple Streams and Institutional Rational Choice. Their criticism of the Stages approach will be incorporated during the study1. Impact models as they have been termed by Rossi and Freeman (1993) can be seen as specific highly technical versions of the stages approach useful for evaluation research. An impact model takes the form of a statement about the expected relationships between a programme and its goal, setting forth the strategy for closing the gap between the objectives set during the planning process and existing behaviour or condition. It must contain a causal hypothesis, an intervention hypothesis, and an action hypothesis. (Rossi and Freeman 1993, p.120) A generalized impact model consists of several hypotheses: the causal hypothesis, the intervention hypothesis and the action hypothesis. Each of the hypotheses specifies essential parts of the policy design, and work as a foundation for ex. ante and ex. post evaluation: The causal hypothesis specifies causal properties of the phenomena to be influenced, i.e. properties of an object, or issue, subject to a policy. The interventions hypothesis describes how a policy may influence the causal relations of the phenomena to cause changes, i.e. what properties to emphasise and influence in order to cause improvements. The action hypothesis describes how it is possible to determine that that the proposed interventions caused the changes, and not other factors. A general impact Model for agriculture and environment has been has been specified in various studies (Onate et al. 2000;Primdahl et al. 2003) It includes the following relationships between policies and outcomes, and covers as well the causal as the intervention hypothesis: Objective
1

obligation

Agricultural practises (land use/ management practises)

environmental outcomes

E.g through participation in the Ph d. course 04/10-11/10 2006 Advanced Theories of the Policy Process and Strategies for their Application, October 2006

Agricultural practises and environmental outcomes represent the causal hypothesis, where as the objectives and obligations refer to an intervention hypothesis, i.e. changes caused by farmers signing agreements based on agri-environmental schemes. Agricultural practises may include as well management practises as land use practises (Onate et al. 2000;Primdahl et al. 2003). In the AEFootprint study the generalized Impact Model for agriculture has been studied (Vesterager et al. 2006b;Vesterager et al. 2006a). In EC-context a parallel to the impact models, the logical diagram of intervention has been introduced. It is based on causal relationships between cause and effect in a programme (European commission 1999) In the context of the planned Ph. D. project, the main rationale for using a Stages Approach is to enable analysis of relationships (as well assumed as real) between policy objects and policy outcomes. The concept of Impact Model (IM) is central to this purpose. Design and implementation of AEM and other agri-environmental policies is not only depending on the quality of the assumed relationship between goal, practises and outcomes. Three other dimensions are of vital importance A) the coherence between different administrative levels, B) policy integration, and C) experiences with Evaluation.

A) The coherence between different administrative levels - In between top down and bottom up, and the concept of subsidiarity The DRDP is designed through a combination of top-down and bottom up processes. From a theoretical perspective, the study will provide insights into opportunities and constraints for the development of the subsidiary principle. As proposed by Buller et. al. (1999) and Billaud et. al. (2002) implementation of regulations at one level may be policy design at another level. The statements are based on empirical material on three levels: EU, National and regional level. Further a local level may be added (European Commission 2005), p.3. The implementation is based on the subsidiary principle. Subsidiarity is highly relevant. It is meaningful in complex AE context to leave concrete AE policy design to member states or regions. This allows a high degree of freedom in designing and implementing policies. Implemented measures may be described according to geographical focus, institutional context, and as results of a specific policy process including stakeholders (Hein et al. 2006) as well as a choice in means of regulation and the regional/national importance of the issue. These parameters may be used in designing a typology for characterizing and comparing different measures, and thus the outcomes of EC regulation. The price use of subsidiarity is inconsistency in objectives (Wildavsky 1979) and in coherence in design and implementation. Transfer and transformation may be used in the analysis of existing policies and ideas and policy approaches between member states and regions. Pedersen (2006) uses the concepts transfer and transformation in connection to analysis of Europeanization and European integration, to produce a

comparison of measures on supranational level. The concepts are based on earlier literature but are actualized in the following way: - Transfer refers to influence transferring on supra national level either through mimic, normative or coercive mechanisms - Transformation refers to influence changing slightly when it is transformed to another political system or institution The concepts are also used by Billaud et. al (2002), p. 22-25. As an example of transfer analysis including the nature geographical properties of areas Bunce et al. (1998) may be mentioned, proposing transfer of schemes from specific parts of UK to a specific part of Spain. The concepts are useful for comparing design and implementation of RDP in European perspective. Most likely the concepts may be extended with a geographical dimension. Relevant questions: On the one side good evaluation results of goal achievements require precise goals, on the other hand precise goals may conflict with the subsidiarity principle how is this dilemma dealt with in practise? What experiences exist with learning, and transfer of ideas and approaches between member states? B) Policy Integration Between environmental and agricultural policies, policy integration in practise The programme can be viewed as a combined agricultural and environmental policy and may therefore be analysed as a case of policy integration in practice. Damary (2002) emphasises the division between regulation within agricultural policy and environmental policy. Research (training, information extension) Environmental rules General agricultural policies Agri-environmental schemes Rural development instruments Market related instruments Busck et. al. (2001) proposes an analytical model with three different kinds of integration: Substantial integration, Spatial integration Institutional integration. Difference between the administrative traditions is of importance when it comes to policy integration. The study offers opportunities to work on cross national comparative basis, including recent studies of Cross compliance as an alternative to AEM. Analysis of Cross compliance includes descriptions of new Farm Advisory Tools (FAT), and new and existing Farm Advisory Systems (FAS), working with agriculture as well as environment. FAS is a typical example of involving 3rd parties, as a mean to communicate regulation between authorities and the farmer (Gunningham, Grabosky, & Sinclair 1998).

When choosing means of regulation the importance of local stakeholders and actors play a decisive role. This is described by Morris (2004), on the Actors network theory (ANT) applied in an AEM context, Billaud et al (2002) p. 30 and (Davos et al. 2002), based on experiences with Costal zone management. Policy integration will be studied by focusing on cross compliance as one way of integration, and AEM as another. The advantages and disadvantages of the two types of regulation will be compared. Relevant questions: What does the role of tradition within agricultural policy and environmental policy mean focusing on -traditional regulation? -third parties? -stakeholder engagement? C) Evaluation - Between two policy programmes, the role and methods of policy evaluation Continuous, independent evaluations are built into the Rural Development Programme (RDP) and provide opportunities to analyse how evaluations are performed in practice and how these practices are based theoretically and methodologically and how results are used in improving the RDP. The existing evaluation praxis in EC is based on a number of questions that must be answered by the evaluator (European Commission 2000;European commission 2002;European commission 1999). The questions address different levels, from the programme to the scheme. The evaluation system is quite comprehensive, and includes proposed indicators and criteria (European commission 2004) It is however also quite flexible, and allows the evaluator to add appropriate subjects and indicators (European commission 2000). From comments by the Danish evaluator it becomes obvious that there are certain shortcomings in the procedure: There is a lack of objectives and goals hierarchy, and quantified goals, and there is a lack of logical diagrams of impact as impact models are termed in EC context. Further there is a lack of data that may satisfy the visions for the evaluation set out by the Commission. For the AEM there is a problem in effect measurement due to the short time horizon of the programme and evaluations compared to a longer time span before effects of the programme may be expected (Kvistgaard Consult 2003). The proposed evaluation procedure may be criticised (Frouws 2001). Based on recent improvements in evaluation theory and interdisciplinary and system theory approaches (Bruckmeier 2001) evaluation should be (p.127): Process oriented non standardised Participatory

A method for employing the best possible evaluation is based on questions within the following four categories(Dauc & Holz 2001 p. 31 ff ). This will help adjust the evaluation to the context Program type Evaluation type Evaluation analyse Participation in the Evaluation The above mentioned approach may be appropriate when several types of policies are evaluated, when several levels are present in implementation and evaluation, many stakeholders are involved in a varying degree and many contexts may be described (European Commission 2005 p. 3.). Relevant questions: What do the above mentioned approaches imply in the highly hierarchical policy contexts of the CAP? How has the evaluations and policy produced so far contributed to the learning dimension in the CAP?

Reference List AEFootprint consortium. AE-Footprint. www.footprint.rdg.ac.uk . 2006. Ref Type: Electronic Citation Billaud,J.-P., Pinton, F., Bruckmeier, K., Reigert, C., Patrcio, T., & da Lima, A. V. (2002) Agricultural development in the European Context. The Agri-Environmental Policy of the European Union The implementation of the Agri-environmental Measures within the Common Agricultural policy in France, Germany and Portugal (eds Bruckmeier,K. & Ehlert,W.), pp. 19-42. Peter Lang, Europischer Verlag der Wissenschaften, Frankfurt am Main Brouwer,F. (2003) Sustaining Agriculture and the Rural Environment. Governance, Policy and Multifunctionality. Edward Elgar, Cheltenham, UK Bruckmeier,K. (2001) Evaluation of Rural development - the History of evaluation Research as a Source of Innovation. Innovating Rural Evaluation Social sciences in the interdisciplinary Evaluation of rural development (ed Bruckmeier,K.), pp. 13-30. Institut fr Internationale Sozialforschung, Berlin/ Freiburg Buller,H., Andersen, E., & Primdahl, P. (1999) Principal findings and recommendations. Implementation and effectiveness of agrienvironmental schemes established under regulation 2078/92 (eds Schramek,J., Biehl,D., Buller,H., & Wilson,G.A.), pp. 189-192. Frankfurt Bunce,R.G.H., Bell, M., & Farino, T. (1998) The Environmentally Sensitive Area Legislation in the United Kingdom and its potential application to the Picos de Europa mountains in north west Spain. Environmental Conservation 25, pp. 219-227 Busck,A., Kristensen.L.S., Primdahl, J., Tom-Petersen, P., & Vejre, H. (2001) Integration af natur og landskabshensyn i landbrugspolitikken En analyse af erfaringer fra de fire medlemslande, England, Tyskland (Slesvig-Holsten), Holland og Sverige. Institut for konomi, Skov og Landskab, KVL, Damary, P. Pan-European Biological and Landscape Diversity Strategy High level Pan-European Conference on Agriculture and Biodiversity towards integrating biological and landscape diversity for sustainable agriculture in Europe. 2002. Le Moulin Martinet, REDD Rseau-Echanges Dveloppement durable. Ref Type: Report Dauc,P. & Holz, B. (2001) A note on the Debate on Innovative and Interdisciplinary Methods for evaluation. Innovating Rural Evaluation Social sciences in the interdisciplinary Evaluation of rural development (ed Bruckmeier,K.), Institut fr Internationale Sozialforschung, Berlin/ Freiburg Davos,C.A., Jones, P. J. S., Side J.C., & Siakavara, K. (2002) Attitudes towards Participation in Coorporative Costal management: Four European Case Studies. Costal Management 30, pp. 209-220

deLeon,P. (1999) The Stages approach to the policy process: What Has It done? Where is it going? Theories of the Policy Process (ed Sabatier,P.), pp. 19-32. Westview Press, Colorado European Commission (2000) Den nye programmeringsperiode 2000-2006: Metodebeskrivelse ARBEJDSPAPIR 3 Overvgnings- og evalueringsindikatorer: Metodevejledning. European Commission (2005) Agri-environment Measures Overview on general principles , Types of measures and application. European commission, Directorate General for Agriculture and Rural Development, European commission,D.A. (1999) Evaluation of rural development programmes 2000-2006 supported from the European Agricultural Guidance and Guarantee Fund. European commission,D.A. (2000) Part A Introduction purpose abd use of the common questions with criteria and indicators. European commission,D.A. (2002) Guidelines for the mid term evaluation of rural development programmes 2000-2006 supported from the European Agricultural Guidance and guarantee Fund. European commission,D.A. (2004) Part D, Explanatory sheets. Frouws,J. (2001) Reflection on Critical Perspectives in evaluating Rural Development Programs. Innovating Rural Evaluation Social sciences in the interdisciplinary Evaluation of rural development (ed Bruckmeier,K.), Institut fr Internationale Sozialforschung, Berlin/ Freiburg Gunningham,N., Grabosky, P., & Sinclair, D. (1998) Smart Regulation Designing Environmental Policy. Oxford University Press, Hein,L., van Koppen, K., de Groot, R. S., & van Ierland, E. C. (2006) Spatial scales, stakeholders and the valuation og ecosystem services. Ecological Economics 57, pp. 209-228 Kleijn,D., Baquero, R. A., Clough, Y., Daz, M., De Esteban, J., Fernndez, F., Gabriel, D., Herzog, F., Holzschuh, A., Jhl, R., Knop, E., Kruess, A., Marshall, E. J. P., Steffan-Deweter, I., Tscharntke, T., Verhulst, J., West, T. M., & Yela, J. L. (2006) Mixed biodiversity benefits of agri-environment schemes in five European countries. Ecology Letters 9, Knop,E., Kleijn, D., Herzog, F., & Schmid, B. (2006) Effectiveness of the Swiss agri environment scheme in promoting biodiversity. Journal of Applied Ecology 43, pp. 120-127 Kvistgaard Consult (2003) Direktoratet for FdevareErhverv: Midtvejsevaluering af det danske landdistriktsprogram, Samlet rapport, 19. december 2003, bilag 3. Direktoratet for FdevareErhverv, Morris,C. (2004) Networks of agri-environmental policy implementation: a case study of England's Countryside Stewardship Scheme. Land Use Policy 21, pp. 177-191

Onate,J.J., Andersen, E., Peco, B., & Primdahl, J. (2000) Agri-environmental schemes and the European agricultural landscapes: the role of indicators as valuing tools for evaluation. Landscape Ecology 15, pp. 271-280 Pedersen,L.H. (2006) Transfer and transformation in processes of Europeanization. European Journal of Political Research, Primdahl,J., Peco, B., Schramek, J., Andersen, E., & Onate, J. J. (2003) Environmental effects of agrienvironmental schemes in Western Europe. Journal of Environmental Management 67, pp. 129-138 Rossi,P. & Freeman, H. (1993) Evaluation : a systematic approach, 5. edn. Sage, Sabatier,P. & Jenkins-Smith, H. C. (1999) The Advocacy Coalition Framework: An Assessment. Theories of the Policy Process (ed Sabatier,P.), pp. 117-166. Westview Press, Colorado Vesterager, J. P., Primdahl, J., Kristensen.L.S., and Vejre, H. Agri-environmental schemes - Impact models and environmental indicators. D 3. 2006a. Ref Type: Unpublished Work Vesterager, J. P., Primdahl, J., Kristensen.L.S., and Vejre, H. Impact Models in Agri-environmental schemes - 12 cases. D 10. 2006b. Ref Type: Unpublished Work Whitfield,J. (2006) Agriculture and environment: How green was my subsidy? Nature News 439, pp. 908-909 Wildavsky,A. (1979) Speaking truth to power. Transaction Books, Oxford Winter,S. (1994) Implementering og effektivitet. Systime, Herning

10