The Politicization of ex post Policy Evaluation in the EU

• A Introduction

  Policy evaluation is not new in European Union (EU) policymaking. The EU has long had a legal requirement for ex post evaluation in relation to expenditure policies, and some regulatory instruments include clauses for ex post review, monitoring or evaluation. Yet the practice of evaluation has often been ad hoc and diverse across policy areas and Commission Directorates-General (DGs), whilst efforts for a more coherent approach have primarily focused on financial accountability. Ex post evaluation has mainly been of a technical nature, attracting little political attention. However, over the last 15 years, the European Commission has gradually built up its efforts in relation to ex post evaluation; and with the presentation of the Commission’s Better Regulation package¹xI use the expression ‘Better Regulation Package’ for three documents adopted together in May 2015 which set out the European Commission’s new Better Regulation approach and tools; Communication from the Commission, ‘Better Regulation for Better Results – An EU Agenda’, 19 May 2015, COM(2015)215 final (which sets out the general philosophy of the new BR approach); European Commission, ‘Better Regulation Guidelines’, 19 May 2015, SWD(2015) 111 final (which sets out the compulsory aspects of the BR strategy) (further referred to as BR Guidelines 2015); and the ‘Better Regulation Toolbox’, attached to these Guidelines, which set out complementary guidance to assist practitioners in the application of the Guidelines (further referred to as BR Toolbox 2015). in May 2015, ex post evaluation has become a key issue on the political agenda. This article analyses this development as a gradual politicization of ex post evaluation in European governance.
  It is important to first clarify the concepts of (policy) evaluation and politicization used in this article. The European Commission describes evaluation in its 2015 Better Regulation Guidelines as “a tool to help the Commission assess the actual performance of EU interventions compared to initial activities”,²xEuropean Commission, ‘Better Regulation Guidelines’, COM(2015) 215 final, p. 49. and defines it as

  an evidence-based judgment of the extent to which an intervention has:

  • been effective [ie do the verified effects correspond to the original objectives?] and efficient [ie where the costs justified?]

  • been relevant given the needs and its objectives [ie do the original objectives still correspond to the needs of the EU?]

  • been coherent both internally and with other EU policy interventions and

  • achieved EU added value [ie compared to what could be achieved by the Member States].

  In the Commission’s use of the concept, ‘evaluation’ normally refers to ex post (i.e., retrospective) evaluation, which can be interim (i.e., at the midterm of an initiative), final (at its conclusion) or ex post in the strict sense (which can take place several years after the intervention has finished). The Commission talks about ex ante evaluation only in relation to expenditure programmes. The ex ante appraisal of new EU (regulatory) initiatives that take place via Integrated Impact Assessments is usually not referred to as ‘evaluation’. Although linguistically there are no particular reasons to reserve the concept of evaluation to ex post and not ex ante appraisal, I will follow the Commission’s predominant use of terminology here by using ‘evaluation’ or ‘policy evaluation’ as a shorthand for ex post evaluation, unless explicitly indicated otherwise, such as in the case of ‘evaluation throughout the policy cycle’, which implies both ex ante and ex post evaluation.
  It is equally useful at this stage to clarify what I understand to be politicization. The narrower, and institutionally focused, use of the concept of politicization is found in the political science and public administration literature that deals with the relationship between the assumed ‘neutral’ character of the bureaucracy and the political imperatives of elected politicians.³xB. Guy Peters, ‘Politicisation: What Is It and Why Should We Care?’, in C. Neuhold, S. Vanhoonacker & L. Verhey (Eds.), Civil Servants and Politics, Basingstoke, Palgrave Macmillan, 2013, pp. 12-24. The more the bureaucracy is steered by political imperatives, the more the administration is politicized. Political control over the administration is desirable, as administrators lack the political elected mandate of politicians. At the same time, strong politicization of the administration may undermine the expertise of the administration and make policy excessively dependent on short-term political interests. Such politicization is particularly problematic when one is dealing with institutions that should profit from a considerable independence, such as courts or independent central banks.
  A broader use of the concept of politicization relates less to institutions and more to the political salience of societal issues. When an issue is politicized, it becomes part of political debate and discussion. More precisely, it will be discussed within the political realm, which includes political (elected) representatives, party politics, interest groups and, most broadly, public debate. Politicization is then measured in terms of both salience (when does an issue rise higher on the political agenda?) and contestation (how polarized are positions about it?).⁴xW. van der Brug et al., ‘A Framework for Studying the Politicisation of Migration’, in W. van der Brug et al. (Eds.), The Politicisation of Migration, London, Routledge, 2015, p. 2. However, as a debate becomes more polarized, interest-based arguments are more likely to take the lead over evidence-based argumentation or search for the truth. The concept of politicization therefore often has a negative connotation. Most narrowly, a politicized debate would be characterized by motives of party politics, or simply politicians’ interest in re-election, at the cost of expertise judgement. Hence, when one talks about the politicization of migration policy, or of crime, it most often comes with warning bells that the debate is not properly informed.⁵xElectoral success rather than best policy is suspected as main driver of action. Similarly, one has talked about the politicization of development or humanitarian aid, to criticize that the primary objective of providing aid is driven and undermined by more dubious, or at least morally less clear, political motives. See M. Haneef Atmar, ‘Politicisation of Humanitarian Aid and Its Consequences for Afghans’, Disasters, Vol. 25, No. 4, 2001, pp. 321-330. The tension between sound evidence and political motives and argumentation has particularly been discussed in science and technology studies, where analysis of the scientification of politics has gone hand in hand with findings about the politicization of science.⁶xT.D. McGarity & W.E. Wagner, Bending Science. How Special Interest Corrupt Public Health Research, Cambridge, Harvard University Press, 2008; M. Everson & E. Vos, ‘The Scientification of Politics and the Politicisation of Science’, in M. Everson & E. Vos (Eds.), Uncertain Risks Regulated, London, Routledge, 2009, pp. 1-18.
  However, it should be said that politicization is not inherently bad. At its most basic level, it means that a topic is brought in the public realm of politics, which should ensure decision-making considering the common good. This positive role of politicization comes best to the forefront when politicization is pitched against its opposite, namely ‘technocratization’. The latter also often has a negative connotation when it is used to refer to situations in which decision-making that should be made in the public realm is unduly kept in the hands of experts and administrators.⁷xS. Raman, ‘Science, Uncertainty and the Normative Question of Epistemic Governance in Policymaking’, in E. Cloatre & M. Pickersgill (Eds.), Knowledge, Technology, and Law, London, Routledge, 2015, pp. 17-32. For a critical analysis of technocratic theory as a normative theory, see C.M. Radaelli, Technocracy in the European Union, London, Longman, 1999, pp. 11-28. At the same time, the shielding of expert advice, or even decision-making, from political influences has often been heralded in modern governance, as it would ensure that policymaking is based on the best available evidence, independent of sectoral interests and short-term thinking that drives politicians.⁸xG. Majone, Regulating Europe, London, Routledge, 1996. Neither politicization nor technocratization is thus inherently bad. The question is about the right balance between the two. I therefore use politicization in this article as a ‘neutral’ concept, not implying a normative judgement, but simply referring to a process in which the topic of evaluation becomes increasingly part of a debate and agenda that is set, influenced by and played out in the political realm and not simply by experts and administrators. At the same time, I assess advantages and pitfalls of such politicization.
  In Section B, I first briefly describe how policy evaluation in the EU developed from a focus on financial accountability that was mainly technical in nature, to a key principle of Better Regulation, identifying the key features of evaluation in the Commission’s approach in recent years, and as part of the Better Regulation package in particular. The following sections will analyse the increasing politicization of evaluation that results from this development. Section C analyses how evaluation becomes more politicized as it becomes a key element of the Better Regulation strategy, which the Juncker Commission has propelled to a key political priority. Section D analyses the politicization that is inherent when evaluation becomes a general principle of Better Regulation, applied to both redistributive and regulatory policies and throughout the policy cycle. Section E studies the politicization that results from the increased participatory nature of EU policy evaluation. I finally conclude with some reflections on the challenges of a more politicized evaluation strategy, particularly at a time when populism delegitimizes many modern governance mechanisms.

• B From Financial Accountability to a General Principle of Better Regulation

  The EU has engaged in project and programme evaluation for several decades now. However, evaluation has been especially focused on expenditure policies, such as structural funds, research and development, the Common Agricultural Policy (CAP) and development aid. Although initial evaluation practices developed ad hoc within different DGs, the increase in the EU budget and EU expenditure during the 1990s, as well as instances of corruption and the legitimacy crisis related to the resignation of the Santer Commission, led to a more systematic approach to evaluation in the European Commission. Legal requirements and control were tightened to ensure financial accountability.
  Evaluation became centrally enshrined in budgetary allocations and the 7-year financial programming cycle. Evaluation standards were developed by DG Budget in 1999 (and revised in 2004) to guide DGs in their evaluation work, particularly when outsourcing evaluation to external consultants.⁹xEuropean Commission, ‘Evaluating EU Activities: A Practical Guide for the Commission Services’, Directorate-General for the Budget, July 2004. All DGs were supposed to develop their evaluation capacity, particularly through the establishment of evaluation units. However, many DGs (especially those not involved in expenditure policy) considered evaluation a formality they had to comply with, rather than a useful exercise they could learn from.¹⁰xS. Højlund, ‘Evaluation in the European Commission: For Accountability or Learning?’, European Journal of Risk Regulation, Vol. 6, No. 1, 2015, pp. 35-46.
  More recently, the European Commission has aimed for a reorientation of its evaluation policy. The reorientation of evaluation policy was first set out in the 2007 Commission Communication “Responding to Strategic Needs: Reinforcing the use of evaluation”¹¹xCommunication from the Commission, ‘Responding to Strategic Needs: Reinforcing the Use of Evaluation’, 21 February 2007, SEC(2007) 213. and was subsequently given more clout by the embedding of evaluation into the broader Better Regulation agenda. The turn from ‘Better Regulation’ to ‘Smart Regulation’ in 2010¹²xCommunication from the Commission, ‘Smart Regulation in the European Union’, 8 October 2010, COM(2010) 543 final. was precisely characterized by the argument that ‘Better/Smart Regulation’ should be taken into account throughout the entire policy cycle and not just at the start of it (where most Better Regulation tools had until then been focused). This means in particular that ex post evaluation should gain a more central place in the policymaking process and should be linked to the ex ante assessment of new policy intervention. The Commission’s 2013 Communication “Strengthening the foundations of Smart Regulation – improving evaluation”¹³xCommunication from the Commission, ‘Strengthening the foundations of Smart Regulation – Improving Evaluation’, 2 October 2013, COM(2013) 686 final. reiterates this approach in catchy terms by talking about the ‘evaluate first’ principle and promoting an ‘evaluation culture’ in the Commission, whilst more concrete proposals were to be proposed in new evaluation guidelines. The Better Regulation Guidelines adopted in May 2015 finally consolidate this new approach, by bringing together guidelines on (ex ante) integrated impact assessments and (ex post) evaluation and putting the issue of Better Regulation on the highest political agenda.
  The key features of the new approach to evaluation resulting from the four Communications (2007 ‘Reinforcing Evaluation’, 2010 ‘Smart Regulation’, 2013 ‘Improving evaluation’ and 2015 ‘Better Regulation Guidelines’) are the following:

  1. Evaluation has to be applied to all types of EU intervention: expenditure policy as well as regulatory intervention, including soft law measures. Although the idea to apply evaluation beyond expenditure policies goes back more than a decade, it is only in the 2007 and 2010 Communications that concrete measures were proposed for a more systematic application of evaluation to regulatory intervention. Between 2002 and 2012 only about one third of legislative measures were evaluated, and whilst the data show some increase, this has not been consistent.¹⁴xE. Mastenbroek, S. van Voorst & A. Meuwese, ‘Closing the Regulatory Cycle? A Meta-Evaluation of Ex-post Legislative Evaluations by the European Commission’, Journal of European Public Policy, Vol. 23, No. 9, 2015, p. 10. The 2015 Better Regulation Guidelines now make evaluation a general principle of Better Regulation.

  2. The ‘evaluate first’ principle locates evaluation firmly within the policy cycle. New EU intervention can only occur after an assessment of past action has been made. Although evaluation of expenditure policy has long been linked to the 7-year financial programme cycle, the 2007 Communication sets the Commission on track to fit evaluation of all its actions into its strategic planning and programming cycle. Most importantly, ex post evaluation should feed back into the EU system of ex ante impact assessments, which has been solidly established since 2003.

  3. The Commission’s emphasis upon the place of evaluation in the policy cycle goes hand in hand with the embedding of evaluation within the Better Regulation agenda, and the ‘Regulatory Fitness and Performance Programme’ (REFIT) in particular. As part of the Better Regulation agenda, the Commission initiated REFIT in December 2012 in order to review the entire stock of EU legislation¹⁵xCommunication from the Commission, ‘EU Regulatory Fitness’, 12 December 2012, COM(2012) 746 final.; to identify burdens, inconsistencies, gaps or ineffective measures, with the aim to ensure “a simple, clear, stable and predictable regulatory framework for businesses, workers and citizens”.¹⁶xCommunication from the Communication, ‘Regulatory Fitness and Performance (REFIT): Results and Next Steps’, 2 October 2013, COM(2013) 685 final, p. 2.
     Evaluation has always had a ‘value for money’ character. Control over the implementation of expenditure policies would ensure a certain level of accountability regarding whether citizens had got what they paid for. However, in the Better Regulation context, the value for money argument develops from ex post accountability to making use of ex post evaluation to decide on the desirability of future action, particularly in the context of a regulatory framework that aims to be as ‘smart’ and ‘thin’ as possible. The Commission considers evaluation a ‘key tool’ in its Better Regulation agenda, not only to ensure better (quality) regulation but also, and particularly, to avoid regulatory burden: “Evaluating the effectiveness and efficiency of EU legislation will improve the quality of policy-making and help to identify new opportunities to simplify legislation and reduce administrative burdens.”¹⁷xCommunication from the Commission, ‘Smart Regulation in the European Union’, p. 4. Or as the 2013 Evaluation Communication states it: “There can be a tendency to look forward and focus on new initiatives. But changes are costly and take time to implement – so they need to be justified and greater attention needs to be paid to looking back before moving forward.”¹⁸xCommunication from the Commission, ‘Strengthening the Foundations of Smart Regulation – Improving Evaluation’, p. 5.

  4. Finally, by placing evaluation centrally in the policy cycle for all EU action, ex post evaluation should ensure, above all, policy learning, and not just (financial) programme or project learning. Evaluation should not only be the remit of a small number of administrators directly involved in a specific programme or project but should feed back into the political decision-making process. This underpins the Commission’s attention to ensuring better communication and transparency so as to increase the number of actors that can be involved in this learning process, being either stakeholders or other institutional actors.

  Having established these key features of the Commission’s new evaluation approach, the following sections more closely analyse how this affects the politicization of evaluation.

• C Policy Evaluation as a Political Priority, Steered from the Top

  I Political Priority

  A key aspect of the politicization of evaluation results from it getting centre stage of the political agenda. More precisely, it becomes a tool of centralization within the Commission, allowing the College of Commissioners, and especially the Commission President and the First Vice President for Better Regulation, to keep bottom-up initiatives from Commission DGs in line with the politically set agenda, and the goal of ‘Better Regulation’ and ‘reducing regulatory burden’.
  Over the last decade, evaluation policy in the Commission has gradually been placed under more centralized control. Initially the key centralizing role was played by DG Budget, which had the most experience with ex post evaluation, and adopted guidelines to steer the various DG practices. However, DG Budget’s steering of evaluation was soft guidance that often did not provide for major change in evaluation practice beyond areas of expenditure policy. The last evaluation guidelines drafted by DG Budget (Guidelines 2004) were presented as a “Practical Guide for the Commission Services” “presenting practical solutions and good practices”.¹⁹xEuropean Commission, ‘Evaluating EU Activities: A practical guide for the Commission Services’.
  As ex post evaluation was extended to regulatory action and became a key political priority, the Secretariat General became the centralized guiding unit to coordinate and steer evaluation in 2009. This facilitates linking ex post and ex ante evaluation, as the Secretariat General is also responsible for monitoring integrated impact assessments. At the same time, it brings evaluation closer to the political decision-making in the Commission. The Secretariat General has hierarchical control over the DGs and sits directly under the Commission’s Presidency. Evaluation becomes a key tool of steering DG initiatives more stringently in function of the policy priorities set out by the College of Commissioners, and the Commission President in particular.
  As the new evaluation guidelines are now included in the Better Regulation Guidelines drafted by the Secretariat General, they become more stringent in nature, compared to the ‘practical solutions and good practices’ previously suggested by DG Budget. As the document states, “the main guidelines set out the mandatory requirements and obligations for each step in the policy cycle whilst the Toolbox provides additional guidance and advice which is not binding unless expressly stated to be so” (emphasis added).²⁰xEuropean Commission, ‘Better Regulation Guidelines’, p. 4.
  The centralization of evaluation within the Commission should be seen in the context of two other processes. Firstly, the Commission’s management in general has become more centralized over the last decade in an attempt to make the Commission both more presidential and political in nature. The Commission President is no longer a primus inter pares among the other Commissioners, but increasingly a primus super pares, with increasing powers over the appointment, organization and dismissal of other commissioners, and the ability to set agendas and define the alternatives at stake.²¹xA. Wille, The Normalization of the European Commission: Politics and Bureaucracy in the EU Executive, Oxford University Press, 2013, p. 65. The Secretariat General has been strengthened to support him in that role,²²xS. Kurpas, C. Grøn & P.M. Kaczyinski, ‘The European Commission after Enlargement: Does More Add Up to Less?’, CEPS Special Report, Centre for European Policy Studies, February 2008; M. Egeberg, ‘The European Commission’, in M. Cini & N. Pérez-Solórzano Borragán (Eds.), European Union Politics, Oxford, Oxford University Press, 2015, p. 129. so that the President has increased control not only over the College of Commissioners but equally over the Commission bureaucracy. At the same time, the Commission has become more political in nature through the gradually strengthened relationship between the European Parliament (EP) elections and the appointment and accountability of the Commission.²³xWille, 2013, p. 66. The 2014 EP elections strengthened this process further by the introduction of a system of ‘Spitzenkandidaten’, in which each political party in the EP put forward its own candidate, who would become Commission President if that party turned out to be the biggest in the Parliament. Although the system of Spitzenkandidaten was not legally set out in the Treaties and the Member States (who retain the power to appoint the Commissioners) could have refused this system, they respected the EP invention of Spitzenkandidaten and appointed the candidate of the political party that won the elections as Commission President, namely Jean-Claude Juncker from the European People’s Party.²⁴xFor more about the politicization resulting from the Spitzenkandidaten system, see C. Antpöhler (2015), ‘Enhancing European Democracy in Times of Crisis? – The Proposal to Politicise the Elections of the European Commission’s President’, in F. Fabbrini, E. Hirsch Ballin & H. Somsen (Eds.), What Form of Government for the European Union and the Eurozone?, London, Hart Publishing, 2015, pp. 217-232; A. Kocharov, ‘In the Image of the State: Constitutional Complexities of Engineering a European Democracy’, in F. Fabbrini, E. Hirsch Ballin & H. Somsen (Eds.), What Form of Government for the European Union and the Eurozone?, London, Hart Publishing, 2015, pp. 233-251. As I will further explain below, Juncker has repeatedly referred to this ‘political mandate’ to defend a more political Commission.
  The second key development explaining the centralization of evaluation in the work of the Commission is the increasing political priority for reducing the regulatory burden. The establishment of the REFIT programme in 2012 was key in this direction, but with the coming into office of the Juncker Commission, this issue becomes even more central to the Commission’s agenda and organization of its work. Juncker, and First Vice President Timmermans (responsible for Better Regulation), interpret the rising Euroscepticism expressed in the EP elections of 2014 as a strong sign that people consider the European regulatory framework too burdensome. The result of the EP elections is thus seen as a mandate for the new Commission to make regulatory burden their primary target.
  In its first Work Programme, the Juncker Commission states,

  “…the bulk of what happens in the EU today and how our citizens judge the EU is based on existing legislation and programmes. Their immediate concerns are with the stock of existing rules, which is why this Commission is making a political priority of lightening the regulatory load while keeping high levels of social, health and environmental protection and consumer choice. We will overhaul the rules to make sure they contribute to the jobs and growth agenda and do not impose unnecessary red tape or administrative burdens, while at the same time bringing the benefits that citizens expect. Where the rules are outdated or out of line with our priorities, we will review and improve them. Where there is unnecessary red tape, we will cut it. Where the rules we have make sense and serve our objectives, we will work actively to ensure they are properly applied, implemented and enforced so they deliver real benefits to citizens. And we will use the other tools we have available to drive jobs and growth, in particular the EU budget which is primarily a means for smart investment in Member States and regions”. (emphasis added)²⁵xEuropean Commission, ‘Commission Work Programme 2015: A New Start’, COM(2014) 910 final, p. 2.

  The result is a Commission that wants to be big on big things, and small on small things. It would be difficult to counter-argue the Commission’s statement that “Citizens expect the EU to make a difference on the big economic and social challenges – high unemployment, slow growth, high levels of public debt, an investment gap and lack of competitiveness in the global marketplace”.²⁶x Ibid. Rising Euroscepticism is surely linked to the failure of the EU to address big social challenges.²⁷xWhether it is the EU level or the Member States to blame for it is another question. The Juncker Commission’s starting point is a promise to focus on these big challenges. At the same time, its focus on the regulatory burden and interpretation of the EP elections as being citizens wanting less EU interference is more open to questioning. Can the EP elections of 2014 really be interpreted as a primary question to reduce red tape for business, rather than dissatisfaction with the way the EU had dealt with the financial crisis and failure to address socio-economic problems appropriately?
  That the Commission sees itself increasingly as a political Commission mandated by the EP elections and that the 2014 elections implied answering the concern of regulatory burden is nicely illustrated by the following quote from the European Commission’s State of the Union in September 2016:

  “The Commission has to take responsibility by being political, and not technocratic. A political Commission is one that listens to the European Parliament, listens to all Member States, and listens to the people.
  And it is us listening that motivated my Commission to withdraw 100 proposals in our first 2 years of office, to present 80% fewer initiatives than over the past 5 years and to launch a thorough review of all existing legislation. Because only by focusing on where Europe can provide real added value and deliver results, we will be able to make Europe a better, more trusted place.”²⁸xEuropean Commission, ‘State of the Union Address 2016: Towards a better Europe – A Europe That Protects, Empowers and Defends’, Speech by the European Commission, Strasbourg, 14 September 2016.

  Put differently, when Juncker proudly sets out what his ‘political Commission’ has achieved after 2 years in charge, his first reference is to the key objective of reducing the regulatory burden.
  Better Regulation, and evaluation in particular, is a key tool to attaining this political priority. The same political argument can thus be found when the Commission sets out the importance of evaluation and fitness checks in the 2015 Better Regulation Communication: “The natural tendency of politicians is to focus on new initiatives. However, the EU is judged not just on new political initiatives, but on the benefits and the burden of existing EU legislation. As such, actively managing existing EU legislation is just as important politically as preparing new initiatives.”²⁹xCommunication from the Commission, ‘Better Regulation for Better Results – An EU Agenda’, p. 10. Evaluation is no longer simply an issue of technocratic learning but a key tool to realize a political priority of a European Commission that is more centralized and motivated to answer what it sees as the lessons to be drawn from the 2014 EP elections.

  II Control, Planning and Scrutiny

  Three institutional developments exemplify the politicization of evaluation by centralizing political control over it.
  Firstly, Juncker immediately translated ‘his political mandate’ from the EP elections into a new way of organizing the Commission, which would allow more centralized control. The Commissioners are organized in project teams, which are each led by a Vice President, thus creating some level of hierarchy among Commissioners. Moreover, a First Vice President has been created with the specific portfolio of Better Regulation, thus allowing it to sit at the nexus of the organizational hierarchy of the Commission. As evaluation has become a key element of the Better Regulation strategy, this organization of the College of Commissioners allows steering of evaluation practices across all DGs and optimizing its added value in function of political priorities.
  Secondly, centralized control is strengthened by better planning of evaluation. A decision to evaluate can result from several elements. Expenditure programmes have legal requirements for cyclical evaluation, whilst regulatory measures may include legal clauses to review or evaluate after a certain period. Moreover, all DGs are now required to set up an evaluation contact point (email address) to answer questions by stakeholders, which may provide input for considering evaluation.³⁰x See also Section V.II regarding the ‘Lighten the Load’ tool in the context of REFIT. Most importantly, the REFIT programme provides a regulatory screening of the acquis comunitaire, and sets out political priorities for evaluation and fitness checks. DGs are expected to develop annual and multi-annual evaluation programmes. The decision for major evaluations and fitness checks, particularly if part of REFIT, is to be politically validated by the Commissioner, whilst non-major evaluations and fitness checks can be endorsed by the Director General’s approval of the DGs Management Plan.³¹xEuropean Commission, ‘Better Regulation Guidelines’, p. 14.
  Thirdly, to ensure centralized control and common evaluation practices across DGs, the Better Regulation package has, for the first time, established scrutiny of evaluation practices, via the Regulatory Scrutiny Board (RSB). The RSB (previously named Impact Assessment Board) until then only had responsibility in relation to (ex ante) impact assessments, but its role is now extended to all fitness checks and all major evaluations.³²x Ibid., p. 9. The qualification ‘major’ leaves much room for interpretation, so scrutiny of evaluations by the RSB is far from being as systematic as it is for impact assessments (for which RSB scrutiny is always required, with some limited exceptions). In 2016, for instance, the RSB discussed seven evaluations³³xAlthough the RSB also checks evaluations ‘indirectly’ as part of its assessment of impact assessments, in 2016, it thus assessed an additional 15 evaluations in this way. European Commission, Regulatory Scrutiny Board Annual Report 2016, p. 17. (including two fitness checks, two programme evaluations and three regulatory evaluations) against 56 impact assessments. Unlike for impact assessments, the RSB has so far not attached positive or negative judgements to individual evaluations, although it intends to do so from 2017 onwards.³⁴x Ibid., p. 16. However, even then, there will not be a procedural ‘sanctioning mechanism’ when an evaluation is judged negatively, unlike for impact assessments, which are sent back to the DG for redrafting if judged negatively by the RSB. Therefore, RSB opinions on evaluations are supposed to create learning in the longer term and do not ensure sanctioned quality control on individual evaluations. Of the seven assessments of evaluations in 2016, for instance, the RSB concluded that there were generally shortcomings regarding critical analysis of coherence, relevance and added value, and there were no systematic conclusions for follow-up action, whilst the analysis of effectiveness and efficiency was more complete in evaluation reports.³⁵x Ibid., p. 17. The scrutiny of evaluation is thus not as tight as that of impact assessments, but there is clearly an intention to streamline evaluation practices across DGs in function of priorities and procedures set out centrally.

• D Evaluation as a General Principle throughout the Policy Cycle

  I Linking ex ante and ex post

  When evaluation becomes a general principle throughout the policy cycle for all types of policy intervention, it will inevitably be more political in nature than when it is simply an ex post tool of financial accountability limited to expenditure programmes.
  It is worth distinguishing between what are generally seen in the academic literature (and partially reflected in EU official discourse) as the two key objectives of evaluation, namely accountability and learning.³⁶xM. Scriven, ‘Beyond Formative and Summative Evaluation’, in M. McLauglin & D.C. Philips (Eds.), Evaluation and Education: At Quarter Century, Chicago, University of Chicago Press, 1991; F.-B. Van der Meer & J. Edelenbos, ‘Evaluation in Multi-Actor Policy Processes: Accountability, Learning and Co-operation’, Evaluation, Vol. 12, No. 2, 2006, p. 201; S. Borrás & S. Højlund, ‘Evaluation and Policy Learning: the Learners’ Perspective’, European Journal of Political Research, Vol. 54, No. 1, 2015, p. 99. Both accountability and learning can happen at a more technocratic or more political level.
  Evaluation focuses more on technocratic accountability when it is primarily financial or legalistic in nature. It is then an exercise in ensuring that the financial accounts are correct and the legal rules have been respected. Much of such cyclical financial and legal accountability does not require high political salience. It is a guarantee against misuse or inefficient use of resources that largely functions through deterrent and permanent technical control rather than high levels of politicization. The political accountability of this process only comes into play when the evaluation report would be used in the EP to highlight waste of taxpayers’ money and bad implementation or to more politically contest the added value of the intervention.³⁷x See also P. Stephenson in this special issue.
  Similarly, learning in evaluation can remain technocratic. When evaluation is focused on projects and programmes, learning is often limited to the officers dealing with it. Experience from evaluation may feed into the next cycle of a programme, but even then this is not straightforward as policy officers dealing with evaluation of the past project/programme may be different than those drafting the new one.
  By making evaluation a general principle throughout the policy cycle, accountability and learning become more political in nature throughout the entire process. The ‘evaluation first’ principle links ex post evaluation inevitably to judgement calls that are more political in nature. Past initiatives are assessed in function of the ability to make judgements on the orientation and re-orientation of future action. Evaluation is not simply done to confirm that the rules have been respected or that the original objectives have been reached, but also to feed in more political judgement calls on what the future objectives should be. This is strengthened by the fact that the Commission’s new evaluation strategy aims explicitly at learning and accountability that goes beyond the individual project or programme. Of particular importance in this regard are ‘fitness checks’, which were introduced in 2010. Fitness checks aim at a comprehensive evaluation of a policy area (i.e. the evaluation of a group of related interventions that are linked by a common set of objectives) rather than of single programmes or acts. Fitness checks help to give higher political leverage to evaluation, as they extend beyond the tiny network of a single regulatory intervention. At the same time, they are particularly embedded in the political priority of the REFIT programme to reduce the regulatory burden.

  II Expenditure and Regulatory Policy

  One can argue that it is not only the intrinsic link between ex ante and ex post evaluation that strengthens the politicized nature of the process but also that evaluation is now generally extended to regulatory intervention and not just expenditure policy. Traditionally, it is often argued that expenditure policy is more politicized in character than regulatory policy, as redistributive choices inevitably stir strong interest politics whilst regulation would often fall under the veil of technical detail³⁸xThe classical argument of the EU as a regulatory state is based on this premise. See Majone, 1996. or get away with the assumption that it is win-win in nature. However, the same does not necessarily hold true for evaluation. Although decisions on new expenditure programmes inevitable involve political choices, assessment of past programmes can be relatively neatly done based on well-established financial accountability rules. Put differently, future ex ante assessment and intervention do not necessarily intervene in the nature of previous ex post evaluation, for which there are well-established rules and criteria. The latter, though, is much less the case for regulatory intervention. The tools for ex post evaluation of regulatory intervention are far less established and less clear-cut.³⁹xH. Summa & J. Toulemonde, ‘Evaluation in the European Union: Addressing Complexity and Ambiguity’, in J. Furubo, R.C. Rist & R. Sandahl (Eds.), International Atlas of Evaluation, London, Transaction Publishers, pp. 407-424; T. Fitzpatrick, ‘Evaluating Legislation: An Alternative Approach for Evaluating EU Internal Market and Services Law’, Evaluation, Vol. 18, No. 4, 2012, pp. 477-499; Mastenbrouk et al., 2015, p. 4. Measuring the costs and benefits of existing regulatory policy is bound to open more questions about how this can be quantified than in the case of expenditure policy. Therefore, evaluation of regulatory action is bound to raise more political debate on the value and method of quantification and how methodology relates to the set objectives. Put differently, in the case of expenditure, there is more likely to be a clearer divide between an ex post evaluation based on a recognized established method, which might then fit (or not fit) into a separate debate on new redistributive intervention; whilst in the case of regulatory intervention, there is more likely a blurring of the debates on the correct target setting and their measurement in both ex post and ex ante evaluation.

  III The Interinstitutional Dimension

  In addition to the intrinsic link between ex ante and ex post evaluation, and the application of evaluation to both expenditure and regulatory policy, there is a third way in which evaluation becomes more politicized when applied as a general principle throughout the policy cycle. With the 2015 Better Regulation package, the Commission increasingly stresses that Better Regulation is not only the Commission’s responsibility. “The European Parliament and the Council should, therefore, mirror the Commission’s commitment to better regulation, as should Member States when transposing and implementing EU law.”⁴⁰xCommunication from the Commission, ‘Better Regulation for Better Results – An EU Agenda’, p. 4. Better Regulation is an interinstitutional commitment and also extends to the Member States. In the Interinstitutional Agreement (IIA) on Better Lawmaking of 2016, the three institutions commit to coordinating their evaluation activities (though without providing any specific procedures),⁴¹xIn contrast to the interinstitutional commitment on impact assessments, where the Council and EP commit to provide their own impact assessment on major amendments. to include evaluation requirements in new regulation and to systematically consider the use of review clauses and sunset clauses.⁴²xEuropean Parliament, Council of the European Union & European Commission, ‘Interinstitutional Agreement on Better Law-Making’, 13 April 2016, OJ L 123/1. This suggests a collaborative effort on Better Regulation that appears to be of a rather technical nature. However, the increasing focus on the interinstitutional dimension of Better Regulation also creates a context in which the institutions are expected to scrutinize each other’s evidence base more actively, which also allows blame-shifting and contestation. Providing evidence, ex ante and ex post, is a duty for all, making the evidence-gathering process potentially more politicized, as it can become more openly the object of (interinstitutional) contestation.

  IV Evaluation as a ‘Neutral’ General Principle of Better Regulation or Subordinated to Pre-Established Political Agenda?

  The Commission presents evaluation (and evidence-based policymaking more broadly) as a general principle of Better Regulation. This appears to give it the aura of a neutral procedural principle. However, there is a strong tension here in the Commission’s approach. On the one hand, it aims to ‘depoliticize’ evaluation and presents it as a neutral exercise of assessment that provides the evidence basis for political deliberation that would only follow subsequently. On the other hand, it has strongly linked evaluation to the political priority of reducing the regulatory burden. Whilst the Commission is proud to flag this as a political priority (and thus politicizes evaluation), it wants to avoid discussion on the value of such a political choice (thus avoiding politicization). The objectives of the evaluation system appear beyond discussion, whilst individual evaluations are said to be neutral as they should simply assess against the originally set objectives of a regulatory act, programme or project without debating such objectives. However, evaluation always implies choices about priorities, what to evaluate and how to do it. As I will analyse in the following section, whilst the Commission is keen to encourage participation to receive information on inefficiencies of existing policy, it is much less keen for the priorities and implicit choices of evaluation to be debated more broadly.

• E Politicization through Participation

  I Consultation

  In the words of the European Commission, “Better Regulation is not a bureaucratic exercise. Citizens, businesses and other stakeholders judge the EU on the impacts of its actions: not just new initiatives, but, even more importantly, the rules already in force” (emphasis added).⁴³xCommunication from the Commission, ‘Better Regulation for Better Results – An EU Agenda’, p. 4. Therefore, “the Commission intends to listen more closely to citizens and stakeholders, and be open to their feedback, at every stage of the process – from the first idea, to when the Commission makes a proposal, through the adoption of legislation and its evaluation” (emphasis added).⁴⁴x Ibid. That the Commission heralds broad consultation is not new, particularly since the 2002 White Paper on European Governance.⁴⁵xEuropean Commission, ‘European Governance – A White Paper’, 25 July 2001, COM(2001) 428 final. However, with the 2015 Better Regulation package, the Commission clearly goes a step further by supporting and institutionalizing participation and transparency throughout the entire policy cycle, including ex post evaluation.
  The 2015 Better Regulation Guidelines have considerably strengthened the participatory dimension of evaluation by requiring that the same principles and tools of consultation are applied to evaluation as to impact assessments. Participatory requirements have been a key feature of the Integrated Impact Assessment system ever since it was set up in 2003 and have always been clearly defined in the Impact Assessment Guidelines. For ex post evaluation, participatory requirements have been at best ad hoc, as part of the requirements set out to external consultants to whom evaluation tasks have been contracted out. The horizontal Evaluation Guidelines did not establish participation as a general principle of evaluation practice. Neither did they provide specific tools for it. According to a study by Mastenbroek et al. covering the period from 2002 to 2012, about 39% of the 216 evaluation reports on regulatory acts did not show stakeholder involvement at all, whilst in 51% stakeholders provided some information to the empirical part of the evaluation. Only in 9% did stakeholders have a more prominent role by providing feedback on various aspects of the study.⁴⁶xMastenbrouk et al., 2015, p. 13.
  Participation has long been perceived as something that came mainly after the evaluation had been made. By “publishing evaluation findings, the Commission is publicly taking responsibility for its actions, acknowledging how an intervention is performing and inviting further feedback”.⁴⁷xEuropean Commission, ‘Public Consultation on Commission Guidelines for Evaluation’, November 2013, p. 13, available at: <http://ec.europa.eu/smart-regulation/evaluation/consultation/index_en.htm>. The invitation for further feedback suggests a participatory approach to evaluation, but only after evaluation reports have been adopted.

  “The purpose of evaluations, namely to promote accountability/transparency and organisational learning, can only be achieved if the information produced by such evaluations reaches those to whom we are accountable to [sic] (general public, parliaments, etc.) or certain intermediaries (journalists) and those who should learn from the results. All evaluation reports of high quality should therefore be disseminated in a manner suited to the different audiences. Active discussion and debate on these findings should be encouraged.”⁴⁸x Ibid.

  Hence, the main effort was on ensuring the publication of evaluation reports online, although the central database of evaluations provided on the Commission’s evaluation website has been far from comprehensive and reports are often scattered around on pages of individual DGs. Evaluation would thus be politicized, but only by ex post political debate by parliament and interest groups to ensure accountability.
  The 2015 Better Regulation Guidelines take a decisively different approach by considering participation also essential during the evaluation process and imposing procedures for that. “Stakeholder’s views [sic], practical experience and supporting evidence can help deliver higher quality and more credible evaluations.”⁴⁹xEuropean Commission, ‘Stakeholder Consultation in the Context of Evaluation’, Better Regulation Toolbox, p. 280, available at: <http://ec.europa.eu/smart-regulation/guidelines/docs/br_toolbox_en.pdf>.
  Evaluations now have to follow the General Principles and Minimum Standards of Consultation, that were first set out in 2002 for ex ante policy drafting, and are now included in the 2015 Better Regulation Guidelines. Procedurally, participation in ex post evaluation is now also organized along the same procedural steps as participation in impact assessments.
  First of all, an interservice steering group (ISG) with representatives from DGs involved in the evaluation is established. The ISG has to draft an evaluation roadmap that summarizes the evaluation design, including clear statements on the subject, purpose and scope of evaluation, the issues to be addressed and the evidence to be gathered, including the consultation strategy. The Roadmap itself has to be made public and open to consultation for at least 4 weeks.⁵⁰xEuropean Commission, ‘Better Regulation Guidelines’, p. 77. Since June 2015 (1 month after the publication of the Better Regulation Guidelines) Evaluation Roadmaps are indeed published for consultation on the Commission’s Better Regulation webpage.⁵¹xEuropean Commission, ‘Roadmaps/Inception Impact Assessments’, available at: <http://ec.europa.eu/smart-regulation/roadmaps/index_en.htm-fbform>.
  However, consultation on the Roadmap is the start, not the end of the participatory process. The Guidelines state that it is essential to consult on all the mandatory evaluation criteria, which are effectiveness, efficiency, relevance, coherence and EU added value.⁵²xEuropean Commission, ‘Better Regulation Guidelines’, p. 72. This assumes that the consultation process extends to the evaluation exercise itself and not simply to the Roadmap setting out its general features. As is the case in ex ante policy drafting, the responsible DG (and the ISG in particular) has some flexibility on how to organize its consultation strategy in the evaluation process.
  However, it has to comply with the following:

  1. The General Principles of Consultation: consult as widely as possible, make the consultation process and how it has affected policymaking transparent, consult at a time where stakeholder views can still make a difference and ensure consistency of consultation processes across services.⁵³x Ibid., p. 65.

  2. The five Minimum Standards of consultation: clear content of consultation; ensure that all relevant parties have an opportunity to express their opinions; ensure adequate publication of its consultation strategy, including compulsory announcement of open consultations on the ‘Your Voice in Europe’ single access point; provide sufficient time for responses; and acknowledgement of feedback (including publication of contributions and results).⁵⁴x Ibid.

  3. The consultation process needs to include a 12 week internet based public online consultation at some point over the lifetime of the evaluation (which, corresponding the minimum standards set out above, needs to be announced on the ‘Your Voice in Europe’ website), unless the Secretariat General allows a derogation.⁵⁵x Ibid., p. 66.

  Participation therefore becomes a general principle throughout the entire evaluation. This implies at least two open online consultations, namely one on the Roadmap (with a consultation period for a minimum of 4 weeks) and a second one further along during the evaluation (with an 8-week consultation period). This “should be complemented by other approaches and tools in order to engage all relevant stakeholders and to target potential information gaps”.⁵⁶xEuropean Commission, ‘Stakeholder Consultation in the Context of Evaluation’, p. 280. Such complementary tools can be the consultation of existing expert groups or Member State committees, questionnaires, hearings or workshops. Each DG is also required to indicate a central evaluation contact point (e.g. an email address), which is responsible for answering questions from stakeholders on the planning, timing and progress of work on evaluations.⁵⁷xEuropean Commission, ‘Planning & The 5 Year Rolling Evaluation Plan’, Better Regulation Toolbox, p. 259, available at: <http://ec.europa.eu/smart-regulation/guidelines/docs/br_toolbox_en.pdf>.
  This new participatory approach to evaluation raises many challenges, some of a mere practical and others of a more substantive nature. I will start with the latter.
  Firstly, the new participatory approach politicizes evaluation in a very different way than was thus far the focus. Until now, interest politics and politicization were supposed to happen mainly once an evaluation report was produced. Evaluation was a rather independent exercise, to be done either by the Commission, or, in most cases, by an independent contractor. Such an ‘independent report’ would provide the evidence to ensure accountability, which could be ensured by parliamentary control and public debate. Hence, politicization happens ex post to evaluation and mainly to ensure accountability. By ensuring participation throughout the entire evaluation exercise, interest politics and politicization become a stronger feature of the entire process.
  Secondly, this raises practical questions, particularly as evaluation is traditionally outsourced to private consultancies. These consultancies lack the extensive experience the Commission has built up in relation to consultation practices for policy drafting and ex ante assessment. Can they quickly adjust to ‘copy’ these practices? Will the mandate for the contractors include detailed provisions on which consultation strategy to follow and who to consult?
  Thirdly, whilst the Minimum Standards require that ‘all relevant parties have an opportunity to express their opinion’, it remains to be seen whether the two online consultations are supposed to automatically cover that requirement.
  Fourthly, extra efforts are planned to ensure feedback on consultation during evaluation. Results of the consultation should be reflected in the contractors’ report (if applicable) and the evaluation staff working document (SWD) that constitutes the Commission’s evaluation report. A synopsis report covering all consultations launched needs to be annexed to the SWD.⁵⁸xEuropean Commission, ‘Stakeholder Consultation in the Context of Evaluation’, p. 281.

  II Participation in REFIT: The REFIT Platform and ‘Lighten the Load’

  In addition to the requirement for online consultations in evaluation, the most explicit institutionalization of participation in evaluation is via the creation of the REFIT Platform. The creation of a platform, including stakeholders, to advise on some aspects of the Better Regulation agenda is a clear element of politicization of issues that have long been approached as technical and bureaucratic in nature. However, it should immediately be stressed that the REFIT Platform has not been set up as a stakeholder platform to advise on all aspects of the BR agenda. Its focus is on the realization of the Regulatory Fitness and Performance Programme.
  The role of the REFIT Platform is to invite, collect and assess suggestions from all available sources on how to reduce the regulatory and administrative burden and make subsequent recommendations to the Commission and Member States.⁵⁹xEuropean Commission, ‘Commission Decision of 19.5.2015 Establishing the REFIT Platform’, C(2015) 3261 final. The language of the Commission Decision setting up the REFIT Platform, as well as its composition, confirms the narrow focus of the Platform. In theory, a platform of stakeholders could have been given the opportunity to voice concerns and make suggestions about the direction of the Better Regulation agenda more broadly. Even if focused on regulatory fitness, one could imagine a platform that can highlight problems of implementation and suggest solutions not only because rules are ‘burdensome’ on business or administrations, but equally because they do not reach their objectives. The latter could lead to suggestions for re-regulation or new regulation. Yet, whilst ‘effectiveness’ is an objective of the Better Regulation agenda, the REFIT Platform mandate does not mention it all. The focus is exclusively on reducing regulatory burden.
  Therefore, whilst aimed at stakeholder involvement, the REFIT Platform is actually expected to not politicize the debate. It has to work within the confines of the political priority already set out, namely reducing the regulatory burden, and is not expected to redefine what ‘better’, ‘fitter’ or ‘more effective’ regulation means. It has little to say on the direction of the Better Regulation agenda, and even when highlighting problems with individual regulatory decisions, the task seems to focus on highlighting when they are burdensome rather than suggesting solutions on how the set policy objectives of regulation can be achieved more effectively.
  This focus is also reflected in the composition of the Platform. It is composed of a ‘government group’, consisting of representatives of the Member States’ administration, and a ‘stakeholder group’, consisting of up to 20 experts, “two of them representing the European Economic and Social Committee and the Committee of the Regions, and the rest business, including from Small and Medium Enterprises (SMEs), and from social partners and civil society organisations having direct expertise in the application of Union legislation”.⁶⁰x Ibid., Art. 4(3). According to the Decision establishing the REFIT Platform, “the appointments [to the stakeholder group] shall ensure, to the extent possible, a balanced representation of the various sectors, interests and regions of the Union and gender”.⁶¹x Ibid., Art. 4(4).
  Among the stakeholders group of the REFIT Platform, one could argue that a balance has been sought between business and non-business interests. Nine members represent business associations: one from Business Europe, one Chamber of Commerce (Denmark), two SME associations (European and Dutch) and five associations representing particular industries (forest, food, household appliances, banking and technology). The ‘non-business’ stakeholder representatives could be divided into a group of six associations with a market-correcting objective (Friends of the Earth Europe, European Trade Union Confederation, European Consumer Association BEUC, European Citizen Action Service, Finance Watch and the British Royal Society of Bird Protection) and a group of three ‘institutional actors’ (a representative from the European Economic and Social Committee, from the Committee of the Regions and from the standardization bodies CEN+CENELEC). One member of the stakeholder group is appointed in a personal capacity (unlike all the others, who are said to represent the common interest of their stakeholder group),⁶²xAccording to Art. 4(3) of the Decision establishing the REFIT Platform, ‘the experts in the stakeholder group shall be appointed in their personal capacity or to represent a common interest shared by a number of stakeholders’. namely the Director of Europa decentraal, a Dutch non-profit knowledge centre that gives advice on the application of EU regulations to local and regional governments.⁶³xThe Platform composition is available online at: <https://ec.europa.eu/info/law/law-making-process/overview-law-making-process/evaluating-and-improving-existing-laws/reducing-burdens-and-simplifying-law/refit-platform_en>.
  Although the stakeholders group shows a relative balance between business and non-business interests, this does not guarantee a balance between opinions on whether Better Regulation should primarily be about reducing the regulatory burden or ensuring efficiency of regulation in terms of reaching its policy objectives. Four members of the stakeholder group (Friends of the Earth, ETUC, BEUC and Finance Watch), part of the Better Regulation Watchdog, have an openly critical approach to the risks of the concern for regulatory burden taking over the Better Regulation agenda and potentially undermining regulatory standards. With business representatives traditionally taking the opposite position, and other actors more ‘neutral’ on the issue, the regulatory burden priority might well be the dominant one even within the stakeholder group.
  The prioritization of the regulatory burden as an objective of REFIT is also reflected in the composition of the government group. Just half of the government representatives come from horizontal administrative units, such as an office of the prime minister, specific BR units, EU legislation units or other public institutions that ensure quality of legislation such as the Conseil d’Etat. The other half all come from ministries of economy or trade. Other departments, such as environment, are not represented in the government group. This obviously reflects where the core of the Better Regulation agenda is also embedded at the national level.
  It is also worth noting that, despite the requirement for the stakeholder group to ensure a balanced representation of regions of the EU,⁶⁴xEuropean Commission, ‘Commission Decision of 19.5.2015 establishing the REFIT Platform’, Art. 4(4). the Platform shows a stronger representation from Northern Europe as opposed to Southern, Central and Eastern Europe. Although there is obviously a representative for each country in the government group, the stakeholder group relies on a stronger experience with Better Regulation agenda’s in Northern Europe. Besides the EU-level associations and institutional actors, the associations come from The Netherlands (2), Germany, UK, Finland and Denmark, with ‘the South’ being present with a Portuguese association.
  The creation of the REFIT Platform goes hand in hand with the new online tool ‘Lighten the load – Have your say’. This website is a tool for citizens and businesses to provide feedback on EU policy and legislation at any moment in time. The Commission wants “to hear what people find irritating, burdensome, or in need of improvement”.⁶⁵xCommunication from the Commission, ‘Better Regulation for Better Results – An EU Agenda’, p. 6. Although the latter suggests that comments may also propose amendment and re-regulation, the focus seems very much on whether the existing regulatory framework is too ‘irritating’ or ‘burdensome’.
  The Commission replies directly to all entries on the Lighten the Load website and will refer the issue further to the REFIT Platform where it is considered if it merits further discussion and potential action. By 28 February 2017, most submissions to the website came from individual citizens, namely 163, followed by submissions by companies (18), business associations (18), public authorities (6), trade unions (1) and consumer organizations (1), with 14 submissions described as ‘other’. However, the majority of submissions do not provide the information the website is aiming for. Often they are statements on non-REFIT-related topics or requests for information, in which case the Commission refers to the appropriate Commission unit. Of the 163 submissions by citizens,⁶⁶xThe data are available online at: <http://ec.europa.eu/smart-regulation/refit/simplification/consultation/contributions_en.htm>. only nine were considered appropriate to be (potentially) taken into consideration for discussion by the REFIT Platform; whilst for a similar amount of cases, the Commission’s reply on the website would engage with the substance of the matter but explain why it would not be taken up further by the Platform. Entries by other categories fared slightly better; yet, most entries would not make it to discussion by the Platform; just one of the six public authority entries, three of the 18 company entries and three of the 14 ‘other’ category. The single entry by a consumer organization was considered worth further discussion, whilst the single entry by a trade union was not. The business associations’ submissions give most cause to potential discussion by the Platform with eight of the 18 submissions being potentially considered for Platform discussion. Additionally, a fair amount of other submissions from this group led to an online engagement on substance by the Commission, although it was not considered appropriate for further Platform discussion. However, overall, after 20 months since the website’s creation, the amount of valuable information that has reached the REFIT Platform via this path appears minimal. As the key participatory tool to receive feedback on how EU policy is implemented, and particularly on how it creates regulatory burden, the website does not yet appear to live up to its expectations.

  III Politicization through the Mobilization of Civil Society on Issues of Better Regulation

  The rise of Better Regulation as an issue on the political agenda has not gone unnoticed by civil society. Civil society organizations, such as environmental organizations, trade unions and consumer organizations, have commented on European governance issues for a long time. However, over the last 15 years, several European civil society organizations have been set up with the particular objective to focus on (some aspect of) the EU Better Regulation agenda. In 2005, the European Civil Action Service (ECAS) was set up with among its main objectives to bring the EU closer to its citizens and increase attention to good governance and transparency of EU policymaking. Transparency International also set up an EU office in 2008.
  In 2009, Corporate Europe Observatory moved its office from Amsterdam to Brussels,⁶⁷xThe Corporate Europe Observatory was originally set up in 1997. intensifying its research on the role of corporate lobbying in EU decision-making and asking for changes in the Better Regulation agenda. In 2011, Finance Watch was created with the support of the EP and in reaction to the financial crisis, in order to allow civil society to play a watchdog role on the quality of EU financial regulation. And finally, and most explicitly, a group of civil society organizations set up the Better Regulation Watchdog in 2015 with the objectives of closely following the developments on Better Regulation; increasing understanding among civil society, media and decision makers about the risks and potential negative impacts of the Better Regulation agenda on social, environmental, labour, consumer and public health standards for citizens and the public interest; and challenging the widely held belief that regulation is a burden for society.⁶⁸xBetter Regulation Watchdog, ‘Who are We’, available at: <www.betterregwatch.eu/about-us/>.
  Better Regulation, and evaluation, is thus no longer the realm of a small group of technocrats, but its wider orientation and societal impact are discussed by civil society organizations, some of which have been set up with the specific objective of focusing on (a part of) the Better Regulation strategy.

• F Conclusion: Ex post Evaluation; Vanguard against or Victim of Populism?

  With the 2015 Better Regulation package, evaluation has been placed centre stage on the Commission’s agenda. Evaluation has become a key tool to steer EU policymaking in function of the political priorities of the Commission and the objective of reducing the regulatory burden. The hierarchical embedding of evaluation, its more systematic planning and its scrutiny via the RSB all contribute to anchoring ex post evaluation to political priorities. The focus on linking ex post and ex ante evaluation, the application of evaluation to regulatory intervention as well as to expenditure policy and increased attention for the interinstitutional dimension of appraisal all contribute to further politicization. Finally, politicization is enhanced by increasing the participatory dimension of evaluation. However, the way the Commission organizes participation in evaluation also illustrates the ambiguous approach to the politicization of evaluation. Whilst interest groups and citizens are encouraged to contribute to evaluation, they are not supposed to discuss or put into question the objectives of the evaluation process itself. Participation has to function within the political parameters already set by the Commission. From that perspective, the Commission presents evaluation as a neutral procedural exercise and it fears politicization that would contest the underlying political orientation of the pretended neutral exercise. However, some civil society organizations have mobilized to do that exactly, that is politicize the debate on evaluation by pointing out that appraisal is not a neutral exercise.
  We are at a crucial moment in time where populism risks undermining the basic principles of liberal democracy and the pillars on which modern governance is built. Evaluation can both fall victim to and be a vanguard against such populism. On the one hand, one has to be cautious that evaluation may end up being a tool to appease populist discourse. The EU tendency to use evaluation strongly in function of reducing the regulatory burden is a risky strategy in this regard. Populist movements all over Europe have taken EU regulatory action as one of their primary targets to justify retreat from and demise of the EU, in order to ‘take back control’ and defend nationalist, protectionist and exclusionary policies. These movements all describe the European regulatory framework as a useless set of rules, simply there to satisfy the interests of a self-serving bureaucracy and corrupt elite. By increasingly stressing the ‘regulatory burden’ of EU policy, the EU risks simply strengthening a populist discourse that is not evidence based. The Juncker Commission’s prioritization of Better Regulation to reduce the regulatory burden in reaction to the rise of Euroscepticism in the 2014 EP elections might therefore backfire. In fact, populism and Euroscepticism have been on the rise since the 2014 elections, and it is doubtful that the Better Regulation agenda has contributed to weakening rather than strengthening these tendencies. From this perspective, the politicization of evaluation is problematic, as it turns a process of (technocratic) learning into a (failed) strategy to appease populism.
  At the same time, evaluation is a key tool of evidence-based policymaking required to falsify the claims of populist discourse. A more participatory approach to evaluation facilitates this role of appraisal. However, such politicization also requires that the objectives of the evaluation process itself are not beyond discussion but open to democratic debate.

Noten

• * The research leading to this article has received funding from the European Research Council under the European Union’s Seventh Framework Programme (FP/2007-2013)/ERC Grant Agreement n. 313642-LASI (“Law, science and interests in European policy-making”).

• 1 I use the expression ‘Better Regulation Package’ for three documents adopted together in May 2015 which set out the European Commission’s new Better Regulation approach and tools; Communication from the Commission, ‘Better Regulation for Better Results – An EU Agenda’, 19 May 2015, COM(2015)215 final (which sets out the general philosophy of the new BR approach); European Commission, ‘Better Regulation Guidelines’, 19 May 2015, SWD(2015) 111 final (which sets out the compulsory aspects of the BR strategy) (further referred to as BR Guidelines 2015); and the ‘Better Regulation Toolbox’, attached to these Guidelines, which set out complementary guidance to assist practitioners in the application of the Guidelines (further referred to as BR Toolbox 2015).

• 2 European Commission, ‘Better Regulation Guidelines’, COM(2015) 215 final, p. 49.

• 3 B. Guy Peters, ‘Politicisation: What Is It and Why Should We Care?’, in C. Neuhold, S. Vanhoonacker & L. Verhey (Eds.), Civil Servants and Politics, Basingstoke, Palgrave Macmillan, 2013, pp. 12-24.

• 4 W. van der Brug et al., ‘A Framework for Studying the Politicisation of Migration’, in W. van der Brug et al. (Eds.), The Politicisation of Migration, London, Routledge, 2015, p. 2.

• 5 Electoral success rather than best policy is suspected as main driver of action. Similarly, one has talked about the politicization of development or humanitarian aid, to criticize that the primary objective of providing aid is driven and undermined by more dubious, or at least morally less clear, political motives. See M. Haneef Atmar, ‘Politicisation of Humanitarian Aid and Its Consequences for Afghans’, Disasters, Vol. 25, No. 4, 2001, pp. 321-330.

• 6 T.D. McGarity & W.E. Wagner, Bending Science. How Special Interest Corrupt Public Health Research, Cambridge, Harvard University Press, 2008; M. Everson & E. Vos, ‘The Scientification of Politics and the Politicisation of Science’, in M. Everson & E. Vos (Eds.), Uncertain Risks Regulated, London, Routledge, 2009, pp. 1-18.

• 7 S. Raman, ‘Science, Uncertainty and the Normative Question of Epistemic Governance in Policymaking’, in E. Cloatre & M. Pickersgill (Eds.), Knowledge, Technology, and Law, London, Routledge, 2015, pp. 17-32. For a critical analysis of technocratic theory as a normative theory, see C.M. Radaelli, Technocracy in the European Union, London, Longman, 1999, pp. 11-28.

• 8 G. Majone, Regulating Europe, London, Routledge, 1996.

• 9 European Commission, ‘Evaluating EU Activities: A Practical Guide for the Commission Services’, Directorate-General for the Budget, July 2004.

• 10 S. Højlund, ‘Evaluation in the European Commission: For Accountability or Learning?’, European Journal of Risk Regulation, Vol. 6, No. 1, 2015, pp. 35-46.

• 11 Communication from the Commission, ‘Responding to Strategic Needs: Reinforcing the Use of Evaluation’, 21 February 2007, SEC(2007) 213.

• 12 Communication from the Commission, ‘Smart Regulation in the European Union’, 8 October 2010, COM(2010) 543 final.

• 13 Communication from the Commission, ‘Strengthening the foundations of Smart Regulation – Improving Evaluation’, 2 October 2013, COM(2013) 686 final.

• 14 E. Mastenbroek, S. van Voorst & A. Meuwese, ‘Closing the Regulatory Cycle? A Meta-Evaluation of Ex-post Legislative Evaluations by the European Commission’, Journal of European Public Policy, Vol. 23, No. 9, 2015, p. 10.

• 15 Communication from the Commission, ‘EU Regulatory Fitness’, 12 December 2012, COM(2012) 746 final.

• 16 Communication from the Communication, ‘Regulatory Fitness and Performance (REFIT): Results and Next Steps’, 2 October 2013, COM(2013) 685 final, p. 2.

• 17 Communication from the Commission, ‘Smart Regulation in the European Union’, p. 4.

• 18 Communication from the Commission, ‘Strengthening the Foundations of Smart Regulation – Improving Evaluation’, p. 5.

• 19 European Commission, ‘Evaluating EU Activities: A practical guide for the Commission Services’.

• 20 European Commission, ‘Better Regulation Guidelines’, p. 4.

• 21 A. Wille, The Normalization of the European Commission: Politics and Bureaucracy in the EU Executive, Oxford University Press, 2013, p. 65.

• 22 S. Kurpas, C. Grøn & P.M. Kaczyinski, ‘The European Commission after Enlargement: Does More Add Up to Less?’, CEPS Special Report, Centre for European Policy Studies, February 2008; M. Egeberg, ‘The European Commission’, in M. Cini & N. Pérez-Solórzano Borragán (Eds.), European Union Politics, Oxford, Oxford University Press, 2015, p. 129.

• 23 Wille, 2013, p. 66.

• 24 For more about the politicization resulting from the Spitzenkandidaten system, see C. Antpöhler (2015), ‘Enhancing European Democracy in Times of Crisis? – The Proposal to Politicise the Elections of the European Commission’s President’, in F. Fabbrini, E. Hirsch Ballin & H. Somsen (Eds.), What Form of Government for the European Union and the Eurozone?, London, Hart Publishing, 2015, pp. 217-232; A. Kocharov, ‘In the Image of the State: Constitutional Complexities of Engineering a European Democracy’, in F. Fabbrini, E. Hirsch Ballin & H. Somsen (Eds.), What Form of Government for the European Union and the Eurozone?, London, Hart Publishing, 2015, pp. 233-251.

• 25 European Commission, ‘Commission Work Programme 2015: A New Start’, COM(2014) 910 final, p. 2.

• 26 Ibid.

• 27 Whether it is the EU level or the Member States to blame for it is another question.

• 28 European Commission, ‘State of the Union Address 2016: Towards a better Europe – A Europe That Protects, Empowers and Defends’, Speech by the European Commission, Strasbourg, 14 September 2016.

• 29 Communication from the Commission, ‘Better Regulation for Better Results – An EU Agenda’, p. 10.

• 30 See also Section V.II regarding the ‘Lighten the Load’ tool in the context of REFIT.

• 31 European Commission, ‘Better Regulation Guidelines’, p. 14.

• 32 Ibid., p. 9.

• 33 Although the RSB also checks evaluations ‘indirectly’ as part of its assessment of impact assessments, in 2016, it thus assessed an additional 15 evaluations in this way. European Commission, Regulatory Scrutiny Board Annual Report 2016, p. 17.

• 34 Ibid., p. 16.

• 35 Ibid., p. 17.

• 36 M. Scriven, ‘Beyond Formative and Summative Evaluation’, in M. McLauglin & D.C. Philips (Eds.), Evaluation and Education: At Quarter Century, Chicago, University of Chicago Press, 1991; F.-B. Van der Meer & J. Edelenbos, ‘Evaluation in Multi-Actor Policy Processes: Accountability, Learning and Co-operation’, Evaluation, Vol. 12, No. 2, 2006, p. 201; S. Borrás & S. Højlund, ‘Evaluation and Policy Learning: the Learners’ Perspective’, European Journal of Political Research, Vol. 54, No. 1, 2015, p. 99.

• 37 See also P. Stephenson in this special issue.

• 38 The classical argument of the EU as a regulatory state is based on this premise. See Majone, 1996.

• 39 H. Summa & J. Toulemonde, ‘Evaluation in the European Union: Addressing Complexity and Ambiguity’, in J. Furubo, R.C. Rist & R. Sandahl (Eds.), International Atlas of Evaluation, London, Transaction Publishers, pp. 407-424; T. Fitzpatrick, ‘Evaluating Legislation: An Alternative Approach for Evaluating EU Internal Market and Services Law’, Evaluation, Vol. 18, No. 4, 2012, pp. 477-499; Mastenbrouk et al., 2015, p. 4.

• 40 Communication from the Commission, ‘Better Regulation for Better Results – An EU Agenda’, p. 4.

• 41 In contrast to the interinstitutional commitment on impact assessments, where the Council and EP commit to provide their own impact assessment on major amendments.

• 42 European Parliament, Council of the European Union & European Commission, ‘Interinstitutional Agreement on Better Law-Making’, 13 April 2016, OJ L 123/1.

• 43 Communication from the Commission, ‘Better Regulation for Better Results – An EU Agenda’, p. 4.

• 44 Ibid.

• 45 European Commission, ‘European Governance – A White Paper’, 25 July 2001, COM(2001) 428 final.

• 46 Mastenbrouk et al., 2015, p. 13.

• 47 European Commission, ‘Public Consultation on Commission Guidelines for Evaluation’, November 2013, p. 13, available at: <http://ec.europa.eu/smart-regulation/evaluation/consultation/index_en.htm>.

• 48 Ibid.

• 49 European Commission, ‘Stakeholder Consultation in the Context of Evaluation’, Better Regulation Toolbox, p. 280, available at: <http://ec.europa.eu/smart-regulation/guidelines/docs/br_toolbox_en.pdf>.

• 50 European Commission, ‘Better Regulation Guidelines’, p. 77.

• 51 European Commission, ‘Roadmaps/Inception Impact Assessments’, available at: <http://ec.europa.eu/smart-regulation/roadmaps/index_en.htm-fbform>.

• 52 European Commission, ‘Better Regulation Guidelines’, p. 72.

• 53 Ibid., p. 65.

• 54 Ibid.

• 55 Ibid., p. 66.

• 56 European Commission, ‘Stakeholder Consultation in the Context of Evaluation’, p. 280.

• 57 European Commission, ‘Planning & The 5 Year Rolling Evaluation Plan’, Better Regulation Toolbox, p. 259, available at: <http://ec.europa.eu/smart-regulation/guidelines/docs/br_toolbox_en.pdf>.

• 58 European Commission, ‘Stakeholder Consultation in the Context of Evaluation’, p. 281.

• 59 European Commission, ‘Commission Decision of 19.5.2015 Establishing the REFIT Platform’, C(2015) 3261 final.

• 60 Ibid., Art. 4(3).

• 61 Ibid., Art. 4(4).

• 62 According to Art. 4(3) of the Decision establishing the REFIT Platform, ‘the experts in the stakeholder group shall be appointed in their personal capacity or to represent a common interest shared by a number of stakeholders’.

• 63 The Platform composition is available online at: <https://ec.europa.eu/info/law/law-making-process/overview-law-making-process/evaluating-and-improving-existing-laws/reducing-burdens-and-simplifying-law/refit-platform_en>.

• 64 European Commission, ‘Commission Decision of 19.5.2015 establishing the REFIT Platform’, Art. 4(4).

• 65 Communication from the Commission, ‘Better Regulation for Better Results – An EU Agenda’, p. 6.

• 66 The data are available online at: <http://ec.europa.eu/smart-regulation/refit/simplification/consultation/contributions_en.htm>.

• 67 The Corporate Europe Observatory was originally set up in 1997.

• 68 Better Regulation Watchdog, ‘Who are We’, available at: <www.betterregwatch.eu/about-us/>.