Many people would agree that talking about the new Performance Monitoring and Evaluation Framework (PMEF) of the CAP Strategic Plan Regulation (SPR) is almost as entertaining as reading your annual tax declaration. At least they have the same number of acronyms.
Jokes aside, the new PMEF is an extremely important tool of policy accountability and learning. It is one of the core parts of the CAP reform that should concern every EU taxpayer, not only statistical offices, evaluation consultants, and other official authorities. Arc2020 has provided an update on the state of play with the preparation of the new PMEF and highlighted some areas which deserve more attention in the future works on the new PMEF, namely: 1.) Supporting the use of digital technologies to monitor and evaluate the CAP; 2.) Building a stronger data infrastructure (interoperability and open data), and 3.) Ensuring transparency of documents and procedures.
PMEF: what is new and what is the same?
The PMEF is composed of a set of elements, including common objectives for both CAP Pillar I and II, common indicators, quantified targets and milestones, data management and reporting activities, and incentive mechanisms to reward good performance and address deviations from planned targets and milestones. Similar to the current programming period, the PMEF is set up at EU level with common elements, but it will need to be further adapted and developed in the specific Member States.
With the apparent shift towards a simplified and result-oriented performance approach of the CAP post-2020, the new PMEF has a number of differences compared to the current programming period. Some examples are the lack of a common intervention logic at EU level, or the lack of a common set of evaluation questions to guide the new PMEF across all the Member States. The withdrawal of these two elements has been justified by the Commission with the new CAP delivery model, based on higher flexibility and subsidiarity for the Member States in terms of designing the CAP Strategic Plans. It is also worth noticing the changes in terms of reporting requirements (e.g. Performance clearance, performance review, CAP monitoring), the role of certifying bodies, or the interpretations of some indicators (e.g. new result indicators rarely going beyond output, and capable of capturing the immediate changes after the delivery of an intervention).
To date, the European Court of Auditors has already expressed several concerns on the “fitness” of the new PMEF (full text here and here). Moreover, looking at the PMEF as a jigsaw puzzle, the positions taken by the Member States on the current proposal range from those who still see it as a bundle of disconnected pieces, those who see fundamental pieces missing to get the ‘big picture’, and those who wants to reduce the number of pieces with a view of simplifying and improving the communication and performance measurement. The experience with the current rural development programmes shows that the construction of the common monitoring and evaluation system is a nitty gritty process: it might take longer than expected and the details might need to be established in ‘implementing acts’. Of course, this means disappointing some Member States that would prefer to see the entire PMEF already carved in stone in the ‘basic acts’, which could be adopted already by the end of 2020. Whatever legislative shape it will take, a lot of work has already been done to put the puzzle together. Below is a short summary.
How is the work on the PMEF proceeding?
Not unlike the other chapters of the CAP reform, the legislative work on the PMEF proceeds in parallel between the Council and European Parliament, who are making amendments and discussing the Commission proposal in their negotiating teams. Beyond the work of the co-legislators, and apart from the ongoing work with statistical offices and research centres, the details of the new PMEF are also being discussed and elaborated in:
• Expert group meetings chaired regularly by the Commission to coordinate and exchange views with Member States on the preparation of legislative proposals;
• Thematic working groups, tools, and workshops between the Commission, representatives of the Member States, and technical support units (e.g. European Evaluation Helpdesk for Rural Development).
There have been at least six Expert Group meetings since the 12 March 2018, when the PMEF was, for the first time, introduced to the Member States. These meetings do not exert decision making powers but provide a forum to clarify open questions and better define numerous elements of the PMEF. Since then, the Member States expressed several concerns on the new PMEF, such as:
• Poor clarity in the methodology and definition of some indicators, e.g. digitizing agriculture (R.3), growth and jobs in rural areas (R.31), or developing the rural bioeconomy (R.32);
• Inconsistencies between the terminologies used in the indicator fiches and the legislative proposals;
• The lack of a common approach in building the CAP intervention logic, i.e. linking interventions to objectives and indicators, and avoiding double counting;
• The need to define unit amount in a flexible way, especially for CAP interventions related to investments and LEADER.
Needless to say that the discussions taking place in these Expert group meetings share many things in common with those happening in the Council. The level of technicalities can sometimes be very intense and difficult to follow. When the use of acronyms goes up, it is very easy to get lost in translations. Nevertheless, it is not impossible to understand some common trends in these discussions.
Some Member States, under the shield of simplification and proportionality, are trying to get away from (often the same) reporting requirements and indicators (e.g. especially on energy savings, water efficiency, biodiversity) without a more ambitious counterproposal. On the other hand, the Commission´s response so far has not been easily accessible in the revised documents. Most likely, we will see higher flexibility and extended definitions of result indicators, whose planned targets and milestones will be easily hit and visually displayed in showy dashboards. It is true that the collection and quality of data are very important limiting factors for a more accurate, ambitious and useful PMEF at EU and Member State level, but there are more than technical and data reasons to explain the downsizing of the “E” in the new PMEF.
In addition to expert groups meetings, various workshops and thematic working groups have been facilitated by the European Evaluation Helpdesk for Rural Development to support the Member States and the Commission in getting ready for the new PMEF. One example is the tool recently published at the end of March to guide the ex-ante evaluators in the appraisal of the planned monitoring, data collection and implementation arrangements of the CAP Strategic Plans.
What still needs to be done for the preparation of new PMEF?
There are numerous other issues concerning the new PMEF, so it is important to touch upon some of the most overlooked issues so far too.
Supporting the use of digital technologies to monitor and evaluate the CAP
Despite the audit study of the European Court of Auditors on the use of new imaging technologies, as well as various research projects suggesting the potential application of digital tools to monitor and evaluate the CAP (e.g. NIVO) there is still large space for the Commission to support the wide uptake in the Member States. Digital technologies like Sentinel satellite constellation, drones, or geo-tagged photos can increase the quality of data collection in terms of volume, velocity, exhaustivity, resolution and identification. Without opening a philosophical debate on big data, it is fair to acknowledge that digital technologies will not be a panacea for the problems in the CAP monitoring and evaluation. However, digital technologies might offer new options for solving systemic problems on data in the CAP and rural development, especially in regionalized countries where official and up-to-date statistics at levels below NUTS-4 are often missing. It is important that this area is adequately developed in the Member States, with the assistance of specialists and support units.
Building a stronger data infrastructure in the CAP Strategic Plan: interoperability and open data
For those who tried assess the evaluation of the RDPs or CAP, one of the biggest challenges in carrying out a sound analysis, is not necessarily about the access to a specific dataset (e.g. FADN), but the interoperability and linkages between multiple data and information systems (e.g. Paying agency, environmental statistics). If the Commission wants to ensure that the Member States continue to build capacity in implementing robust evaluation methodologies, it is important to go on with investing in open and linked data systems. Open data in CAP Strategic Plans shall not be confused with free access to private and sensitive information about farmers. Some legislation on open data in European Structural and Investment Funds is already in place. However, there exist tools, standards and principles which can improve the format and access of open government data, and make it more easily used by independent evaluators. These could be promoted also in capacity buildings and tools to appraise the monitoring arrangements of the CAP Strategic Plans.
Ensuring transparency of documents and procedures on the new PMEF
It is true that numerous institutions (e.g. Commission, Managing Authorities, NRN) are working in parallel on evaluation activities of both the current and future programming periods. This requires extra human efforts and create higher workload to the personnel. However, this can also put pressure on and challenge some neglectable aspects, like ensuring the transparency of documents and the inclusion of civil society, NGOs or researchers through the process. Ensuring a transparent approach means making important documents like the minutes of meetings, presentations, and agenda publicly available and in a timely manner. Often, this is not only a good practice, but also a legal requirement, e.g. art. 13(3) to the Commission Decision establishing rules on the transparency of Expert Groups. A high transparency of documents shall not be circumscribed at EU level, where internal working documents are sometimes available only to officials, but also at national and regional level. To the extent that the communication about the performance of the CAP matters to society at large, it is important to open the PMEF up to scrutiny right from its preparatory phase.
More on ARC2020 on CAP
Council of Ag Ministers meet on Covid19 | Green Lines, State Aid, CAP Extension
