Quality project design: the NEW Logframe according to the July 2015 edition of the European Commission PRAG manual

The Logframe is the key tool for successful project design – as well as demanding exercise – to say the least!  PRAG is the European Commission crucial document for procedures in the external action domain. Massimo Rossi[1], Independent consultant and Lecturer on Planning and Evaluation comments on the new format of the Logframe and underline the differences with respect to the previous version. 

Do you want to increase the quality of your proposals? Take advantage of the EuropaBook experts’ network.

Stay in touch with EuropaBook news: register to the newsletter.

 

The EC published an updated Procedures and Practical Guide (PRAG) in 2015 (European Commission, 15 July 2015). This Guide introduced a number of innovations in the Logical Framework Matrix (Logframe) without any consultation with the concerned actors, using, e-g. the capacity4dev or other on-line tools, nor did it make any reference to methods, such as Project Cycle Management (PCM), that includes the Logical Framework Approach (LFA), or such as Results Based Management (RBM) or Managing for Development Results (MfDR). 

Additionally, this decision was taken without presenting a clear rationale in the PRAG and, more in general, without the previously planned publication of an updated Project and Programmes Cycle Management (PPCM), replacing the present in-use PCM Guidelines (European Commission, March 2004), as announced by the EC many times.

The PRAG introduces a revised Logframe , but, shifting the focus from only one Project Purpose to multiple Specific objectives/Outcomes implies changing not only a tool as the Logframe, but also a method as the LFA, “coming back “ to over-ambitious design approaches. The latter, presenting many possible specific objectives and pretending to address too many problems, was abandoned by the EC in the first PCM Manual (EC, 1993), as well as by other Donors and Agencies, based on the evidence of risks of failure linked to over-ambitious projects and programmes.

The aim of the present article is to contribute to the clarification on the innovations introduced by the new PRAG, and more in general, to the improvement of methods and tools for PPCM.

  1. Considerations on the revised Logframe

The following considerations refer to various issues of the Logframe such as: Specific objective(s)/Outcome(s), Outputs, Results, Activities, Baseline, Current value, Target, Means and costs, Assumptions.

Specific objective(s)/outcome(s)

In the PRAG’s Intervention Logic (IL) (“Results Chain” in the  “Indicative Logframe” presented in the EC Unified Action Document Template[2] (UADT) (European Commission, 1 January 2015) column the level between Outputs and Overall objectives is “Specific objective(s)/Outcome(s)”. This refer to possible multiple purposes, as clearly indicated by the parenthesis following outcome and specific objective in the title and by the example provided in the text: “Outcome = Oc (Oc 1; Oc 2; etc.)” (European Commission, 15 July 2015, Annex E3d, page 1). In UADT text and annexed Logframe reference is clearly made to Specific objective(s)/Outcome(s).

Specific objective(s)/Outcome(s) are defined in the IL column as follows:  “The direct effects of the project which will be obtained at medium-term and which tend to focus on the changes in behaviour resulting from project” (bold is in the text)”. Another definition is provided in the legenda of the Logframe: “ “Outcome” means the likely or achieved short-term and medium-term effects of an Action’s outputs” (European Commission, 15 July 2015, Annex E3d, page 1). This definition is not consistent with that one provided in the IL because it refers to medium and short-term periods.

In PCM Guidelines (European Commission, March 2004) and in ROM Handbooks (European Commission, April 2012, European Commission, March 2015) the PRAG’s Specific objective(s)/Outcome(s) are called “Project Purpose” or “Outcome”, using the singular. In particular, the ROM 2015 Handbook illustrates a Logframe with an “Outcome” singular,  despite the reference made to UATD (using Outcomes plural) in the presentation. In the same PRAG, in different sections of the document, reference is made to only one Outcome/Purpose:

in § 3.3 concerning Full Application Evaluation Grid: “The outcome of the action” (European Commission, 15 July 2015, Annex E5b, page 4) .

in the Glossary no definition is provided for outputs or for Specific Objective/Outcome, but target groups are defined as follows: “ The groups/entities that will be directly positively affected by the project at the Project Purpose level” (PRAG, 15 July 2015, Annex A1a Glossary, page 10).

Concerning the issue of unique Purpose or multiple Purposes, development aid Manuals in use, based on RBM (FAO, 2012), MfDR (IFAD, 2008 UNDP, 2009) or LFA (DFID, 2011) agree on the identification in the Results Chain (RC) of only one Purpose, called Outcome. In addition to these Manuals, Theory of Change (ToC) (DFID, April 2012) refers, in its RC/sequence of change, to only one Outcome. ToC also identifies the need to add an intermediate level between the Outputs and the Outcome: the “Missing middle”. The mentioned manuals are based on a worldwide evidence: successful actions use modest design models with only one Purpose, and failures are very often related to over-ambition.

In conclusion, Purpose/Outcome, the more relevant level of the IL/RC since it refers to the benefits for the target groups, is not clearly addressed in the PRAG. That circumstance is a source of confusion among development actors for correct target group identification and targeting.

Outputs

Outputs are defined in the IL of the PRAG Logframe: The direct/tangible outputs (infrastructure, goods and services) delivered by the project” (bold is in the text). A different definition is provided in the legenda of the  Logframe: ““Output” means the products, capital goods and services which result from an Action’s activities” (European Commission, 15 July 2015, Annex E3d, page 2).

The IL sequence is: outputs  → outcomes → overall objective.

During the implementation of the EC Research and Technological Development Framework Programmes the design approach used over the years was the Work Packages approach (WPs). WPs sequence identified the level of Outputs (called “Deliverables”) as an intermediate level between Activities (called Tasks) and Results. The IL sequence was: deliverables → results → objectives.

EuropeAid LFA and WPs approach present different positive aspects. Based on the mutual learning from the latter, it was proposed that deliverables be adopted as a useful intermediate level in the IL of LFA and that LFA be used to improve the WP approach, shifting in this way from WPs to Logical Work Packages (LWPs) (Rossi M. Sycamore C. 2006). LWPs were applied by the Associazione per la Promozione della Ricerca Europea (APRE) in project proposal designs presented to the EC.

The proposed IL sequence was: deliverables → results → purpose→ overall objectives.

In the 2012 ROM Handbook (European Commission, April 2012), Outputs are indicated as intermediate levels between Activities and Results. In this Handbook, efficiency analysis is related to achievement of Outputs and effectiveness analysis is related to achievement of Results.

The IL sequence was: outputs → results → purpose → overall objectives.

EC new Research and Innovation Programme Horizon 2020 confirmed the WPs approach, but in the WPs sequence the level of Results was unfortunately eliminated (European Commission, May 2015), and therefore the IL sequence is: deliverables →objectives.

In UADT the IL sequence is the same as in PRAG: outputs  → outcomes → overall objective. For “Results” it is specified: “ In line with OECD terminology the term results should be understood as covering the three levels of results chain, i.e output, outcome and impact” (European commission, 1 January 2015, note 19, page 13).

In the 2015 ROM Handbook (European Commission, March 2015) the level of Results was unfortunately eliminated, and the IL sequence is: outputs → outcome → overall objective.

Results

In the revised PRAG Logframe results disappear. This lack of intermediate level between outputs/deliverables and purpose/outcome is consistent with the Horizon 2020 new WPs approach, UATD and and with the 2015 ROM Handbook. But “Results”, not “Outcomes” is used in the following sections of the PRAG:

The § 4.3, where reference is made to “Expected results” and not to Outputs within the Full Application Evaluation Grid.

The legenda of the Logframe, where the definition of Indicators refer to Results not to Outputs: “Indicator” is the quantitative and/or qualitative factor or variable that provides a simple and reliable means to measure the achievement of the Results of an Action” (European Commission, 15 July 2015, Annex E3d, page 2).

Activities

Activities were included in the Logframe in the more recent PCM Manual as “Optional within the matrix itself” (European Commission, March 2004, page 73),  whereas they were included in the Logframe without remarks in the previous Manual (European Commission, 2001).

In the PRAG Activities are included in the Logframe without any remarks either. This inclusion is inconsistent with the UADT, with the 2015 ROM Handbook and with RBM that do not mention activities in the Results Chain of the Results Matrix.

 

Baseline, Current value, Target

Positive innovations are:

The inclusion of Baseline, Current value and Target as new columns in the Logframe.

The inclusion of gender specificity.

Despite the positive aspects, the innovations are not presented in a clear way and they are inconsistent with the ROM 2015 Handbook presentation :

In the  Logframe, Baseline e Current value are presented in two different columns.In the legenda of Logframe , Baseline e Current value are presented as follows :“Baseline” means the starting point or current value of the indicators” (European Commission, 15 July 2015, Annex E3d, page 2).

The three columns of the ROM 2015 Handbook are: Indicators, Baselines, Targets.

 

Means and costs

Means and costs were included in the Logframe in PCM Manual in 2001, but no more in the PCM Manual 2004 edition.

PRAG introduces Means and costs in the Logframe again. UADT does not include Means and costs in the Logframe.No other Aid Development Manual includes these issues in the Logframe or in the Results Matrix.

Assumptions

ToC emphasises the high relevance of analysis of Assumptions expressed by different stakeholders, making Assumptions explicit: “ Getting depth and critical thinking on assumptions is widely agreed to be the crux of the theory of change”(DFID, April 2012, page 22).

In an opposite perspective, RBM eliminated the Assumptions column from the RM, rigidly focusing only on a linear cause-effect relationship, as presented in the RC. The RM, also called Results Framework, is an example of over-simplification of any complex reality:  “A result framework, a simplified version of the traditional logical framework, depicts causes and effect relationships in development projects” (World Bank-FAO Investment Center, 2010, page 35).

It is therefore a positive concern that PRAG includes Assumptions as a column of the Logframe.

Conclusions and proposals

The innovations introduced in the revised Logframe as a PRAG Annex, concerning the multiple specific objectives to be identified as well as other specific aspects, are not clearly presented and justified, and they are not consistent with the in-use 2004 PCM Manual, the 2015 UADT and the 2015 ROM Handbook.

Uncertainty and confusion are in this way generated and consequently the quality of formulation, management, monitoring and evaluation will be affected. Additionally, the innovations introduced in the revised Logframe are not consistent with Manuals produced by other donors.

A revision of the EC operational guidance is on-going, as stressed in a 2014 EC Call for tender concerning methodological support regarding the programming, design, implementation, monitoring and evaluation of development actions: “A set of three documents (Common features of Budget Support and Project Modality, core methodological guidelines and would replace the previous sets of guidelines on Project Cycle Management (2004), Support to Sector Programmes (2007) and the previous Budget Support Guidelines (2010, already redesigned and published in 2012) bringing them together into one coherent set of Manuals” (European Commission, 2014, page 3, underlining is in the text).

The background section of Terms of reference of this Call for tender concludes that the methodological revision and the knowledge sharing process, quoting “It will also facilitate the harmonisation of practices by developing and sharing EC guidance with other donors” (page 4).

In line with the mentioned EC ongoing comprehensive methodological revision and with the need for harmonization, it is proposed that the EC:

Learn from criticism (as the remarks contained in the present article) and “embrace the error”.

Recognize in particular that the Purpose/Outcome, the more relevant level of the IL/RC since it refers to the benefits for the target groups, is not clearly presented in the PRAG. Additionally, PRAG, presenting Specific objectives/Outcomes is inconsistent with the ROM 2015 Handbook illustrating a Logframe with only one Outcome. That circumstance is a source of confusion among development actors for correct target group identification and targeting.

Publish as soon as possible an errata corrige note in PRAG, presenting a revised Logframe.

 

Bibliography

DFID, 2011, Guidance on using the revised Logical Framework.

DFID (by Isabel Vogel), April 2012, Review of the use of “Theory of Change” in the international development, Review Report.

European Commission, International Cooperation and Development, PRAG Practical Guide, 15 July 2015.

European Commission, Horizon 2020, 21 May 2015, Research and Innovation Actions (RIA), Research proposal, Part B.Brussels.

European Commission, Directorate-General for International Cooperation and Development, March 2015, ROM Handbook Results Oriented Monitoring, Brussels.

European Commission, 1 January 2015, Unified Action Document Template, Brussels.

European Commission, 2014, restricted Call for tender Methodological and knowledge sharing support  to enhance capacities for quality and results in EU external assistance, Publication Reference: EuropeAid/135767/C/SER/Multi, Terms of Reference.

European Commission-EuropeAid, Updated version, April 2012, Handbook for Results-oriented Monitoring of EC External Assistance (projects and programmes) , Brussels.

European Commission-EuropeAid Co-operation Office, March 2004, Guidelines on Aid Delivery Methods, Volume 1: Project Cycle Management, Brussels.

European Commission-EuropeAid Co-operation Office, March 2001,Manual-Project Cycle Management,  Brussels.

European Commission, February 1993, Project Cycle Management, Integrated Approach and Logical Framework-Manual, Brussels.

FAO, 2012, Guide to the Project Cycle, Quality for Results, FAO, Rome.

IFAD, 2008, An overview of managing for development results at IFAD, Rome.

Rossi M. Sycamore C., 2006, Utili idee di ricerca e ricerca di progetti utili. Alcune indicazioni per la progettazione nel VII Programma Quadro di Ricerca e Sviluppo Tecnologico: applicazioni dell’Approccio del Quadro Logico, “Formazione e Cambiamento, N. 43, 0ttobre 2006.

UNDP, 2009, Handbook on Planning, Monitoring and Evaluating for Development Results, New York.

World Bank-FAO Investment Center, 2010, The use of monitoring and evaluation in agriculture and rural development projects, Rome.

 

[1] Independent consultant. Lecturer on Planning and Evaluation at Pisa and at Roma Sapienza Universities.Massimo1949@gmail.com
The author is grateful to Roberto Carpano for taking the time to provide suggestions. This text is an updated version of the article Design methods and tools: considerations on the revised Logframe presented in the EC PRAG, published on 10 November 2015 inTools and Methods series, and includes references to the EC March 2015 ROM Manual and to the EC Unified Action Document Template.

[2] The UATD is presented in 2015 ROM Handbook as “Standard Action Document Template” (European Commission, March 2015, page 34).

 

Author: 
Massimo Rossi