Tool 24 | OECD-DAC evaluation questions

Monitoring and evaluation should be planned from the start of the process.  This helps to identify aspects to monitor, which will make the later evaluation easier and more effective.  Agreeing who will be involved in monitoring and evaluation also helps to identify the appropriate measures and tools.

The planning team should ensure that monitoring and evaluation responsibilities are identified at an early stage.

Stakeholders potentially involved should agree with measures and responsibilities.

The beneficiaries are potentially all impacted by a strategic planning process and also those supporting the process.

The OECD Development Assistance Committee has developed a useful set of questions to help understand important aspects of a development activity. The questions form a useful framework for undertaking monitoring.

The questions are:

Relevance: The extent to which activities meet policies and priorities. Specific sub questions include:

1. To what extent are objectives still valid?
2. Are objectives consistent with overall goals?
3. Are activities and outputs consistent with intended impacts and effects?

Effectiveness: The extent of achievement of the objectives.

1. To what extent are the objectives met or likely to be met?
2. What are factors influencing this?

Efficiency: Were the outputs achieved with good use of the inputs.

1. Were activities cost-effective?
2. Were activities achieved on time?
3. Was implementation efficient related to alternatives?

Impact: This refers to the changes as a result of project activities.  They can be positive or negative, intended or unintended.  It should also pick up influences from external factors, e.g. the overall economic situation.  The scale of impact should be assessed.  OECD, DAC sub-questions include:

1. What has happened as a result of the programme or project?
2. What real difference has a project made to the beneficiaries?
3. How many people have been affected?

Sustainability:   This is to do with whether activities are likely to continue after specific programmes and / or projects cease.  It includes social, economic and environmental sustainability. Questions include:

1. Are the activities likely to continue after project activities cease?
2. What were major factors helping or hindering implementation?  Are they likely to continue?

Advantages

1. Provide a good set of questions which explore important aspects of development
2. Wide familiarity and acceptance

Disadvantages

1. Needs refinement to meet needs of specific activities.

Tool24.pdf

This tool has strong links with monitoring and evaluation tools, SMART objectives, and logical frameworks.

Generally accessible documents:

OECD/DAC (1991) Principles for evaluation of development assistance, Paris Organisation for Economic Co-operation and Development, Development Assistance Committee.

OECD (2002) Glossary of key terms in evaluation and results based management = Glossaire des principaux termes relatifs à l’évaluation et la gestion axée sur les résultats = Glosario de los principales términos sobre evaluación y gestión basada en resultados, Paris, DAC Working Party on Aid Evaluation, Organisation for Economic Co-operation and Development.

Academic documents (access may be limited):

Chianca, Thomaz (2008) The OECD/DAC criteria for international development evaluations : an assessment and ideas for improvement, Journal of MultiDiscipinary Evaluation, vol. 5, no. 9, pp. 41-51.