Instructions for Preparing Evaluation Reports
1
2
3
4
5
6
7NUMBER 1
VERSION 1.0
NOV 2012
HOW-TO NOTE
Preparing Evaluation Reports
Monitoring and Evaluation Series
INTRODUCTION
Create
This Note supplements USAID Automated Directives System (ADS)
evaluation
Chapter 203 and provides current good practice in preparing evaluation
reports that
reports, the main deliverable for most evaluations. Following these
practices will help to establish clear expectations for evaluation reports
are clear,
during the preparation of evaluation statements of work and the in-briefing
credible, and
of the evaluation team. These practices also serve as a guide for reviewing
the quality of draft evaluation reports submitted by the evaluation team.
useful.
This Note is also a resource for USAID partners and independent
evaluators of USAID programs and projects. An
evaluation report template
and
sample evaluation report covers
are available as additional resources.
BACKGROUND
The most important outcome of an evaluation is that it is used to inform
decisions and improve USAID projects and programs. A key factor in using
evaluation findings is having a well-written, succinct report that clearly and
quickly communicates credible findings and conclusions, including easy-to-
understand graphics and consistent formatting.
How-To Notes
REQUIREMENTS
are published by the
USAID’s Evaluation Policy and ADS 203 provide guidance on evaluation
Bureau of Policy,
report structure and content, and steps in the process of creating a report.
Planning and Learning
These are listed in Table 1. The report must present a well-researched,
and provide guidelines
thoughtful and organized effort to objectively evaluate a USAID activity,
and practical advice to
project or program. Findings, conclusions and recommendations must be
USAID staff and
based in evidence derived from the best methods available given the
partners related to the
evaluation questions and resources available. The evaluation methods,
Program Cycle. This
limitations, and information sources must be documented, including by
How-To Note
providing data collection tools and the original evaluation statement of work
supplements USAID
as annexes to the main report. Finally, the findings should be shared
ADS Chapter 203
transparently and widely, to ensure accountability and to promote learning
from USAID’s experience.
www.usaid.gov
NUMBER 1
VERSION 1.0
NOV 2012
HOW-TO NOTE
Preparing Evaluation Reports
Monitoring and Evaluation Series
INTRODUCTION
Create
This Note supplements USAID Automated Directives System (ADS)
evaluation
Chapter 203 and provides current good practice in preparing evaluation
reports that
reports, the main deliverable for most evaluations. Following these
practices will help to establish clear expectations for evaluation reports
are clear,
during the preparation of evaluation statements of work and the in-briefing
credible, and
of the evaluation team. These practices also serve as a guide for reviewing
the quality of draft evaluation reports submitted by the evaluation team.
useful.
This Note is also a resource for USAID partners and independent
evaluators of USAID programs and projects. An
evaluation report template
and
sample evaluation report covers
are available as additional resources.
BACKGROUND
The most important outcome of an evaluation is that it is used to inform
decisions and improve USAID projects and programs. A key factor in using
evaluation findings is having a well-written, succinct report that clearly and
quickly communicates credible findings and conclusions, including easy-to-
understand graphics and consistent formatting.
How-To Notes
REQUIREMENTS
are published by the
USAID’s Evaluation Policy and ADS 203 provide guidance on evaluation
Bureau of Policy,
report structure and content, and steps in the process of creating a report.
Planning and Learning
These are listed in Table 1. The report must present a well-researched,
and provide guidelines
thoughtful and organized effort to objectively evaluate a USAID activity,
and practical advice to
project or program. Findings, conclusions and recommendations must be
USAID staff and
based in evidence derived from the best methods available given the
partners related to the
evaluation questions and resources available. The evaluation methods,
Program Cycle. This
limitations, and information sources must be documented, including by
How-To Note
providing data collection tools and the original evaluation statement of work
supplements USAID
as annexes to the main report. Finally, the findings should be shared
ADS Chapter 203
transparently and widely, to ensure accountability and to promote learning
from USAID’s experience.
www.usaid.gov
HOW-TO NOTE
Preparing Evaluation Reports
TABLE 1: EVALUATION REPORT REQIREMENT
(from the USAID Evaluation Policy and ADS 203)
Report should be…
A thoughtful, well-researched, well-organized, and objectively evaluate what worked, what
did not, and why.
Executive
Include a 3 to 5 page Executive Summary that provides a brief overview of the evaluation
purpose, project background, evaluation questions, methods, findings, and conclusions.
Summary
Evaluation
Address all evaluation questions in the statement of work.
Questions
Methods
Explain evaluation methodology in detail.
▪
Disclose evaluation limitations, especially those associated with the evaluation
▪
methodology (e.g. selection bias, recall bias, unobservable differences between
comparator groups, etc.).
NOTE: A summary of methodology can be included in the body of the report, with the full
description provided as an annex.
Findings
▪
Present findings as analyzed facts, evidence and data supported by strong quantitative or
qualitative evidence and not anecdotes, hearsay or people’s opinions.
▪
Include findings that assess outcomes and impacts on males and females.
Recommendations
Support recommendations with specific findings.
▪
Provide recommendations that are action-oriented, practical, specific, and define who is
▪
responsible for the action.
Annexes
Include the following as annexes, at minimum:
Statement of Work.
▪
▪
Full description of evaluation methods.
All evaluation tools (questionnaires, checklists, discussion guides, surveys, etc.).
▪
▪
A list of sources of information (key informants, documents reviewed, other data
sources).
Only if applicable, include as an annex Statement(s) of Differences regarding any significant
unresolved differences of opinion on the part of funders, implementers, and/or members of
the evaluation team.
Quality Control
Assess reports for quality by including an in-house peer technical review with comments
provided to evaluation teams.
Transparency
Submit the report to the Development Experience Clearinghouse (DEC) within three
▪
months of completion.
Share the findings from evaluation reports as widely as possible with a commitment to full
▪
and active disclosure.
Use
Integrate findings from evaluation reports into decision-making about strategies, program
priorities, and project design.
2
HOW-TO NOTE
HOW-TO NOTE
Preparing Evaluation Reports
Preparing Evaluation Reports
STEPS IN THE PROCESS
4. Submit to DEC and Share Findings Widely
1. Define Report Requirements in the
USAID Program Offices must ensure that
Evaluation Statement of Work and Final
evaluation final reports (or reports submitted by
Work Plan
evaluators to USAID as their final drafts) are
All evaluation statements of work (SOW) should
submitted within three months of completion to
clearly define requirements and expectations for
the Development Experience Clearinghouse at
the final evaluation report. All of the items in
http://dec.usaid.gov. The actual submission can
Table 1 must be included as requirements for
be done by USAID staff or by the evaluation
the final report. Ensure that all requirements in
team with USAID concurrence (once an
the SOW are also included in the final evaluation
opportunity has been provided for USAID or
work plan that is put in place once the
others to include a Statement of Differences, if
evaluation team is on board. Adjustments can be
appropriate). In addition to submission to the
made at this time, as long as the minimum
DEC, USAID should also consider how to share
requirements are met, and additions can be
the evaluation report widely to facilitate broader
included such as defining when the first draft will
learning. This could include posting the report
be due, how many days USAID will have to
on the USAID mission website, translating a
review and provide comments, and when the
summary into local language, and hosting
final report will be submitted.
presentations of the evaluation findings.
2. Review First Draft
5. Use Evaluation Findings to Inform
Program Offices must ensure that evaluation
Decisions
draft reports are assessed for quality by
USAID must integrate evaluation findings into
management and through an in-house peer
decision making about strategies, program
technical review and comments provided to the
priorities, and project design. While the Program
evaluation teams. USAID staff may consider
Office in a mission should ensure this happens, it
including implementing partners and other direct
is the responsibility of all USAID staff.
stakeholders in the review process. Tools such
as the USAID Evaluation Report Checklist can
be used.
3. Final Draft and Statement of Differences
Evaluation reports are independent products and
therefore the evaluation team leader reviews the
comments and determines which to incorporate
into the final draft. Once the final draft is
submitted to the USAID mission or office, the
content should not be changed without the
permission of the evaluation team leader.
USAID, other funders, implementing partners,
and other members of the evaluation team can
decide to include a Statement of Differences as an
annex to the report, if there are differences
related to the evaluation findings or
recommendations.
3
HOW-TO NOTE
Preparing Evaluation Reports
the project was started, a description of the
CONTENT AND STRUCTURE
beneficiary population, geographic area of the
General Style
project, and the underlying development
When writing a report, the evaluation team must
hypothesis, or causal logic, of the project or the
always remember the primary audience: project and
broader program of which the project is a part.
program managers, policymakers, and direct
If a results framework (for strategies, objectives
stakeholders. The style of writing should be easy to
or programs) or logical framework (for projects)
understand and concise while making sure to
is available, this should be included here. For
address the evaluation questions and issues with
projects designed under the
project design
accurate and data-driven findings, justifiable
guidance
released in 2011, the evaluation team
conclusions, and practical recommendations.
should have access to the final Project Appraisal
Document and related annexes (which includes a
Report Sections and Content
logical framework and original monitoring and
At a minimum, all reports should include the
evaluation plans, among other things). This
following sections: Executive Summary (3 to 4
information provides important context for
pages); Evaluation Purpose and Questions (1 to 2
understanding the evaluation purpose, questions,
pages); Project Background (1to 3 pages); Evaluation
methods, findings and conclusions.
Methods and Limitations (1 to 3 pages, with full
version provided in an annex); Findings, Conclusions
Methods and Limitations
and Recommendations (15 to 25 pages); and,
This section should provide a detailed
Annexes. Reports may include additional content,
description within one to three pages of the
split the sections up differently, or present the
evaluation methods and why they were chosen.
sections in a different order.
If more space is needed, additional detailed
information on the methods should be provided
Executive Summary
in an annex. The reader needs to understand
The Executive Summary, between three to four
what the evaluation team did and why to make
pages in length, should stand alone as an
an informed judgment about the credibility of
abbreviated version of the report. All content of
the findings and conclusions and the underlying
the full report should be summarized, and the
evaluation design including the data collection
Executive Summary should contain no new
and analysis methods.
information.
Evaluation Purpose and Questions
Evaluation methods should correspond directly
The evaluation purpose should be clearly defined
to the questions being asked and should
at the beginning of the report. It should describe
generate the highest quality and most credible
in about one page or less why the evaluation is
evidence possible, taking into consideration time,
being conducted now, how the findings are
budget and other practical considerations.
expected to be used, what specific decisions will
be informed by the evaluation, and who the main
This section should provide information on all
audiences are for the evaluation report. The
aspects of the evaluation design and methods,
evaluation questions are linked to the purpose,
including tradeoffs that led to selection of
and should be listed here. Good practice is to
specific data collection and analysis methods, a
limit the questions to three to five that are clear,
description of data availability and quality, and
focused, and that will directly inform specific
sampling strategies (purposive, random, etc.),
decisions.
including how interview subjects or site visits
were selected. Just as important as describing
Project Background
the evaluation methods is describing any
This section should summarize the project being
limitations in data collection and analysis, data
evaluated in one to three pages, including the
quality, access to data sources, or any other
original problem the project is designed to
factors that may result in bias. To show the
address, any changes that have occurred since
4
HOW-TO NOTE
Preparing Evaluation Reports
relationship between the evaluation questions
Annexes
and methods, it is useful to include a chart that
All evaluation reports must include the following
lists each evaluation question, the corresponding
as annexes: 1) the Evaluation Statement of
evaluation method to be used for data collection
Work, 2) Detailed description of the evaluation
and analysis, data sources, sample sizes, and
design and methods, 3) copies of the actual data
limitations.
collection tools such as survey or interview
questions, 4) a list of information sources
Findings, Conclusions and
(including documents reviewed, sites visited, and
Recommendations
key informants, assuming they gave permission
Findings, conclusions, and (if requested in the
to be identified)., and 5) disclosure of any
evaluation statement of work)
conflict of interest by including a signed
recommendations, make up the main body of
statement by evaluation team members that
the report, synthesizing what was learned during
attests to a lack of conflict of interest or
the evaluation and presenting it in an easy to
describes an existing conflict of interest relative
understand and logical fashion. Findings are
to the project being evaluated. Additional
empirical facts based on data collected during
annexes can be included at the discretion of the
the evaluation and should not rely only on
evaluation team and USAID, and in some cases
opinion, even of experts. Conclusions synthesize
implementing partners, including, if applicable,
and interpret findings and make judgments
any Statements of Differences with the evaluation
supported by one or more specific findings.
conclusions.
Recommendations, if applicable, are specific
actions the evaluation team proposes be taken
by program management that are based on
findings and conclusions. The reader should be
able to discern what evidence supports the
conclusions and recommendations. Whenever
possible, data should be presented visually in
easy to read charts, tables, graphs, and maps to
demonstrate the evidence that supports
conclusions and recommendations. All graphics
must have a title, be clearly labeled, and include
a caption.
5
