STEP 3: Data Checking

The third step of your metrics-based evaluation will aim to check your earlier findings, nuance them or add first-hand insights from interviewees, personal observations or detailed analysis of a specific case. Step 3 allows you to cross-check your findings through a process of triangulation.

Triangulation is a powerful technique that helps you cross-check or validate your findings by using two or more sources or methods to study the same phenomenon. It can be used in both quantitative and qualitative (inquiry) studies. By combining observations, theories, methods, and empirical data, triangulation can help to overcome the weakness or intrinsic biases and the problems that come from single-method, single-observer and single-theory studies.

According to Freudenberger, K. (1998) in Rapid Rural Appraisal (RRA) and Participatory Rural Appraisal (PRA), Baltimore: Catholic Relief Services, some of the ways to help triangulate and reduce bias include:

• Using team members with different experiences and perspectives
• Continuously cross-checking information using different methods and types of informants
• Actively identifying bias at the end of each day
• Relying on a range of key informants
• Varying times of data collection

Step 3 could also employ quantitative techniques but is probably more likely to involve qualitative techniques. That allows for a mixed-methods approach.

Triangulation Tools

Only four (out of many possible) methods are discussed here in any detail. Other useful triangulation tools are set out as part of a Theory-based evaluation framework. All of these methods are helpful for checking your data and going beyond mere hunches about how things are going:

• Key Informant Interviews
• Participant Observation
• Document analysis
• Case Study

Note that steps 2 and 3 could in some cases be reversed. Thus participant observation could be used to provide an impressionistic understanding of how your project is proceeding. This could be followed up by a quantitative survey.

Where to next?

Click here to return to the top of the page, here to return to step 3 of theory-based evaluations, and here for a worked example of a metrics-based evaluation.