STEP 3: Data Checking
The third step of your metrics-based evaluation will aim to check your earlier findings, nuance them or add first-hand insights from interviewees, personal observations or detailed analysis of a specific case. Step 3 allows you to cross-check your findings through a process of triangulation.
Triangulation is a powerful technique that helps you cross-check or validate your findings by using two or more sources or methods to study the same phenomenon. It can be used in both quantitative and qualitative (inquiry) studies. By combining observations, theories, methods, and empirical data, triangulation can help to overcome the weakness or intrinsic biases and the problems that come from single-method, single-observer and single-theory studies.
According to Freudenberger, K. (1998) in Rapid Rural Appraisal (RRA) and Participatory Rural Appraisal (PRA), Baltimore: Catholic Relief Services, some of the ways to help triangulate and reduce bias include:
- Using team members with different experiences and perspectives
- Continuously cross-checking information using different methods and types of informants
- Actively identifying bias at the end of each day
- Relying on a range of key informants
- Varying times of data collection
Step 3 could also employ quantitative techniques but is probably more likely to involve qualitative techniques. That allows for a mixed-methods approach.
Only four (out of many possible) methods are discussed here in any detail. Other useful triangulation tools are set out as part of a Theory-based evaluation framework. All of these methods are helpful for checking your data and going beyond mere hunches about how things are going:
Note that steps 2 and 3 could in some cases be reversed. Thus participant observation could be used to provide an impressionistic understanding of how your project is proceeding. This could be followed up by a quantitative survey.