How to use the 1-2-3 Method
Step 1: Choosing what type of evaluation
Your first decision relates to the kind of evaluation you will end up undertaking mid-way through and/ or at the close of your project.
How do you decide which aspects of the 1-2-3 Method you’d like to use? Try asking yourself the following questions:
Have your funders already told you to use a particular evaluation method (e.g., logframe) and provide a report in a particular format?
You will have to start with the logframe. You will nonetheless still be able to choose from a wide range of data collection and analysis tools in response to donor demands.
Are your funders willing or keen, to allow you to develop a Theory of Change approach?
Consider a ToC approach as long as you keep your funders in the loop about the approach you intend to take. See here for our comparison of the differences between the ToC versus the logframe.
Are you funded mainly by other types of donation, such as individual charitable giving?
If so, you may be able to choose how much you want to dip your toes into M&E waters. You may also wish to go for one of the quick appraisal methods.
Are you already engaged fully in a project without having given much thought to M&E or even some kind of self-evaluation?
Quick appraisal techniques may again be of interest.
If you are accountable to an external donor or want to do quite a rigorous evaluation, opt for the Full 1-2-3 Method using one of the following:
If, however, you are not accountable and are seeking to do a rapid appraisal, you may opt for the Quick 1-2-3 Method:
- Quick Appraisals and Self-Evaluations by NGDOs
- Quick Appraisals and Self-Evaluations by Volunteer Organisations
You can, of course, also combine different types of evaluation.
Step 2: Select Preferred Data Collection/Analysis Method
Having decided on the most appropriate type of evaluation, you can move onto step 2: the selection of your preferred data collection and analysis methods.
Bear in mind that different types of evaluation have different consequences for the kind of data collection tools you employ.
- the logframe lends itself well to harder data collection gathering tools. These may include quantitative techniques involving number crunching, the inspection of hospital records or school attendance sheets, project records, local government data, measurement of percentage increases in uptake of a particular drug or course, coding, and statistical correlations.
- the Theory of Change lends itself to the collection of softer data, including the opinions and judgments of stakeholders, beneficiaries and other project participants.
- the same is true of the quick appraisals/self-evaluations, although here, the emphasis is on more rapid appraisal techniques which do not require as much planning, staff or time.
Which data types are most suitable for your needs? Ask yourself:
- What am I trying/ hoping to show?
- What information do I need and who is likely to have it? Why not draw up a list of key questions and key people to consult?
- What resources are available? Try to identify the human, financial and material resources and skill-sets available to your organisation and your local partners or target community/ group
- What data collection methods are most suitable for gathering the necessary information with the resources available?
- Consider how much quantitative data (if any) is required
- Weigh up different data collection methods in terms of their costs and benefit
Keep things simple
The 1-2-3 Method suggests no more than three techniques are required. Larger organisations can, of course, do more. But, as a rule, for a smaller organisation it should be enough to choose one type of evaluation, one data collection method and another data collection method to confirm it or nuance your findings.
So, step 2 requires you to select and use one of a range of data collection methods. Some may opt to collect hard data (e.g. through quantitative techniques), while others may seek out softer variables and the opinions of project participants (via a focus group, the most significant change technique, pre and post-event quizzes, vox pops, an appreciation line, a graffiti wall or a time diagram …).
Step 3: Triangulate Using Another Data Collection/ Analysis Method
Step 3 either involves more data collection, data analysis or cross-checking. It allows you to make your own fact-based/ first-hand judgments or to cross-check the opinions of others with other evidence you have collected. You will choose one of a range of methods for data checking/ further data collection, notably interviews with key participants, community interviews, extended participant observation, and short participant observation.
As will be shown later, all of these latter techniques allow you to cross-check your findings through a process of triangulation. You can check with participants why they spoke or responded in a particular way in a focus group or survey.
You can also go into more depth and secure a range of perspectives on your project over the short to medium term. To illustrate, participant observation allows you to check for yourself that the project is working and making a difference and that the claims made by beneficiaries and stakeholders are evidence-based.
You should only choose one data collection method for step 2 and one for step 3. In some instances, you may decide only to use only one method in total from steps 2 and 3. That is absolutely fine. The use of interviews, surveys, a focus group or participant observation or even an appreciation line can be valuable in itself.
With such a wide, almost overwhelming, choice of methods available to you, it can be hard to decide how best to proceed. It is a good idea to talk to your funders or to other NGOs which have used particular data collection tools.
To help you make these tough decisions, our PDF table breaks these research methods into categories.
The table sets out the key types of evaluation you might consider, the type of framework or theory you will have to develop and the kind of methods you can use to collect and check data. The list is by no means exhaustive and there are many additional permutations. Some data collection methods appear under more than one type of evaluation. Others could equally have been duplicated but have not been in order to streamline the 1-2-3 approach as far as possible.
Some techniques are arguably better suited to being used before, during or after the project or development intervention: see Europlanet (no date, p. 12). However, it is worth stressing that there are no hard and fast rules in this regard.
Note: technology-led methods are not discussed here due to issues relating to internet access in many parts of the developing world.