STEP 2 – Quick Appraisal Methods
Now that you have your M&E framework or at least some broad ideas about what you are trying to achieve and how you will work out whether or not you are ‘succeeding’, you are ready to engage in step 2: one or more quick appraisal methods.
Rapid Appraisal Methods are useful for providing rapid information for project management decision-making, for shedding light on complex social changes, for understanding people’s motivations, and for providing context that helps to make sense of other data.
- Low cost
- Can be conducted quickly
- Provide flexibility to explore new ideas
- Findings usually relate to specific communities or localities; it is, as such, often difficult to generalise from such findings
- Less valid, reliable, credible and nuanced than formal surveys
So, it’s worth remembering that while you are not going to capture everything with these techniques, you should get some valuable data from what is working or not. You should get a snapshot of the status of the project at a particular point in time.
You should nonetheless be aware of the fact that the technique you choose may show the project in a particularly favourable or unfavourable light. Thus, for example, mini-surveys which only target locals who are (un)enthusiastic about the project will lead to certain conclusions. Equally, your assessment of attitudes towards a project to create a fair trade, open market space for local traders may reach different conclusions if conducted in the rainy or dry season.
The key is to make the most of without overstating the significance of your findings.
The following techniques are considered here:
- Graffiti Wall
- Appreciation Line
- Time Diagram
- Straw Poll
- Palm on Chest
- Moodbox/ Pebbles in Box
- Pre- and Post Quizzes
Before engaging in some of the activities listed above, you may decide to start with some warm-up, ice-breaking activities. These include:
- Physical ranking scales: ask participants to stand along a line representing different levels of experience/attitude, height, age, number of brothers and sisters, etc. This gets everyone moving and talking to each other.
- Names and introductions: pair up participants and get them to speak to one another for one minute about themselves. Your partner has to report back on what he has learned (age, name, village). Keep this brief. Only one or two people may be asked to report back.
- This can be used to gather initial thoughts on how a project/ programme is progressing.
- Participants write or draw their thoughts on a post-it and then stick it the wall.
- They may need prompts so you could give them different colours of post-its: green for what they like about the project or where they think it is succeeding, red for their doubts or where they think the project is failing/ not addressing their needs, yellow for their recommendations or other comments.
- You could equally ask them to fill in post-its to set out what they think the project is about.
- Take a photo of the wall once it has been completed.
- You can then ask for suggestions as to how to group the comments into themes (like, dislike, way forward)
- Post-its can then be moved and clustered around particular themes or project strengths/ weaknesses. This may go into your annual report.
- Take a photo of the graffiti wall now.
- This tool allows participants to express an opinion on an issue or aspect of a project on a scale of 1 to 10.
- The line can be written on paper (ten or so A4 pages lined up together) or can take the form of a long piece of string.
- Participants can write notes on the A4 pages or add post-its to explain why they have given a particular score. It can help to visualise areas of consensus and disagreement.
- Take a photo of this line; you may wish to include it in your annual report.
- You cannot assume that positive or indeed negative ratings are genuine or justified. If you want to increase your chance of getting more honest opinions, you may wish to leave the room for a few moments while participants post their ratings on the wall or peg them to the string you have pinned to that wall.
- Once you have this appreciation line, it is worth probing participants for their reasons for giving your project (or some aspect of it) a high or low rating.
- You may find that their ratings have a ring of truth or that they are based on false assumptions about what your project is intended to achieve. This exploration of assumptions can feed into a Theory of Change approach but it is also useful for quick appraisals.
- Are participants simply misguided in assuming that your project should be about X and not Y or should it really be about X? Can you deliver more of whatever is being highly ranked? How can you address the areas that are receiving lower rankings?
This is usually used to chart changes in people’s lives over time. It can also be employed to identify common points where the effects of a project or programme began (or failed) to be felt.
Time diagrams can be drawn up, possibly including key project milestones, and participants can be asked to write out and stick up post-its on to the timeline saying how (if at all) their lives were affected at any point in time.
Facilitators should stress that participants should feel free to be honest regarding their experience of the project. In countries where literacy rates are low, the post-its could be given three Xs where high impact is felt, two Xs for medium and one for low impact.
|Phase 1||Phase 2||Phase 3||Phase 4||Phase 5||Phase 6||Phase 7||Phase 8|
|Consultation of local community on location of well||Geological survey part-funded locally||Hire of small rig drills||Digging||Casing||Insertion of water pump||Wide consultation on usage and access||Well open to community|
Straw polls often take the form of unofficial, even impromptu, votes. They can be used by NGOs as a way of gauging opinion on key questions and assumptions underpinning their projects. They tend to be based on only one or a small number of pre-set questions. They are particularly useful if done with the same participants at the start and end of an activity, training programme or other output.
You could, for example, start a training project by asking the following questions:
Please put your hand up (or please place you hand open on your chest if using palm on chest) if you think that:
- The major cause of death of children under five in Africa is diarrhoea.
- Breastfeeding is important to ensuring child survival rates.
- A third of African children now sleep under insecticide-treated malaria nets.
Note down how many people thought that these statements were true and how many thought them false.
At the end of your activity, ask the same questions to see what learning has taken place.
Note: A USAID report in 2011 noted that the major cause of death is actually infection linked to the delivery process, particularly sepsis (not diarrhea or malaria). Breastfeeding improves survival and guards against stunted growth. Also, only 2% of African sleep under treated malaria nets.
Palm on Chest
- This technique, suggested by Dr Yael Nazé and posted on the Europlanet astronomical website, offers an extremely useful and quick method of assessing the knowledge or opinions of participants. The facilitator will stand at the front of their audience which will be organised in rows rather than a circle.
- The facilitator will then ask questions designed to elicit a yes/no or true/ false approach. Participants will place the palm of their hand open on their chest for yes/ true or will place their clenched fist on their chest for no/ false. Note down the responses. You may ask the same questions at the start and end of the exercise.
- This technique can help you gauge the thinking and level of knowledge of participants and can help your gear an event or piece of training accordingly.
- It is more discreet than asking individuals to raise their hands or speak out and can be particularly valuable when issues are sensitive (e.g parental abuse of children, teenage pregnancies, female genital mutilation) or where there is a danger that participants might be bullied or victimised if they express their own views publicly.
Mood Box/ Pebbles in a Jar
Mood boxes are most commonly associated with politics. Members of a political party or members of the public are asked to put a ball in one or other side of a glass or perspex box. The name of the candidate or political idea is clearly posted on each side of the box, as per below. See below for examples
The mood box can easily be adapted for M&E purposes. Thus, you could distribute marbles, tokens or ping pong balls and ask participants in a project to place their ball/ token/ marble in one or other side of the box. They could be agreeing or disagreeing with a statement, approving or disapproving a particular practice (e.g. women having the right to drive in Saudi Arabia) or voting on an issue. Needless to say, it may be best not to use this technique for important votes as it is not a secret ballot and could be subject to tampering, particularly in cases where you are using local materials rather than marbles or ping pong balls. The advantage of this method is that you often do not even need to count the votes as it should be visually obvious which side has won. It can also help you gauge emotions/ attitudes. For example, how do you feel about the road being built close to your village? Happy/Sad.
At a more local community level, you may consider a similar method to dropping pebble stones in a box like the one above or, if that is not available, in a jar/ bag/hat/ basin. One of the receptacles must be clearly marked ‘yes’ and the other ‘no’ (or perhaps agree/ disagree; important/ not important; lost/ left to go etc).
The disadvantage of these rudimentary measures is that you may have to count the votes. Make sure there is an observer from both camps present as you count, especially if it is a vote that might have resource implications for participants. Take a photo of the box, the jar or the piles of assembled votes as this could be included in your evaluation report.
Alternatively place the mood/ pebble box near the entrance of an event you are hosting, count the votes, empty the box then ask people to vote again on their way out. Or have one box at the entrance and another at the exit. Same question, potentially different results and hence potentially evidence of an outcome: changed thinking or behaviour.
Pre- and Post-quizzes
For the best summary of how to conduct pre and post quizzes, see Europlanet’s guide.
These quizzes are carried out at the start and end of your training course, presentation, event or even display. They help you assess how much participants/ have or have not learned. They can help you to look for new ways to get your message across.
Design a pre-quiz ahead of the event focusing on true/ false and multiple-choice questions.
The post-quiz is carried out at the end of your training session must be identical so that it can test what participants have learned. You may, for example, be training participants on permaculture, bee-keeping, candle-making, coding. Whatever the topic, your first quiz is designed to establish the baseline: what do participants know?
The post-quiz tells you what participants have learned as a group. There may be ways of linking the pre and post quizzes. One way to do this is to take a bag full of Scrabble letters A to Z (one of each letter) and ask each participant to choose a letter then turn it over.
Participants use this letter as an identifier but they do not reveal it at any time and return it to the bag at the end of the process.
So now you can break down results according to individuals. If you want to, go further and consider responses male versus female. You might give a separate bag to males; this will be full of bits of paper on which a number from one to ten is marked. That becomes their identifier.
Mentimeter (https://www.mentimeter.com/) is online interactive presentation tool that allows presenters to quickly and in a fun way, gather audience responses. It can be used to collect anonymously a wide variety of data on a particular event, question or problem.
It is not always suitable when working in the field but can work where you and your audience have a decent internet connection and access to a smartphone, tablet or computer. Ideally you should have the capacity to present results via your laptop of phone but this is less essential
- Make sure the internet is working reliably on your computer.
- Sign into your Mentimeter account (free to register).
- Create your questions within the Mentimeter site
- During your presentation, use the Mentimeter website to select the question you want to use.
- After your audience has connected to the site using a special access key, they can respond to your questions on their own phones/tablets/computers. The software counts their answers, giving you immediate feedback. You can also choose to display the results to your audience.
For more detailed advice on how to use mentimeter, see the very useful advice below from Cardiff University: