STEP 3 – Quick Data Checking

The third step is about going further, triangulating and confirming (or infirming) your findings. If you have gone for an appreciation line or graffiti wall at the start of your project, you could repeat this technique towards the end of your project (step 3).

Any of the activities in step 2 could be used in step 3. However, you may wish to give your quick appraisal more depth. If so then you may wish to engage in an abbreviated version of some of the techniques already highlighted:

Mini-survey

Small focus group

Most Significant Change (streamlined)

Short Participant Observation

Snapshot or short interviews

Mini Surveys

These are good just before of just after an event and can help to gauge learning, changes in attitudes and potentially in behaviour.

They are useful for gaining rapid insights from a large number of participants but not for gaining in-depth insights.

Mini-surveys involve the same broad techniques as surveys/ questionnaires. The key difference is that you must reduce the number of questions.

At most, ten minutes, preferably five, should be the amount of time taken to fill in a mini-survey. So, think hard about what you really want to know and ask closed not open questions.

Make sure you have enough copies of the survey, as well as spare pens and pencils.

You may issue the same survey before and after your event. But do leave enough time for completion and take account of the fact that some people may arrive late or leave early.

If you are keen to compare pre and post-event responses, consider using the scrabble bag technique. take a bag full of Scrabble letters A to Z (one of each letter) and ask each participant to choose a letter then turn it over.

Participants use this letter as an identifier but they do not reveal it at any time and return it to the bag at the end of the process.

So now you can break down results according to individuals. If you want to, go further and work out make versus female responses. To do this you might give a separate bag to males; this will be full of bits of paper on which a number from, say, one to ten is marked. That becomes their identifier.

Small Focus Groups

Our previous section has discussed in detail how and why to conduct focus groups.

From a quick appraisal perspective, the only thing that changes is the size of the group and the level of organisation required. With such small groups, the project coordinator or a member of his/ her team might feel confident enough to take on the process of facilitating this group. All the usual dangers apply (one dominant voice, a general reticence/ reluctance to speak).

But a small group is usually more manageable. You may wish to engage in an ice-breaking activity such as names and introductions right at the start. If the participants already know each other, you might ask them to tell each other their middle names or a joke or something they find funny or sad.

Most Significant Change (Streamlined)

The Most Significant Change technique has been set out in detail here and need not be repeated below. The key point to retain for quick appraisals is that you are only focusing here on a very small set of people (up to 5).

You can speak to them together in a circle and allow them to choose the most significant stories. Or you can speak to them individually, perhaps audio-recording or filming these stories and, with permission and anonymisation, posting them on your website or including them in your reports.

Could you be accused of selection bias? Why these individuals? Why these stories? Are there other stories that are more significant and potentially that have more negative connotations for your project. Quite possibly. But at least you are engaging with M&E, working to ensure that you are doing no harm and looking to see what difference your project is making.

.For a full version of the MSC technique, click here

Short Participant Observation

Participant observation involves immersing yourself in the daily life of a village or community. In some cases, you may embed yourself within and organisation. You observe while seeking to go unnoticed (an almost impossible task). Your aim is to see how local actors behave and how they have (or have not) learned or benefited from your project.

Here the focus is on short spells of participant observation, perhaps ranging from 3 days to 2 weeks. Needless to say, 3 days may not be enough time for you to blend into your ‘new’ environment unless of course you are already very well known to your host community/ target audience. 2 weeks should generally be enough unless you are dealing with highly sensitive local issues.

For a more detailed account of how to conduct participant observation, click here.

Snapshot or Short Interviews

According to Europlanet, snapshot interviews are ‘very brief, focused interviews, which are used in conjunction with an event to gather impressions quickly, like a photo of a moment in time’.

Europlanet offers instructions as well as a case study example of this technique which helps you get a quick picture of attitudes towards a particular event, display, conference or other development.

These interviews usually last only a couple of minutes and can involve 10 to 40 interviewees in any single day. They involve short one to one interviews, based on about four questions. The key thing is to to keep it brief.

For details of how to conduct full-scale semi-structured interviews, click here. As you will see below, the snapshot interview process is more streamlined.

How to Conduct a Snapshot Interview

Before

• Decide what information you are looking for and whether snap interviews are best suited to these ends. If you are interested in getting a quick snapshot of the attitudes towards a particular event or development then snap interviews are likely to be useful.

• Prepare your list of about three to five questions. Most if not all of these will usually be set questions. Examples include:

♦ what do you like about this event?

♦ do you think that the solutions proposed here are the right ones?

♦ what could be improved?

♦ what have you learnt?

♦ how do you think you might act differently as a result of coming here going well with the project?

♦ what could be improved and how?

♦ on a scale of 1 to 10, how useful/ interesting was this event?

Other questions will be tailored to specific villages or segments of a community (e.g. officials, teachers, mothers, children, elders, religious leaders, health workers). There would, for example, be little point asking an African schoolboy about the politics of the region. He could, however, provide valuable insights into questions of local schooling and family life.

• Bring a pad of paper and pen. Alternatively, you may use a dictaphone (assuming a microphone is passed around), mobile phone or iPad recorder. But remember that will mean a lot of transcription later. With community interviews, it is often best to ask a colleague to act as your assistant and take copious notes of what is said as well as of non-verbal signs (sighs, laughter, applause).
• Prepare 20 to 40 interview sheets, each with the same 3 to 5 questions and space on which to write down responses, and fill in sheet one for each respondent.
• If appropriate, arrange for an interpreter to be present at the right time and venue and have enough local currency with you to pay him/ her. Keep on good terms with this interpreter as you will need his/ her help with transcription and potentially future interviews.

During

• Approach potential interviewees at the event and ask them for two minutes of their time. make sure you are dressed appropriately and have some kind of clipboard!
• Start your snapshot interview by explaining in one line the purpose of the interview and stressing that it is anonymous. Ask your core questions:

♦ What did you expect this event would be like?

♦ What part of the event did you like the most?

♦ What did you like the least?

♦ Was there anything that surprised you?

• Once you have asked your core questions, do not develop follow-up questions. You are looking for similarities and differences across the range of interviewees and you will introduce bias if one interviewee is encouraged to talk or ends up talking for longer than others.

After

• If you have written out your sheets neatly, you may not need to re-write. Otherwise, you will need to write them up or transcribe them. This may require the help of your interpreter. You may also ask a colleague to help with the time-consuming business of transcription.

• Read through the interview data (along with other interview write-ups/ transcripts). Look at the answers to your core questions on what is going well, what could be improved, how have attitudes or behaviours changed, was the project right to assume X or Y. Coding techniques may be used if you are confident with this approach.

• Start to input this data into your evaluation or annual report. Some of the material may just help you understand the particular event but some of it may feedback into your understanding of your own assumptions and potential outcomes. Look out for nuggets that help you with your evaluation/ indicators.

• Assumptions:  Our first assumption was confirmed by 50 % of snapshot interviewees who claimed that they had no understanding of the need to boil water for three minutes continuously in order to remove cysts.

• Outcomes: 15 percent of visitors said they would change their cooking practices in light of this ‘new’ information.

Click here to return to the top of the page and here to move on to Quick appraisal methods for volunteers/ volunteer organisations.