The Benefits of Being Wrong

Yesterday, at the fieldwork halfway point, Asimwe (my research assistant) and I decided to feedback our research findings to the children and young people in Moshi. The discussion went thus; "Asimwe, I really feel like we're starting to saturate our categories. Let's feedback to the children to check that we're understanding them correctly. If they feel like we understand correctly, we may need to sample outside of the population of children to see if we find some counter examples to provide us with more questions. If they disagree with things we are saying, that will be useful for developing the depth of our categories." 

The beauty of feeding back your findings to participants is that people are very quick to correct you if you are wrong. They are the experts in their own lives, to use the cliche, so our findings being wrong is inevitable to some extent. But I wasn't prepared for the extent to which we might be wrong!

When reflecting on how we had got some things wrong, it occurred to me that the children are still very used to researchers coming and asking them about their problems. These sorts of questions inevitably distill only one side of life. Even though throughout our discussions with children we have tried very hard not to lead questions or ask just about "problems", this is generally children's first thought when we ask questions - either because life is overwhelmingly difficult, or because they know we come from NGOs who have the capacity to help them, or maybe both. But, when we present their lives to them through the lens of what we have seen in the data, the children cannot help themselves pointing out where we may be mistaken. For example, while on the street, children speak a lot about the difficulties earning money, we have heard many a break-down of their daily budget - "we spend 500 on breakfast, 500 on lunch, 500 for dinner and then the little we have left, we use for something else." But when explaining our findings last night we learned that some children are able to save money and they use their savings to buy new clothes at times of festivals - Christmas or Eid. They save money with the Mama Ntilie (the women who cook local street food) or local shop owners in what sounds like a Credit Union format. We knew from our observations that the children had good relationships with some of the Mamas in the market, but we would not have guessed that they worked together to keep savings and access capital. This is just one of many new routes for us to explore relating to children's relationships and social networks on the street, all thanks to us being wrong.

Failing quickly
It is common to feedback research findings at the end of fieldwork, or during a return trip to the field after the researcher has almost completed their analysis. However, with Grounded Theory Method (GTM), since you are analysing your data all the time, this provides golden opportunities to sense check your findings as you go along.

My husband is into business and he loves the GTM because it reminds him of a concept of building businesses by "failing quickly," a concept articulated in The Lean Startup by Eric Reis (Andy's lean review of this book is here). Since GTM is not about generalising your findings for the whole population (where asking the same questions many times is a key research method) but instead building substantive theory about a particular phenomena, you need not focus on number of interviews but instead saturating your findings until you stop hearing new things. This allows more time for considering data and sense checking your findings with participants so that you "fail quickly" and stop pursuing incorrect avenues. It can also be time saving because you do not find yourselves with a mass of data which all says the same thing in the hope of making a population wide claim.

*Disclaimer* there is definitely a place for large sample sizes and generalising data across populations, and varying sample sizes depending on how confident you need to be about your findings! But from previous experience, collecting enough data to be confident about your findings for the population is difficult, often requiring large sample sizes. This puts a lot of pressure on getting the questions exactly right and having research assistants you can trust 100% to not fabricate the odd questionnaire in order to finish work early or because they're scared of the consequences of coming back empty handed. For the research projects in-between where a lot of data is gathered but not enough to claim generalisation across the population (which includes a lot of social science research), deciding on a sample size can seem arbitrary. 

No more superfluous research
On a final note, the children I work with in Moshi and Arusha are used to being interviewed a lot, mainly due to many high profile NGOs working in the area with intense annual reporting requirements. As a consequence the children have become very skeptical about research and complain about the impact of raising their expectations by coming to speak to them and then dashing their hopes when they see that nothing comes of what they have told researchers. Most of this research is conducted using that "in-between methodology," where a large number of children are asked the same questions, which may or may not be the right questions, and which may or may not produce useful findings. The researcher leaves and the following year, another person comes and asks (what seems to the children) almost identical questions.

I've been that person, asking those questions, and for me the learning was extremely important and I gained a lot from these research projects. But as time goes on, I find myself becoming very against research that seems superfluous. This also leaves me with a large weight of responsibility regarding what I do with my data with my limited capacity, and what happens to the participants in Moshi and Arusha post-fieldwork. I feel like more teaching should focus on how to exit fieldwork well and the responsibility that researchers have towards their participants to use their time and information well, especially with marginalised populations. If you have any ideas or examples about how to leave a good legacy following your research, I'd love to hear about it in the comments below.

Being wrong is scary
I have to admit that last night I felt nervous. Telling the children what we feel we saw in the data and what we had been thinking seemed like a scary step. It is difficult to face a group of people and welcome them to tell you where you're wrong. We were opening ourselves up to uncertainty and anything could have happened. Thankfully for me, I could send Asimwe to do all the talking on my behalf, and she's a lot braver than I am! But it seems I'm not alone in this fear, and (naturally) I have found a TED Talk to illustrate my point.

Kathryn Schultz tells us that being wrong could be the most "intellectual, moral and creative step [we can] make." True to what she describes in her talk at TED 2011, we found risking being wrong to be a key step in developing our thinking. I am looking forward to doing it again!

Blog title image found here.