One thing that has become clear to me over the years is that people are creatures of stories.
From young to old, initiate to master, we use the art of storytelling to navigate the world. As children we learn about the world through story. Aesop’s tale of the turtle and the hare, for example, helps us learn morals that can help us live happier lives. Likewise, our parents often tell us rudimentary tales about ‘the little child who touched the fire’ to help us protect ourselves. Some of these stories are at least a thousand years old, such as the Swedish story of Näcken who will draw you to him and drown you in a lake… Intended to stop children playing by the stream and falling in by mistake. Personally, I was fortunate to have tales about a princess who lived her own life and shirked society’s expectations of her – something that helped me learn complex concepts of feminism at a young age. (I think in the end she lived a happy life with her seventeen-or-so dragons in the wilds, which makes me raise an eyebrow at my current brood of 3 cats settled in the woods of Sweden but I digress…)
Whether it be to learn safety or morality, societal norms or new ideas, stories help us access concepts in a way that feels familiar and less complex. They translate ideas. This is the concept that I want to focus on today.
Monitoring and Evaluation (M&E) frequently feels like a dark art. A black box. A difficult-to-grasp concoction of methods and magic that feels so far out of reach. These are all phrases that I have heard said to me before I walk into a training I am going to deliver, or even just expressed to me by those bringing about change in programmes, and I think that is concerning. Nothing should feel so complicated that it cannot be explained in simple terms. Personally, I think if you cannot explain something simply you simply do not understand it enough! One of the best governance advisers I ever worked with was one who did not buy into the jargon of the profession, and simply used plain language.
Anyone who has read my blog knows that I shirk M&E jargon. I subscribe to Useful Theories of Change that use language like ‘goods and services provided’ instead of ‘outputs’; I advocate for facilitation in the image of ‘Socrates’ or a ‘Troll’; but most crucially I use a lot of analogies and stories to explain complex ideas. This is done for a reason.
Discussion of adaptive indicators is about as far from intuitive as M&E gets, and so I explain the different types with a story about crossing a river and encountering different scenarios (including a sea monster). Complex systems feel, well, complex until you explain them as flocks of birds. Nothing needs to be difficult because we can use analogies and stories to translate an idea from confusing jargon to an accessible image.
Let me give some quick examples.
1. Proxy Measurement
Standard explanation: it is not always possible to measure something directly because it is either unsafe to measure it or obstructed. As such proxy measurement selects indications that are indirect: that suggest the presence of the change we wish to see to a degree that it is reasonable to assume the change has taken place.
Additional explanation: this is much like ghost hunting. We cannot report that there is a ghost present by spotting one as we are unable to see ghosts. Instead, we must proxy measure! If there are indirect indications of ghost activity, such as ectoplasm and objects flying around the room, it is reasonable to draw the conclusion that there is a ghost even if we have not directly seen it.
2. Outcome Harvesting
Outcome Harvesting collects (“harvests”) evidence of what has changed (“outcomes”) and, then, working backwards, determines whether and how an intervention has contributed to these changes. (taken from betterevaluation.com)
This is like planting seeds in a communal garden, and your evaluator is a botanist. The botanist arrives in the garden after a year and makes note of what flowers have grown: tulips, snapdragons, and foxgloves. They then work out when those seeds must have been planted, by whom, and whether that person was the one watering them or not. If it was in fact you who planted and watered the foxgloves, then you contributed to the foxgloves growing and flowering. If it was in fact someone else and your flowers never grew, then they can still help you understand why that may have been the case.
Granted, my stories and analogies are often a little silly. However, they are memorable (I trained a NATO team who said that my analogies are the only reason they still remember how adaptive indicators work). This is so, so important for us as M&E folk. It is our responsibility to put the ‘me’ in ‘MEL’ because we are so reliant on our programme teams to engage in the MEL process, from collecting observations to performing calculations to designing a theory of change. Our programme teams are our power, and if they don’t understand what we are trying to do then we have already failed. Analogy and story is a powerful tool in an M&E professional’s arsenal for making our work accessible. Much like how we learn not to play with fire or to be patient with ourselves through stories and analogies as children, we can access complex evaluation concepts through stories too (Michael Quinn-Patton is incredible at this if you have ever listened to him talk or read his work).
This is not restricted to explaining M&E mechanics, though it is certainly helpful. I extend this to just about every aspect of my M&E work and it has never failed me. I use stories to connect the M&E profession to people who may be sceptical: I simply tell stories of how, for example, adaptive mechanics on a Rule of Law programme empowered someone to dramatically shift direction for the better. I use stories to explain programme progress: stories of change make complex change accessible by using a narrative format. I weave story-telling into contribution analysis in a participatory mechanism called a contribution court. All of these approaches connect humans and ideas in the way we are most familiar with, and it gives so much more dimension to how teams work. Even just using stories to explain change brings fire to a programme in a way that simple numbers do not in isolation (and often cannot due to statistics going wobbly: see my analogy involving cows and coyotes here).
Let me tell you a story to illustrate this (see what I did there?). Details are tweaked or omitted for anonymity but the message is the same (see what I did there again?). I worked on a programme some time ago that was working in the general governance space. Its delivery team was incredibly small but incredibly efficient, they just struggled to communicate this out: a problem not uncommon for governance-focused programming. Their reporting was made up of numbers that felt divorced from reality. Their SRO could see that they were delivering… Something… well. Certainly, all the milestones were hit, but it was hard to explain exactly what that change meant. This is where stories come in. The team had an attitude of reticence towards M&E as it felt unhelpful and detached; it was something to appease the eye of Sauron. This was entirely accurate: their way of measuring and reporting change needed some work, but the first barrier was getting close to that.
Instead of walking in with methods and jargon, I started by telling them a few stories. I told them stories of programmes similar to them that encountered similar issues and how they were overcome, and I told a few stories about my ways of working to ground M&E as an idea. This was not an approach they were used to, and it immediately set a different tone of dialogue. They shared their worries and shared their confusion over purely quantitative reporting which we subsequently addressed… With stories! In short, a bespoke change frame was created to allow cross-comparison between initiatives in different countries and equipped workstreams with qualitative reporting. Instead of counting beans, they were using Stories of Change to write the narrative of what was happening (robustly verified, triangulated, and contribution traced). These stories were a game-changer. The team finally had a method to explain the complex change they were nurturing in a way that their SRO immediately could access. The team were empowered to showcase the impressive change they had caused and captured attention from other channels. All it took was introducing stories.
My programmes live and breathe analogies and stories because they are so incredibly powerful. And I am not alone. A colleague was recently telling me how he explained concepts of freedom of speech through analogy: when shopping, it is possible that you might pick up the wrong bag of sugar. However, we don’t react in anger when this happens! We do not tear open the bag and throw the sugar to the ground, trying to destroy it. We simply realise it is not the sugar we want and put it back on the shelf, then retrieve the one we do want. This simple analogy is why he does not repress posts on a blog: people are free to say it, and others are free to ignore it. That is, you can put that sugar back on the shelf. This simple analogy got through to a group in a way that lengthy lectures never had. It is hard to disagree with a story when it aligns so well to a situation.
This is why I emphasize the qualitative in mixed-methods approaches. We need it. Not just because it adds meaning to numbers but because it makes complex change accessible. It allows ideas to translate no matter your background, language, or beliefs. I highly recommend integrating it into your methods if you have not already. If you have: tell me your stories. I would love to hear them.
Huge thanks to my wonderful colleague Bassel for some wonderful conversations that inspired me to finally put these thoughts to paper. Thanks also to Lauren Sweeney for always being a fellow champion of using stories and analogy.