(Above: example theory of change – International Institute for Environment and Development)
WFD’s Director of Research and Evaluation, Graeme Ramshaw, reflects on this year’s evaluation conference circuit and how to better integrate theories of change into democracy assistance work.
It’s conference season for evaluators. This means a lot of time spent travelling but also a lot of time spent thinking about monitoring and evaluation concepts and methods. In the past few weeks, I’ve been thinking a lot about theories of change and what they mean for democracy assistance.
This idea has been around for a while now, but I’m not sure anyone outside the evaluation community fully understands its purpose or the potential benefits the theory of change approach brings to development programming. Most seem to use the concept merely as a means of converting their rigid logframes into more compelling diagrams, with arrows connecting various coloured shapes to illustrate dynamism and change. But I think this misses the point.
For me, the theory of change approach is meant to dig into the assumption column—ever present in the standard logframe format but often ignored or misused. At WFD we are certainly guilty of rarely explaining how we expect change to occur at the different stages of our logic model. Others in the democracy assistance community may believe their explanations are more robust, but I think as a whole we haven’t thought enough about how democratic institutions develop and change.
But, before we go any further, we need to make a couple of big assumptions about the nature of the work we do and the impact this has on our theory of change. At the top, we have to assume that institutions like parliaments and political parties matter for democratic development and democratic outcomes. We have to believe that the structure and function of these institutions make a difference in how citizens experience democracy. And at the bottom, we have to assume that parliamentary process and political party development can be learned through a variety of tools or methods, with individual or group learning acting as a sufficient catalyst for institutional change of some kind.
These are big assumptions, but they are supported to a certain degree by our own democratic experience. We have to start from somewhere, because it only gets more complicated when we discuss outcomes. For instance, do we really know how institutions like parliaments and political parties change? And if we do, do we know what role we as outside actors play in catalysing or facilitating that change and how that affects the outcome of that change?
I don’t think we really do. This doesn’t mean there are no theories; there are many. Traditional approaches to parliamentary and political party development are largely based on the premise that structure matters in determining performance of institutions. The theory is that if you create the right form, function will follow. Many still subscribe to this theory, but it has proved problematic in practice. How many parliamentary research centres have been created that generate no research? How many committees created or ‘strengthened’ still don’t perform any meaningful function?
Critics of this approach point to the inherent difficulties in simply transferring structures from one context ‘where it works’ to another without any understanding of why it had worked previously. Indeed, a new generation of parliamentary and political party assistance trumpets the innovation of incorporating incentives into their approach. They argue that institutions are not monolithic but composed of individuals whose incentive structures must be re-shaped to enable change to occur.
While certainly more nuanced than the traditional structural approach, it remains no less prescriptive. We are still imposing ‘best practice’ on them; we’re just smoothing our path to implementation by getting local support first. This is more effective in the short-term, certainly; but if the incentive structures did not facilitate a certain structural set-up prior to our engagement, how long will it survive after we’ve left, if we don’t also address the culture and norms of the institution itself?
Indeed, reliance on political economy analysis (PEA) as the tool for informing programme design only reinforces this trend in my view. In many applications I’ve witnessed, the PEA isn’t used to suggest how we can develop a specific model for a particular context to produce our intended outcome. Rather, it’s used to determine the best method for inducing an institution to accept a predetermined solution. The question wrongly being asked is: who do we need to convince?
The Problem-Driven Flexible Approach tries to mitigate this by avoiding prescription and engaging beneficiaries in identifying locally-rooted solutions to the problems that surface. This idea has a lot of potential for democracy assistance, as we really don’t know what combination of individual, organisational, structural, or contextual factors actually influence the performance of any given parliament or political party. We can identify deficiencies easily, but our solutions based on UK experience or otherwise may not be universally applicable. Being honest about what we don’t know is probably a better approach than assuming we intrinsically understand how institutions like parliament and political parties develop and change for the better.
Our partnership with the University of Oxford is looking at these gaps in our knowledge, using our unique position at the nexus of research and practice to think more deeply about the institutional change theories that underpin democracy assistance programmes. We want to get a better understanding of the conditions under which different theories are more or less successful at explaining why parliaments or parties developed the way they did. This means digging deeper into the theoretical bases for a variety of international democracy assistance programmes and the changes they aim to achieve.
We at WFD articulated a new theory of change last year, and it’s significantly better than anything we had before. But we don’t want to stop there. We want our next strategic review, planned for 2018, to have more and better evidence on which to base our programming decisions. This will give us confidence that while everything we do won’t necessarily work, we will at least have a sound basis from which to learn from our failures, as well as our successes.