Are we measuring what really matters?

Susan Dodsworth

Research Fellow – International Development Department University of Birmingham

At Westminster Foundation for Democracy’s recent conference – Deliberating Democratisation: Examining Democratic Change and the Role of International Democracy Supportone of the more contentious panels addressed the role of research in democracy support.

During that panel, I discussed how the quality of evidence available to academics shapes its utility to practitioners and policy-makers.

There’s both good news and bad news in this area. The good news is that there have been significant advances in how democracy is measured. Thanks to work like that of the Varieties of Democracy Project, we now have much more transparent, more finely grained indicators of democracy. The bad news is that measures of democracy only tell half the story. We also need better measures of democracy support.

This is bad news because our primary source of data on democracy support is the OECD Development Assistance Committee’s (DAC) database on aid flows. For academic researchers, this is the ‘default’ approach to measuring democracy support. Why? Because this is what we can access. Organisations that support democracy are often very reluctant to share their data, in part because it can put those they are trying to help at risk.

Our reliance on OECD data constrains research and limits its usefulness in three ways. First, reducing democracy support to money spent tends to produced recommendations along the lines of ‘spend more on X and less on Y.’ This has some value, but there are limits to its utility. Ultimately, it’s more useful to know how to spend money, rather than how much money to spend.

Second, we are limited by what is – and is not – captured in the coding of these databases. The OECD data only captures the primary purpose of aid. But we know that aid generally, and democracy support in particular, is often multi-purpose. Aid flows have also been reported in a way that creates blind spots. As I pointed out in a recent policy paper, it’s essentially impossible to calculate how much democracy support is invested in civil society – a significant problem given how important civil society is in democratisation.

Third, and perhaps most importantly, we might be missing what really matters. When you talk to democracy supporters about their work they will explain that ‘You can’t buy political will,’ or emphasise that ‘It’s not about the money.’ Instead, they (and their strategic documents, like this one) stress the importance of relationships, or of intangible resources like respect and recognition. The difficulty, of course, is that these things are extremely difficult to measure.

If research is to play a constructive role in democracy support, we need to have better measures of it. The challenge is to work out how we can get those better measures. This won’t be an easy task and, unless researchers and practitioners work together, it will probably be an impossible one. That’s why I hope we’ll see more events – like WFD’s conference – that create opportunities for better dialogue and – hopefully – more sustained collaboration between researchers and practitioners.

Highlights from Deliberating Democratisation conference

(Photo: Jo-Ann Kelly)

You may also like