2013 SIAA Conference: Is impact measurement the highway to hell?
by Martial Paris, Wise Philanthropy Advisors
The annual conference of the Social Impact Analyst Association (SIAA) took place in Paris on 10 December. SIAA was founded in 2011 to ‘build an active international community of social impact analysts’. The conference theme, which attracted over 100 participants, was ‘Beyond Measurement’: Analysts should not measure impact for its own sake but integrate it in a larger process, so that organizations can improve their work.
‘Impact’ can have many definitions. The following one identifies clearly the key components: ‘a set of significant and long-term changes – whether positive or negative, expected or not – that were caused directly or indirectly by an action.’ I do not have the space to go into more technical terms, but I will say that when we speak of impact measurement, in most cases we are really talking about tools that help measure the quality of a project in a given time period, but not its long-term effects. One of the reasons is that we need to distinguish between contribution and attribution: through the various projects that they implement, organizations contribute (in the best-case scenario) to improving the condition of beneficiaries. However, there are many other factors that can cause change, and the project will only be one of those factors.
That being said, it makes sense to go beyond sheer impact measurement, but the real question is: how do we define ‘beyond’? Where does impact measurement take us? According to various points of view, it could be heaven or hell.
The road to the hell of impact measurement is paved with good intentions. There are numerous tools and attempts to standardize measurement. Each tool strives to ease the work of donors and organizations. The Foundation Center has thus developed a database that includes over 150 tools. Each of them has its apostles who belong to a specific parish.
The goal is not to say which tool is the best, because the answer depends on the strategy defined by the supported organization and the donor.
However, there is a trend that can take us from that hell to a paradise in which organizations define a strategy that is revised after their results are measured (as opposed to a recent NPC study showing that 80 per cent of charities do not use any planning models; a paradise where donors use not only their heart but also their head when they engage in philanthropy (only 2 per cent do so according to another NPC study).
The road to this paradise is not a highway. At this stage, it looks more like a path, and it has to be approached from a global and pragmatic perspective. Impact measurement is not an end in itself but the end of a process that begins with planning one’s philanthropic commitment.
Martial Paris, is director of strategy and impact at Wise, philanthropy advisors and a member of SIAA.