Social Value Measurement - When should we use high levels of rigour?
By Craig Foden - Social Value Technical Manager at Social Value International
A question I’m often asked is: “Do we always need to measure social value with a high level of rigour?”. This is a good question to ask; organisations want to make informed decisions which improve the wellbeing of their stakeholders but do not have unlimited resources for analysis.
The brief answer to this question is no; it is not essential to apply high levels of rigour in every situation. Instead, rigour should be proportionate to the decisions being made. This is what we mean by the phrase “enough precision for the decision”: the level of evidence should be proportionate to its purpose, no more and no less.
The significance of the potential impacts, and the resources needed to reverse the decision will determine how much rigour is required. For example:
A low-impact, easily reversible decision such as changing the start time of a training course. This will be based on some information you collect (for example, participant feedback that a later start time would allow them to pick their children up from school), but it would be disproportionate to conduct a very in-depth analysis with external verification, control groups, and so on.
A high-impact, difficult to reverse decision. This may be an organisation choosing to start an entirely new activity based on forecast data or choosing to end another activity based on evaluation data. Or it could be decisions made by a government to enhance social value through new legislation which will impact an entire sector of the economy. Here, the analysis should be conducted with a high level of rigour and include external verification.
In practice, decisions will fall along a spectrum, and you will need to decide for yourself how much rigour is appropriate to use. A useful concept is that of “impact risk” – the risk that decisions made being sub-optimal, meaning that they don’t result in effective use of resource.
This impact risk is based on:
The impact of the decision on the wellbeing of stakeholders
The likelihood of these impacts occurring
How easily the decision can be reversed
Any trade-offs, such as the resources available to conduct the analysis
Rigour – not just collecting more data
Another misconception is that increasing rigour simply means collecting more data. Instead, it is based on to what extent the data is:
Complete – how many material outcomes which stakeholders experience have been included in the analysis? An outcome is material if it is both relevant and significant to stakeholders. If all material outcomes are included, then the analysis would have a high level of completeness. If material information is omitted, there is a risk that different (potentially sub-optimal) decisions will be made.
Accurate – the degree of precision used to collect the information. This means elements of the analysis like sample size, which indicators are used, which method is used to value outcomes, and so on. In general, as stakeholder involvement increases, so do levels of accuracy.
Use of the Principles of Social Value
No matter how much rigour is used, all Principles of Social Value should be applied. When taken together, the Principles provide a framework so that organisations of all sizes and sectors can effectively use information to make better, more-informed decisions.
More information on the Principles can be found here: https://www.socialvalueint.org/principles