I’ve written before about the growing use of more rigorous quantitative approaches to community and third sector research. A great example is the way the School for Social Entrepreneurs used control cohorts to evaluate its pioneering Match Trading programme. By doing so, they were able to demonstrate real and long-lasting improvements in trading performance over three consecutive years. Better yet, they achieved this despite the last nine months of the programme overlapping with the Covid-19 pandemic and global economic slowdown.
All together now
In this month’s blog, I want to focus on another novel approach to quantitative community research: aggregation – and the value of taking The Beatles’ sage advice; all together now!
My first example is a very nice piece of analysis by Clare Leckie, Rebecca Munro, and Mark Pragnell on behalf of the Rural Services Network. Despite the impression given by Rishi Sunak earlier this summer, not all rural areas are leafy and well-to-do. Deprivation is not a solely urban phenomenon.
In their report, Rural as a region: the hidden challenge for Levelling Up, Leckie et al combine data from across rural England and present it alongside the nine English regions. The results are stark. If rural England were a region, it would be more populous than London or the south-east. English rural local authorities are home to one fifth of the population, nearly 5m workers, and half a million businesses. Treated in this way, the authors argue the case for Levelling Up in rural areas is greater than any other part of the country.
The long and winding road
My second example of aggregation highlights the importance of looking again (winding back?) around existing evidence. In a very impressive re-analysis of survey data, which was funded by the National Institute for Health and Care Research and led by a team of researchers based at Lancaster University, the Universities of Newcastle and Liverpool, and the London School for Hygiene and Tropical Medicine. Community empowerment and mental wellbeing combines data from 150 Big Local areas around England, over three years (2016, 2018 and 2020). It explores the relationship between mental wellbeing, collective control and social cohesion. The research team find:
- mental wellbeing increased temporarily in 2018 but fell back again in 2020;
- the 2018 increase occurred only in men and those with a degree-level education; women and lower educated participants saw no change;
- in general, those with higher levels of collective control and area-belonging had better mental wellbeing than those who did not.
As intriguing as these findings are, there are important caveats to bear in mind:
- the analysis is based on only 217 participants, fewer than 1.5 people, on average, per Big Local area;
- only 80% (173) of respondents were actual residents in Big Local areas;
- the overall demographic profile of respondents was not representative of the resident population in Big Local areas.
A day in the life
The authors suggest that the decline in wellbeing in 2020 could be explained by the Covid-19 pandemic when the mental wellbeing of the whole country declined. This is where the value of having a control cohort really comes into its own (as with the SSE analysis of Match Trading above), because it allows like-for-like comparisons to be made when circumstances change in individuals’ experiences and daily lives. While this can be more difficult to do with survey data Power to Change has elsewhere shown how ‘synthetic counterfactuals’ can be built that allow robust causal statements to be made.
There is of course one big drawback with this sort of aggregation approach to analysis. For those of us interested in understanding what’s working in hyper-local communities, the reasons for change are often deeply rooted in the specific history and people who make up each area. This can get lost if we assume that all 150 Big Local ‘interventions’ have a similar impact on people, regardless of geography, history or demography.
Recent additions to our Repository include reports on the role of higher education institutes in the climate action agenda, what community businesses can do to understand their social impact, and lessons learned from the UKRI’s Citizen Science Exploration Grant programme. As always, email richard.harries@icstudies.org.uk if you have a publication you’d like to submit.
Institute for Community Studies Posted on: 22 August 2022