Ethics & Engagement across the Wellcome Trust Major Overseas Programmes
Participatory statistics is a set of methods that enable local people to generate statistics for local level planning, learning and reflection, but which can also be aggregated at wider levels and feed into national level policy processes. Participatory statistics have been used in the design, monitoring and evaluation, and impact assessment of policies, programmes and projects in a number of developing countries.
Proponents claim they are a ‘win-win’ approach that both produces rigorous and generalisable conclusions in a cost-efficient manner, while empowering local people in a sphere of research that has traditionally been highly extractive and externally controlled
Participatory research is often seen as valuable for exploring qualitative changes in relationships and processes, and providing a rich picture of how local context affects issues under investigation. It has been seen as limited however, in its ability to produce data that is ‘representative’ or can be compared across context.
Participatory statistics seeks to get past these perceived limits and be ‘the best of both worlds’, producing data that combines local validity and representativeness. Participatory group-based and visual methods are used for data collection, analysis and reflection, and are adapted to improve standardisation and comparability across sites. These are combined with the use of statistical principles in the selection of sites and informants to ensure that findings can be aggregated and compared across settings.
A diverse set of tools can be used to generate of quantitative data and quantify qualitative data for whole communities through key informants in an efficient way. Tools include; score-cards to assess services, wealth-rankings to analyse poverty, proportional piling to estimate food and income, and a range of others.
Who Counts? The power of participatory statistics, (Edited by Jeremy Holland, 2013 Practical Action Publishing, Rugby, UK). A comprehensive introduction to participatory statistics and their use in international development practice. The book provides a range of detailed case examples of how participatory statistics have been used in monitoring, evaluation and learning, participatory impact assessment, and to influence policy in a wide range of countries. It includes four chapters exploring examples of using participatory statistics for evaluation and three chapters looking at impact assessment.
Who Counts? Participation and numbers. 14 pages (Chambers, 2008, Chapter in ‘Revolutions in Development Inquiry’ London: Earthscan. p 15 - 129). Accessible summary of key developments in participatory statistics detailing the range of methods used, the methodological challenges and trade-offs and signposting a range of practical examples and case studies. Includes accounts of using participatory numbers to conduct large scale surveys in a range of countries including at a national level in Malawi, Uganda. (this Chapter is a shortened, adapted version of the IDS working paper highlighted below).
How to generate statistics and influence policy using participatory methods in research: reflections on work in Malawi 1999-2002. 59 pages (Barahone, C and Levy, S., IDS working paper 212, 2003) “
Describes work in Malawi on food insecurity that combined statistical principles and participatory methods to generate ‘participatory numbers’ to inform national level policy on food security interventions. The authors make a case for the methods being at least as rigorous and reliable as statistics obtained from traditional survey methods.
Who Counts? The quiet revolution of participation and numbers. 45 pages (Robert Chambers, 2007, IDS Working Paper 296, IDS: Brighton). Provides an introduction and rationale for this growing body of work, with a description of key methods used, and a range of practical examples.(a version of this paper appears as the Chapter highlighted above in ‘Revolutions in Development Inquiry) https://www.ids.ac.uk/files/Wp296.pdf
Currently, there is no manual or guide that brings together all the recent innovations in participatory statistics work. A number of guides to ‘participatory impact assessment’ however, do share an emphasis on generating quantitative data through participatory methods, including the following:
Participatory Impact Assessment – A Guide for practitioners. 63 pages (Catley et al, 2007, Feinstien International Centre, Boston: Tufts University). Comprehensive guide focused on participatory impact assessment for livelihoods interventions making use of community identified impact indicators. Outlines how to use participatory approaches to measure impact in the absence of baseline data, and how qualitative data from participatory tools can be systematically gathered and numerically presented to give ‘representative’ results of project impact. http://fic.tufts.edu/assets/Part_Impact_10_21_08V2.pdf
NGO-IDEAS toolbox and manual
This set of tools and manual have been iteratively developed in a long-term collaboration between 40 non-govermental organisations (NGOs) from South Asia, East Africa and the Philippines and 14 German NGOs working in the field of international development.
Measuring change and results in voice and accountability work. 82 pages (Holland and Thirkell 2009, DFID working paper 34). Outlines the rationale, a framework, indicators and methods for assessing voice and accountability in an international development context
A series of video’s capture a panel discussion introducing participatory statistics hosted at the Overseas Development Institute in 2013
Introduction to participatory statistics (24 minutes)
Jeremy Holland introduces participatory statistics in a panel discussion (24minutes). He describes the development of the methods over the last two decades, discusses some practical examples and explains some of the trade-offs involved in making use of the methods. https://www.youtube.com/watch?v=tq_AfnQuXSI
Introduction part 2 (20 minutes)
Robert Chambers adds to the introduction in the same panel discussion (20 minutes) including an explanation of the rigour of group visual-synergy methods, and a range of practical examples of the methods used. The second half of the video explores resistance to adopting these promising new methods more widely.
Case Studies and examples
Measuring Empowerment? – Ask Them: Quantifying qualitative outcomes from people’s analysis - insights for results based management from the experience of a social movement in Bangladesh. 108 pages. (Jupp and Ali, 2010, SIDA). A detailed but accessible account of the process used to develop a self-assessment process to measure ‘empowerment’ for a social movement in Bangladesh focused on land-rights. Describes how participatory methods generated qualitative statements of change that were drawn on to develop a self-assessment matrix used by social movement groups. A parallel process drew on these self-assessments to analyse trends across groups and over time and perform statistical analysis. (A Chapter summarising this work by Jupp and Ali appears in the Holland 2013 book detailed above)
A range of examples are also featured in the Holland 2013 book detailed above including
Community groups monitoring impact with participatory statistics in India: reflections from an international NGO collective. 10 pages (Causemann et al 2013 in Holland 2013: pages 113-123). Outlines a process used for self-assessment of changes in income and income inequality, and collaborative development planning by the Karwar Rural Women and Children Development Society working for social justice and sustainable development with indigenous populations in in Karnataka state. (The NGO-IDEAS toolbox highlighted above provides more detail on some of the methods used in this work).
Scoring perceptions of services in the Maldives: instant feedback and the power of increased local engagement. 11 pages (Riemenschneider, Barca and Holland 2013 in Holland 2013: p 125 – 136). Describes an evolving evaluation process that combined group and survey based score-cards with conventional surveys and instant analysis and feedback of evaluation trends to stakeholders for validation and learning.
This resource list was produced by Robin Vincent for the Wellcome Trust linked Community of Practice around evaluation of community and public engagement August 2014, firstname.lastname@example.org