e-MOPs

Ethics & Engagement across the Wellcome Trust Major Overseas Programmes

Case studies for evaluation - introductory resources

Introduction

Case Studies usually explore a complex single instance (of an individual, organisation, project, event) taken as a whole in its context - providing a comprehensive description and analysis of that instance using multiple sources of data.

Case studies are useful when there is a concern to understand ‘how’ and ‘why’ questions and the process or project being investigated needs to be explored as a meaningful whole within its real life setting. In evaluation, explanatory case studies are useful to explain the presumed causal links in real-life interventions that are too complex for survey or experimental designs. Case studies can also be used in a variety of other ways, including exploratory case studies to illuminate aspects of a situation or setting about which little is already known. 

The need to understand context usually means there are many factors and aspects of a situation that need to be taken into account even while there may not be comprehensive data on all of them. In this situation triangulation of evidence is crucial. It is important to build up a chain of evidence supporting the claims made about the case, but also to test rival explanations for the characteristics and events described. In turn, for evaluation as much as for other case studies, this means it is important to develop clear theories about how change is brought about by an intervention (its theory of change). Making clear the assumptions about how changes happen also supports the development of clear evaluation questions to guide data collection and analysis.

A definition from Yin (2009) suggests that a case study is an empirical enquiry that:

  • investigates a contemporary phenomenon in depth and within its real life context, especially when
  • the boundaries between the phenomenon and context are not clearly evident
  • copes with the technically distinctive situation in where there will be many more variables of interest than data points, and as a result
  • relies on multiple sources of evidence with data needing to converge in a triangulating fashion

Case studies are often misunderstood as a purely qualitative, descriptive method and criticised as being ‘unrepresentative’ compared to survey and other quantitative methods. However, the case study is not a ‘sample’, and the aim of a case study is not to enumerate the frequencies of one or two key variables in a way that can be seen as representative of a larger population to which the findings can be ‘generalised’. Rather, case studies, like experiments, follow a logic of ‘replication’  - where it is the theory (of how change happens) which is tested and generalised to new cases in empirical studies (in technical terms the case study uses ‘analytical generalisation’ rather than the ‘statistical generalisation’ of randomised assignment to treatments).

On the other hand, many descriptive accounts are often wrongly called ‘case studies’ when none of the procedures that make up a rigorous case-study approach have been applied (teaching ‘case-studies’ are also usually a different thing from the rigorous case studies used in research or evaluation). Case studies, like any other method can be done rigorously or less so, depending on how carefully and thoughtfully they are designed and conducted. In many ways they are more difficult than other methods to do well, since they attempt to triangulate and combine a range of different types of data and a range of approaches to gather that data.

Comparing or contrasting a range of case studies can increase the confidence that the factors and processes being demonstrated through the case are more widely applicable.

Recently, a powerful new approach for systematic case comparison has been developed, called Qualitative Comparative Analysis. This approach supports research and analysis into complex cases in a way that preserves their integrity as wholes embedded in their own contexts, but which also allows comparison and generalisation across many examples to draw out key combinations of factors that seem to make a difference (for some selected links on this more particular application of case based methods see here)

Resources

The resources below combine general introductions to case study methods, with some that are more focused on using case-studies in evaluation.

 

Introductions

Case Study Research: Design and Methods, (book 219 pages) Fourth Edition, California: SAGE. Robert Yin 2009

The definitive guide to conducting rigorous case studies, outlining the distinctive advantages of the case study approach in comparison with other methods. Includes a clear step-by-step treatment of the selection of cases, different types of case study and case study design, gathering evidence and data, analysis and reporting, and criteria for quality, with a range of practical examples and links to illustrative reference materials.  Does not focus only on the use of the case study methods in evaluation, but is valuable for clearly outlining the components of a rigorous case study approach.

Case study introduction pages and links on Better Evaluation website: http://betterevaluation.org/plan/approach/case_study

Short introduction to case study methods in evaluation and related links

Case study methods and evaluation when and why? (presentation 18 slides) Catherine Walshe: http://www.ceco.org.uk/resourcedocs/case-study.pdf

Very short presentation outlining why and where case studies in evaluation may be preferred over experimental designs. Outlines some of the key characteristics of a case study

Video Introduction to case studies (part 1 of 3). Graham Gibbs of Huddersfield University (19 minutes): https://www.youtube.com/watch?v=gQfoq7c4UE4

General introduction to the basics of the case study as a distinctive method, explains different types of case studies and provides examples.

Case Study Evaluations, Washington: World Bank Operations Evaluation Department. (paper 20 pages). https://ieg.worldbankgroup.org/Data/reports/oed_wp1.pdf

Clear overview introduction to the distinctive character of case studies, the range of types of case study in evaluation with short illustrative examples of these types drawn from World Bank development assistance programmes. Usefully outlines the rationale for selection of different types of case study for evaluation, depending on the evaluation questions that need to be answered. (Borrows heavily from the GAO guide and Yin books highlighted on this page).

 

Guides 

Case Study Evaluations, (guide/paper 154 pages). United States General Accounting Office (1990) http://www.gao.gov/special.pubs/10_1_9.pdf

Clear and thorough overview of methodological issues related to the use of case studies in evaluation (not a guide to case study design), including the distinctive advantages of the case study approach, and procedures for selection of cases, data gathering and analysis, and common challenges. Looks in turn at six different types of case studies – Illustrative, exploratory, critical instance, programme implementation, programme effects and cumulative, and reviews their strengths and weaknesses. Originally aimed at government decision makers to help them critically assess the contribution of case studies, it includes many short illustrative examples and clear summary tables, checklists and guidelines.

Case study on-line guide - Europeaid: http://ec.europa.eu/europeaid/evaluation/methodology/tools/too_cas_som_en.htm

Detailed overview of case studies and why, how and when they are useful

Case study on-line guide Colarado State University: http://writing.colostate.edu/guides/guide.cfm?guideid=60

Detailed overview of case studies, their history, relative merits, design and conduct with an extensive bibliography

 

Examples of using case studies

Two companion books to Yin’s ‘Case study research: design and methods’ highlighted above give a comprehensive set of example case studies.

Applications of Case study research (book 264 pages) Yin (2011), Thousand Oaks, CA: Sage. Includes 21 detailed examples of case studies, including a range of different types of case study, which are clearly described and critically appraised.

The case study anthology (book 296 pages) Yin (2004), Thousand Oaks, CA: Sage

Accounts of a range of different types case studies and the methods used, many of them extended and detailed examples of the cases summarised in boxes in Yin’s ‘case study research’.

Both the World Bank and US GAO guides highlighted above provide a range of short illustrative case study examples.

This resource list was produced by Robin Vincent for the Wellcome Trust linked Community of Practice around evaluation of community and public engagement August 2014, robvconsult@gmail.com

© 2017   Created by Dina Rippon.   Powered by

Badges  |  Report an Issue  |  Terms of Service