Reporting quantitative information in qualitative research: guidance for authors and reviewers
The absence of numbers is often seen as a defining feature of qualitative research 1. Despite this, qualitative researchers frequently use numbers or semi-quantification (e.g. terms such as ‘many’ or ‘most’) when writing up their data. We have found that different qualitative researchers can have strongly held, but opposing, views on whether and how quantitative information should be expressed in what are essentially qualitative studies (that is, studies reporting concepts, themes and ideas arising from interviews, group discussions or other linguistic material). This editorial note seeks to provide clarity for Addiction authors and reviewers on the Journal's current position and preferences.
First, let us consider why researchers might want to convey quantitative or semi-quantitative information in reports from qualitative studies. The use of numbers to describe sample characteristics (e.g. number of participants and key demographics) is essential and uncontroversial. Beyond that, it has been argued that quantification or semi-quantification can improve the transparency of data analysis 2, give precision to statements 3, enable patterns in the data to emerge with greater clarity 4 and increase the meaning of key findings by providing focus 5.
Quantification or semi-quantification can, however, be problematic. For example, if not everyone has been asked exactly the same questions in the same way, reporting or alluding to the frequency of a given response or emergent theme will probably misrepresent the data, even within the sample studied. In addition, the use of numbers tends to detract from the more valuable, detailed and nuanced data that are collected in qualitative work 6. A more conceptual concern, meanwhile, is that presenting numbers implies an ‘objective’ and measurable reality that is incompatible with the goals of qualitative research 3. Finally, presenting numbers can sometimes lead both the author and the reader to make inappropriate generalizations about the findings, albeit unconsciously 3, 6.
Ritchie & Lewis argue that if the study ‘is of the scale and design recommended for qualitative research, it will not support any statements about prevalence or distribution other than within the sample itself’ (6, p. 311). Here, the phrase ‘within the sample itself’ is crucial. This is because qualitative researchers may legitimately wish to identify the relative importance, frequency or strength of emerging findings within the specific data set that they are analysing. For example, a qualitative researcher who finds that 27 participants of a sample of 30 spontaneously report finding X while three spontaneously report finding Y may wish to express this as ‘nearly all participants highlighted X, but only a few participants identified Y’.
Pattern recognition can be an important aspect of qualitative data analysis. When qualitative researchers code or categorize their data to identify patterns in the form of events, experiences or themes, etc. they are, in practice, undertaking a form of counting 5. We might refer to this as ‘semi-quantification’, as it occurs when researchers imply a vaguely defined range of numbers by using words such as ‘a few’, ‘some’, ‘several’ or ‘many’ (to refer to participants) or ‘rare’, ‘occasional’, ‘common’ or ‘typical’ (to refer to events, experiences or themes) 5. This type of semi-quantification, when performed judiciously, can enable the researcher to draw attention to regularities, peculiarities and idiosyncrasies in the data; but it is not meant to convey generalizability beyond the study population 7.
- Include a clear statement reminding readers that no inferences can be drawn about the prevalence of phenomena observed beyond the sample. While this might seem unnecessary, the journal has a very diverse readership, many of whom will not be academic researchers and, in any event, it is useful for authors to convey that they are fully aware of this limitation.
- By and large, limit reporting of raw numbers to features that have been assessed for all the participants in a manner that allows for comparison. If, for example, in the course of interviews with 10 people, three reported a particular point of view but not all the participants had an opportunity—because of the flow of the interviews—to report such a view, it is better not to enumerate this.
- Avoid percentages unless the study has a sample size greater than 50 or thereabouts. Some qualitative researchers argue that such a sample size cannot be a qualitative study, but Addiction recognizes that there are many different types of qualitative study and relatively large samples are appropriate for some of these.
- Do not use terms that have a specific quantitative meaning, e.g. ‘majority’, ‘minority’, ‘most’, without providing a specific justification for why such language is appropriate. We believe this kind of language will rarely be appropriate.
- If some form of semi-quantification is required, it is often better to use appropriately non-specific terms such as: ‘a few’, ‘several’, ‘some’, ‘many’ etc. As with point 4, explicit justification should always be given.
When submitting qualitative papers to Addiction, it might help to refer to this editorial note to assist reviewers who may otherwise provide inconsistent recommendations for revision. We will keep this guidance under review and welcome suggestions for improvement.
Declaration of interests
Joanne Neale: None. Peter Miller has no affiliations which he believes constitute a conflict of interest. In the past 5 years he has been funded by charitable foundations, government departments and a pharmaceutical company via an intermediary charity. Travel expenses have been paid by charities, the Society for the Study of Addiction and university sources. He holds no stocks in any related companies. Robert West has received travel funds and hospitality from, and undertaken research and consultancy for, pharmaceutical companies that manufacture or research products aimed at helping smokers to stop. These products include nicotine replacement therapies and Zyban (bupropion). This has led to payments to him personally and to his institution. He undertakes lectures and training in smoking cessation methods which have led to payments to him personally and to his institution. He has received research grants from medical charities and government departments.
