Use of heuristics during the clinical decision process from family care physicians in real conditions
Funding information: Health Research Fund of the Ministry of Economy and Competitiveness, Grant/Award Numbers: PI13/01175, PI10/01468; Funding for open access charge: Universidad de Granada / CBUA
Abstract
Rationale aims and objectives
The available evidence on the use of heuristics and their relationship with diagnostic error in primary care is very limited. The aim of the study is to identify the use of unknown thought and specifically the possible use of Representativeness, Availability and overconfidence heuristics in the clinical practice of primary care physicians in cases of dyspnoea and to analyse their possible relationship with diagnostic error.
Methods
A total of 371 patients consulting with new episodes of dyspnoea in Primary Care centres in Spain were registered. Based on specific operational definitions, the use of unconscious thinking and the use of heuristics during the diagnostic process were assessed. Subsequently, the association between their use and diagnostic error was analysed.
Results
In 49.6% of cases, the confirmatory diagnosis coincided with the first diagnostic impression, suggesting the use of the representativeness heuristic in the diagnostic decision process. In 82.3% of the cases, the confirmatory diagnosis was among the three diagnostic hypotheses that were first identified by the general physicians, suggesting a possible use of the availability heuristic. In more than 50% of the cases, the physicians were overconfident in the certainty of their own diagnosis. Finally, a diagnostic error was identified in 9.9% of the recorded cases and no statistically significant correlation was found between the use of some unconscious thinking tools (such as the use of heuristics) and the diagnostic error.
Conclusion
Unconscious thinking manifested through the acceptance of the first diagnostic impression and the use of heuristics is commonly used by primary care physicians in the clinical decision process in the face of new episodes of dyspnoea; however, its influence on diagnostic error is not significant. The proposed explicit and reproducible methodology may inspire further studies to confirm these results.
1 INTRODUCTION
The clinical decision-making process in Primary Care (PC) is performed under conditions of greater uncertainty,1, 2 than in other clinical settings because in PC symptoms and signs are often poorly defined, the early stages of clinical processes predominate, and it is often not possible to identify a clear diagnostic code. In addition, there is very limited time available per patient,3 which makes the clinical decision-making process even more difficult, given the cognitive limitation involved, which may lead to a preference for the use of so-called system 1 (rapid, non-conscious) strategies over system 2 (analytical, reflective) strategies.4 As a result, general practitioners are likely to use ‘unconscious thought’5 during the diagnostic process in PC, which is called by various names and assessments (‘gut feelings’, first impressions, intuition, heuristics), depending on the authors. However, the empirical evidence on the use of such procedures in clinical practice is very limited and, sometimes, contradictory.
Thus, Herbert Simon defined ‘intuition’ as recognition (‘the observable fact that people reach solutions to problems suddenly’); for this author, the process by which decision-makers collect and evaluate all the information, weigh its weight according to certain criteria, and combine it to maximise the chances of achieving the objectives (‘optimization’) is not very feasible in the real world; therefore, human beings usually choose to use simple strategies that are sufficiently adequate for the proposed ends.6 Subsequently, Kahneman and Tversky identified certain practical rules or ‘heuristics’ that are very efficient in saving time and cognitive effort,7 which allow for greater speed in the decision-making process. Gigerenzer, for his part, defines a heuristic as a simple decision strategy that ignores part of the available information, focusing only on some relevant predictors.6 Finally, Wooley and Kostopoulou consider that the concept of clinical intuition goes beyond the experience of ‘first impressions’,8 including in it also the feeling of difficulty or discomfort (gut feeling) derived from tacit knowledge acquired through experience (what Gigerenzer calls recognition heuristic),9 or the ‘aha’ phenomenon, in which a problem is solved after a process of maturation or incubation (when analytical thinking has previously been unable to solve it).10
This type of ‘unconscious thinking’ strategies, also identified as the in attentional deliberation effect,11 is considered by some authors to be cognitive illusions or simply irrational. Thus, in the Conceptual Framework of Heuristics and Biases proposed by Kahneman and Tversky, they have usually been considered sources of error, an approach shared by many authors who have researched clinical decision and error.12 Gigerenzer, by contrast, in his Fast and frugal heuristic framework attempts to understand when and how people's reliance on simple heuristic decisions can result in intelligent and successful goal-oriented behaviour; in these cases, decision-makers rely on a repertoire of heuristics (what he calls an adaptive toolbox) in which each heuristic is tailored to a particular situation.9
More than 100 heuristics have been described in the literature,13 with significant discrepancies regarding the benefit of their use in clinical decision-making: while some authors consider them a useful resource,14-16 the majority opinion in the medical literature over the last 30 years is that these heuristics and cognitive processing are the primary cause of diagnostic error.12, 17 In the clinical setting, getting it wrong, either by over- or under-diagnosis, increases the risk of harmful effects and unnecessary costs associated with ‘diagnostic error’, defined as error that is incorrect, mistimed, or ignored.18
A recent study of 100 cases of diagnostic errors found at least one cognitive error in 74% of cases.19 A quasi-systematic review of the use of heuristics in clinicians shows that representativeness, availability, and overconfidence are some of the most used heuristics in clinical practice.20 However, they do not find an answer whether the use of these heuristics produces systematic errors and advocate evaluating their use in real clinical practice conditions.
The representativeness heuristic refers to ‘the degree of correspondence between a sample and a population that makes us think an event is likely if it seems representative of a larger class’.21 The availability heuristic makes judgements about the likelihood or frequency of certain events based on how easy it is to recall examples of them.22 The overconfidence heuristic occurs when one overestimates one's own skills and abilities.23
An overview review of the use of these three heuristics in clinical decision-making in PC,24 found little empirical evidence of their use in clinical practice, finding that most studies were conducted in laboratory conditions using vignettes, with reasonable doubts about their applicability in real clinical practice. This scarcity of studies in real practice is even greater in PC (where only 6 of the 49 studies were conducted). The available evidence does not allow us to know to what extent the bias derived from the use of heuristics is a relevant factor in Diagnostic Error.25
In view of all these questions, this paper explores the use of heuristics in clinical decision-making by general practitioners in everyday clinical practice and their potential relationship with diagnostic error.
Given the lack of previous operational definitions to measure the clinical use of heuristics of representativeness, availability, and overconfidence, a previous article made an approximation to their operational definition based on the first diagnostic impression (representativeness), the identification of possible differential diagnoses (availability) and the degree of confidence in the proposed diagnosis on the part of the doctors participating in the study (overconfidence).26 The objectives of the present study are: To analyse the possible use of the Representativeness, Availability and Overconfidence heuristics in the decision-making process of PC doctors in the case of patients presenting with new episodes of dyspnoea, and to study the possible relationship (or association) between unconscious thinking applied through this type of heuristic and diagnostic error.
2 METHODS
2.1 Scope of the study
Four PC centres in Granada and six centres in Madrid (Spain) were selected. A total of 371 cases were registered in the period 2012–2016, with 23 participating physicians, 11 of whom were women and 12 men. The mean age was 49 years (range: 29–53 years) and mean experience in PC was 22 years (range: 4–32 years).
2.2 Study design
Applying the methodology proposed in the protocol, 26 a prospective study of new episodes of dyspnoea identified by the attending physician was designed. This methodology is adapted from the study protocol for patients with dyspnoea seen in Dutch hospitals developed by Zwaan et al27 Dyspnoea was selected because it is a prevalent reason for consultation in the PC care process, there are multiple diagnostic alternatives, and it has been previously studied. The follow-up time for each patient was from the first visit for a new episode of dyspnoea to the time when the confirmatory diagnosis was made. For each patient presenting with a new episode of dyspnoea, the physician completed two questionnaires at different times (at the first visit and when the confirmatory diagnosis was made). All the patient's data and the patient's diagnostic process were recorded in the questionnaire. The existence of this parallel register was decided to avoid using the same hardware as the electronic medical record to always maintain the confidentiality of the clinical information.
2.3 Collection of information
At the time of seeing a patient with a new episode of dyspnoea, the family doctor filled in his or her first impression of the case (First Diagnostic Impression or FDI). At the end of the consultation and before seeing a new patient, he/she completed the rest of the questionnaire, which included questions about the three most probable diagnoses (differential diagnosis or DD) and the probable diagnostic judgement (JD). Finally, the physician was asked to estimate his or her confidence in having made the correct diagnosis in terms of probability from 0 to 100%. Depending on the clinical picture, each doctor could request further diagnostic tests after the first visit or conduct new clinical encounters, which were recorded in the electronic medical record. Once the episode of dyspnoea had ended, a second questionnaire was filled in again, including the confirmatory diagnosis (CD) and the time elapsed from onset to diagnosis, information filled in by the family doctor himself.
2.4 Clinical audit process
A structured search of the literature was carried out to obtain clinical practice guidelines on the care of dyspnoea in PC; as no such guidelines were available, a guide was drawn up based on the existing literature, which was agreed with the participating doctors. Based on this, an evaluation questionnaire was drawn up and completed by the evaluators after reviewing the electronic medical records.
Each of the cases was evaluated by two evaluators, Primary Care physicians, participants in the study, with extensive accreditations in clinical care. If there were differences between the assessment of the two assessors, the case was reviewed by a third assessor. The latter were authorized to access it as they were also participants in the project but belonged to different centres.
They reviewed each of the episodes and completed the questionnaires which included information on their personal assessment of whether the appropriate tests had been ordered, whether the diagnostic process was correct, and whether the diagnosis was correct, following the methodology developed by Zwaan et al.27
2.5 Operational definition of the heuristics
For the operational approach to the use of the Representativeness and Availability heuristics, the previously published study protocol was used, which analyses the cognitive aspects of the diagnostic process of dyspnoea by primary care physicians,26 the definition of which is described below:
2.5.1 Representativeness
The possible use of the representativeness heuristic is considered when the Confirmatory Diagnosis of dyspnoea coincides with the FDI made by the physician when identifying a new episode of dyspnoea, before initiating any clinical intervention (anamnesis, physical examination, or request for diagnostic tests. It would form part of what has been called ‘gut feeling’28 or ‘first diagnostic impression’,29 identifying the degree of similarity of the sample (the new case) with the population (the set of cases of that diagnosis).
2.5.2 Availability
The availability heuristic is considered likely to be used when the confirmatory diagnosis falls within the three diagnostic hypotheses included in the Differential Diagnosis, made after the anamnesis and physical examination and before making the diagnostic judgement. It would identify the diagnostic options that are most quickly retrieved from memory.30
2.5.3 Overconfidence
The presence of ‘Overconfidence’ in the diagnosis was estimated if the confidence in the diagnosis was higher than average in the study subjects (75% on a scale from 0 to 100%; Figure 1).
2.6 Operational definition of diagnostic error
In the clinical audit, the assessors assessed whether the general practitioner ordered the appropriate tests, whether these were correctly interpreted, whether the diagnostic process was correct and whether the final diagnosis was correct. In cases where the confirmatory diagnosis was not assessed as correct by the assessors, a diagnostic error was considered to have occurred.
2.7 Statistical analysis
Statistical analysis of the data collected was performed using the R statistical package. The frequency of the use of heuristics, the concordance between diagnoses in their different phases, First Diagnostic Impression, Differential Diagnosis, Diagnostic Judgement and Confirmatory Diagnosis, as well as the frequency of diagnostic errors were estimated. A bi-variate analysis was performed to analyse the relationship between the use of each of the three heuristics and the diagnostic error or success. Associations are expressed as OR with their 95% confidence interval, together with the Chi-square and Fisher tests.
3 RESULTS
3.1 Diagnostic errors
Following the peer audit, 33 diagnostic errors were identified out of 371 recorded cases (9.9%).
3.2 Use of heuristics
3.2.1 Representativeness
The first diagnostic impression matches the confirmatory diagnosis in 183 of the cases (49.6%).
3.2.2 Availability
The confirmatory diagnosis matches one of the differential diagnoses in 304 of the registered cases, representing 82.3%. In most cases, the physician determined as the Confirmatory Diagnosis one of the three that most quickly appeared in his or her mind.
3.2.3 Overconfidence
In 54 of the cases (14.5%), the physicians felt confident in their Diagnostic Judgement below 50%. In 103 cases (27.7%), the confidence was between 50 and 70%. Finally, in 214 cases (57.8%) the doctors showed above-average confidence in their Diagnostic Judgement (more than 75.1%; Figure 2).
3.3 Correlation of heuristics with diagnostic error
3.3.1 Representativeness
Seventeen of the 33 errors detected, showed concordance between the First Diagnostic Impression and the Confirmatory Diagnosis (51.5%). However, Fisher's test (p-value: 0.12) does not show a statistically significant association between concordance and error.
3.3.2 Availability
In 25 of the 33 diagnostic errors, the Confirmation Diagnosis is included among the three Differential Diagnosis options (75.8%). Fisher's test (p-value: 0.09) shows no statistically significant correlation between concordance and diagnostic error, although the association (OR = 1.73, CI = 1.09–1.96) indicates a high probability of diagnostic error when the Confirmation Diagnosis is included among one of the three Differential Diagnosis options.
3.3.3 Overconfidence
In 22 of the 33 diagnostic errors (66.7%), the diagnosing physician showed above-average confidence in the diagnosis. The estimate of the association was OR 1.44, CI = 1.18–1.77, the frequency of making diagnostic errors by physicians was 44% more frequent when they were overconfident (understood as…? than when they were…?) However, as with the previous heuristics the association is not statistically significant (p-value: 0.08).
4 DISCUSSION
Clinical decision making is an extremely complex process, given its difficulty and the consequences for the patient, which can determine their life or death. The Institute of Medicine's report on diagnosis in medicine considers that everyone will experience at least one diagnostic error in their lifetime, often with devastating consequences.31
The uncertainty of the problem at hand (especially in the context of PC), the limited time available and the cost involved in searching for information may result in a greater use of system 1 by physicians,30 opting for intuitive and heuristic decisions, allowing quick decisions to be made. In any case, the effect of unconscious thinking on clinical decision-making is contradictory,11 the evidence on the use of heuristics in medicine is very scarce and the instruments and operational definitions for their assessment are insufficiently validated.25, 26, 32 Most of the knowledge regarding clinical decision-making and the use of intuitive strategies is based on studies conducted under experimental conditions, using simulations or vignette assessment, which are unlikely to be representative of real-life clinical encounters.29
To deepen our understanding of the use of heuristics by general practitioners, an empirical approach has been developed that may improve our understanding of the scope of heuristics cited in routine clinical practice, and which, in contrast to previous literature ‘in laboratory conditions’, studies PA clinical decision-making in its daily life with its organisational, temporal, and cognitive constraints.
The results suggest an extensive use of heuristics (or in any case unconscious modes of thinking) by primary care physicians: in almost half of the cases seen, the first diagnostic impression coincided with the Confirmatory Diagnosis, before any clinical intervention was made. It is known that physicians generate their diagnostic hypotheses very early, in just seconds, with very little information27, 33 and that even this decision prior to gathering more information is associated with greater diagnostic accuracy.29, 34 The difficulty in considering this as a first impression, a heuristic of representativeness or recognition, or gut feeling stems from the lack of agreement in the scientific community on how to define these forms of unconscious thought (unknown thought). Regardless of what they are called, what they seem to demonstrate is that in many cases the decision regarding the problem that afflicts a patient is predetermined before information is gathered. Probably one of the factors that determines this is the prolonged knowledge of patients in PC (longitudinally), one of the keys to Primary Care doctors achieving a high degree of success in the management of patients' problems and protecting them from pathogenesis and overtreatment.35, 36 In our study, most of the doctors had been treating the same list of patients for more than 10 years.
Also in our study, it was observed that in more than 80% of the cases the Confirmatory Diagnosis coincided with one of the three initial diagnostic hypotheses, which could suggest that the ease or speed with which the possible diagnosis comes to mind determines the Confirmatory Diagnosis, without being associated with statistical significance to a higher percentage of error.
The percentage of diagnostic error identified (9%) is slightly lower than that reported by Zwaan et al37 in their study of 247 cases of dyspnoea seen in hospital (11.3%), confirming the commission of diagnostic errors identified through clinical audits. However, the percentage of these errors leading to serious patient harm is almost non-existent in our study, while in Zwaan's hospital study it was 4%, suggesting both the lower severity of cases seen in PC, the effect of longitudinally and the options for correction of previous hypotheses involved in continuous patient care over time.37 The second contribution of this research is the lack of a statistically significant association between the use of heuristics and diagnostic error. If these results are confirmed in subsequent studies, it would call into question the classical approach of authors such as Croskerry13 who consider that the systematic use of heuristics or mental shortcuts entails a greater risk of error and validate the hypothesis of authors such as Gigerenzer and Graissmaier,38 who argue that heuristics are a useful and effective way of resolving clinical dilemmas. Some authors add that heuristics can provide a richer and broader knowledge base for making intuitive judgements and decisions.39
The study has several limitations. Firstly, the number of participating physicians is small, but given that the methodology developed is explicit, it can be replicated in subsequent studies with larger populations, bearing in mind in any case that direct observation of clinical decision making under real conditions is complex and costly, and it will hardly be possible to design studies with large populations. Secondly, the lack of previous studies in real clinical conditions makes it difficult to compare results, but this approach provides information on what happens in real conditions, not in laboratory experiments. In this sense, this research initiates a novel approach to the study of the use of heuristics in the clinical decision process in primary care and its relationship with diagnostic error. Thirdly, there is probably a Hawthorne effect on the part of the participants in this study, as well as a certain learning bias in the knowledge that they were going to be evaluated in their attention to this type of problem (dyspnoea), and that they could improve their intervention as a result. It could also be considered as a limitation the bias that could have been produced by the evaluator's knowledge of the doctor responsible for the case evaluated, an unavoidable aspect as the structure of the electronic medical record does not allow ‘blind’ review of the same. In any case, the percentage of errors identified (like Zwaan's study) leads us to believe that there has not been an excessively positive assessment of the cases studied. Another potential limitation of the study is that the operational definitions used to characterize the availability, representativeness and overconfidence heuristics may reflect other constructs or heuristics. However, we believe that our operational definition presents a high concordance to the conceptual framework of the heuristics selected in this case during clinical practice. It should be noted that in no case does this study aim to ascertain the ability of the participating physicians to identify cognitive biases, who, as Zwaan,27 points out, are not able to agree on when a cognitive bias is present.
The results seem to confirm the use of unconscious thinking by family physicians in the care of new episodes of dyspnoea, probably using heuristics, including representativeness, availability, and overconfidence. However, the use of these mental and intuitive shortcuts to establish a diagnosis does not seem to lead to an increase in diagnostic errors. Further research should confirm or disconfirm these results, which are in any case relevant for doctors and patients and useful for the training process of the former.
ACKNOWLEDGEMENT
The authors received funding for the study from the Health Research Fund of the Ministry of Economy and Competitiveness (PI10/01468 and PI13/01175).
CONFLICT OF INTEREST
The authors declare there is no conflict of interest.
AUTHOR CONTRIBUTIONS
All the authors participated in the conception and design of the work. The data were collected by CFA and SM, together with their design and interpretation, which were later evaluated by SM, JMM and AFA. The final version was approved by all authors.
ETHICAL APPROVAL STATEMENT
The project was approved by the Regional Research Ethics Commission of the Granada and Madrid provinces.
Open Research
DATA AVAILABILITY STATEMENT
The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.
