British Journal of Radiology 74 (2001),949-951 © 2001 The British Institute of Radiology
Error in radiology
P Goddard, MD, FRCR1,2,
A Leslie, MRCP, FRCR3,
A Jones, MRCP, FRCR1,
C Wakeley, FRCS, FRCR1 and
J Kabala, MRCP, FRCR1
1Bristol Royal Infirmary, 2University of the West of England and 3University Hospital, Lewisham, UK
 |
Abstract
|
|---|
The level of error in radiology has been tabulated from articles on error and on "double reporting" or "double reading". The level of error varies depending on the radiological investigation, but the range is 2–20% for clinically significant or major error. The greatest reduction in error rates will come from changes in systems.
|
Introduction
|
|---|
Error in medicine has become headline news in recent years. Politicians and lawyers have been known to remark that no mistakes are acceptable in medicine and that removal of "bad" doctors will remove the tarnish of error from the National Health Service. At a recent British Institute of Radiology meeting entitled "How much error is acceptable in radiology?", figures were presented regarding the currently reported level of error in radiological investigations, since it is important to know the present situation to determine what is fair and reasonable. A search of the medical literature for articles including "error" or "mistakes in radiology" in the title produces few papers, but articles on "double reporting" or "double reading" can also be informative on the subject of error. This paper reports on the level of error or variance documented in publications on error and "double reading" in radiology.
|
Method
|
|---|
Literature for the past 20 years was searched on Medline using the Knowledge Finder search engine [1]. Representative papers covering double reading and reporting of errors for a range of general radiological procedures were reviewed, excluding breast screening procedures. The simplest measure of error is the remaining percentage when the accuracy has been taken away from the total. Thus, if the accuracy was 85% then the error would be 15%. Accuracy is defined as true positives and true negatives divided by all results (true positive, true negative, false positive and false negative). This definition of error was adhered to wherever possible, but where only partial information was available in the reports this was also tabulated (Table 1
).
In papers on double reading, the divergence between the two observers was taken as being proportional to error. Whenever the paper included an indication of the clinical importance of the divergence, this was also recorded. Minimum error was assumed to be 50% of the divergence rate between the two observers where no overall consensus was reached, or a minimum of 100% of the individual divergence from the consensus view. Where the result is given as an individual percentage concordance with the consensus view, this is equal to the accuracy. The error is then taken to be the total minus the concordance in percentage.
|
Results
|
|---|
Chest radiology, CT, barium studies and MRI were taken as examples covering a broad field of radiology. Results obtained from the published literature have been tabulated (Table 1
).
|
Discussion
|
|---|
Many radiologists would be very surprised at the level of error described in this short report, but, as a recent leader in the British Medical Journal reminded us, "we are human and humans err" [13]. The leader continues: "suspend every doctor today who makes an error today and the error rates in the NHS tomorrow will be exactly the same as today's". It must be realized that error is purely a measure of variance from the perfect result. All radiologists will have experienced the situation where discussion at a clinicoradiological meeting resulted in further information being obtained from a radiograph in addition to that already supplied in a report. In such circumstances the original report must be deemed as being in "error". This is, of course, an everyday occurrence or such meetings would not be considered worthwhile. It could be argued that the radiological process has not been completed until such a meeting has taken place and therefore no "error" has occurred and no detriment suffered by the patient. Perhaps "variance" is a better term in this context since it is less emotive, but "error" is the usual term and has therefore been used in this report. This paper examines the level of error in radiological reporting as described in the literature and does not purport to indicate whether or not this discrepancy actually led to any harm.
Colloquially, error implies that the practitioner could have done better. This may be the case for part of the error as recorded in these papers, but some discrepancy from the final findings is always inevitable. Because of the overlap between abnormal and normal populations, there is almost no test in which there is a perfect separation between normal and abnormal results.
When double reporting, the divergence or discrepancy between two observers is related to error since, although one of the observers may be correct, at least one must be considered as less correct. Where individual concordance figures are given, the error is then taken to be 100% minus the percentage concordance. This result can be termed the divergence and, where individual figures are given, divergence equals error. In some papers the overall concordance of two radiologists is given, but not their individual performance. For such papers the mean error rate will be half the rate of divergence, although the individual error rate of each radiologist could range from zero to the entire divergence. Further clinical information (endoscopy, biopsy, follow-up) was also available in several papers.
Divergence that would potentially alter management can be considered as major error. Papers on multiple reading of diagnostic images show that there is considerable improvement with the addition of the second reader and a further slight improvement with the addition of a third [5, 8, 12]. Some error still remains however many readers are added.
Sensitivity in the reporting of CT of the abdomen and pelvis in one series [5] improved from 79% to 85% with replicated reading (separate double reporting), although specificity was not significantly changed. Accuracy improved from 84% to 88.5% and error dropped from 16% to 11.5%. It is not possible to tell from the paper how much of the remaining error was potentially avoidable, but replicated reading clearly reduced error by an avoidable 4.5%.
Misinterpretation [3] of recognized findings may be due to a number of factors, including:
- individual variation, including overlap of findings between normal and abnormal populations;
- technical factors;
- varying co-operation;
- false localization;
- inaccurate differential diagnosis and incorrect clinical information [6].
Errors can be reduced by improvements both in knowledge and in systems.
Improved knowledge and skills may include:
- awareness of history and clinical symptoms [3,6];
- comparison with previous studies [3];
- systematic analysis of all anatomical compartments [3];
- careful selection of the initial and subsequent radiological or clinical investigation [3].
Systems changes include:
- improvement in working conditions and in the time available for reporting;
- equipment alteration to prevent accidental error [13];
- double reporting [2, 6, 8, 12];
- regular, recorded dialogue between clinicians and radiologists [2, 6].
|
Conclusion
|
|---|
Since error is inevitable, it is counterproductive to censure doctors for an individual error. It is the overall practice and error rates that should be examined to determine whether or not an individual is at fault. The greatest reduction in error rates is likely to come from changes in systems.
Received for publication March 9, 2001.
Revision received May 25, 2001.
Accepted for publication June 28, 2001.
|
References
|
|---|
-
Knowledge Finder. North Andover, MA: Aries Systems Corporation (www.kfinder.com).
-
Shaw NJ, Hendry M, Eden O. Inter-observer variation in interpretation of chest X-rays. Scott Med J 1990;35:140–1.[Medline]
-
Vock P. Frequent errors in the interpretation of chest x-ray films. Schweiz Med Wochenschr 1987;117:928–32. [In German.][Medline]
-
Roszler MH, McCarroll KA, Rashid T, Donovan KR, Kling GA. Resident interpretation of emergency computed tomographic scans. Invest Radiol 1991;26:374–6.[Medline]
-
Fultz PJ, Jacobs CV, Hall WJ, Gottlieb R, Rubens D, Totterman SM, et al. Ovarian cancer: comparison of observer performance for four methods of interpreting CT scans. Radiology 1999;212:401–10.[Abstract/Free Full Text]
-
Leslie A, Jones A, Goddard P. Influence of clinical information on the reporting of CT by radiologists. Br J Radiol 2000;73:1052–5.[Abstract]
-
Bechtold RE, Chen MY, Ott DJ, Zagoria RJ, Scharling ES, Wolfman NT, et al. Interpretation of abdominal CT: analysis of errors and their causes. J Comput Assist Tomogr 1997;21:681–5.[Medline]
-
Markus JB, Somers S, O'Malley BP, Stevenson GW. Double-contrast barium enema studies: effect of multiple reading on perception error. Radiology 1990;175:155–6.[Abstract/Free Full Text]
-
Arfeen S, Salter RH, Girdwood TG. A negative double-contrast barium meal—qualified reassurance. Clin Radiol 1987;38:49–50.[Medline]
-
De Smet AA, Tuite MJ, Norris MA, Swan JS. MR diagnosis of meniscal tears: analysis of causes of errors. AJR 1994;163:1419–23.[Abstract/Free Full Text]
-
Mackenzie R, Keene GS, Lomas DJ, Dixon AK. Errors at knee magnetic resonance imaging: true or false? Br J Radiol 1995;68:1045–51.[Abstract]
-
Wakeley CJ, Jones AM, Kabala JE, Prince D, Goddard PR. Audit of the value of double reading magnetic resonance imaging films. Br J Radiol 1995;68:358–60.[Abstract]
-
Berwick D. Not again! BMJ 2001;322:247–8.[Free Full Text]
This article has been cited by other articles:
|
|
|
|
 
K. Peldschus, P. Herzog, S. A. Wood, J. I. Cheema, P. Costello, and U. J. Schoepf
Computer-Aided Diagnosis as a Second Reader: Spectrum of Findings in CT Studies of the Chest Interpreted as Normal
Chest,
September 1, 2005;
128(3):
1517 - 1523.
[Abstract]
[Full Text]
[PDF]
|
|
|
Top
Abstract
Introduction
