This Article Abstract Full Text (PDF) Services Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via Google Scholar Google Scholar Articles by Brealey, S Articles by Longsworth, D Search for Related Content PubMed PubMed Citation Articles by Brealey, S Articles by Longsworth, D

The costs and effects of introducing selectively trained radiographers to an A&E reporting service: a retrospective controlled before and after study

¹ York Trials Unit, Department of Health Sciences, University of York, York YO10 5DD, ² York Hospital, Wigginton Road, York YO31 8HE, ³ Department of Health Sciences, University of York, York YO10 5DD, ⁴ The Queen Elizabeth Hospital, Gayton Road, Norfolk PE30 4ET and ⁵ The Derwent Practice, Norton Road, Malton YO17 9RF, UK

	Abstract

Top Abstract Introduction Methods Results Discussion References

 
The costs and effects of introducing selectively trained radiographers reporting accident and emergency (A&E) radiographs of the appendicular skeleton in a district general hospital were assessed using a retrospective controlled before and after design. Reference standard reports were compared with a random stratified sample of 200 A&E and 200 general practitioner (GP) reports before and after the intervention. GP reports were used as a non-intervention, non-equivalent control group. An A&E specialist registrar judged whether incorrect A&E reports might have a clinically important effect on patient management. The effect of incorrect A&E reports on outcome was assessed by patient re-attendance to the hospital because of missed abnormalities. The annual, average and incremental costs of radiographers and radiologists reporting A&E radiographs were calculated and a sensitivity analysis was undertaken. The introduction of the radiographers resulted in a 1% (95% CI –7.9 to 5.9) fall in A&E radiograph reporting accuracy and 11% (95% CI –33.7 to 11.3) reduction of cases in which incorrect A&E reports might have a clinically important effect on patient management. Only two A&E reports (one before and one after the intervention) affected patient outcome in that a fracture missed at the first visit resulted in patient re-attendance to the X-ray Department. There was a saving of £361 per annum to the X-ray Department. In conclusion this study provides further evidence that selectively trained radiographers can accurately report A&E plain radiographs and at no additional cost.

	Introduction

Top Abstract Introduction Methods Results Discussion References

 
A shortage in doctors and the need to address increasing demand in a healthcare system with scarce resources brought about changes in government policy that promote the flexible and creative use of allied healthcare professional skills [1–4]. In X-ray departments, a national shortage of radiologists [5, 6] and a continuing rise in the level of activities undertaken [7] provided a climate that encouraged the development of radiographers in a written reporting role [8, 9]. Furthermore, reporting plain radiographs should increase radiographer job satisfaction and enhance their professional standing [10]; radiologists' time could also be freed to perform other more specialist and complex investigations [11].

There is evidence that selectively trained radiographers can accurately report Accident and Emergency (A&E) radiographs [12–14] resulting in a growing number performing this activity [15, 16]. Despite an emphasis on the clinical contribution of radiologists' reports to the management of A&E patients [17–19] this has not been evaluated for radiographer reporting; although there is evidence that radiographer reports do not adversely affect A&E patient outcome in terms of re-attendance to the X-ray Department [20]. The College of Radiographers vision was that radiographers in a reporting role should revolutionise the cost-effective management of patients [21] yet no robust cost analysis of radiographers reporting A&E plain radiographs has been performed [22].

In February 1995, following a period of postgraduate training in plain radiograph interpretation, two carefully selected radiographers in our hospital began to report A&E radiographs judged normal or insignificantly abnormal by the casualty officers. The X-ray department performs more than 7500 examinations per month, of which 29% are referred from A&E. This corresponds with a report which found that 27% of the radiology workload was generated by referral from A&E [23]. The radiographers report the appendicular skeleton, i.e. upper limb (shoulder girdle to fingers), lower limb (hip to toes) and foreign body examinations, but do not report the axial skeleton (pelvis, spine, head), chest, thorax and abdomen. Their introduction to the A&E reporting service was an opportunity to assess whether this skill mix initiative was an effective and efficient use of scarce resources [24]. The aim of this study was to assess the effect of introducing radiographer reporting on the accuracy of A&E reports, the subsequent effects on A&E patient management and outcome, and to examine the associated costs.

	Methods

Top Abstract Introduction Methods Results Discussion References

 
Assessment of accuracy
To assess the effect on accuracy a random sample of A&E plain radiograph reports for the period February 1993 to January 1997 (2 years before and after the intervention) was selected. This was to minimize the effects of statistical regression and maturation, ensure a similar sample of reports were compared before and after the intervention, and to reflect the population of reports on A&E plain radiographs in clinical practice. To further underpin the study, so that a change in A&E radiograph reporting accuracy can be attributed to the intervention rather than other historical events, a random sample of general practitioner (GP) plain radiograph reports were selected as a non-equivalent, non-intervention control group. The GP reports are non-equivalent because these referrals are for a different type of patient with different pathology. It is a non-intervention control group as the radiographers do not report GP radiographs. The samples of A&E and GP reports were stratified by time and body area. This was to control for the effect on accuracy of reporting on different pathology at different times of the year and for radiographs that the radiographers do and do not report. The body areas that the radiographers do not report were included to capture the wider implications of introducing this intervention. For example, radiologists not reporting A&E radiographs of the appendicular skeleton might free them to spend more time reporting on the axial skeleton, which could affect the accuracy with which these radiographs were reported.

To select the reports a random sample of 3 days for each quarter were chosen. The Systems and Networks Service Department provided a list of all A&E and GP examinations performed for the selected days. A random sample of 25 A&E and 25 GP reports were selected for each quarter stratified by body area. To assess the accuracy of the reports all the radiographs were made available so that a consultant radiologist with 11 years experience in radiology and a special interest in skeletal radiology could produce a reference standard report. The radiologist reported in normal viewing conditions using the same format as during clinical practice and had access to previous radiographs but not the original and previous reports to prevent bias [25]. The arbiter who compared reports for concordance, blind to who had produced each report, was a consultant radiologist at a different hospital with 10 years experience and a special interest in skeletal radiology. Normal was defined as within normal limits, no bony injury, or a minor abnormality such as an old fracture. Abnormal was defined as all clinically relevant abnormalities. Reports were only judged concordant if they agreed on the presence, location and type of abnormality [26].

Assessment of management
For incorrect A&E reports an A&E specialist registrar decided whether this might have a clinically important effect on patient management compared with the reference standard report. Knowledge that only discordant pairs of reports were used might affect the registrar's decision about an expected difference in patient management so for every two pairs of discordant reports a convenience sample of a concordant pair of reports was included in the assessment. The concordant sample of reports was then excluded from the analysis. When comparing pairs of reports the registrar could view the plain radiographs and was provided with information from X-ray request cards, case notes, and casualty cards and chose one of the following options:

• No difference in management (treatment or advice);
  
• Clinically unimportant difference in management – the patient may have been treated differently but this would not affect patient outcome;
  
• Clinically important difference in management – the patient may have been treated differently and patient outcome would be affected.
  

Assessment of outcome
Patient re-attendance to A&E or the X-ray department was used as a proxy for outcome, as it may indicate that significant pathology was missed at the first visit. For A&E patients whose report was judged to be incorrect the casualty X-ray card and patient case notes were searched to identify whether a patient re-attended the A&E department within 3 months of the first visit. The Radiology department computer system was also searched to establish whether a patient re-attended for further radiographic examinations or other procedures within 1 year of the first visit. The process of searching case notes and the Radiology department computer identified three possible outcomes for patients:

• Did not re-attend A&E or X-ray department;
  
• Re-attended X-ray department for an unrelated examination;
  
• Re-attended X-ray department for a related examination.
  

When a patient did not re-attend A&E or the X-ray department it was assumed that any error in the report had a negligible effect on patient outcome. When a patient re-attended for a further X-ray examination that was unrelated to the same symptomatic body area as the first examination it was assumed their re-attendance was not due to an error in the report. For those patients who re-attended for a repeat X-ray examination of the same anatomical area or for a different imaging procedure it was important to establish whether an error in the report at the first visit resulted in their re-attendance. To do this a consultant radiologist compared the report from the first visit with reports of the subsequent examinations. When a discrepancy was found between the first and subsequent reports all examinations were reviewed in consensus by a group of three consultant radiologists. The reference standard radiologist or radiologist responsible for the report of radiographs under review was not included in this group to eliminate bias [25].

Assessment of costs
A cost analysis of radiographers and radiologists reporting A&E plain radiographs of the appendicular skeleton was performed from the perspective of the X-ray department. This included the capital costs of providing radiographers with the equipment and facilities to report, costs of training radiographers, and the overhead costs of the rooms used for reporting activity. There were no additional medicolegal or insurance costs. Consumable costs were excluded as they were negligible and the same for both professions. The cost of auditing radiographer reporting was excluded because there was already protected time to perform audits. The cost of radiographers on the rare occasion (once every 2 weeks) seeking advice from radiologists was excluded as radiologists also sometimes seek the opinion of their colleagues.

The costs were inflated to the base year 2001/2 using appropriate indices [27] and the capital costs were annuitized to produce the annual equivalent cost (AEC) [28]. When calculating the cost of radiographer and radiologist reporting it was important to estimate the time spent reporting and typing reports. The two radiographers and radiologists at the hospital were therefore asked to report again the A&E radiographs of the appendicular skeleton that the reference standard reported and record the time it took using a stopwatch. The time a radiologist spent dictating their report to tape was assumed to be the time a medical secretary spent entering the report into the computer. The radiographers also recorded the time it took to record and enter their report into the computer. To value the time spent reporting it was essential to weight for differences in the clinical productivity of the two professional groups. Radiologists and radiographers, respectively, spend 34 h and 26 h a week on clinical activity so 1 clinical minute for a radiologist is equivalent in value to 1.3 clinical minutes for a radiographer. This variation in clinical activity occurred because radiologists allocated around 15% of their time to administrative duties and clinical audit. In contrast the radiographers spend around 50% of their time on reporting, 25% on clinical activities such as general radiography and fluoroscopy, and 25% on administrative duties such as quality assurance, radiation protection, staff recruitment and organizing rotas.

A considerable number of assumptions can be made about the costs that should be included in radiographer and radiologist reporting so a series of scenarios were constructed that recognized the uncertainty of these assumptions.

Analyses
Radiograph reporting performance was presented using sensitivity, specificity, and accuracy with 95% confidence intervals (CI) around a single proportion and for the difference in proportions between two independent groups (i.e. before and after samples). The hypothesis tests used were the Chi-squared test or Fisher's Exact test when the total number of observations were less than 20 or any of the expected frequencies were less than 5. To analyse the effect of incorrect reports on A&E patient management the 95% CI when testing for a difference in proportions between two independent samples were calculated.

The cost analyses required the time it took in clinical minutes for radiographers and radiologists to report to be multiplied by their hourly rate. This cost was then multiplied by the number of A&E radiographs of the appendicular skeleton performed during 2001/2 (n=9713). The sum of the capital, training and overhead costs was then added to produce the annual cost of reporting. The average cost of reporting was the annual cost divided by the number of A&E radiographs of the appendicular skeleton performed during 2000/1. The incremental cost was the difference in cost between radiographer and radiologist reporting. Sensitivity analysis tested the impact of changes in assumptions about the inclusion of certain costs.

	Results

Top Abstract Introduction Methods Results Discussion References

 
Effect on accuracy
As shown in Table 1, both before and after the introduction of radiographer reporting, specificity was high in the 90s, but sensitivity only around the 50s. This indicated that normal A&E radiographs were correctly identified but nearly half of A&E radiographs judged to be abnormal by the reference standard were reported as normal. There was also more variation in sensitivity than specificity before and after the intervention. Table 2 shows that following the introduction of radiographer reporting there was a clinically insignificant difference of 1% in A&E radiograph reporting accuracy with 95% CI from –7.9% to 5.9%. For the non-equivalent, non-intervention control group there were no statistically significant differences before and after the intervention in GP radiograph reporting sensitivity, specificity, and accuracy for the different body areas and in total. The difference in GP radiograph reporting accuracy overall was 0% (95% CI –8.9 to 7.9). This is evidence that there were no other historical events that might have affected the accuracy with which radiographs were reported in the X-ray Department and underpins the validity of the results about the effects of this intervention on the reporting accuracy of A&E radiographs.

	Discussion

Top Abstract Introduction Methods Results Discussion References

A previous study found that the costs of introducing radiographer reporting in four NHS trusts ranged from nil to £15 000 per annum with radiographers spending around 0.5 whole time equivalent reporting [22]. The authors acknowledged that the data were limited to staff costs and were unreliable. Our analyses showed that introducing radiographer reporting saved the X-ray Department £361 per annum and further cost savings could be made as radiographers acquire the same experience as the radiologists and if secretaries type radiographers' reports. We included capital costs in the cost of radiographer reporting because it was not practical for the radiographers to use the equipment and facilities available to the radiologists so modification of an existing room and provision of additional equipment was necessary. Furthermore we included the cost of radiographer training but not radiologist training as this too was an additional cost fundamental to introducing radiographer reporting at our hospital. The cost of radiographer reporting may vary though between hospitals depending on how this change in service delivery is implemented. Excluding the capital costs (£416 per annum) and training costs (£684 per annum) from our analyses would result in a base case saving to the X-ray Department of £1461 per annum.

There were also the resource management implications and opportunity cost of radiographer and radiologist reporting. This concerns the activities that radiographers and radiologists relinquish to report A&E radiographs. To explore whether it was reasonable to assume that radiographer reporting freed radiologists to make better use of their time and expertise we conducted a retrospective assessment of different healthcare professional attitudes towards radiographer reporting [29]. The qualitative findings revealed that radiographer reporting A&E radiographs of the appendicular skeleton reduced their involvement in general radiography and they no longer worked in A&E or speciality areas such as CT or angiography. The radiographers also commented that reporting radiographs takes up "25% to 50%" of their time. In contrast the radiologists thought that radiographer reporting frees only a little of their time, possibly "about hours per week". Because this activity was shared between eight consultant radiologists, as opposed to two radiographers, this might explain why only a little of the radiologists' time appears to have been freed. These findings suggest that whereas the reporting of radiographs had a dramatic impact on activities performed by the radiographers, there was a negligible change to the radiologists' work. Not prospectively collecting data meant it was difficult to accurately establish the affect of introducing radiographer reporting on changes in staff workload and the time allocated to different activities in the X-ray department.

Another limitation of this study was the validity of a single consultant radiologist as the reference standard [30], although there was consensus in opinion among the radiologists in the X-ray Department that this radiologist was an acceptable local standard. The reference standard report was also judged to be correct for seven of the eight cases reviewed by the panel of radiologists when establishing the reason for patient re-attendance. For the other case, whereas the reference standard reported a "small bony flake adjacent to the lateral epicondyle" for a radiograph of the elbow, the consensus panel judged it to be a "fleck of calcium". This slight difference in interpretation was not clinically important, and both reports agreed there was "no convincing radial head fracture". There was also the sample size to consider. As an example, following the introduction of the intervention A&E radiograph reporting accuracy fell by 1%, which we can assume to be clinically unimportant. The wide confidence interval shows that 95% of the time the true difference ranges from –7.9% to 5.9% accuracy; a fall of 8% in accuracy might be judged clinically important. The width of the confidence interval affects our certainty about the true difference in A&E radiograph reporting accuracy before and after the intervention and whether this was clinically important.

Other limitations include the completeness and accuracy of information extracted from case notes when assessing the effect of incorrect reports on A&E patient management. Indeed, there were notes missing for two A&E patients before and five A&E patients after the intervention, although it is unlikely this affected the results. The method of assessing patient outcome was also restricted with the loss of three radiographs before the intervention preventing the consensus review. Patient re-attendance as an adverse event and proxy for patient outcome also ignores the effects of false positive reports and the morbidity of patients who suffer but do not re-attend. Furthermore patients who attended A&E may not have been local people so might have re-presented at a different hospital resulting in an underestimation of patient re-attendance as an adverse event. Because we selected a random stratified sample of A&E reports any underestimation in re-attendance should be comparable before and after the intervention so should not affect our conclusions. The study is also only applicable to selectively trained radiographers reporting in a district hospital.

Nevertheless, this study is a valuable contribution to the evidence base as unlike previous studies it addresses accuracy, patient management and outcome, and costs [24]. It is also more pragmatic and rigorous in design, including independent reporting of radiographs by the reference standard for all radiographs and independent comparison of reports. The time period for data collection, the selection of a random stratified sample of reports, and the control group all help to eliminate threats to validity. All costs of radiographer and radiologist reporting to the X-ray department were comprehensively identified, measured and valued, and sensitivity analysis conducted acknowledging uncertainty in key assumptions. In conclusion this study provides further evidence that selectively trained radiographers can accurately report A&E plain radiographs and also at no additional cost.

	Acknowledgments

 
The authors are very grateful to the staff in the X-ray Department for their assistance in the conduct of this study only with their support was this study feasible.

Received for publication July 14, 2004. Revision received November 10, 2004. Accepted for publication January 5, 2005.

	References

Top Abstract Introduction Methods Results Discussion References

 

1. Department of Health. National Health Service and Community Care Act. London: Stationary Office, 1990.
2. Secretaries of State for Health, Wales, Northern Ireland and Scotland. Working for patients. London: Stationary Office, 1989.
3. Department of Health. The NHS Plan: A plan for investment. A plan for reform. London: Stationary Office, 2000.
4. Department of Health. Meeting the challenge. A strategy for the allied health professions. London: Stationary Office, 2000.
5. The Royal College of Radiologists. Staffing and standards in departments of clinical oncology and clinical radiology. London: Royal College of Radiologists, 1993.
6. Brindle M. Debate intensifies over non-radiologists' roles. Diagnostic Imaging Europe 1996;43–7.
7. The Royal College of Radiologists. Clinical radiology: A workforce in crisis. London: The Royal College of Radiologists, 2002.
8. The Royal College of Radiologists. Medico legal aspects of delegation. London: The Royal College of Radiologists, 1993.
9. The College of Radiographers. X-ray examination requests by nurse practitioners and radiographic reporting by radiographers. London: College of Radiographers, 1995.
10. Hughes H, Hughes K, Hamill R. A study to evaluate the introduction of a pattern recognition technique for chest radiographs by radiographers. Radiography 1996;2:263–88.[CrossRef]
11. Craven C, Barber J. Role extension: Where are we now? Radiography Today 1995;61:12–3.
12. Robinson PJA. Plain film reporting by radiographers — a feasibility study. Br J Radiol 1996;69:1171–4.[Abstract]
13. Loughran CF. Reporting of fracture radiographs by radiographers: the impact of a training programme. Br J Radiol 1994;67:945–50.[Abstract]
14. Brealey S, King DG, Crowe MTI, Crawshaw I, Ford L, Warnock N, et al. Accident and Emergency and General Practitioner plain radiograph reporting by radiographers and radiologists: a quasi-randomized controlled trial. Br J Radiol 2003;76:57–61.[Abstract/Free Full Text]
15. Paterson AM. Role development - towards 2000: a survey of role developments in radiography. London: College of Radiographers, 1995.
16. Price RC, Le Masurier SB, High L, Miller LR. Changing times: a national survey of extended roles in diagnostic radiography. Br J Radiol 1999;72:7.
17. De Lacey G, Barker A, Harper J, Wignall B. An assessment of the clinical effects of reporting accident and emergency radiographs. Br J Radiol 1980;53:304–9.[Abstract]
18. Snow DA. Clinical significance of discrepancies in roentgenographic film interpretation in an acute walk-in-area. J Gen Intern Med 1986;1:295–9.[Medline]
19. Thomas HG, Mason AC, Smith RM, Fergusson CM. Value of radiograph audit in an accident service department. Injury 1992;23:47–50.[CrossRef][Medline]
20. Robinson PJA, Culpan G, Wiggins M. Interpretation of selected accident and emergency radiographer examinations by radiographers: a review of 11 000 cases. Br J Radiol 1999;72:1–6.[Medline]
21. The College of Radiographers. Reporting by radiographers: A vision paper. London: College of Radiographers, 1997.
22. Piper K, Paterson AM, Ryan C. The implementation of a radiographic reporting service for trauma examinations of the skeletal system, in Four National Health Service Trusts. Research & Development Directorate, South Thames Regional Office (NHSE) 1999.
23. Audit Commission. Improving your image: how to manage radiology services more effectively. London: Stationary Office, 1995.
24. Brealey S. Measuring the effects of image interpretation: an evaluative framework. Clin Radiol 2001;56:341–7.[CrossRef][Medline]
25. Brealey S, Scally AJ. Bias in plain film reading performance studies. Br J Radiol 2001;74:307–16.[Abstract/Free Full Text]
26. Robinson PJA. The nature of image reporting. In: Paterson A, Price R, editors. Current topics in radiography 2. London: W B Saunders, 1996:70–82.
27. Netten A, Curtis L. Unit costs of health and social care 2002. Kent: Personal Social Services Research Unit, 2002:187.
28. Drummond MF, Stoddart GL, Torrance GW. Methods for the economic evaluation of health care programmes. Oxford: Oxford Medical Publications, 1997.
29. Brealey S, King D, Warnock N. An assessment of different health care professionals' attitudes towards radiographers' reporting A&E films. Radiography 2002;8:27–34.[CrossRef]
30. Brealey S, Scally AJ, Thomas NB. Methodological standards in radiographer plain film reading performance studies. Br J Radiol 2002;75:107–13.[Abstract/Free Full Text]

This Article Abstract Full Text (PDF) Services Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via Google Scholar Google Scholar Articles by Brealey, S Articles by Longsworth, D Search for Related Content PubMed PubMed Citation Articles by Brealey, S Articles by Longsworth, D