First published online October 12, 2006
British Journal of Radiology (2007) 80, 256-260
© 2007 British Institute of Radiology
doi: 10.1259/bjr/82769410
Practical assessment of the display performance of radiology workstations
D P Thompson, MSc
C J Koller, MSc
and
J P Eatough, PhD
The University Hospital Of North Staffordshire NHS Trust, Stoke-on-Trent, Staffordshire, UK
Correspondence: Mr D P Thompson, University Hospital of North Staffordshire, North Staffordshire Royal Infirmary, Princes Road, Stoke–on–Trent, Staffordshire, ST4 7LN, UK. E-mail: david.thompson{at}uhns.nhs.uk
 |
Abstract
|
|---|
The performance of 14 primary clinical display monitor workstations in use in the Radiology Department of a large acute NHS Trust was assessed using the methods and guidelines described by the American Association of Physicists in Medicine Task Group 18. Tests undertaken included the measurement of ambient light, display uniformity, luminance ratio, luminance response, maximum luminance and spatial resolution. Four display monitors failed to meet at least one of the test's guideline tolerances. In addition a number of display monitors were found to be operating at settings that might reduce their useful life span. These devices were either replaced or recalibrated by the installers, or were subject to local adjustment to ensure applicable standards were met. Consequently the study suggests that quality assurance testing of display monitors used for image reporting is necessary and valuable to ensure that images are viewed at an appropriate standard.
|
Background
|
|---|
Over the last decade there has been an increase in the use of digital imaging and Picture Archiving and Communication Systems (PACS) within hospitals. This has resulted in the increased use of electronic display monitor workstations for both presentation and evaluation of diagnostic images. The Ionising Radiations Regulations 1999 (IRR99) [1] require that any equipment used in connection with a medical exposure should be so maintained as to restrict radiation exposure whilst achieving performance; and in particular a quality assurance programme be provided to achieve these objectives. The approved code of practice clarifies that these requirements apply not only to radiation equipment but also to ancillary equipment associated with the radiation exposure and with image production. In the process of image optimization each step of the imaging chain must be considered, including the way in which the final images are displayed; although it is this final stage that is often overlooked [2].
The recognized standard for digital communications in medicine is DICOM (Digital Imaging and Communications in Medicine). Part 14 of this standard refers to the Grey Scale Display Function (GSDF) for display monitors [3]. The GSDF is designed to account for the response of the eye in order to ensure perceptual linearization of the grey scale [3]. The GSDF effectively ensures that the step from black to 90% grey appears equal to the step from 60% grey to 50% grey for example [4].
The successful implementation of the DICOM GSDF enables the perceived relative contrast between areas in any given image to appear near identical when viewed on any calibrated display monitor. However, the DICOM GSDF, whilst accounting for image perception, is not designed to address a number of other issues that are important to the quality of the displayed image. These include the ambient lighting level in the environment, the maximum luminance of the display, the luminance ratio, luminance uniformity and spatial resolution; tests which have been described by the American Association of Physicists in Medicine (AAPM) Task Group 18 (TG18) in their publication "Assessment of Display Performance for Medical Imaging Systems" [5]. This publication on test methodology has been cited in the recently published Institute of Physics and Engineering in Medicine (IPEM) Report 91 [6], which provides national guidelines on the level and frequency of quality assurance testing, including that recommended for display monitors. The published tolerance levels within the AAPM test procedures account for the two different types of display monitors classified by the report. Primary devices are display monitors used for the interpretation of medical images and are those typically used in radiology. Secondary devices on the other hand are used for viewing medical images for purposes other than providing medical interpretation. These types of display monitor are used by general medical staff and medical specialists [5]. Consequently the AAPM has assigned tighter tolerances for primary compared with secondary units.
The principal aim of this study was to assess the primary clinical display monitors within a large NHS Trust against the AAPM criteria to identify units performing outside the recommended tolerance levels. In addition it was hoped that the experience gained in performing routine quality assurance (QA) testing would progress the development of a Trust-wide programme for clinical image display monitors and potentially maximize the life span of the devices.
|
Methods and materials
|
|---|
Several tests, based on the AAPM protocols, were selected to form the basis of a display monitor's assessment using the AAPM Task Group 18 digital test patterns [5]. The tests selected for assessment encompassed the physical parameters likely to affect image quality, and together with the AAPM guideline tolerances for primary display monitors are given in Table 1
. Test images were downloaded from the AAPM TG18 website (http://deckard.duhs.duke.edu/tg18) and saved on the Trust's digital archive using lossless compression. The test patterns could then be sent to specific display monitors when required.
Fourteen primary liquid crystal displays (LCDs) were tested using the AAPM guidelines. The measurement of light was performed using a calibrated Unfors P11 light meter, capable of measuring both luminance (cd m–2) and illuminance (lux). A foam baffle surrounding the photodiode detector was used in order to minimize the effect of specular and diffuse reflections. Each display monitor was allowed to warm up for 30 min and their surface was cleaned prior to measurement as recommended by the guidelines. A majority of the testing was performed outside normal working hours as this was the only time at which use of the display monitors could be guaranteed given their extensive clinical workload.
Display monitor testing was undertaken in minimized ambient lighting levels, readily attainable in each location and designed to replicate the conditions used for clinical reporting; typically less than 5 lux. The tests were performed initially using the display monitor's contrast and brightness settings found on arrival, along with the default window width and level settings for each DICOM image. The time taken to perform a full survey on each unit was approximately 20 min excluding the time taken for the device to warm up. Some test procedures were repeated following adjustments made to the display monitors in an attempt to meet the AAPM tolerance guidelines, improve the overall image quality or maximize the life span of the unit. This was done using the AAPM TG18 QC test pattern in accordance with the associated guidelines to assist in the correct setting of the contrast and brightness levels.
The following parameters were recorded for this study:
- Ambient light falling on the display monitor was measured using the lux meter positioned at the centre of the screen with the device in "stand by".
- Luminance uniformity was assessed by making measurements in areas of uniform pixel value at the four corners and centre of the screen. Uniformity is defined as the deviation of brightness expressed as a percentage of the average value, and was calculated using the following expression:
where Lmax(U) and Lmin(U) are the maximum and minimum uniformity luminance values recorded.
- Spatial resolution was evaluated across the whole display monitor using Nyquist and half-Nyquist frequency high contrast bar patterns. Prior to measurement the image was adjusted appropriately such that the test image was mapped onto the display at the ratio 1:1.
- The maximum (Lmax), minimum (Lmin) and ambient (Lamb) luminance levels, corresponding to 100% white, 100% black and the environmental light reflected by the display monitor screen when in "stand by" mode, were measured using the light meter. The display monitor's luminance ratio was calculated using the following expression:
- The luminance response of the display monitor was assessed by measuring the change in luminance levels between patterns with equally spaced pixel values across the full range available to the device. By indexing the luminance levels against Just Noticeable Difference (JND) values [3] it was possible to determine the performance of the display monitor against the DICOM GSDF. The luminances of the grey scales tested are determined by the Look Up Table (LUT) found either on the computer video card or the display monitor itself [4]. The LUT on some devices can be altered via appropriate calibration techniques.
A comprehensive methodology for each test procedure used in this study is given in the AAPM Task Group 18 report [5].
|
Results and outcomes
|
|---|
Test results for all 14 display monitors obtained with the set-up found on arrival are shown in Table 2. The compliance of two of the devices with respect to the DICOM GSDF is shown in Figure 1. In addition the luminance response of a typical non-calibrated standard desktop computer display monitor was assessed for comparison purposes and the results are shown in Figure 2.
View this table:
[in this window]
[in a new window]
|
Table 2. Monitor QA measurements recorded at levels found prior to testing. Monitors listed in appropriate performance order with the best first
|
|
View larger version (15K):
[in this window]
[in a new window]
|
Figure 1. The luminance response of 2 of the 14 monitors, expressed as the change in luminance between successive grey scales(dl l–1) and terms of just noticeable difference (JND). The shaded regions correspond to the ±107 AAPM tolerance for the assessment of the DICOM GSDF. (a) and (b) correspond to monitors A and G1, respectively.
|
|
The contrast and brightness levels of a number of display monitors were adjusted after the initial tests. This was done so as to either improve performance (devices H and J) or attempt to prolong the display monitor's life (devices E and F).
The ambient lighting of each room containing clinical displays was assessed and was found to be satisfactory. Recorded illuminance values ranged and between 2 lux and 9 lux and were therefore within the guideline value of 10 lux given by the AAPM [5]. From the assessment of luminance uniformity and resolution, all display monitors met the AAPM's minimum standard for these factors (Table 2
).
The maximum luminance levels recorded for the display monitors, with the contrast and brightness levels set at those found prior to testing, ranged from 65 cd m–2 to 372 cd m–2 (Table 2); four of which failed to reach the AAPM recommended standard (devices G1, H, I and J).
The AAPM Task Group 18 guidelines state that the maximum luminance should not differ by more than 5% between display monitors on a single workstation [5]. Table 2 shows display monitors G1 and G2 failed to meet this criterion by a significant margin. As a result the supplier was informed and the display monitors were subsequently replaced. Further work performed with regards to maximum attainable luminance included slightly reducing the brightness levels of devices E and F, which were found on their maximum brightness setting, so as to prolong their useful life span (Table 3). This was undertaken at the request of, and in discussion with, the PACS project manger whilst ensuring the maximum luminance still exceeded the 170 cd m–2 stipulated by the AAPM, based on the recommendations of the American College of Radiology [7].
View this table:
[in this window]
[in a new window]
|
Table 3. Display monitor QA measurements recorded after adjustment. For monitors E and F brightness levels, which had previously been set to maximum, were reduced to prolong their life. The brightness level of display monitor H was increased in an attempt to meet the AAPM tolerance level for maximum luminance. For display monitor J contrast and brightness levels were increased to improve overall image quality
|
|
In total, of the 14 primary display monitors assessed during initial measurements of luminance ratio, four devices failed to meet the standard expected as described above (devices G1, H, I and J). Remedial action was performed using the TG18 QC test pattern on display monitor J which resulted in the luminance ratio reaching a level consistent with that expected of a device of its classification and an increase in the maximum luminance by a factor of two (Table 3). This was still below the 170 cd m–2 threshold. However its performance was consistent with display monitor I, a device of the same make and model. As a direct consequence of these findings the supplier of display monitors J and I was contacted and they are now believed to be reviewing the specification of their displays against the AAPM guidelines. A decision regarding the continual use of these displays will be dependent on the conclusions of the supplier. Display monitor H was also identified as having a low luminance ratio (Table 3) and will therefore be carefully assessed to ensure there is no significant change in performance in the coming months.
Each display monitor's GSDF was compared with the DICOM part 14 standard through the use of graphs, two of which can be seen in Figure 1. The plots show the relative change in luminance levels of the display as a function of changing JND values. The tolerance for a primary device is ±10% and is represented by the shaded region. Results for a vast majority of the display monitors were satisfactory, with all points falling within the AAPM tolerance range, an example of which can be seen in Figure 1a. Figure 1b illustrates that display monitor G1 had several points exceeding the AAPM guidelines and as a result the manufacturer was called to recalibrate it. However, because the device was also part of the dual-display monitor system that had significantly different luminance values, the device was replaced.
|
Discussion and conclusion
|
|---|
The ambient light in which clinical display monitors are operated is an important parameter when determining the ability of the clinician to detect low contrast details. Excessive background light levels and extraneous light sources can illuminate the display surface. The superposition of light from such sources, and that emitted by the display monitor, can result in the decrease of perceived contrast in the image [8]. Several authors have directly demonstrated the effect of ambient light on soft copy reporting through degradation of the detection limit for low contrast details [9, 10].
All results were obtained in lighting conditions that were fully optimized using the available measures during the analysis of clinical images, i.e. light boxes were switched off and blinds and doors were shut. The change from hard to soft copy reporting can result in a substantial decrease in the luminance of an object when viewed using a primary display monitor compared with that of a film and light box operating with typical values of between 1500 cd m–2 and 3000 cd m–2. Clinicians should be aware of this change and actively encouraged to minimize ambient lighting accordingly so as to maximize the difference between the intensity of the image and ambient lighting to account for the limitations of the display monitor.
The effect of maximum luminance on perceived image quality and its variation over time are significant factors when evaluating the overall performance of a display. Previous studies performed with light boxes [11, 12] have demonstrated that a decrease in the luminance reduces the detection of low contrast objects. This is as a result of insufficient light being received by the retinal cones in the eye [13, 14] and therefore justifies regular assessment, especially when considering the range of luminance levels attainable by current display monitors. However, although high luminance levels are desirable when soft copy reporting, the continual use of a display monitor at its maximum brightness will significantly reduce its useful life span as the output of the backlight reduces over time for LCDs [15].
To ensure a "true" comparison of images on multi-display monitor workstations is achievable, the maximum difference between devices should be no more than 5% [5]. Consequently display monitors for such workstations should have almost identical performance to ensure accurate comparison and reporting of clinical images. This may limit Imaging Departments from freely moving and connecting display monitors to different stations around the Trust.
Seven of the 14 primary display monitors assessed were identified as operating at a level well above the minimum standard regarding maximum luminance [7]. Currently an investigation is being undertaken into the two display monitors which had their brightness levels reduced as a means of prolonging their life (devices E and F). This is in an attempt to ensure that low contrast detection accuracy and the GSDF has not been compromised since reducing the luminance levels to 170 cd m–2. If the results of the study are satisfactory then the brightness settings of the other five devices may also be reduced.
It seems reasonable to believe that a display monitor's fundamental characteristics such as its brightness and resolution will inherently affect reporting accuracy, based on the experiences of imaging with film [11, 12]. However, there is a need for work to be undertaken to confirm the extent to which factors outlined in the Task Group 18 report, such as the device's luminance response and uniformity, have a statistically significant effect with regards to clinical work. In addition, the guideline tolerance values within the report may require confirmation that they remain suitably quantified for the classification of current display monitor workstations.
Reporting display monitors are used to assess images acquired using different radiographic imaging techniques, irrespective of whether they required high or low radiology doses. To ensure the requirements of the IRR99 [1] legislation is adhered to, the implementation of some form of QA programme is both necessary and valuable in ensuring quality standards are met and maintained. Whilst Imaging Departments with limited resources may find the task of initiating a suitable QA programme for a large Trust somewhat daunting, consideration should be given to the results of this study. Several display monitors in this report, each taking 20 min to evaluate, were identified as performing at an inappropriate standard, enabling remedial action to be undertaken to potentially prolong their life and improve image quality.
|
Acknowledgments
|
|---|
The authors would like to thank Mr J Payne and Mr D Butler for helpful comments and assistance during the undertaking of the project.
Received for publication February 14, 2006.
Revision received June 1, 2006.
Accepted for publication August 7, 2006.
|
References
|
|---|
- Ionising Radiations Regulations 1999. SI No. 3232. London: HMSO, 1999
- Brettle DS, Bacon SE. A method for verified access when using soft copy display. Br J Radiol</pub-tl>2005. 78:749–51.
- NEMA. Digital Imaging and Communications in Medicine (DICOM) Part 14: Grayscale standard display function. Virginia: National Electrical Manufacturers Association, 2003
- Eichelberg M, Riesmeier J, Kleber K, Holstein J, Oosterwijk H, Jensch P. Consistency of soft copy and hardcopy: preliminary experiences with the new DICOM extensions for image display. http://wuerlim.wustl.edu/mesa/software/consistency/display-consistency.pdf
- Samei E, Badano A, Chakraborty D, Compton K, Cornelius C, Corrigan K, et al. Assessment of display performance for medical imaging systems. Report of the American Association of Physicists in Medicine (AAPM) Task Group 18, Medical Physics Publishing, Madison, WI, AAPM On-Line Report No. 03, April 2005
- The Institute of Physics and Engineering in Medicine, College of Radiographers, National Radiological Protection Board. Recommended standards for the routine performance testing of diagnostic X-ray imaging systems, IPEM Report No. 91. York: IPEM, 2005
- American College of Radiology. ACR Standards for Teleradiology. In: Standards 2001–2001. Reston, VA: American College of Radiology, 2001: 9–17
- Wade C, Brennan PC. Assessment of monitor conditions for the display of radiological diagnostic images and ambient lighting. Br J Radiol 2004;77:465–71.[Abstract/Free Full Text]
- Goo JM, Choi JY, Im JG, Lee HJ, Chung MJ, Han D, et al. Effect of display monitor luminance and ambient light on observer performance in soft copy reading of digital chest radiographs. Radiology 2004;232:762–6.[Abstract/Free Full Text]
- Fuchsager MH, Schaefer-Prokop CM, Eisenhuber E, Homolka P, Weber M, Funovics MA, et al. Impact of ambient light and window settings on the detectability of catheters on soft-copy display of chest radiographs at bedside. AJR Am J Roentgenol 2003;181:1415–21.[Abstract/Free Full Text]
- Wang J, Gray JE. Detection of small low-contrast objects in mammography: effect of viewbox masking and luminance. AJR Am J Roentgenol 1998;170:105–8.[Abstract/Free Full Text]
- Kimme-Smith C, Haus AG, DeBruhl N, Bassett LW. Effects of ambient light and view box luminance on the detection of calcifications in mammography. AJR Am J Roentgenol 1997;168:775–8.[Abstract/Free Full Text]
- Commission of the European Communities. Quality criteria for diagnostic radiographic images. Working Document (2nd edn), XII/173/90, Brussels, Belgium: CEC, 1990
- Hartmann E, Stieve FE. Quality control of radiographic illuminators and associated viewing equipment. In: Moores BM, Stieve FE, Eriskat H, Schibella H, editors. Report 18, Technical and physical parameters for quality assurance in medical diagnostic radiology. Tolerances, limiting values and appropriate measuring methods. London: The British Institute of Radiology, 1989: 135–7
- CCFL Characteristics. Maxim Integrated Products, Inc. 2005. http://www.maxim-ic.com/appnotes.cfm/appnote_number/3528
This article has been cited by other articles:
|
|
|
|
 
BJR review of the year -- 2007
Br. J. Radiol.,
April 1, 2008;
81(964):
265 - 269.
[Full Text]
[PDF]
|
|
|
Top
Abstract
Background
