British Journal of Radiology (2009) 82, 69-72
© 2009 British Institute of Radiology
doi: 10.1259/bjr/13585245
British Journal of Radiology 82 (2009),69-72 ©2009 The British Institute of Radiology
Interobserver variability in the measurement of abdominal aortic calcification using unenhanced CT
D J BOWDEN
1
S R I AITKEN, MSc
1
I B WILKINSON, DM, MA, MRCP
2 and
A K DIXON, MD, FRCR, FRCP
1
1 Department of Radiology, University of Cambridge School of Clinical Medicine and 2 Clinical Pharmacology Unit, Addenbrooke's Hospital, Box 111, Hills Road, Cambridge CB2 2SP, UK
Correspondence: David Bowden, Department of Radiology, University of Cambridge School of Clinical Medicine, Addenbrooke's Hospital, Box 111, Hills Road, Cambridge CB2 2SP, UK. E-mail: davidjbowden{at}gmail.com
 |
Abstract
|
|---|
Arterial calcification is well recognized as being associated with an increased risk of adverse cardiovascular events. Numerous methods for its quantification have been published, with no consensus on the technique used. In order to assess the reproducibility of a novel technique for quantifying aortic calcification, we measured the interobserver variability between two observers analysing the abdominal aortas of 34 volunteer patients. Using non-contrast abdominal CT images together with commercial imaging software, the quantity of calcium in a pre-determined section of aorta was calculated for each patient, and the difference in values obtained between the two observers compared using a Bland–Altman plot. Minimal interobserver variability was observed, with a significant difference in results occurring for only two patients. This protocol therefore represents a reliable technique that may be applied as a future standard in order to facilitate comparison between studies.
|
Introduction
|
|---|
The importance of arterial calcification and its influence on cardiovascular events are well known. In particular, coronary artery calcification has been associated with a markedly increased risk of myocardial infarction and death [1]. Similar relationships have also been established between calcification of the aorta and adverse cardiovascular events: (i) aortic calcification can be used as a marker for coronary artery disease; (ii) it has been found to be positively related to systolic blood pressure; and (iii) it is associated with increased morbidity and mortality in patients with end-stage renal failure [2–4]. Calcification of the aortic media is recognized from the fourth decade onwards [5] and is also a common feature of those with renal disease; this may also help to explain why antihypertensive drugs sometimes fail to reduce arterial stiffness, with subsequent adverse effects on outcome [6].
As part of a study investigating the possible relationship between aortic calcification and arterial stiffness, the amount of calcification in a pre-determined section of abdominal aorta was quantified by analysis of images obtained by unenhanced CT of volunteer patients. No standardized protocols have been established for aortic calcification measurement and, in those studies that have been published, there has been a wide variation in the methods used and a lack of demonstration of reproducibility [7], with the result that direct comparison between studies has been difficult. One recent study of the quantification of aortic calcification demonstrated good reproducibility [8]; however, the subjects used all suffered from abdominal aortic aneurysms or symptomatic peripheral vascular disease, and underwent CT angiography as part of their further assessment. As a result, the methodology and results obtained cannot easily be translated into normal subjects undergoing unenhanced CT. In addition, several studies have described the quantification of aortic calcium using the "aortic calcification index", in which the proportion of the aortic circumference covered by calcification is calculated from unenhanced CT images [9, 10]. However, these studies involved the subjective identification and subsequent selective analysis of only those cross-sectional images with the most extensive atherosclerosis, and such a method does therefore not easily allow objective quantification of calcium content in entire sections of aorta.
In order to obtain reliable data and enable future comparisons between studies, the interobserver variability of the proposed calcium quantification method was assessed. If such variation is acceptable, a standard protocol can therefore be established that will facilitate future comparisons.
|
Methods and materials
|
|---|
75 subjects were recruited for a separate study into the possible relationship between aortic calcification and stiffness. Subjects were recruited from a community-based database of healthy volunteers, NHS outpatient clinics and general practice referrals to the hypertension service. All 75 subjects underwent thoracic and abdominal CT imaging as part of the study, and the images obtained from 34 of the 75 subjects were chosen randomly (from the order that the images were recorded onto hard copies) to be assessed for calcification by two independent observers. The protocol was approved by the local research ethics committee and informed written consent was obtained for each subject.
Imaging was performed using a 16-slice CT system. 1.5 mm slices were obtained of the thorax, abdomen and pelvis; the images were then analysed using a Leonardo workstation with Syngo software (Siemens Medical Solutions, Forcheim, Germany). The volume of aortic calcification was measured within a defined section of abdominal aorta as follows:
- Images were viewed using a window preset of "mediastinum". The level of the aortic bifurcation was chosen as the starting point for analysis, and was determined on axial CT images as the most distal point at which the aorta is still approximately circular with no clear evidence of bifurcation.
- This slice and the 33 slices proximal to it were selected for analysis, thereby representing a 4.95 cm cylindrical section of aorta.
- The Syngo Volume Calculation tool enables the calculation of a volume from a series of axial images. This is achieved by defining the volume of interest (VOI) on each axial slice and by limiting the minimum and maximum Hounsfield unit (HU) values for calculation. In this study, individual areas of calcification within the aortic wall were identified on each slice and selected as a VOI by the investigator labelling them with the cursor (Figure 1
). The software automatically prevented the inclusion of voxels that fall outside pre-set threshold limits of 130–3000 HU. In this way, only calcification within the aortic wall was included in the volume calculations.
- Once all 34 slices had been analysed, the total volume of calcification within the section of aorta was calculated automatically by the software summating the VOIs from each slice. This process was subsequently repeated for each of the 34 subjects.
Each observer was blind to the results of the other during data collection. In order to assess interobserver variability, two graphical statistical methods were used. Firstly, data obtained from both observers for each subject were plotted against each other, and a line of equality was drawn (Figure 2). This allowed an initial subjective assessment of the degree of variability present. In addition, the average of the two observers' readings, as well as the difference in their values, was calculated for each subject. These data were subsequently recorded on a Bland–Altman plot — a graphical statistical tool commonly used to compare two sets of measurements — thereby allowing the degree of variation present to be analysed objectively (Figure 3).
View larger version (13K):
[in this window]
[in a new window]
|
Figure 2. Plot of Observer 1 data against Observer 2 data for each subject. The line of equality is also shown and illustrates the minimum variation in results between observers.
|
|
View larger version (11K):
[in this window]
[in a new window]
|
Figure 3. Bland–Altman plot of the difference in calcification volume against the mean for each subject, as measured by two independent observers. SD, standard deviation.
|
|
|
Results
|
|---|
Minimal interobserver variability was observed with the protocol used. In Figure 2
, there is little deviation from the line of equality (a theoretical line on which all points would lie if the two observers obtained identical readings) and the agreement between observers appears to be excellent. However, because of less clustering of data, the Bland–Altman plot illustrated in Figure 3 allows differences to be assessed objectively and shows that, out of 34 subjects analysed, only two measurements were significantly different and fell outside the limits of agreement (±1.96 standard deviations from the mean). In each case, explanations for the variation can be proposed. In the case of Subjects A and B, calcification was extremely heavy around the region of the aortic bifurcation, with the result that any difference between observers in the position chosen to start measurements would result in a disproportionately large variation in the final result. Each point represents a subject, with the difference between observers' values and the limits of agreement being shown on the y-axis. The mean difference between observers' results is shown on the x-axis. Significant variation occurred in only two individuals (Subjects A and B).
For Subject A, there was a 4.5 mm difference between the observers in the estimation of the level of aortic bifurcation, thereby explaining the discrepancy in results. Similarly, in the case of Subject B, there was a 3 mm disagreement. Depending upon an individual's anatomy, estimating the precise point of bifurcation may be challenging and somewhat subjective. If the distal abdominal aorta is especially tortuous, the possibility for variation between observers increases as the cross-section of the aorta adopts a more ovoid shape in the raw axial images. The precise point of bifurcation then becomes harder to determine. For Subjects A and B, therefore, the concentration of calcification close to the aortic bifurcation and the tortuosity of the aorta combined to cause a significant disagreement in results.
One further potential source of significant variability originated in cases in which the aorta appears almost directly apposed to the lumbar vertebral bodies in the axial images. In these cases, because of their close apposition, the software is unable to automatically differentiate between bone and calcified aortic wall when an area of calcification is selected, and automated volume calculation is not possible. For such individuals, a circular region of interest (ROI) has to be manually drawn using the software's interactive tool around the aortic wall for each slice, taking care not to include any vertebral bone or other areas of calcification. The software then summates all voxels within the drawn ROIs that fall between the threshold limits of 130–3000 HU. As extra care is needed not to include unwanted regions of vertebral bone, there is the potential for significant disagreement between observers; however, in the subjects analysed in this way, this did not appear to be a problem, as none fell outside the limits of agreement.
|
Discussion
|
|---|
The technique applied above results in minimal interobserver variability. This is most likely explained by a number of factors. Firstly, the software used for the calculations is simple and quick to use (approximately 5 min per subject), with areas of calcification being easy to identify and highlight as a VOI. One feature that helped to ensure areas of calcium were not missed was the automatic inclusion by the software of areas of calcification in each slice that were contiguous with areas highlighted in neighbouring slices. In this way, a long segment of calcification within the aortic wall could be quickly and easily included in the volume calculation, and errors were reduced.
Secondly, the only major possible source of variation between observers was the choice of starting position for measurements. In none of the subjects where the starting point was the same for both observers did the end result vary. Other published techniques, such as those reported by Jayalath et al [8], involve the selection of both start- and end-points for measurement, and therefore increase the possibility of errors occurring. In this study, although there was variation in the choice of aortic bifurcation position, this was minimal and well within acceptable limits. As discussed earlier, for those subjects in whom calcification was heavily concentrated around the aortic bifurcation, the choice of starting slice was more critical. Indeed, in a study of automatic detection of aortic calcification, Isgum et al [11] also discovered that large calcifications around the bifurcation could be problematic.
|
Conclusions
|
|---|
Numerous methods for quantifying aortic calcification have been published; however, there has been a lack of consensus with regard to technique, and comparisons between studies have been difficult. This study has shown that the selection of a fixed and standardized section of aorta for measurement, combined with the use of commercial imaging software, is a robust technique that should enable its use in future studies, with minimal interobserver variability.
|
Acknowledgments
|
|---|
Special thanks are due to Dr Carmel McEniery, Clinical Pharmacology Unit, Addenbrooke's Hospital, Cambridge, UK for her work in recruiting subjects for the original study.
Received for publication January 13, 2008.
Revision received February 25, 2008.
Accepted for publication February 29, 2008.
|
References
|
|---|
- O'Malley PG, Taylor AJ, Jackson JL, Doherty TM, Detrano RC. Prognostic value of coronary electron-beam computed tomography for coronary heart disease events in asymptomatic populations. Am J Cardiol 2000;85:945–8.[CrossRef][Medline]
- Yamamoto H, Shavelle D, Takasu J, Lu B, Mao S, Fischer H, et al. Valvular and thoracic aortic calcium as a marker of the extent and severity of angiographic coronary artery disease. Am Heart J 2003;146:153–9.[CrossRef][Medline]
- Danielsen R, Sigvaldason H, Thorgiersson G, Sigfusson N. Predominance of aortic calcification as an atherosclerotic manifestation in women: the Reykjavik study. J Clin Epidemiol 1996;49:383–7.[CrossRef][Medline]
- London GM, Guerin AP, Marchais SJ, Metivier F, Pannier B, Adda H. Arterial media calcification in end-stage renal disease: impact on all-cause and cardiovascular mortality. Nephrol Dial Transplant 2003;18:1731–40.[Abstract/Free Full Text]
- Dixon AK, Lawrence JP, Mitchell JRA. Age-related changes in the abdominal aorta shown by computed tomography. Clin Radiol 1984;35:33–7.[CrossRef][Medline]
- Guerin AP, Blacher J, Pannier B, Marchais SJ, Safar ME, London GM. Impact of aortic stiffness attenuation on survival of patients in end-stage renal failure. Circulation 2001;103:987–2.[Abstract/Free Full Text]
- Jayalath RW, Mangan SH, Golledge J. Aortic calcification. Eur J Vasc Endovasc Surg 2005;30:476–88.[CrossRef][Medline]
- Jayalath RW, Jackson P, Golledge J. Quantification of abdominal aortic calcification on CT. Arterioscler Thromb Vasc Biol. 2006;26:429–30.[Free Full Text]
- Kabaya T, Nitta K, Kimura H, Kawashima A, Narusawa K, Nihei H. Increased aortic calcification index in hemodialysis patients. Nephron 1999;81:354–5.[CrossRef][Medline]
- Nitta K, Akiba T, Uchida K, Otsubo S, Takei T, Yumura W, et al. Serum osteoprotegerin levels and the extent of vascular calcification in hemodialysis patients. Nephrol Dial Transplant 2004;19:1886–9.[Abstract/Free Full Text]
- Isgum I, van Ginnekan B, Olree M. Automatic detection of calcifications in the aorta from CT scans of the abdomen: 3D computer-aided diagnosis. Acad Radiol 2004;11:247–57.[Medline]
This article has been cited by other articles:
|
|
|
|
 
C. M. McEniery, B. J. McDonnell, A. So, S. Aitken, C. E. Bolton, M. Munnery, S. S. Hickson, Yasmin, K. M. Maki-Petaja, J. R. Cockcroft, et al.
Aortic Calcification Is Associated With Aortic Stiffness and Isolated Systolic Hypertension in Healthy Individuals
Hypertension,
March 1, 2009;
53(3):
524 - 531.
[Abstract]
[Full Text]
[PDF]
|
|
|
Top
Abstract
Introduction
