British Journal of Radiology (2004) 77, S201-S208
© 2004 British Institute of Radiology
doi: 10.1259/bjr/66587930
Fusion of contrast-enhanced breast MR and mammographic imaging data
C P Behrenbruch, PhD1,3,6,
K Marias, PhD1,2,
P A Armitage, PhD1,
M Yam, PhD1,
N R Moore, MD, FRCP, FRCR3,
R E English4,
P J Clarke, MD, FRCS5,
F J Leong, MD, PhD, ARPS6 and
J M Brady, PhD, FRS, FREng1
1 Medical Vision Laboratory, Engineering Science, Oxford University, Parks Road, Oxford OX1 3PJ, 2 Department of Surgery, Royal Free and University College Medical School, UCL, London NW3 2QG, 3 Magnetic Resonance Imaging Centre, John Radcliffe Hospital, Headley Way, Oxford OX3 9DU, 4 Breast Care Unit and 5 Department of Surgery, Churchill Hospital, Oxford OX3 7LJ and 6 Mirada Solutions Ltd, Level 1, 23–38 Hythe Bridge Street, Oxford OX1 2ET, UK
 |
Abstract
|
|---|
Increasing use is being made of Gd-DTPA contrast-enhanced MRI (CE-MRI) for breast cancer assessment since it provides three-dimensional (3D) functional information via pharmacokinetic interaction between contrast agent and tumour vascularity, and because it is applicable to women of all ages as well as patients with post-operative scarring. CE-MRI is complementary to conventional X-ray mammography, since it is a relatively low-resolution functional counterpart of a comparatively high-resolution 2D structural representation. However, despite the additional information provided by MRI, mammography is still an extremely important diagnostic imaging modality, particularly for several common conditions such as ductal carcinoma in situ (DCIS) where it has been shown that there is a strong correlation between microcalcification clusters and malignancy. Pathological indicators such as calcifications and fine spiculations are not visible in CE-MRI and therefore there is clinical and diagnostic value in fusing the high-resolution structural information available from mammography with the functional data acquired from MRI. This article is a clinical overview of the results of a technique to transform the coordinates of regions of interest (ROIs) from the 2D mammograms to the spatial reference frame of the contrast-enhanced MRI volume. An evaluation of the fusion framework is demonstrated with a series of clinical cases and a total of 14 patient examples.
|
Introduction
|
|---|
This paper introduces a system that has been developed to perform data fusion between two-dimensional (2D) X-ray mammography and 3D contrast-enhanced MRI (CE-MRI) of the breast. The objective was to develop a non-rigid registration and visualization framework that makes use of both the high spatial resolution of mammography and the physiological (functional) information provided by CE-MRI. In general, there are several reasons for attempting to relate mammograms to "functional" images such as contrast-enhanced breast MRI. One is the desire to correlate underlying pathological processes with different visual manifestations. For example, microcalcifications present in a mammogram may meaningfully correlate with an enhancing region in MRI, despite the fact that calcifications are not radiologically visible in MRI. The majority of patients undergoing pre-operative breast MRI scan already have a mammogram and therefore fusion of the information in both studies can support the diagnosis and provides a continuity of evidence in patient management. Younger patients often have both an MRI scan and a mammogram due to the limited effectiveness of using mammography for assessing pre-menopausal women due to the X-ray attenuation characteristics of glandular tissue. Fusion of the two modalities is a means of providing the clinician with a means of assessing a mammogram which is otherwise difficult to interpret. Finally, post-operative tissue changes may be difficult to assess with mammography alone and are often better evaluated with MRI. This is largely due to the high X-ray attenuation of scar tissue.
The technological details of the method have been published in the literature [1, 2] and this paper will not cover the processes in any detail. Instead this is intended as an overview of the technique and a highlight of some its clinical benefits.
|
Methods
|
|---|
The principal concept behind this work is the registration of a highly compressed 2D projective representation of the breast with an uncompressed volume-based acquisition. In the case of the MRI acquisition, the patient lies facedown in the scanner with the breasts pendulous (elongated slightly by gravity) and uncompressed in the breast coils. This is entirely different from the situation in mammography, where the patient normally remains upright with the breasts compressed, (one at a time), between two plates. The underlying challenge in the breast 2D–3D image fusion process therefore is to compensate for these differences in breast shape.
Previous work related to modelling breast compression has used a cross-section approach to uncompress the breast to enable point correspondence between mammogram MLO (medial–lateral oblique) and CC (cranial–caudal) views [3, 4]. All of the patients in our study had both a MLO and a CC mammogram, as is required for breast cancer screening in the UK. However, in our approach, the objective is to uncompress the mammograms into the reference frame of prone-acquired 3D MRI volumes and therefore a different strategy is required. The objective is to use the "uncompression" of the mammogram images not only to facilitate a framework for point correspondence between mammograms, but also to enable pathology that is visible in the mammogram to be compared with MRI. As a consequence, the uncompressed shape is very different to that proposed by Kita et al as the reconstructed shape resembles the prone-acquired MRI breast, rather than the idealized upright shape of the breast. This is illustrated in Figure 1
where an uncompensated 3D shape reconstruction from an MLO and CC mammogram is compared with the model-based technique of Kita et al and a reconstruction based on the mammograms registered to the MRI functional projections. At first glance, Figure 1c), a 2-view reconstruction using the registered X-rays, appears somewhat unusual. For comparison, Figure 2 show the same registered and reconstructed views together with the segmented MRI projections. Not only is there good shape correspondence, but also it is possible to reconcile, albeit approximately, the location of the pathology between the two different modalities.
View larger version (24K):
[in this window]
[in a new window]
|
Figure 1. (a) A 3D shape reconstruction based on the two uncompressed mammogram views (elliptical volume interpolation between views). Due to the fact that both the cranial-caudal and medial lateral oblique planes are compressed, the shape appears somewhat cylindrical. (b) A shape reconstruction based on the technique described in [4]. (c) The shape of the breast based on the two mammograms registered to the MRI volume projections.
|
|
View larger version (74K):
[in this window]
[in a new window]
|
Figure 2. (A) and (D) are the original cranial-caudal and medial-lateral oblique mammograms, respectively. (B) and (E) show the segmented MRI functional projections. (C) and (F) depict the original mammograms registered to the MRI projections based on the shape of Figure 1c .
|
|
|
Results
|
|---|
The technique was applied to 14 sets of patient data. In two cases, we were unable to assess the registration due to the fact that features such as calcifications were not present in both mammogram views. This represents a limitation of our method. In the remaining 12 cases covered a wide range of pathologies with very different structural and tissue properties.
Our evaluation of the performance of our registration, although limited, is quite simple. In cases where pathology was visible in either/both of the mammogram views and the MRI volume, or a biopsy/localization has been performed, we compare the centroid of the backwards transformation from the MRI volume to the two mammographic views on the basis of a lesion (or region of interest, ROI) centroid. This analysis centroid is then compared against the centroid of a manual segmentation by the clinician. On the assumption that different classes of lesion will result in different deformation results (and corresponding errors) we have grouped each case into 7 radiological appearances. These are shown in Table 1. It is not surprising that larger pathologies exhibit a more substantial registration error as the use of these regions as a salient feature for registration is more ambiguous by virtue of our method for producing landmarks for deformation. Similarly, it is sometimes problematic to evaluate cases with calcifications as there may not be corresponding enhancement in the MRI volume for all clusters. These considerations have limited the size of this study.
View this table:
[in this window]
[in a new window]
|
Table 1. A comparison of the registration error in 12 cases and categorized by 7 distinct radiological appearances
|
|
The set of results illustrated in Table 1 is somewhat difficult to compile as the deformations are calculated in millimetres, but the voxel dimensions are not uniform across the various MRI acquisitions. Therefore, we have quoted the error relative to a voxel as it better illustrates the fact that in many of the cases, the MRI scans were acquired using transverse slicing and hence the voxel resolution in the MLO projection is poorer. This error trend is quite visible when the MLO voxel error is compared against the CC error. To further understand the relative significance of this error, we compare the estimated error in millimetres (converted on the basis of the average voxel dimension in both projections) to the average calculated displacement field computed from the registration transformation function.
On average across all seven classes of pathology, the registration error relative to the size of the deformation is less than 20%. Although this is a preliminary result, it provides an indication of the extent of the deformation necessary to achieve registration. Once again, however, the extent of the local displacement of a region varies between the two projective views but also depends greatly on the shape and size of the breast and to a certain extent, the slice orientation of the MRI acquisition.
The maximum registration error we encountered was in the vicinity of 5–6 voxels, which in the case of a transverse MRI acquisition (with 2–3 mm voxels in the MLO projection), equates with a maximum error of typically 10 mm (depending on the projection direction). Of course, in cases where the pathology is clearly visible in both modalities, this can itself be a registration landmark – hence in some instances, very good registration results were achieved with a perceived accuracy of 2–3 mm. However, it must be considered that in these cases, the centroid error for larger lesions is also likely to be a somewhat inaccurate reference point as the extent of the visible lesion in the MRI volume will, when projected, underestimate the size of the region in the mammogram.
Several specific cases are notable. Figure 3 illustrates a case where no lesion was visible in a dense mammogram but was noted with MRI. When this ROI was localized on the mammogram with fusion (Figure 4), both a repeat mammogram and subsequent biopsy confirmed malignancy (Figure 5). It is well known that younger women typically have denser breasts, as do those women placed on hormone replacement therapy and in such situations mammography is frequently unreliable [5–7]. There is evidence to suggest that MRI-based screening may be a more appropriate first-line approach to accurate detection and diagnosis for these cases (particularly if a patient falls into a high-risk category) [8]. Correlation between the two studies allows the mammographic data to be utilized in the interpretation, something not possible if it was considered in isolation.
View larger version (67K):
[in this window]
[in a new window]
|
Figure 3. Initial assessment of the mammograms with the suspected region of interest highlighted by the clinician in the lateral mammogram (b) along with a corresponding region in a slice of the contrast-entranced MRI volume (c). No lesion was obviously apparent in the cranial–caudal mammogram (a).
|
|
View larger version (37K):
[in this window]
[in a new window]
|
Figure 4. Registration results between both mammogram views and the MRI projection with corresponding region of interest location highlighted.
|
|
View larger version (69K):
[in this window]
[in a new window]
|
Figure 5. The initial clinical assessment of the extended cranial-caudal mammogram (a) did not reveal any suspicious regions of interest. The medial-lateral oblique mammogram (b) was determined to have a slightly denser region (circled) but did not reveal any pathology with needle biopsy. After registration with the contrast-entranced MRI projections, the centroid of the region visible in the MRI was projected back onto the mammograms (indicated by +). Biopsy of this region tested positive for malignancy.
|
|
We examined scans from a 26-year-old female with history of ductal carcinoma in situ (DCIS; treated with wide local excision and radiotherapy) who presented for a routine mammogram screening. This revealed three distinct clusters of calcifications, largely visible in both views. Previous surgery complicated mammography interpretation as scarring can mask pathology (scar tissue has unfavourable X-ray projection characteristics). The patient was sent for a follow-up MRI.
However, the clinicians were interested to know the correspondence between the calcification clusters visible in the mammogram (despite the dense tissue) and the CE-MRI acquisition. Figure 6 depicts 4 years of mammograms (both views) that have been registered using the technique described in Marias et al [9]. In this way, it is possible to compare the changes in tissue density due to both the surgical treatment (the excision was followed by a breast reconstruction using muscular tissue which is clearly evident in the sequence) and possible evolution of pathology.
View larger version (75K):
[in this window]
[in a new window]
|
Figure 6. 4 years of patient mammograms registered using the techniques described in [9]. At year 4, three clusters of calcifications are visible in both the cranial-caudal (top row) and medial-lateral oblique (bottom row) mammograms. Comparing the location of cluster "1" to the surgical excision area (circled regions in the mammograms of year 2) suggests calcification due to surgical scarring rather than malignancy.
|
|
This first stage analysis of the mammographic history of the patient was a useful exercise. First, the surgeon felt that the calcification cluster located near the previous excision site (Figure 6, cluster labelled "1") was a relatively low-risk of cancer since surgical scarring may produce a slight dusting of calcifications. Certainly the shape and distribution characteristics of the microcalcifications were consistent with this expectation. The second cluster of calcifications (Figure 6, labelled as "2") was somewhat contentious due to the location, particularly with respect to the nipple plane of the breast. The remaining cluster was morphologically the most pathologically indicative of DCIS.
The two mammogram views were registered to the CE-MRI volume in the manner described previously and the location of the calcifications were colour coded and presented to the radiologist in a 3D slice-based visualization of the MRI volume. This is shown in Figure 7 where the "trouble" case of calcifications (Figure 7, "3") is visualized as a spherical region of interest (indicating the approximate location and extent of the cluster).
View larger version (85K):
[in this window]
[in a new window]
|
Figure 7. 3D visualizations of the location of the three calcification clusters in the reference frame of the MRI volume (labelled according to the previous figure).
|
|
By providing a set of visual prompts to the radiologist we had hoped to highlight regions that required additional scrutiny. Since one expects the contrast-enhancement characteristics of the calcifications to be very subtle in the CE-MRI acquisition this provides a useful additional piece of information to the radiologist. Upon re-examination of the MRI contrast sequence, the radiologist concluded that there was a region of slightly higher enhancement at the indicated position of the third cluster. Although the enhancement were subtle, there was enough quantitative evidence (from the contrast profile) to indicate that the suggested location of the calcification cluster in the reference frame of the MRI was potentially significant. Biopsy of the calcification cluster proved positive for DCIS.
|
Discussion
|
|---|
In this article we have endeavoured to present a novel concept in breast data fusion. To our knowledge, such an approach has not yet been attempted in a clinical application and therefore there are naturally a number of issues that require further assessment. Our data fusion approach incorporates a combination of MRI contrast-enhancement projection (to produce a "pseudo" mammogram), boundary and internal landmark detection and a two-stage registration process using radial basis functions. The internal deformation of the breast (due to mammography compression) is compensated by the correlation of internal landmarks between the MRI projection and the mammogram, using a novel wavelet-based scale-space analysis. By combining boundary and internal landmark information, the registration approach is able to cater for cases where there may or may not be internal landmarks due to the involution characteristics of the breast.
Despite the difficulty with validating non-rigid registration, particularly for multimodal applications, we have been able to demonstrate that the error due to registration (compared against a clinical evaluation) is typically less than 20% of the local displacement resulting from elastic deformation, an encouraging result. One of the major difficulties is validating a system that has a large number of components and a relatively high degree of complexity.
DCIS is a commonly occurring form of breast cancer that can be difficult to assess mammographically. A clear indicator of DCIS is often the presence of calcifications with a "jagged" morphology [10, 11], which are usually quite clearly visible in at least one of the mammogram views. A difficulty with DCIS is the often impalpable nature of the tumour during clinical examination. From a surgical perspective, therefore, 3D localization and visualization is very useful. This is because of the high dependency on the palpability of the tumour to guide the resection process. An obvious way of localizing the tumour in 3D would be a CE-MRI acquisition. Unfortunately, the sensitivity of CE-MRI to DCIS is very low due to the relatively small amount of tumour vascularity and hence the contrast-enhancement profile of the pathology is usually too subtle for reliable detection [12]. Of course, an additional complication with imaging this kind of pathology using MRI is the relatively small size of the cancer and hence the limited resolution of MRI (and partial volume effects), which further compounds the issue. However, if the locations of the calcifications in the mammogram views could be transformed to the spatial coordinates of the MRI volume, it might be possible to prompt the radiologist with regions of interest that should be more clearly scrutinized for suspicious (although subtle) enhancement and morphology.
Fusion can aid the radiologist in understanding the pathological relationship between mammography and MRI, particularly when there may be ambiguity in the diagnostic value of either modality. The cases demonstrated highlight situations where fusion of two modalities is indeed synergistic for assessment of each modality in isolation would not provide the same information. Interpretation of mammograms of dense breast is aided with the fusion of CE-MRI data. Similarly, subtleties in a CE-MRI study are given more significance when correlated with an abnormality noted on mammography.
An area of significant debate in breast MRI is the extent to which DCIS enhances with gadolinium DTPA. Some radiologists suggest that there may be a subtle enhancement characteristic that can be used to detect DCIS. We are interested to pursue this as a larger study to see whether a definitive quantification of DCIS enhancement can be obtained. This could lead to the development of software tools that not only aid the localization of mammogram-visible microcalcifications in the MRI acquisition, but also provide analysis tools that help the radiologist localize DCIS-characteristic enhancement. In the interim, we use calcification localization in conjunction with some simple segmentation techniques based on region-growing local voxel contrast characteristics to produce visualizations of DCIS that we feel are useful to both the radiologist and ultimately, the surgeon. An example of such a visualization is shown in Figure 8.
View larger version (74K):
[in this window]
[in a new window]
|
Figure 8. A rendering of the MRI volume (post-contrast sagittal acquisition) with the region corresponding to calcifications in the mammograms indicated by arrows. In terms of understanding the pathology, 3D morphology is much more meaningful than a slice-based representation. Additionally, models such as this may provide the surgeon with a useful location reference during excision.
|
|
Mathematical and clinical validation of non-rigid registration is a problem that is an ongoing burden to the computer vision community. We have provided a measure of validation against deformation characteristics, clinician segmentation and in a few cases we have had the benefit of a localization study. However, future research effort could involve further validation exercises. Given that the majority of women presenting for breast MRI already have a mammogram, it is conceivable that quite a large clinical study could be performed on cases where the radiological diagnoses were more certain. Correlation with histological information from a core biopsy or guided fine needle aspiration is also another way of ascertaining whether diagnostic accuracy improves with this technique.
If such a system is to be clinically implemented and considered useable by a radiologist, the key to success will ultimately lie in the software implementation and interface. The techniques presented in this paper have been presented as automatic, however in practice there is a need to be able to provide the clinician with interactivity at each step. Not only is this likely to improve the clinician's faith in the registration process, but it enables the "expert opinion" to influence the quality of the result. To this end, our software implementation of the techniques presented in this paper (as a system) allows user input at each step of the process.
Our ongoing effort in this research is aimed at developing a more robust registration framework in the context of a larger and (as indicated in the discussion) more diverse validation study. We have implemented the data fusion framework as a software package, which is currently installed at the John Radcliffe Hospital, Oxford, where it is used on a regular clinical basis. From our clinical experience with our software tools, we have found that there is a considerable gulf between a theoretical fusion concept with the implication of automatic performance, and a clinically useable (and accepted) system. Our long-term objective is to produce a system capable of enabling the radiologist to do a rapid and accurate comparison between mammography and MRI.
|
Acknowledgments
|
|---|
The authors would like to thank many people for their contribution to this work. This includes Dr Jérôme Declerck (Mirada Solutions Ltd.) for useful discussions about this work, Mr Dermot Dobson and the staff at the John Radcliffe Hospital. C Behrenbruch would like to gratefully acknowledge the Commonwealth Scholarship and Fellowship Program for his doctoral funding at Oxford University. P Armitage and JM Brady acknowledge the EPSRC for funding (GR/M54995).
|
References
|
|---|
- Behrenbruch CP, Marias K, Armitage PA, Yam M, Moore A, English RE, et al. Fusion of contrast-enhanced breast MR and mammographic imaging data. Medical Image Analysis 2003;7:311–40.[CrossRef][Medline]
- Behrenbruch CP, Marias K, Armitage PA, Brady JM, Clarke J, Moore N. The generation of simulated mammograms from contrast-enhanced MRI for surgical planning and postoperative assessment. In: Proceedings of IWDM (the International Workshop in Digital Mammography), Toronto, Canada: Kluwer Academic Publishers, 2000.
- Kita Y, Highnam R, Brady M. Correspondence between different view breast X-rays using a simulation of breast deformation. IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 1998:700–7.
- Yam M, Brady JM, Highnam RP, English RE, Kita Y. Reconstructing microcalcification clusters in 3-D using a parameterized breast compression model. Computer Assisted Radiology and Surgery (CARS), Paris, 1999.
- Kavanagh AM, Mitchell H, Giles GG. Hormone replacement therapy and accuracy of mammographic screening. Lancet 2000;355:270–4.[CrossRef][Medline]
- Leung W, Goldberg F, Zee B, Sterns E. Mammographic density in women on postmenopausal hormone replacement therapy. Surgery 1997;122:669–73; discussion 673–4.
- Litherland JC, Evans AJ, Wilson AR. The effect of hormone replacement therapy on recall rate in the National Health Service Breast Screening Programme. Clin Radiol 1997;52:276–9.[CrossRef][Medline]
- Warren RML, Hayes C, the Advisory Group of the UK study of MRI screening for breast cancer – MARIBS. Localization of breast lesions shown only on MRI – a review for the UK study of MRI screening for breast cancer. Br J Radiol 2000;73:123–32.[Abstract]
- Marias K, Behrenbruch CP, Brady JM, Parbhoo S, Seifalian AM. Multi-scale landmark selection for improved registration of temporal mammograms. In: Proceedings of IWDM (International Workshop in Digital Mammography), Toronto, Canada: Elsevier, 2000.
- Lanyi M. Diagnosis and differential diagnosis of breast calcifications. Berlin: Springer-Verlag, 1996.
- Wolfe JN. Xenoradiography of the breast. Springfield, Illinois: Thomas Publishing, 1982.
- Rankin SC. MRI of the breast. Br J Radiol 2000;73:806–18.[Abstract]
Top
Abstract
Introduction
