This Article Abstract Figures Only Full Text (PDF) Services Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by McLean, R G Articles by Elison, B S Search for Related Content PubMed PubMed Citation Articles by McLean, R G Articles by Elison, B S

Comparison of new clinical and scintigraphic algorithms for the diagnosis of pulmonary embolism

Department of Nuclear Medicine, The Wollongong Hospital, Wollongong NSW 2500, Australia

	Abstract

Top Abstract Introduction Material and methods Discussion Conclusion References

 
Since the publication of the modified Prospective Investigation of Pulmonary Embolism Diagnosis (PIOPED) criteria for the diagnosis of pulmonary embolism (PE), new clinical and scintigraphic diagnostic algorithms (the McMaster clinical criteria, the PisaPED simplified scintigraphic grading and the Miettinen logistic regression analysis) have been reported although the results have not been reproduced in other sites. Ventilation–perfusion lung scintigraphy was performed in 238 consecutive patients with a provisional diagnosis of PE. Scans were reported as normal/very low, low, intermediate or high probability for PE using standardized criteria. Each patient received a clinical grading of probability of PE as low, moderate or high using the McMaster clinical criteria. Using the PisaPED criteria (an alternate simplified scintigraphic grading system using the perfusion scan alone) each scan was also graded as normal/near normal, abnormal but not PE, or abnormal and PE. Using the logistic regression algorithm of Miettinen each scan received a numerical probability of PE. Frequencies for differing levels of probability of PE varied widely between the various algorithms. Cross tabulations revealed correlation of the standardized criteria with the Miettinen grading but not with the McMaster or the PisaPED gradings. We were unable to reproduce similar results using the McMaster clinical grading or the PisaPED simplified scintigraphic grading although the Miettinen logistic regression formula gave comparable results. New algorithms are not automatically transferable to new environments.

	Introduction

Top Abstract Introduction Material and methods Discussion Conclusion References

 
In a provocative paper in the late 1970s, Robins suggested that lung scintigraphy was over-diagnosing (and therefore resulting in the over-treatment of) pulmonary embolism and that "the Emperor may have no clothes" [1]. However, it was not until more than 10 years later that the results of the Prospective Investigation of Pulmonary Embolism Diagnosis (PIOPED) study were published, a study that was anticipated to answer the question about the Emperor's clothing. Unfortunately, it showed that although the "Emperor" was "well-clothed" in two areas — normal lung scan means no pulmonary embolism and high probability lung scan means high probability of pulmonary embolism — but in other areas, the clothing had significant holes — low probability lung scan does not exclude pulmonary embolism and intermediate probability lung scan means intermediate probability of pulmonary embolism and is of little use [2].

Nevertheless, the study provided fertile ground for the development of a range of further investigations, techniques and algorithms. These included Doppler ultrasound of the lower limbs, serum D-dimer investigation, lower limb plesythmography and CT scan techniques, particularly CT pulmonary angiography.

With respect to lung scintigraphy, the data and results were re-analysed leading to the so-called "modified PIOPED criteria" which were subsequently shown to be more accurate than the original PIOPED criteria. [3–5]. Further analysis of subgroups from the initial PIOPED trial was also undertaken in an effort to refine the criteria in order to reduce the number of intermediate probability lung scan readings and to improve the specificity of low probability lung scans [6–8].

Taking a different approach, one group decided to "go back to basics" and used a simplified classification technique for the perfusion scan alone (not using a ventilation scan at all), the PisaPED study [9], while Miettinen et al utilized a different conceptual approach by undertaking logistic regression analysis of the PIOPED data [10]. Finally, a group from McMaster University, some members of which had involvement with the initial PIOPED study decided to look more critically at whether pulmonary embolism could be predicted from clinical data, contrary to the long-held belief and teaching that clinical signs are notoriously unreliable in this condition [11].

However each of the three approaches, the PisaPED simplified perfusion scan grading and the logistic regression analysis grading of Miettinen et al and the clinical classification system of the McMaster group, was developed and used primarily in its institution of origin and no significant replication or validation by other users has been reported in the literature thus far.

Therefore we decided to undertake a prospective clinical and scintigraphic study in which the three classification systems were compared with refined modified PIOPED criteria, developed in our institution. The purpose was to assess newly reported algorithms in settings other than those in which they were developed to see if the same results could be obtained.

	Material and methods

Top Abstract Introduction Material and methods Discussion Conclusion References

 
Patients
238 consecutive patients undergoing ventilation–perfusion scintigraphy for detection of pulmonary embolism at the Nuclear Medicine Department, Wollongong Hospital between October 2000 and April 2001 were enrolled. The group comprised 124 females (mean age 60 years, range 17 to 88) and 114 males (mean age 63.5 years, range 18 to 91). Patients were excluded if the study was a progress study, if the study was technically unsatisfactory or if only a perfusion study was performed.

On arrival in the Nuclear Medicine Department prior to undergoing the scan, a clinical assessment was undertaken by an experienced nuclear medicine physician or advanced training registrar, interrogating the patient and supplementing clinical and other details from the patient's notes, in order to obtain a clinical grading of probability of pulmonary embolism using the McMaster classification (see below).

Subsequently, ventilation–perfusion scintigraphy was performed. The patients were imaged in the supine position on either a General Electric (GE, Milwaukee, WI) Starcam 3000 ACT, 4000 XCT or 3200 XRT gamma cameras using standard GE low energy, high resolution, parallel hole collimators and a 128 x 128 matrix. The lung ventilation study was performed using ⁹⁹Tc^m Technegas. Technegas was administered until a count rate of approximately 2200 counts per second was reached. Images with 200 000 counts were acquired for anterior, posterior, left and right posterior oblique as well as left and right anterior oblique projections. The lung perfusion scan was performed using 180 MBq of ⁹⁹Tc^m macroaggregated albumin. This activity results in approximately 10–15 kilocounts per second. Images each with 600 000 counts were acquired for the same projections as the ventilation images. Using this protocol we found the image quality to be good, free from artefacts and no studies needed to be rejected on the basis of technically poor images. All images were of sufficient quality to allow physicians to detect any ventilation defects, and therefore determine the presence of any mismatched ventilation–perfusion defects.

When the scan had been performed, it was read by the physician in association with the registrar. The reading and subsequent grading of probability of pulmonary embolism was undertaken using three methods: the locally refined modified PIOPED criteria, the PisaPED criteria, and the logistic regression analysis method of Miettinen et al.

Clinical classification (McMaster)
The development and application of the clinical classification system is described by Wells et al [11]. Basically the algorithm comprises categorization at three stages. Firstly a clinical history is taken from the patient and a physical examination is performed. On the basis of pre-defined sign and symptom criteria the patient is categorized as having a severe, typical or atypical presentation for pulmonary embolism. Then the possibility that an alternative diagnosis is as likely as or more likely than pulmonary embolism is taken into consideration. Finally the risk factors for venous thromboembolism (recent surgery, immobilization, family history, cancer, etc.) are considered. The outcome of this classification system is that the patient is graded as having a low, moderate or high probability of pulmonary embolism.

Local scintigraphic grading (RM-PIOPED)
This grading system, developed locally, comprises "Refined Modified PIOPED criteria", supplementing the modified PIOPED criteria with results from later subgroup analyses that refined the criteria in several key areas. The grading system is given in Table 1.

• normal;
  
• near normal;
  
• abnormal not consistent with pulmonary embolism (perfusion defects other than wedge shaped); or
  
• abnormal consistent with pulmonary embolism (single or multiple wedge shaped perfusion defects).
  

For the purposes of the analysis the near normal results were considered as low probability and the latter two categories were considered as intermediate and high probability of pulmonary embolism, respectively. Although the category of "abnormal not consistent with pulmonary embolism" does not correspond entirely with intermediate probability (and probably should be considered as lower than intermediate probability) it was classed as equivalent to intermediate probability for expediency to allow at least comparison of the other categories.

Logistic regression analysis method (Miettinen)
This method comprises the application of logistic regression analysis. The form of the equation and the coefficients were determined from analysis of the original PIOPED data. The probability (P) of pulmonary embolism is given by:

where e is the base of the natural logarithm (Napier's constant) approximately equal to 2.71828 and S is the scoring function derived by Miettinen using regression analysis and takes the form:

The observable variables are: x₁ = the number of large or moderate mismatched perfusion defects; x₂ = the maximum number of matched perfusion defects in any lung region; x₃=1 if x₁1 and 0 otherwise; x₄=1 if x₂1 but 0 otherwise; and x₅=1 if there are any perfusion defects with corresponding but larger chest X-ray abnormalities.

For the purposes of the current study the probability of pulmonary embolism was calculated using an MS-Excel Spreadsheet based on the expressions given above.

Analysis
The measure of agreement between the rating systems was assessed using Cohen's kappa with linear weighting. Kappa provides a measure of the degree to which two judges concur in their respective sorting of items into mutually exclusive categories. Given that the categories in this case can be considered more ordinal then nominal, a linear weighted kappa was calculated. Values range between 0 (no agreement) and 1 (complete agreement).

For the calculation of the logistic regression analysis, an a priori value of 34% for the incidence of pulmonary embolism in the population was used.

Results
The McMaster clinical classification scheme resulted in 149 (62.6%) patients being classified as low, 57 (23.9%) as moderate and 28 (11.8%) as high probability of having pulmonary embolism. A further 4 (1.7%) did not fall into any category and were presumed to be low, the original indication for the scan being questionable.

The RM-PIOPED classification system resulted in 19 (8%) scans being considered normal, and 47 (19.7%) being very low, 100 (42%) low, 43 (18.1%) intermediate and 29 (12.2%) high probability of pulmonary embolism.

The PisaPED classification system resulted in 38 (16%) of scans being normal, and 48 (20.2%) being near normal, 68 (28.6%) abnormal PE– and 84 (35.3%) abnormal PE+.

The McMaster and the RM-PIOPED classifications have similar numbers of scans in the moderate/intermediate (23.9% and 18.1%, respectively) and high (11.8% and 12.2%, respectively) categories. However the PisaPED classification gives 28.6% as being abnormal but not consistent with PE (roughly equivalent to the moderate or intermediate probability) and 35.5% of studies as consistent with pulmonary embolism (high probability).

The Miettinen analysis gives different numbers depending on where the cut-off probabilities between the gradings are arbitrarily drawn. If the probability range for intermediate is 0.25 to 0.75, the respective proportions for intermediate and high are 14% and 8% while if the values are 0.33 and 0.66, the respective values are 3% and 18%. This is shown in Figure 1.

View larger version (16K): [in this window] [in a new window]   Figure 1. The distribution of lung scan probabilities from the Miettinen algorithm with boundaries between low and intermediate, and intermediate and high probability of pulmonary embolism set at 0.33 and 0.66, and 0.25 and 0.75, respectively.

Correlation between the Miettinen and RM-PIOPED gradings are shown in detail in Figure 2. The poor correlation between the McMaster classification and the Miettinen criteria with significant numbers of McMaster low probability studies being Miettinen high probability and vice versa is shown in Figure 3. The correlation between the PisaPED and the Miettinen model is similarly poor for the high probability PisaPED studies that show equal numbers of Miettinen low and high probability results (Figure 4).

View larger version (14K): [in this window] [in a new window]   Figure 2. Comparison between Miettinen probability of pulmonary embolism and the classification of the lung scans according to the RM-PIOPED criteria.

View larger version (15K): [in this window] [in a new window]   Figure 3. Correlation between the Miettinen-predicted probability of pulmonary embolism and the probability based on the McMaster clinical classification.

View larger version (14K): [in this window] [in a new window]   Figure 4. Comparison of Miettinen-predicted probability of pulmonary embolism (PE) and the PisaPED classification based on the perfusion scan alone.

With an assumed incidence of 34% the sum of the Miettinen model probabilities of pulmonary embolism for all patients in the study yields an overall incidence of 26%. Revising the assumed incidence down and recalculating the overall incidence generated by the Miettinen model fails to reconcile this difference for any reasonable value of assumed incidence.

	Discussion

Top Abstract Introduction Material and methods Discussion Conclusion References

 
It must be clearly stated up front that no grading system is being held up as being correct as there has been no "gold standard" test performed. It is also clear from many previous studies that it is difficult to enrol all patients with a likely diagnosis of pulmonary embolism in a study that requires all to undergo pulmonary angiography, and this therefore leads to various referral biases. Similarly using final clinical diagnosis as a surrogate for pathological diagnosis is flawed. We have used scintigraphic diagnosis using established criteria as the benchmark against which the newer algorithms have been compared simply because it is the most established and widely used diagnostic test in common clinical practice, and one that can be held up as being reasonably predictive of the diagnosis of pulmonary embolism. Furthermore two of the studies used comparison with established scintigraphic criteria as their methodology.

The primary aim of the study has been to see how new classification systems function when introduced into an unfamiliar environment, when compared with their function or utility in the environments in which they were developed.

It is a not infrequent finding that when new technologies, either diagnostic or therapeutic, are introduced into a new practice situation, the results are less impressive than those that are obtained in the context in which the test or procedure was developed.

There are many reasons for this, the most obvious being that those who develop the test or procedure have a much greater level of skill and experience which allows them to perform the study or procedure at a higher level of skill. Another reason relates to referral bias which may influence the underlying incidence of disease (or its complexity or severity) in patients referred for the test or procedure.

In our study, we were unable to reproduce the results of the McMaster group. Clinical classification did not predict scan result, particularly for the patients subsequently shown to have high probability scans.

In our hands, the PisaPED classification also performed poorly — it did not correlate with clinical classification, it correlated poorly with our revised scintigraphic criteria and it did not reduce the number of patients with intermediate probability scans. To nuclear medicine physicians who are used to dealing with reasonably complicated grading systems, it would appear surprising that such a simple system could effectively predict pulmonary embolism. While this still could be true, it might be necessary to train the observers to a much higher level as to what constitutes a "wedge shaped defect".

Miettinen's logistic regression analysis model is appealing as it relies less on individual interpretations and allows a numerical probability to be generated. The model did not correlate with the clinical model or with the PisaPED model although it did correlate well with our refined scintigraphic criteria. Depending on where boundaries between low, intermediate and high probability are located, the number of patients in the intermediate probability can be significantly reduced.

Therefore the possibility of the Miettinen model replacing the RM-PIOPED criteria arises. This will require a number of conditions to be met. First a larger study will need to confirm these preliminary findings. Second the referring physicians will need to be convinced that the new system of reporting is similar or superior to the system that they have been used to (with minor modifications) for a number of years. In the short term it is likely that the older semiquantitative reporting will still be required and the cut-off levels that allow conversion from a numerical probability to low, intermediate or high probability will need to be explored. Third, further investigation is required to determine the optimal value of the a priori assumed incidence of pulmonary embolism for our patient population. Finally, given the increasing use of CT pulmonary angiography and the improvement in this technology a head-to-head trial of the Miettinen system against this alternative would help convince many of the value of the simplified system.

	Conclusion

Top Abstract Introduction Material and methods Discussion Conclusion References

 
In our hands, the results of the McMaster study and PisaPED study could not be reproduced. However the logistic regression analysis model of Miettinen correlated well with our refined modified PIOPED scintigraphic criteria. Application of the model to a larger group a larger group of patients and referring physician education will be necessary before it can replace the existing reporting criteria.

Overall the study shows that the new algorithms developed in centres of excellence cannot necessarily be implemented in other sites and perform to the same standard from the start. This is not a new finding but it is valuable to have it re-confirmed in a context in which established procedures (clinical assessment, interpretation of perfusion lung scans) are being used in new ways rather than in the context of entirely new diagnostic tests.

	Footnotes

 
Address correspondence to Dr R McLean, PO Box 1043, Dubbo, NSW 2830, Australia.

Received for publication May 27, 2003. Revision received November 28, 2003. Accepted for publication February 3, 2004.

	References

Top Abstract Introduction Material and methods Discussion Conclusion References

 

1. Robins ED. Over diagnosis and over treatment of pulmonary embolism; the emperor may have no clothes. Ann Intern Med 1977;87:775–81.
2. PIOPED Investigators. Value of the ventilation perfusion scan in acute pulmonary embolism. JAMA 1990;263:2753–9.[Abstract]
3. Freitas FE, Sarosi MG, Nagle CC, Yeomans ME, Freitas AE, Juni JE. Modified PIOPED criteria used in clinical practice. J Nucl Med 1995;36:1573–8.
4. Sostman HD, Coleman RE, Delong DM, Newman GE, Paine S. Evaluation of revised criteria for ventilation-perfusion scintigraphy in patients with suspected pulmonary embolism. Radiology 1994;193:103–7.[Abstract/Free Full Text]
5. Worsely DF, Alavi A. Comprehensive analysis of the results of the PIOPED Study. J Nucl Med 1995;36:2380–7.[Abstract/Free Full Text]
6. Stein PD, Relyea B, Gottschalk A. Evaluation of individual criteria for low probability interpretation of ventilation-perfusion lung scans. J Nucl Med 1996;37:577–81.[Abstract/Free Full Text]
7. Stein PD, Henry JW, Gottschalk A. Small perfusion defects in suspected pulmonary embolism. J Nucl Med 1996;37:1313–6.[Abstract/Free Full Text]
8. Gottschalk A, Stein PD, Henry JW, Relyea B. Matched ventilation, perfusion and chest radiographic abnormalities in acute pulmonary embolism. J Nucl Med 1996;37:1636–8.[Abstract/Free Full Text]
9. Miniati M, Pistolesi M, Marini C, Di Ricco G, Formichi B, et al. Value of perfusion lung scan in the diagnosis of pulmonary embolism: results of the prospective investigative study of acute pulmonary embolism diagnosis (PisaPED). Am J Resp Critical Care Med 1996;154:1387–93.[Abstract]
10. Miettinen OS, Henschke CI, Yankelevitz DF. Evaluation of diagnostic imaging tests: diagnostic probability estimation. J Clin Epidemiol 1998;51:1293–8.[CrossRef][Medline]
11. Wells PS, Ginsberg JS, Anderson DR, Kearon C, Gent M, et al. Use of a clinical model for safe management of patients with suspected pulmonary embolism. Ann Intern Med 1998;129:997–1005.[Abstract/Free Full Text]

This article has been cited by other articles:

				BJR Review of the Year - 2004 Br. J. Radiol., March 1, 2005; 78(927): 181 - 185. [Full Text] [PDF]

This Article Abstract Figures Only Full Text (PDF) Services Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by McLean, R G Articles by Elison, B S Search for Related Content PubMed PubMed Citation Articles by McLean, R G Articles by Elison, B S