Pretest Risk Assessment in Suspected Acute Pulmonary Embolism

Original Investigations Pretest Risk Assessment in Suspected Acute Pulmonary Embolism1 Clifford R. Weiss, MD, Edward F. Haponik, MD, Gregory B. Diette, MD, MHS, Barry Merriman, MA, John C. Scatarige, MD, Elliot K. Fishman, MD Rationale and Objectives. To assess the pretest practices of US clinicians who treat patients with acute pulmonary embolism (PE). Materials and Methods. We surveyed 855 practicing physicians selected randomly from three professional organizations. We asked participants to estimate how often and by what method they determine the likelihood of PE before they request confirmatory studies. Participants reported their awareness of four published clinical practice guidelines dealing with acute PE and selected options for further diagnostic testing after reviewing clinical data from three hypothetical patients presenting with low, intermediate, and high probability of acute PE. Results. We received completed surveys from 240 physicians practicing in 44 states. Although most (98.3%) report that they assess pretest probability of PE before testing, slightly more than half do so routinely. A total of 72.5% prefer an unstructured approach to pretest assessment, whereas 22.9% use published prediction rules. Most (93.0%) are aware of at least one published guideline for assessing acute PE, but only 44.2% report using one or more in daily practice. Respondents who use published prediction rules, estimate pretest probability routinely, or use at least one practice guideline were more likely to request additional testing when reviewing a low probability clinical scenario. No differences in testing frequency or preferences were observed for intermediate or high probability clinical scenarios. Conclusions. The majority of clinicians we surveyed use an unstructured approach when estimating the pretest probability of acute PE. With the exception of low probability scenario, clinicians agreed on testing choices in suspected acute PE, regardless of the method or frequency of pre-test assessment. Key Words. Acute pulmonary embolism; pretest risk assessment; clinical practice guidelines; diagnostic algorithms; survey. © AUR, 2008 Acute pulmonary embolism (PE) is a common, potentially lethal, but treatable condition with protean clinical manifestations (1–3). Accurate and timely diagnosis Acad Radiol 2008; 15:3–14 1 From the Russell H. Morgan Department of Radiology and Radiological Sciences (C.R.W., J.C.S., E.K.F.) and Department of Medicine, Division of Pulmonary and Critical Care Medicine (G.B.D.), The Johns Hopkins University School of Medicine, 601 N Caroline Street, Room 3254, Baltimore, MD 21287-0801; Division of Pulmonary and Critical Care Medicine, Wake Forest University School of Medicine, Winston-Salem, NC (E.F.H.); Department of Epidemiology, Johns Hopkins University, Bloomberg School of Public Health, Baltimore, MD (B.M.). Received June 15, 2007; accepted July 13, 2007. Address correspondence to: C.R.W. e-mail: [email protected] © AUR, 2008 doi:10.1016/j.acra.2007.07.019 of PE is essential because both the over- and undertreatment of acute PE carries significant risk (4). Because the prevalence of PE in patient populations in whom it is suspected varies widely and may be as low as 18% (5– 8), the appropriate selection of patients for diagnostic imaging can be challenging. The recent increased use of diagnostic imaging, particularly computed tomography (CT) pulmonary angiography in hospitalized and emergency department patients suggests that clinicians are lowering their threshold for requesting confirmatory radiologic testing in acute PE (9, 10). A simple and effective means of clinical risk assessment to guide subsequent, expensive diagnostic testing would be useful (2, 11). 3 WEISS ET AL Three models for clinical pretest risk assessment in acute PE have been proposed: the Canadian model of Wells and others (8, 11), the Geneva model (7), and the Pisa model (12, 13). Each of these methods assigns patients to low-, intermediate-, or high-risk categories based on history, clinical findings, and chest radiography. After the PE risk is defined, published practice guidelines can be applied to direct subsequent confirmatory laboratory and radiological testing (3). Although the utility of such clinical prediction rules in patients with suspected acute PE has been studied and validated (6, 14 –21), it is not known whether US physicians regularly use them in practice. In the case of PE, physicians may prefer to base pretest assessment on their own unstructured judgment (22, 23), even though such “gestalt” impressions suffer from inadequate interobserver reliability (23). In addition, it is not known whether clinicians are aware of and adhere to published guidelines when evaluating patients with suspected acute PE. We examined the self-reported diagnostic approaches of three groups of US clinicians who are likely to encounter patients with acute PE in their clinical practices. Our specific objectives were to: 1) determine how often and by what method the clinicians evaluate the pretest likelihood of acute PE; 2) assess the awareness of published practice guidelines dealing with acute PE; and 3) examine the effect of the reported pretest practices on diagnostic testing choices in hypothetical patients. MATERIALS AND METHODS Study Population and Sample Size The Joint Committee on Clinical Investigation, our institutional review board, approved the study design and survey instrument, waiving the need for informed consent. We selected emergency medicine specialists, pulmonologists, and general internists as our survey population as we felt that these groups’ physicians would commonly encounter PE in their clinical practices. As well, we felt that having three distinct groups of physicians would improve the generalizability of our study and would allow us to determine if practice patterns were dependent on specialty. Assuming power of 0.80, an alpha equal to 0.05, and wishing to discern an absolute difference of 20% between physician groups, we determined that a sample size of about 103 respondents per physician group would suffice. Our own experience (24) and published evidence on physician response rates to mail question- 4 Academic Radiology, Vol 15, No 1, January 2008 naires available at the time this study was planned (25, 26) suggested that we could achieve a response rate of about 50%. We decided to survey a minimum of 250 participants per physician group. We acquired the names of physicians from the membership lists of three medical specialty organizations: the American Board of Emergency Medicine, the American College of Chest Physicians, and the American College of Physicians. Inclusion criteria included: membership in good standing with the organization; board certification; and current, active practice in one of the 50 US states or Puerto Rico. For participants from the American College of Physicians, we specified the subcategory of general internal medicine. Physicians who were in training, retired from practice, or were no longer caring for patients were excluded. Within these parameters, the organizations provided us with the following samples, chosen at random from their respective pools of eligible members: American Board of Emergency Medicine, 250 emergency medicine physicians (EM); American College of Chest Physicians, 255 pulmonologists (PUL); and American College of Physicians, 350 general internists (GIM). Survey Instrument We developed a 12-page questionnaire which has been previously described (27, 28). Physicians were instructed to base their responses on their own clinical experience and practices during the preceding 12 months. Physician and Practice Characteristics We asked the physicians to classify their clinical practice (eg, private group, academic, solo), to estimate the percentage of their working time involved with direct patient care, to specify the relative mix of inpatients and outpatients they encounter in their practices, and to estimate the number of patients they see per month, on average, with suspected acute PE. Participants also noted their age and the year in which they completed their primary residency or were certified in their specialty. Method and Frequency of Estimating the Pretest Probability of PE We asked participating clinicians to indicate how they estimate the clinical likelihood or probability of PE before requesting confirmatory diagnostic studies. Participants chose from five possible responses: 1) I use a standardized, published set of prediction rules to calculate a pretest probability and to classify my patients into low-, in- Academic Radiology, Vol 15, No 1, January 2008 termediate-, and high-probability categories. 2) I do not use published prediction rules. I rely on my own clinical criteria to classify my patients into low-, intermediate-, and high-probability categories. 3) I do not stratify patients into low-, intermediate-, and high-probability categories. I decide subjectively whether I think that the patient may have a PE or not. 4) I never estimate the pre]-test probability or likelihood of acute PE in my patients. 5) Other. The participants were then asked to estimate how often they determined the clinical likelihood or probability of acute PE before requesting confirmatory studies in those patients. Clinicians that answered ⱖ80% of the time were categorized as “very frequent” estimators, those who answered 79%–50% of the time were categorized as “frequent” estimators, and those who answered ⱕ49% of the time were categorized as “infrequent” estimators. Awareness of Specific Practice Guidelines The participants indicated their level of awareness (not aware of it, aware of it, have read it, and use it in my practice) of the following four published review articles/clinical practice guidelines: 1) American Thoracic Society (1999 practice guidelines) (29); 2) Fedullo and Tapson (2003 review article) (3); 3) American College of Emergency Physicians (ACEP) (2003 practice guidelines) (30); and 4) British Thoracic Society (2003 practice guidelines) (31). Clinical Scenarios We created three clinical scenarios of hypothetical patients who presented for medical care with a chief complaint of dyspnea. The scenarios (Appendix 1) contained sufficient clinical information to allow classification as low, intermediate, or high probability for pulmonary embolism, based on the Fedullo and Tapson (3) adaptations of the criteria of Wells et al (8). Case 1 was low probability (1.5 points), Case 2 was high probability (6.5 points), and Case 3 was intermediate probability (3 points). Respondents were asked to respond to the following three questions, based on their current clinical practice: 1) Which one test would you choose as the first step in evaluating this patient for acute PE? 2) Which one test would you choose if the patient had presented after midnight or on a weekend? 3) Which one test would you choose if the chest x-ray had revealed a left lower lobe infiltrate (Case 1), a left pleural effusion (Case 2), small pleural effusions and moderate pulmonary vascular congestion (Case 3)? For each of these questions, respondents were given the following choices: ventilation-perfusion scan- PRETEST RISK ASSESSMENT IN SUSPECTED ACUTE PE ning, computed tomographic pulmonary angiography (CTPA), D-dimer, ultrasound of the lower extremities, conventional pulmonary angiography, echocardiogram, or no testing for acute PE is needed. Survey Method The survey was conducted between September 2004 and February 2005. Details of the method used have been previously reported (27, 28). Data Analysis The results for closed-ended questions were expressed as a percentage of the total responses and were analyzed by individual physician specialty group. Numerical responses for open-ended questions were expressed as a mean with range and standard deviation. After the responses of the three groups were found to be comparable, the responses of the three groups were also pooled, and analyzed in aggregate. Statistical analyses were performed using SAS Version 8.2 (SAS Institute, Cary, NC). The chi-square test was used to compare these responses to the various demographic and practice characteristics: the organization affiliation; academic versus nonacademic practice setting; the percentage of time spent caring for inpatients (ⱕ50% versus ⱖ51%); the number of patients with possible acute PE encountered per month (ⱕ4 patients versus ⱖ5 patients per month); and the proximity to training (primary residency completed before or after 1990). We chose 1990 as the approximate time when helical CT technology was becoming clinically available. Associations for which the P value was .05 or less were considered statistically significant. RESULTS Survey Responses Of the 855 physicians selected, 32 notified us that they were retired from practice or were no longer seeing patients, and 17 surveys were undeliverable and returned by the post office. Of the 806 potentially eligible participants, we received completed surveys from 240 physicians (29.8%) practicing in 44 states. Physician and practice characteristics have been summarized previously (27, 28). Method and Frequency of Pretest Assessment All but four respondents, 98.3%, reported that they assess the pretest probability of PE before obtaining diag- 5 WEISS ET AL Academic Radiology, Vol 15, No 1, January 2008 Table 1 Method of Estimating the Pretest Risk Assessment in Acute PE* Characteristic (Number) All respondents (n ⫽ 239) Emergency medicine physicians (n ⫽ 84) Pulmonologists (n ⫽ 77) General internists (n ⫽ 78) Practice: Nonacademic (n ⫽ 204) Academic (n ⫽ 35) Proportion of time spent in direct patient care activities ⱕ75% (n ⫽ 53) ⱖ76% (n ⫽ 178) Proportion of time devoted to caring for hospitalized inpatients: ⱕ50% (n ⫽ 174) ⱖ51% (n ⫽ 64) Patients with possible acute PE encountered in practice: ⱕ4/month (n ⫽ 127) ⱖ5/month (n ⫽ 110) Residency completed: Before 1990 (n ⫽ 159) After 1989 (n ⫽ 75) Standardized Prediction Unstructured “Gestalt” Unstructured “Gestalt” Rules (With 3-Group Approach (With Approach (With Stratification)† 3-Group Stratification)‡ 2-Group Stratification)§ Never¶ Other Method Number (%) Number (%) Number (%) Number (%) Number (%) 55 (22.9) 128 (53.3) 46 (19.2) 4 (1.7) 6 (2.5) 21 (25.0) 15 (19.5) 19 (24.1) 43 (51.2) 48 (62.3) 37 (48.8) 14 (16.7) 14 (18.2) 18 (22.8) 2 (2.4) 0 (0) 2 (2.5) 4 (4.8) 0 (0) 2 (2.6) 44 (21.5) 11 (31.4) 110 (53.7) 18 (51.4) 40 (19.5) 6 (17.1) 4 (2.0) 0 (0) 6 (2.9) 0 (0) 16 (29.6) 38 (21.4) 25 (46.3) 98 (55.1) 10 (18.5) 35 (19.7) 0 (0) 4 (2.3) 2 (3.8) 3 (1.7) 41 (23.4) 14 (21.9) 90 (51.4) 37 (57.8) 33 (18.9) 13 (20.3) 4 (2.3) 0 (0) 6 (3.4) 0 (0) 28 (21.9) 27 (24.6) 66 (51.6) 60 (54.6) 25 (19.5) 21 (19.1) 3 (2.3) 1 (0.9) 5 (3.9) 1 (0.9) 33 (20.6) 20 (26.7) 87 (54.4) 38 (50.7) 31 (19.4) 15 (20.0) 4 (2.5) 0 (0) 4 (2.5) 2 (2.7) PE: pulmonary embolism. *Survey question: When you evaluate a patient in whom you suspect acute PE, how do you estimate the clinical likelihood or probability of PE before requesting confirmatory studies? (Please choose the one response that best describes your practice). †Use a standardized, published set of prediction rules to calculate a pretest probability and to classify patients into low-, intermediate-, and high-probability categories. ‡Do not use published prediction rules. Rely on own clinical criteria to classify patients into low-, intermediate-, and high-probability categories. §Do not stratify patients into low-, intermediate-, and high-probability categories. Decide subjectively whether I think that the patient may have a PE or not. ¶Never estimate the pretest probability or likelihood of acute PE in my patients. nostic testing (Table 1). A large majority (72.5%) use their own unstructured clinical judgment to assess their patients, whereas only 22.9% of respondents reported using published prediction rules. No significant differences in response frequency to this question were evident among the demographic and practice characteristics we studied, and the approaches to pretest assessment were similar among the three physician groups (Fig 1). The frequency with which pretest clinical assessment is conducted in acute PE was inconsistent (Table 2). Only 54.8% of all respondents were “very frequent” estimators of pretest probability, and general internists were the least 6 likely to do so (GIM 43.2%, EM 59.5%, PUL 61.3%, P ⫽ .05). Physicians completing training before 1990 were more likely to be “very frequent” estimators of pretest probability (before 1990 ⫽ 58.8%, after 1989 ⫽ 44.4%, P ⫽ .04). Physicians who completed training after 1989 were twice as likely to report being “infrequent” estimators of pretest probability (after 1989 ⫽ 27.8%, before 1990 ⫽ 13.1%, P ⬍ .01). “Very frequent” estimators were more likely to use published prediction rules to assess pretest probability of acute PE than were respondents who are “frequent” or “infrequent” estimators (31.2% vs 15.9% versus 14.7%, respectively, P ⫽ .01). Academic Radiology, Vol 15, No 1, January 2008 PRETEST RISK ASSESSMENT IN SUSPECTED ACUTE PE Figure 1. Clinicians’ approach to estimation of the pretest probability of pulmonary embolism. Table 2 Frequency with which Pretest Risk Assessment is Estimated* Characteristic (Number) All respondents (n ⫽ 228) Emergency medicine physicians (n ⫽ 79) Pulmonologists (n ⫽ 75) General internists (n ⫽ 74) Practice: Nonacademic (n ⫽ 195) Academic (n ⫽ 33) Proportion of time spent in direct patient care activities ⱕ75% (n ⫽ 50) ⱖ76% (n ⫽ 171) Proportion of time devoted to caring for hospitalized inpatients: ⱕ50% (n ⫽ 164) ⱖ51% (n ⫽ 63) Patients with possible acute PE encountered in practice: ⱕ4/month (n ⫽ 120) ⱖ5/month (n ⫽ 106) Residency completed: Before 1990 (n ⫽ 153) After 1989 (n ⫽ 72) Very Frequent† Number (%) Frequent† Number (%) Infrequent† Number (%) 125 (54.8) 47 (59.5) 46 (61.3) 32 (43.2) 63 (27.6) 19 (24.1) 20 (26.7) 24 (32.4) 40 (17.5) 13 (16.5) 9 (12.0) 18 (24.3) 102 (52.3) 23 (69.7) 54 (27.7) 9 (27.3) 39 (20.0) 1 (3.0) 32 (64.0) 91 (53.2) 14 (28.0) 46 (26.9) 4 (8.0) 34 (19.9) 83 (50.6) 42 (66.7) 48 (29.3) 14 (22.2) 33 (20.1) 7 (11.1) 62 (51.7) 62 (58.5) 35 (29.2) 27 (25.5) 23 (19.2) 17 (16.0) 90 (58.8) 32 (44.4) 43 (28.1) 20 (27.8) 20 (13.1) 20 (27.8) *Survey question: In your practice, approximately what percentage of the time do you estimate the clinical likelihood or probability of the diagnosis before you request confirmatory studies? †Clinicians that answered ⱖ80% of the time were categorized as “very frequent” estimators, those who answered 79%–50% of the time were categorized as “frequent” estimators, and those who answered ⱕ49% of the time were categorized as “infrequent” estimators. Awareness of Published Practice Guidelines The vast majority (93%) of respondents reported either using, reading or being aware of at least one of the four listed practice guidelines/review articles referenced in the survey, although far fewer (44.2%) report actually using one or more in their daily practice (Table 3). Respondents were more likely to use or to have read guidelines from their specialty-specific literature than either of the other guidelines (PUL: American Thoracic Society 36.4% versus Fedullo 19.5% versus ACEP 0% and EM: American 7 WEISS ET AL Academic Radiology, Vol 15, No 1, January 2008 Table 3 Awareness of Published Guidelines* Guidelines American Thoracic Society (Guidelines)† Fedullo and Tapson: The Evaluation of Suspected Pulmonary Embolism (review article)‡ The American College of Emergency Physicians (guidelines)§ British Thoracic Society (guidelines)¶ Any of the above Response Option Use it in my practice Have read it Aware of it Not aware of it Use it in my practice Have read it Aware of it Not aware of it Use it in my practice Have read it Aware of it Not aware of it Use it in my practice Have read it Aware of it Not aware of it Use it in my practice Have read it Aware of it Not aware of it Total Responses Pulmonologists General Internists Emergency Medicine n ⫽ 240 n ⫽ 77 n ⫽ 79 Physicians n ⫽ 84 Number (%) Number (%) Number (%) Number (%) 42 (17.5) 69 (28.8) 66 (27.5) 63 (26.3) 48 (20.0) 73 (30.4) 59 (24.6) 60 (25.0) 43 (17.9) 33 (13.8) 41 (17.1) 122 (50.8) 10 (4.2) 10 (4.2) 39 (16.3) 180 (75.0) 106 (44.2) 87 (36.3) 30 (12.5) 17 (7.1) 28 (36.4) 35 (45.5) 8 (10.4) 6 (7.8) 15 (19.5) 37 (48.1) 18 (23.4) 7 (9.1) 0 (0) 4 (5.2) 18 (23.4) 54 (70.1) 3 (3.9) 6 (7.8) 14 (18.2) 53 (68.8) 35 (45.5) 36 (46.8) 5 (6.5) 1 (1.3) 10 (12.7) 19 (24.1) 27 (34.2) 23 (29.1) 19 (24.1) 19 (24.1) 20(25.3 21 (26.6) 3 (3.8) 1 (1.3) 12 (15.2) 63 (79.8) 4 (5.1) 1 (1.3) 14 (17.7) 60 (76.0) 27 (34.2) 25 (31.7) 15 (19.0) 12 (15.2) 4 (4.8) 15 (17.9) 31 (36.9) 34 (40.5) 14 (16.7) 17 (20.2) 21 (25.0) 32 (31.1) 40 (47.6) 28 (33.3) 11 (13.1) 5 (6.0) 3 (3.6) 3 (3.6) 11 (13.1) 67 (79.8) 44 (52.4) 26 (31.0) 10 (11.9) 4 (4.8) PE: pulmonary embolism. *Survey question: Please indicate your awareness of the following four (4) published review articles or clinical practice guidelines that have been proposed for evaluating patients with suspected acute pulmonary PE. Also, indicate whether you have read them and you use them in your practice. Physicians were asked to check all that apply. We report the highest level of awareness. †American Thoracic Society (Guidelines) Am J Resp Crit Care Med 1999; 160:1043–1066. ‡Fedullo and Tapson (review article) The evaluation of suspected pulmonary embolism. N Engl J Med 2003; 349:1247–1256. §American College of Emergency Physicians (Guidelines) Ann Emerg Med 2003; 41:257–270. ¶British Thoracic Society(Guidelines) Thorax 2003; 58:470 – 484. Thoracic Society 4.8% versus Fedullo 16.7% versus ACEP 47.6%), and not surprisingly for three groups of US-based physicians, respondents were the least aware of the British Thoracic Society Guidelines. Those respondents who see more patients per month (ⱖ5) in whom acute PE is suspected are more likely to use at least one of the practice guidelines listed in their daily practice (P ⬍ .003). The awareness of published practice guidelines and method of predicting pretest probability of PE are related. Of the 22.9% of respondents who use published prediction rules to calculate pretest probability of PE, 76.4% report using one of the four guidelines presented in the survey. In contrast, of the 72.5% of respondents who use an unstructured approach to assess the probability of PE, only 35.6% report using at least one of the listed guidelines in practice. 8 Performance on Hypothetical Clinical Scenarios As the likelihood of PE increased in the scenarios, respondents requested more diagnostic testing (Tables 4a, b). Testing choices shifted from “no testing” and D-dimer in the low probability scenario to imaging (computed tomography pulmonary angiography [CTPA] and ventilation-perfusion scanning [VIP]) in the intermediate-and high-probability cases, with the most imaging ordered for the latter. The relationships of clinical scenario testing choices to clinicians’ self-reported methods of pretest assessment are summarized in Fig 2. Respondents who report using published prediction rules to calculate pretest probability of acute PE were more likely to request further testing when faced with a low probability clinical scenario than those respondents using an unstructured approach (64.8% versus 47.1%, P ⫽ .02). No significant differences were seen for the intermediate- Scenario Type Total Responses Total Responses Response (Weekdays) (Nights/Weekends) Option Number (%) Number (%) Low probability n ⫽ 238 104 (43.7) 70 (29.4) 42 (17.7) 12 (5.0) 10 (4.2) No testing D-dimer CTPA VIP Other* n ⫽ 239 8 (3.4) 29 (12.1) 154 (64.4) 27 (11.3) 21 (8.8) n ⫽ 239 10 (4.2) 34 (14.2) 167 (69.9) 19 (8.0) 9 (3.8) No testing D-dimer CTPA VIP Other* n ⫽ 239 0 (0) 13 (5.4) 176 (73.6) 38 (15.9) 12 (5.0) n ⫽ 239 2 (0.8) 18 (7.5) 187 (78.2) 24 (10.0) 8 (3.4) High probability Standardized Prediction Unstructured “Gestalt” Unstructured “Gestalt” Rules (with 3 group Method (with 3 group Method (with 2 group stratification)† stratification)‡ stratification)§ Number (%) Number (%) Number (%) Findings ⫽ left lower lobe infiltrate n ⫽ 238 170 (71.43) 31 (13.0) 28 (11.8) 6 (2.5) 3 (1.3) Findings ⫽ pleural effusions and pulmonary vascular congestion n ⫽ 233 37 (15.9) 28 (12.0) 119 (51.1) 10 (4.3) 39 (16.7) Findings ⫽ pleural effusion n ⫽ 239 3 (1.3) 9 (3.8) 201 (84.1) 13 (5.4) 13 (5.4) n ⫽ 54 19 (35.2) 19 (35.2) 10 (18.5) 5 (9.3) 1 (1.9) n ⫽ 128 68 (53.1) 37 (28.9) 17 (13.3) 5 (3.9) 1 (0.8) n ⫽ 46 24 (52.2) 10 (21.7) 6 (13.0) 3 (6.5) 3 (6.5) n ⫽ 55 1 (1.8) 7 (12.7) 35 (63.6) 7 (12.7) 5 (9.1) n ⫽ 128 2 (1.6) 15 (11.7) 87 (68.0) 14 (10.9) 10 (7.8) n ⫽ 45 3 (6.7) 6 (13.3) 26 (57.8) 6 (13.3) 4 (8.9) n ⫽ 55 0 (0) 3 (5.4) 40 (72.7) 7 (12.7) 5 (9.1) n ⫽ 128 0 (0) 7 (5.5) 96 (75.0) 21 (16.4) 4 (3.1) n ⫽ 45 0 (0) 2 (4.4) 33 (73.3) 9 (20.0) 1 (2.2) CTPA: computed tomography pulmonary angiography; VIP: ventilation perfusion scanning. *Other: pulmonary angiography ⫹ ultrasound ⫹ echocardiogram. †Use a standardized, published set of prediction rules to calculate a pretest probability and to classify patients into low, intermediate, and high probability categories. ‡Do not use published prediction rules. Rely on own clinical criteria to classify patients into low, intermediate, and high probability categories. §Do not stratify patients into low, intermediate, and high probability categories. Decide subjectively whether I think that the patient may have a PE or not. PRETEST RISK ASSESSMENT IN SUSPECTED ACUTE PE Intermediate probability No testing D-dimer CTPA VIP Other* n ⫽ 238 118 (49.6) 68 (28.6) 34 (14.3) 13 (5.5) 5 (2.1) Total Responses (with additional findings) Number (%) Academic Radiology, Vol 15, No 1, January 2008 Table 4a Scenario Performance versus Method of Pretest Assessment 9 WEISS ET AL Academic Radiology, Vol 15, No 1, January 2008 Table 4b Scenario Performance versus Frequency of Pretest Assessment and Awareness of Guidelines Scenario Type Response Option Low probability No testing D-dimer CTPA VIP Other* Intermediate probability No testing D-dimer CTPA VIP Other* High probability No testing D-dimer CTPA VIP Other* Very Frequent† Number (%) Frequent† Number (%) Infrequent† Number (%) Uses any Guideline Does not use any Guideline n ⫽ 124 53 (42.7) 48 (38.7) 13 (10.5) 7 (5.7) 3 (2.4) n ⫽ 124 1 (0.8) 14 (11.2) 87 (70.2) 14 (11.3) 8 (6.5) n ⫽ 124 0 (0) 6 (4.8) 92 (74.2) 18 (14.5) 8 (6.5) n ⫽ 63 36 (57.1) 11 (17.5) 11 (17.5) 4 (6.3) 1 (1.6) n ⫽ 63 3 (4.8) 5 (7.9) 40 (63.5) 8 (12.7) 7 (11.1) n ⫽ 63 0 (0) 2 (3.2) 48 (76.2) 12 (19.1) 1 (1.6) n ⫽ 39 22 (56.4) 7 (18.0) 7 (18.0) 2 (5.1) 1 (2.6) n ⫽ 40 3 (7.5) 10 (25.0) 19 (47.5) 4 (10.0) 4 (10.0) n ⫽ 40 0 (0) 4 (10.0) 27 (67.5) 7 (17.5) 2 (5.0) n ⫽ 106 42 (40.0) 37 (35.2) 17 (16.2) 7 (6.7) 2 (1.9) n ⫽ 106 5 (4.7) 11 (10.4) 76 (71.7) 8 (7.6) 6 (5.7) n ⫽ 106 0 (0) 5 (4.7) 82 (77.4) 13 (12.3) 6 (5.7) n ⫽ 133 76 (57.1) 31 (23.3) 17 (12.8) 6 (4.5) 3 (2.3) n ⫽ 133 3 (2.3) 18 (13.5) 78 (58.7) 19 (14.3) 15 (11.3) n ⫽ 133 0 (0) 8 (6.0) 94 (70.7) 25 (18.8) 6 (4.5) CTPA: computed tomography pulmonary angiography; VIP: ventilation perfusion scanning. *Other: pulmonary angiography ⫹ ultrasound ⫹ echocardiogram. †Clinicians that answered ⱖ80% of the time were categorized as “very frequent” estimators, those who answered 79%–50% of the time were categorized as “frequent” estimators, and those who answered ⱕ49% of the time were categorized as “infrequent” estimators. and high-probability clinical scenarios. Most of the increased testing with the low probability scenario was D-dimer. The same trends were noted with “very frequent” estimators and respondents who use at least one practice guideline in daily practice. After the chest x-ray findings of left lower lobe infiltrate were added to the low probability scenario, making the diagnosis of pneumonia more likely, 71.3% of respondents requested no further testing. DISCUSSION The determination of pretest likelihood based on clinical parameters has been recommended broadly for the accurate and cost-effective evaluation of suspected acute PE. Yet, our study demonstrates that approximately three of four clinicians who responded to our questionnaire do not use published prediction rules to stratify their patients into low-, intermediate-, and high-probability categories. Instead, most prefer an unstructured method to stratify patients into two or three probability categories, even though they are aware of published guidelines. It is noteworthy that the unstructured “gestalt” approach was not 10 associated with detectable suboptimal approaches to our case scenarios. Although it remains to be established whether patient outcome is affected adversely by the clinician’s method of performing a pretest assessment of PE likelihood, these observations suggest that the simple, subjective approach may suffice. As reported, the overall range of adherence to published pretest probability guidelines is 22.9%, within the range of physician adherence to diagnosis and treatment guidelines for other conditions such as asthma and hypertension (32– 42). However, because only 1.7% of respondents report never estimating a pretest probability, it is clear that the practice of pretest clinical assessment in acute PE has diffused widely among US clinicians. Moreover, approximately 93% of the responding clinicians report that they are aware of at least one of the four major guidelines that were available for the diagnosis of acute PE at the time of this survey, and that 84.6% of respondents felt that the pretest probability of PE was either “extremely” or “very” important when they considered testing options in diagnosing acute PE (27). It is not surprising that physician groups were far more likely to have read or to use the guidelines published in their specialty- Academic Radiology, Vol 15, No 1, January 2008 PRETEST RISK ASSESSMENT IN SUSPECTED ACUTE PE Figure 2. Physicians requesting further testing for acute pulmonary embolism (any test). specific literature. Interestingly, the rigorous, evidencebased British Thoracic Society guidelines were the least used or read of all of the guidelines listed, which suggests that literature penetration varies with geographic, as well as specialty boundaries. Half of respondents estimate the pretest probability of acute pulmonary embolism ⱖ80% of the time. That general internists were the least likely to estimate the pretest probability among our three physician groups, is consistent with previous reports that general practitioners adhered less closely to published practice guidelines than did specialists (32– 42). Physician nonadherence with clinical practice guidelines is not uncommon (4, 43), and may be explained by difficulty overcoming the inertia of previous practice habits, a lack of awareness of the recommendations, or disagreement with them (40, 44). General internists may therefore be the best audience for future focused education regarding PE. We noted with interest that physicians who completed residency after 1989 were significantly less likely to calculate routinely a pretest probability of PE than their more senior counterparts. Because these younger physicians were trained during an era in which evidence-based medicine and the impact of pretest clinical suspicions on interpretation of diagnostic tests have been emphasized, we had expected that they would be more likely to calculate pretest probability for PE than their colleagues who were trained before 1990. On the other hand, these younger physicians were also trained in an era when CT was becoming the imaging procedure of choice for suspected PE (27), and whereas VP is reported in terms of probabilities, CT angiography is reported in terms of di- chotomous positive/negative results. Physicians who were trained during the CT era may have more trust in CT technology to provide them with an absolute answer and, consequently, feel less of a need to perform detailed pretest assessments than do older physicians. Importantly, data from the PIOPED II trial demonstrate that the interpretation of CT results varies based on pretest assessment of acute pulmonary embolism, and that the predictive value of CTPA drops significantly when imaging results are inconsistent with pretest clinical probability (21). Analysis of the testing choices in the clinical scenarios provided additional insights regarding the process of patient assessment. As the probability of PE increased, respondents appropriately obtained further confirmatory testing. The type of testing varied with the likelihood of PE: D-dimer assays were favored for the low-probability case, and imaging (particularly CTPA) was preferred for the intermediate- and high-probability scenarios. We hypothesized that in Case 1 (low probability, 1.5 points), physicians who used formal grading systems would be less likely to obtain further diagnostic testing. Remarkably, these clinicians were more likely to obtain further tests than their counterparts who used a less structured “gestalt” approach to risk assessment. The reasons for this apparent discordance are unclear. It is possible that the former group with more familiarity with PE guidelines had heightened suspicions and awareness of PE, which outweighed the effects of grading patient risk factors, leading to a paradoxically lower threshold to test. The higher use of D-dimer testing in these patients is consistent with their increased adherence to guidelines. 11 WEISS ET AL Despite variability in their self-reported approaches to pretest probability estimates, clinicians had similar approaches to hypothetical patients with higher risks for PE. That physicians who rely on clinical “gestalt” performed well on case scenarios is reassuring, and suggests that simple recognition of risks for PE without a formal probability calculation, the approach favored by most (78%) clinicians, is sufficient for appropriate patient management. This finding is also consistent with the recent report of Kabrhel and colleagues, who found that the subjective component of the Canadian Pulmonary Embolism Score accounted for nearly all of the predictive value of the Canadian Pulmonary Embolism Score (45). This question, “Are alternative diagnoses less likely than PE?” is especially relevant to our low-probability scenario in which “gestalt” appeared superior to a more standardized approach. This observation also suggests that the increasing likelihood of PE, the complexity of the setting, or other factors drive the approach to further testing and outweigh effects of the method of probability estimation. Our study does have some limitations, including a lower than anticipated response rate, possible recall bias, and a participant pool which was limited to general internists, emergency medicine physicians, and pulmonologists practicing in the United States. These limitations have been more fully explored in previous reports (27, 28). If the responses to this survey represent those of highly motivated clinicians with particular interest in PE, then it is likely that these data represent an overestimate of PE awareness and of clinicians’ performance. It is possible that respondents’ approaches to our case scenarios may not reflect their approaches to clinical uncertainties in real patients. However, the progressive increase in testing of scenarios with higher likelihood of PE suggests that clinically relevant gradations of risk were recognized by survey respondents. Moreover, such clinical vignettes have been demonstrated to be a valid tool for measuring the quality of clinical practice (46 – 48). This approach has been used effectively previously to assess clinicians’ approaches to deep vein thrombosis (49) and to PE (50 –52). In summary, most clinicians responding to our survey estimate the pretest likelihood of acute PE before requesting confirmatory studies, but prefer an unstructured approach for this assessment rather than using published prediction rules. Only half of clinicians calculate a pretest probability of PE routinely, even though nearly all are aware of current practice guidelines. Physicians who are “very frequent” estimators or who use published prediction rules were more likely to request testing, usually 12 Academic Radiology, Vol 15, No 1, January 2008 D-dimer, in the low probability scenario. No differences were observed in the intermediate- or high-probability categories, regardless of frequency or method of pretest assessment. Although these data may suggest areas for focused education, they demonstrate that the unstructured “gestalt” pretest assessment practiced by most clinicians is probably sufficient both for patient safety and for costeffective diagnosis of acute pulmonary embolism. ACKNOWLEDGMENTS The authors thank Evanus Forrester for her generous assistance in preparing the mailing lists; Chris O’Keefe, BS, for preparation of the response database; and Sheila C. Weiss, RD, for her assistance in preparing and mailing the surveys. REFERENCES 1. Moser KM. Venous thromboembolism. Am Rev Respir Dis 1990; 141: 235–249. 2. Perrier A, Bounameaux H. Cost-effective diagnosis of deep vein thrombosis and pulmonary embolism. Thromb Haemost 2001; 86:475– 487. 3. Fedullo PF, Tapson VF. Clinical practice. The evaluation of suspected pulmonary embolism. N Engl J Med 2003; 349:1247–1256. 4. Hagen PJ, van Strijen MJ, Kieft GJ, et al. The application of a Dutch consensus diagnostic strategy for pulmonary embolism in clinical practice. Neth J Med 2001; 59:161–169. 5. Value of the ventilation/perfusion scan in acute pulmonary embolism. Results of the prospective investigation of pulmonary embolism diagnosis (PIOPED). The PIOPED Investigators. JAMA 1990; 263:2753–2759. 6. Miniati M, Bottai M, Monti S. Comparison of 3 clinical models for predicting the probability of pulmonary embolism. Medicine (Balt) 2005; 84:107–114. 7. Wicki J, Perneger TV, Junod AF, et al. Assessing clinical probability of pulmonary embolism in the emergency ward: a simple score. Arch Intern Med 2001; 161:92–97. 8. Wells PS, Anderson DR, Rodger M, et al. Derivation of a simple clinical model to categorize patients probability of pulmonary embolism: increasing the models utility with the SimpliRED D-dimer. Thromb Haemost 2000; 83:416 – 420. 9. Prologo JD, Gilkeson RC, Diaz M, et al. CT pulmonary angiography: a comparative analysis of the utilization patterns in emergency department and hospitalized patients between 1998 and 2003. AJR Am J Roentgenol 2004; 183:1093–1096. 10. Ost D, Khanna D, Shah R, et al. Impact of spiral computed tomography on the diagnosis of pulmonary embolism in a community hospital setting. Respiration 2004; 71:450 – 457. 11. Wells PS, Ginsberg JS, Anderson DR, et al. Use of a clinical model for safe management of patients with suspected pulmonary embolism. Ann Intern Med 1998; 129:997–1005. 12. Miniati M, Prediletto R, Formichi B, et al. Accuracy of clinical assessment in the diagnosis of pulmonary embolism. Am J Resp Crit Care Med 1999; 159:864 – 871. 13. Miniati M, Monti S, Bottai M. A structured clinical model for predicting the probability of pulmonary embolism. Am J Med 2003; 114:173–179. 14. Wolf SJ, McCubbin TR, Feldhaus KM, et al. Prospective validation of Wells Criteria in the evaluation of patients with suspected pulmonary embolism. Ann Emerg Med 2004; 44:503–510. 15. Tamariz LJ, Eng J, Segal JB, et al. Usefulness of clinical prediction rules for the diagnosis of venous thromboembolism: a systematic review. Am J Med 2004; 117:676 – 684. Academic Radiology, Vol 15, No 1, January 2008 16. Righini M, Bounameaux H. External validation and comparison of recently described prediction rules for suspected pulmonary embolism. Curr Opin Pulm Med 2004; 10:345–349. 17. van Belle A, Buller HR, Huisman MV, et al. Effectiveness of managing suspected pulmonary embolism using an algorithm combining clinical probability, D-dimer testing, and computed tomography. JAMA 2006; 295:172–179. 18. Eisner MD. Before diagnostic testing for pulmonary embolism: estimating the prior probability of disease. Am J Med 2003; 114:232–234. 19. Quiroz R, Kucher N, Zou KH, et al. Clinical validity of a negative computed tomography scan in patients with suspected pulmonary embolism: a systematic review. JAMA 2005; 293:2012–2017. 20. Perrier A, Roy PM, Sanchez O, et al. Multidetector-row computed tomography in suspected pulmonary embolism. N Engl J Med 2005 ;352: 1760 –1768. 21. Stein PD, Fowler SE, Goodman LR, et al. Multidetector computed tomography for acute pulmonary embolism. N Engl J Med 2006; 354: 2317–2327. 22. Runyon MS, Webb WB, Jones AE, et al. Comparison of the unstructured clinician estimate of pretest probability for pulmonary embolism to the Canadian score and the Charlotte rule: a prospective observational study. Acad Emerg Med 2005; 12:587–593. 23. Rodger MA, Maser E, Stiell I, et al. The interobserver reliability of pretest probability assessment in patients with suspected pulmonary embolism. Thromb Res 2005; 116:101–107. 24. Scatarige JC, Diette GB, Merriman B, et al. Physician satisfaction with high-resolution CT services provided by radiologists: results of a nationwide survey of pulmonary subspecialists. AJR Am J Roentgenol 2003; 180:585–589. 25. Cummings SM, Savitz LA, Konrad TR. Reported response rates to mailed physician questionnaires. Health Serv Res 2001; 35:1347–1355. 26. Asch DA, Jedrziewski MK, Christakis NA. Response rates to mail surveys published in medical journals. J Clin Epidemiol 1997; 50:1129 – 1136. 27. Weiss CR, Scatarige JC, Diette GB, et al. CT Pulmonary angiography is the first-line imaging test for acute pulmonary embolism: a survey of US clinicians. Acad Radiol 2006; 13:434 – 446. 28. Scatarige JC, Weiss CR, Diette GB, et al. Scanning systems and protocols used during imaging for acute pulmonary embolism how much do our clinical colleagues know? Acad Radiol 2006; 13:678 – 685. 29. Tapson VF, Carroll BA, Davidson BL, et al. The diagnostic approach to acute venous thromboembolism. Clinical practice guideline. American Thoracic Society. Am J Respir Crit Care Med 1999; 160:1043–1066. 30. Clinical policy: critical issues in the evaluation and management of adult patients presenting with suspected pulmonary embolism. Ann Emerg Med 2003; 41:257–270. 31. British Thoracic Society guidelines for the management of suspected acute pulmonary embolism. Thorax 2003; 58:470 – 483. 32. Hernandez DV, Schmaling KB. Understanding and resolving adherence problems. Clin Rev Allergy Immunol 2004; 27:65–73. 33. Adams RJ, Fuhlbrigge A, Guilbert T, et al. Inadequate use of asthma medication in the United States: results of the asthma in America national population survey. J Allergy Clin Immunol 2002; 110:58 – 64. 34. Legorreta AP, Christian-Herman J, O’Connor RD, et al. Compliance with national asthma management guidelines and specialty care: a health maintenance organization experience. Arch Intern Med 1998; 158:457– 464. 35. Vollmer WM, O’Hollaren M, Ettinger KM, et al. Specialty differences in the management of asthma. A cross-sectional assessment of allergists’ patients and generalists’ patients in a large HMO. Arch Intern Med 1997; 157:1201–1208. 36. O’Dowd LC, Fife D, Tenhave T, et al. Attitudes of physicians toward objective measures of airway function in asthma. Am J Med 2003; 114: 391–396. 37. Karras DJ, Kruus LK, Cienki JJ, et al. Evaluation and treatment of patients with severely elevated blood pressure in academic emergency departments: a multicenter study. Ann Emerg Med 2006; 47:230 –236. 38. Milchak JL, Carter BL, James PA, et al. Measuring adherence to practice guidelines for the management of hypertension: an evaluation of the literature. Hypertension 2004;4 4:602– 608. PRETEST RISK ASSESSMENT IN SUSPECTED ACUTE PE 39. Worrall G, Chaulk P, Freake D. The effects of clinical practice guidelines on patient outcomes in primary care: a systematic review. CMAJ 1997; 156:1705–1712. 40. Cabana MD, Rand CS, Powe NR, et al. Why don’t physicians follow clinical practice guidelines? A framework for improvement. JAMA 1999; 282:1458 –1465. 41. Gupta L, Ward JE, Hayward RS. Clinical practice guidelines in general practice: a national survey of recall, attitudes and impact. Med J Aust 1997; 166:69 –72. 42. Lomas J, Anderson GM, Domnick-Pierre K, et al. Do practice guidelines guide practice? The effect of a consensus statement on the practice of physicians. N Engl J Med 1989; 321:1306 –1311. 43. Mottur-Pilson C, Snow V, Bartlett K. Physician explanations for failing to comply with “best practices.” Eff Clin Pract 2001; 4:207–213. 44. Larkin M. Noncompliance with “best practices” may be justifiable. Lancet 2001; 358:1433. 45. Kabrhel C, McAfee AT, Goldhaber SZ. The contribution of the subjective component of the Canadian Pulmonary Embolism Score to the overall score in emergency department patients. Acad Emerg Med 2005; 12:915–920. 46. Peabody JW, Luck J, Glassman P, et al. Measuring the quality of physician practice by using clinical vignettes: a prospective validation study. Ann Intern Med 2004; 141:771–780. 47. Peabody JW, Luck J, Glassman P, et al. Comparison of vignettes, standardized patients, and chart abstraction: a prospective validation study of 3 methods for measuring quality. JAMA 2000; 283:1715–1722. 48. Dresselhaus TR, Peabody JW, Luck J, et al. An evaluation of vignettes for predicting variation in the quality of preventive care. J Gen Intern Med 2004; 19:1013–1018. 49. Riddle DL, Hillner BE, Wells PS, et al. Improving the diagnostic process for deep vein thrombosis in orthopaedic outpatients. Clin Orthop Relat Res 2005: 258 –266. 50. Iles S, Hodges AM, Darley JR, et al. Clinical experience and pre-test probability scores in the diagnosis of pulmonary embolism. QJM 2003; 96:211–215. 51. Wigton RS, Hoellerich VL, Patil KD. How physicians use clinical information in diagnosing pulmonary embolism: an application of conjoint analysis. Med Decis Making 1986; 6:2–11. 52. Aberegg SK, Haponik EF, Terry PB. Omission bias and decision making in pulmonary and critical care medicine. Chest 2005; 128:1497– 1505. APPENDIX 1: HYPOTHETICAL CLINICAL SCENARIOS Case 1: A 38-year-old, otherwise healthy, male presents with recent onset of dyspnea and a dry cough. On examination, he is febrile to 100°F, normotensive at 120/68 and has a heart rate of 102 beats/minute. His lung examination and chest x-ray are both unremarkable. Case 2: A 72-year-old female with a history significant for well-controlled hypertension, osteoporosis, and for recently treated Stage IIB breast cancer presents with acute-onset shortness of breath, pleuritic chest pain, and mild hemoptysis 5 days after discharge from the hospital following surgical fixation of a hip fracture. She is normotensive (135/75), afebrile but is tachypneic (22/minute) and tachycardic (120 BPM). The saturated oxygen level is 90% on 3 L oxygen by nasal cannula. Her lung examination demonstrates a few rales at the left lung base. Her 13 WEISS ET AL chest x-ray reveals minimal elevation of the left hemidiaphragm. Case 3: A 65-year-old woman with a medical history significant for congestive heart failure, a 40 pack/year smoking history, and a prior treated deep vein thrombosis of the lower extremity, presents with sudden onset of shortness of breath, chest pain, and cough. On examina- 14 Academic Radiology, Vol 15, No 1, January 2008 tion, her heart rate is 120 beats/minute, her respiratory rate is 22/minute, and a saturated oxygen level was 94% on room air. Her lung exam demonstrates a few crackles at the left base, and trace bipedal edema. Her electrocardiogram is negative. Her chest x-ray demonstrates cardiomegaly, discoid atelectasis in the left base, and no evidence of an infiltrate or pulmonary edema.