Original Articles |
From the Division of Cardiovascular Diseases, Mayo Clinic, Rochester, Minn (M.S.); Duke Clinical Research Institute, Durham, NC (S.M-B, E.P.); Mid America Heart Institute/UMKC, Kansas City, Mo (J.A.S.); and Denver VA Medical Center, Denver, Colo (J.S.R.).
Reprint requests to Dr Mandeep Singh, Mayo Clinic, 200 First St SW, Rochester, MN 55905. E-mail singh.mandeep{at}mayo.edu
Received November 28, 2007; accepted June 12, 2008.
| Abstract |
|---|
|
|
|---|
Methods and Results— In-hospital mortality after percutaneous coronary intervention on 309 351 patients from the National Cardiovascular Data Registry admitted from January 1, 2004, to March, 30, 2006, was studied. Using the Mayo Clinic Risk Score equation, we assigned predicted probabilities of death to each patient. The area under the receiver-operating characteristics curve was 0.884, indicating excellent discrimination overall as well as among subgroups, including gender, diabetes mellitus, renal failure, low ejection fraction, different age groups, and multivessel disease. Ninety-seven percent of patients undergoing percutaneous coronary intervention had a Mayo Clinic Risk Score <10, indicating low to intermediate risk. The Mayo Clinic Risk Score model initially slightly underpredicted event rates when applied in National Cardiovascular Data Registry data (observed 1.23% versus predicted 1.10%), but this underprediction was corrected after recalibration. The recalibrated risk score discriminated (c index=0.885) and calibrated well in an National Cardiovascular Data Registry validation data set consisting of procedures performed between April 1, 2006, and March 30, 2007.
Conclusions— Seven variables can be combined into a convenient risk scoring system before coronary angiography is performed to predict in-hospital mortality after percutaneous coronary intervention. This model may be useful for providing patients with individualized, evidence-based estimates of procedural risk as part of the informed consent process before percutaneous coronary intervention.
Key Words: angiography angioplasty complications risk factors
| Introduction |
|---|
|
|
|---|
Clinical Perspective see p 36
This internally validated model includes 7 clinical and easily obtainable noninvasive laboratory variables. To date, however, the MCRS has not been validated against an external PCI data set. The National Cardiovascular Data Registry (NCDR) Cath PCI Registry is designed to measure and promote quality improvement for interventional cardiology with an emphasis on patient and lesion selection criteria, procedural performance, and in-hospital outcomes/complications. Thus, it presents an ideal opportunity to validate the MCRS for in-hospital mortality after PCI. Once the validity of the MCRS is confirmed, it may be used as a tool to inform patients of their estimated risk from PCI and become an integral part of the informed consent process.
| Methods |
|---|
|
|
|---|
|
The definitions used by the NCDR are similar to those used in the original MCRS for in-hospital death and other baseline variables included in the model, except cardiogenic shock. Detailed baseline patient demographic, angiographic, and procedural variables used in NCDR have been previously described (Table 1).5
|
Statistical Methods
Crude in-hospital mortality and morbidity rates were calculated for the entire NCDR analysis population. In addition, raw mortality rates for each level of baseline demographic and laboratory characteristics were compared by use of Pearsons
2 test for all categorical variables and Wilcoxons test for age and ejection factor. Using the Mayo Clinic multivariable risk factor equation,4 we calculated predicted probabilities of death for each patient in the NCDR population. Patients with the same predicted mortality score were grouped together, and within each group, the observed mortality rate and 95% approximate binomial CI were calculated. The observed versus expected in-hospital mortality rates for these groups were plotted, and the calibration of the model was assessed with the Hosmer-Lemeshow method. The observed versus expected in-hospital mortality rates for these 23 groups were plotted, and the calibration of the model was assessed with the Hosmer-Lemeshow statistic with 23 df.
Model discrimination was assessed with the area under the receiver-operating characteristics curve, or c statistic, for the entire population and within prespecified patient subgroups. Approximate 95% CIs were constructed using variance estimates described by DeLong et al.6
Because the MCRS prediction equation did not calibrate as well as expected in the external NCDR population, we compared the regression coefficients from the MCRS and NCDR and refined the analysis to include recalibration of the equation using the NCDR population. Logistic regression was used to model mortality as a function of the MCRS to determine a new coefficient for an additional "recalibration" term in the prediction equation. To account for the nonlinear relationship between the Mayo Clinic integer risk score and mortality, both linear and quadratic terms were included in the new logistic model. Internal validation of this recalibrated quadratic MCRS equation was performed using additional NCDR data consisting of PCI procedures performed between May 22, 2004, and March 30, 2007 (primarily after April 1, 2006). Patients with the same predicted mortality score were again grouped together, and the observed versus expected in-hospital mortality rates were plotted. Discrimination was assessed once more with the c statistic for the entire population, and calibration was again confirmed with the Hosmer-Lemeshow test. For the NCDR population, we ran a logistic regression on mortality using the variables from the Mayo Clinic model, with continuous variables categorized according to the MCRS equation. The regression coefficients were then converted to integers by dividing by 0.37 and rounding. All statistical analyses were performed by the Duke Clinical Research Institute with SAS version 9.0.
| Results |
|---|
|
|
|---|
10, indicating that most PCI procedures were performed in patients at low to intermediate risk (Figure 2).
|
|
|
|
|
|
|
|
| Discussion |
|---|
|
|
|---|
Model Without Angiographic Variables
This study represents a significant advance over previous models that rely on angiographic variables known to predict outcomes after PCI.7–14 Even the recently published New York State model includes variables such as stent thrombosis and left main coronary artery disease, which can be obtained only after diagnostic coronary angiography.15 In our report, 7 clinical and laboratory variables can be summed to make a simple risk score that any healthcare provider can use at the time of initial contact with the patient to better comprehend the attendant risks from PCI. Because PCI procedures are often done in the same setting as diagnostic angiography, our model can allow an accurate assessment of periprocedural mortality before the initial angiogram at a time when the patient is not sedated and is in a better position to discuss the risks and benefits of treatment with the physician. Given that our model discriminated in-hospital mortality well despite not including angiographic variables, it may have an important role in informing patients of the risks of PCI should their coronary anatomy indicate that such a procedure is technically feasible and clinically indicated.
Role of Angiographic Variables in Risk Prediction
The need and utility of angiographic variables for risk prediction are not clear. We had previously reported our original Mayo Clinic risk adjustment model that had angiographic variables. That model was later validated in the National Heart, Lung, and Blood Institute Dynamic Registry.16 Even with the angiographic variables, the original Mayo Clinic risk model did not perform optimally at the extremes (high- and low-risk patients), with significant overprediction of procedural complications in patients with prior coronary artery bypass graft surgery. Underscoring the modest improvements of clinical risk stratification associated with angiographic data, Holmes et al7 forced pertinent angiographic variables (ACC/AHA type C lesion, calcification, bifurcation lesion) into the previous New York State model. Except for the ACC/AHA type C lesion, other angiographic variables did not significantly add to the prognostic information already obtained by the model. Most of the significant prognostic variables in that model were clinical and were similar to those of our new MCRS. To study the utility of the angiographic variables, we previously compared the ACC/AHA lesion classification with the original MCRS.17 The lesion classification did not predict procedural success and was inferior to the MCRS, further underscoring that the most relevant prognostic information can be gathered with patients demographic characteristics, presentation, and a few noninvasive laboratory tests. Angiographic variables, however, were determined to be useful in predicting elevation of creatine kinase-MB after a PCI procedure, an outcome of marginal clinical significance compared with mortality.18
Present Study
We recently constructed a practical pre-PCI risk model based solely on a baseline clinical and noninvasive assessment that could serve as a simple risk assessment aid for patients and physicians before coronary angiography is performed.4 We undertook that study with the specific aim of excluding variables that, thus far, have limited the application of published PCI risk models in the process of informed consent, when a prediction of risk before the angiogram is needed. The original MCRS risk model discriminated well but had only modest calibration when applied to the NCDR data set. To improve the calibration of the model across the entire risk spectrum of patients, we included an additional, simple set of recalibration terms. Our refined model still needs only easily obtained clinical information so that healthcare providers can use the model for gathering pertinent prognostic information with which to obtain higher-quality, evidence-based informed consent.
Information about risks from a revascularization procedure is critical in the planning stages, before the patient is sent to the cardiac catheterization laboratory. This assumes even greater relevance given the excellent long-term outcomes achieved by aggressive medical therapy and the need for patients to have a clearer understanding of the risks they face with the conduct of a PCI.19 The MCRS model initially tended to underpredict event rates when applied in NCDR data (observed 1.23% versus predicted 1.10%), but this underprediction was corrected after recalibration. It is difficult to point to any specific reasons for overprediction for low-risk PCI cases and underprediction of risk for higher-risk categories. In general, models have a tendency to predict a higher risk of death than is observed for the patients with the least severe illness and a lower risk of death than is observed for the most ill patients. First, there may be some unmeasured covariates that can modify the risk in the model. Second, we observed higher risk in the original model at a very young age and low serum creatinine that may partly explain the higher risk offered to low-risk patients. Third, lower risk assigned to higher-risk patients can be due to noninclusion of angiographic variables. Fourth, the effects of the continuous variables included in the MCRS were different in the NCDR (linear versus nonlinear effects). Differences in operator and institutional volumes and disparities in other processes of care, including volume, experience, and availability of onsite surgery, can be surrogates of quality of care and can potentially sway the results in favor or against our model. Ivanov et al20 emphasized the need for periodic recalibration of an existing model. Among 3 strategies to assess mortality after coronary artery bypass graft surgery, recalibration of the original model was recommended for risk prediction. The use of a ready-made model or development of a new model worked well for temporal benchmarking and in instances of profound disparities in the case mix and event rates. It is likely that a different case mix, exclusion of angiographic (left main/multivessel disease, thrombus) or procedural variables (exigency), or improved reportage of risk factors (upcoding, eg, for ejection fraction) have contributed to modest calibration of the model. Recently, Hubacek et al21 externally validated the Michigan mortality model in the Alberta Provincial Project for Outcome Assessment in Coronary Heart Disease (APPROACH). The original Michigan model underestimated the in-hospital mortality for the highest-risk patients with acute coronary syndrome undergoing PCI. With the addition of indication for the procedure (myocardial infarction) and ejection fraction, a significant improvement in the calibration of the extended Michigan model was observed. The new, recalibrated MCRS model is ready for application in routine clinical practice and should be explicitly tested to quantify the impact of accurate periprocedural mortality estimates on the quality of the informed consent process, on patients comfort with medical decision making, and on clinicians practice of PCI.
Study Limitations
Several features of the present study warrant consideration when our results are applied in clinical practice. First, validation of a prediction rule developed on a different database carries the risk that not all variables have similar definitions across data sources. As a result of differences in definitions across studies, it is possible that the calculated probabilities of procedural complication may be inflated in the registry. However, the NCDR is the largest national registry of patients undergoing PCI, and the data definitions used in this study have therefore become the de facto standard for classifying patients clinical status. Second, the ACC NCDR is a voluntary clinical quality assurance data repository with small and variable data audits and adjudication. This limitation is somewhat mitigated by the large sample size and the outcome of interest (in-hospital mortality) with minimal ascertainment bias. It is also possible that other variables such as operator volume could be significantly associated with adverse events after PCI and require further exploration in future studies.22,23 Finally, although the MCRS has high discrimination in predicting in-hospital mortality, no predictive model can ameliorate the effect of chance and unanticipated circumstances and complications inherently encountered in invasive treatments.
Conclusions
External validation of MCRS using the multicenter ACC NCDR confirms the broader potential applicability of this score to informed consent and medical decision making. The recalibrated MCRS had high discrimination for in-hospital mortality using 7 simple clinical and noninvasive variables. Most of the variables can be obtained at the time of first contact with the patient. Risk stratification may help the operator to individualize the risk of procedural death from PCI and to counsel patients at the time of PCI.
| Acknowledgments |
|---|
Drs Spertus and Peterson have a research contract with the American College of Cardiology.
Disclosures
None.
| References |
|---|
|
|
|---|
2. Hochman JS, Sleeper LA, Webb JG, Sanborn TA, White HD, Talley JD, Buller CE, Jacobs AK, Slater JN, Col J, McKinlay SM, LeJemtel TH. Early revascularization in acute myocardial infarction complicated by cardiogenic shock: SHOCK Investigators: Should We Emergently Revascularize Occluded Coronaries for Cardiogenic Shock? N Engl J Med. 1999; 341: 625–634.
3. Holmes DR Jr, White HD, Pieper KS, Ellis SG, Califf RM, Topol EJ. Effect of age on outcome with primary angioplasty versus thrombolysis. J Am Coll Cardiol. 1999; 33: 412–419.
4. Singh M, Rihal CS, Lennon RJ, Spertus J, Rumsfeld JS, Holmes DR. Bedside estimation of risk from percutaneous coronary intervention: the new Mayo Clinic Risk Scores. Mayo Clin Proc. 2007; 82: 701–708.
5. Shaw RE, Anderson HV, Brindis RG, Krone RJ, Klein LW, McKay CR, Block PC, Shaw LJ, Hewitt K, Weintraub WS. Development of a risk adjustment mortality model using the American College of Cardiology–National Cardiovascular Data Registry (ACC-NCDR) experience: 1998–2000. J Am Coll Cardiol. 2002; 39: 1104–1112.
6. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988; 44: 837–845.[CrossRef][Medline]
7. Holmes DR Jr, Berger PB, Garratt KN, Mathew V, Bell MR, Barsness GW, Higano ST, Grill DE, Hammes LN, Rihal CS. Application of the New York State PTCA mortality model in patients undergoing stent implantation. Circulation. 2000; 102: 517–522.
8. Kimmel SE, Berlin JA, Strom BL, Laskey WK. Development and validation of simplified predictive index for major complications in contemporary percutaneous transluminal coronary angioplasty practice: the Registry Committee of the Society for Cardiac Angiography and Interventions. J Am Coll Cardiol. 1995; 26: 931–938.[Abstract]
9. Moscucci M, Kline-Rogers E, Share D, O'Donnell M, Maxwell-Eward A, Meengs WL, Kraft P, DeFranco AC, Chambers JL, Patel K, McGinnity JG, Eagle KA. Simple bedside additive tool for prediction of in-hospital mortality after percutaneous coronary interventions. Circulation. 2001; 104: 263–268.
10. O'Connor GT, Malenka DJ, Quinton H, Robb JF, Kellett MA Jr, Shubrooks S, Bradley WA, Hearne MJ, Watkins MW, Wennberg DE, Hettleman B, O'Rourke DJ, McGrath PD, Ryan T Jr, VerLee P. Multivariate prediction of in-hospital mortality after percutaneous coronary interventions in 1994-1996: Northern New England Cardiovascular Disease Study Group. J Am Coll Cardiol. 1999; 34: 681–691.
11. Qureshi MA, Safian RD, Grines CL, Goldstein JA, Westveer DC, Glazier S, Balasubramanian M, O'Neill WW. Simplified scoring system for predicting mortality after percutaneous coronary intervention. J Am Coll Cardiol. 2003; 42: 1890–1895.
12. Resnic FS, Ohno-Machado L, Selwyn A, Simon DI, Popma JJ. Simplified risk score models accurately predict the risk of major in-hospital complications following percutaneous coronary intervention. Am J Cardiol. 2001; 88: 5–9.[Medline]
13. Rihal CS, Grill DE, Bell MR, Berger PB, Garratt KN, Holmes DR Jr. Prediction of death after percutaneous coronary interventional procedures. Am Heart J. 2000; 139: 1032–1038.[CrossRef][Medline]
14. Singh M, Lennon RJ, Holmes DR Jr, Bell MR, Rihal CS. Correlates of procedural complications and a simple integer risk score for percutaneous coronary intervention. J Am Coll Cardiol. 2002; 40: 387–393.
15. Wu C, Hannan EL, Walford G, Ambrose JA, Holmes DR Jr, King SB III, Clark LT, Katz S, Sharma S, Jones RH. A risk score to predict in-hospital mortality for percutaneous coronary interventions. J Am Coll Cardiol. 2006; 47: 654–660.
16. Singh M, Rihal CS, Selzer F, Kip KE, Detre K, Holmes DR. Validation of Mayo Clinic risk adjustment model for in-hospital complications after percutaneous coronary interventions, using the National Heart, Lung, and Blood Institute dynamic registry. J Am Coll Cardiol. 2003; 42: 1722–1728.
17. Singh M, Rihal CS, Lennon RJ, Garratt KN, Holmes DR Jr. Comparison of Mayo Clinic Risk Score and American College of Cardiology/American Heart Association lesion classification in the prediction of adverse cardiovascular outcome following percutaneous coronary interventions. J Am Coll Cardiol. 2004; 44: 357–361.
18. Singh M, Rihal CS, Lennon RJ, Garratt KN, Mathew V, Holmes DR Jr. Prediction of complications following nonemergency percutaneous coronary interventions. Am J Cardiol. 2005; 96: 907–912.[CrossRef][Medline]
19. Boden WE, O'Rourke RA, Teo KK, Hartigan PM, Maron DJ, Kostuk WJ, Knudtson M, Dada M, Casperson P, Harris CL, Chaitman BR, Shaw L, Gosselin G, Nawaz S, Title LM, Gau G, Blaustein AS, Booth DC, Bates ER, Spertus JA, Berman DS, Mancini GB, Weintraub WS. Optimal medical therapy with or without PCI for stable coronary disease. N Engl J Med. 2007; 356: 1503–1516.
20. Ivanov J, Tu JV, Naylor CD. Ready-made, recalibrated, or remodeled? Issues in the use of risk indexes for assessing mortality after coronary artery bypass graft surgery. Circulation. 1999; 99: 2098–2104.
21. Hubacek J, Galbraith PD, Gao M, Humphries K, Graham MM, Knudtson ML, Ghali WA. External validation of a percutaneous coronary intervention mortality prediction model in patients with acute coronary syndromes. Am Heart J. 2006; 151: 308–315.[CrossRef][Medline]
22. Malenka DJ, McGrath PD, Wennberg DE, Ryan TJ Jr, Kellett MA Jr, Shubrooks SJ Jr, Bradley WA, Hettlemen BD, Robb JF, Hearne MJ, Silver TM, Watkins MW, O'Meara JR, VerLee PN, O'Rourke DJ. The relationship between operator volume and outcomes after percutaneous coronary interventions in high volume hospitals in 1994-1996: the northern New England experience: Northern New England Cardiovascular Disease Study Group. J Am Coll Cardiol. 1999; 34: 1471–1480.
23. Hannan EL, Racz M, Ryan TJ, McCallister BD, Johnson LW, Arani DT, Guerci AD, Sosa J, Topol EJ. Coronary angioplasty volume-outcome relationships for hospitals and cardiologists. JAMA. 1997; 277: 892–898.
Related Article
This article has been cited by other articles:
CLINICAL PERSPECTIVE
The Mayo Clinic Risk Score model is based solely on preprocedural clinical and noninvasive assessments and is less reliant on subjective assessment or angiographic data. In this study, we confirm the validity of the Mayo Clinic Risk Score and show that this model can be used as a tool to inform patients of their estimated risk from percutaneous coronary intervention and can become an integral part of the informed consent process. External validation of Mayo Clinic Risk Score using the multicenter National Cardiovascular Data Registry confirms the broader potential applicability of this score to informed consent and medical decision making. The recalibrated Mayo Clinic Risk Score had high discrimination for in-hospital mortality using 7 simple clinical and noninvasive variables. Most of the variables can be obtained at the time of first contact with the patient. Risk stratification may help the operator to individualize the risk of procedural death from percutaneous coronary intervention and to counsel patients at the time of percutaneous coronary intervention.
Circ Cardiovasc Intervent 2008 1: 36-44.
![]() |
S. R. Dixon, C. L. Grines, and W. W. O'Neill The year in interventional cardiology. J. Am. Coll. Cardiol., June 2, 2009; 53(22): 2080 - 2097. [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Home | Subscriptions | Archives | Feedback | Authors | Help | Circulation Journals Home | AHA Journals Home | Search Copyright © 2008 American Heart Association, Inc. All rights reserved. Unauthorized use prohibited. |