ijcs International Journal of Cardiovascular Sciences Int J Cardiovasc Sci 2359-4802 2359-5647 Sociedade Brasileira de Cardiologia FC4dBtr6jR9bdRq6F3Qtgyh S2359-56472021005009209 S2359-56472022000200174 ijcs.20210019 10.36660/ijcs.20210019 ORIGINAL ARTICLE Validation of the Grace Risk Score to Predict In-Hospital and 6-Month Post-Discharge Mortality in Patients with Acute Coronary Syndrome 0000-0001-7216-3624 Neves Vítor Boniatti 1 0000-0002-9875-2176 Roman Raquel Melchior 2 0000-0001-7688-092X Vendruscolo Tiago 2 0000-0001-9844-3747 Heineck Gilberto 2 0000-0002-5712-9505 Mattos Carlos Alberto Santos de 2 0000-0001-8553-8384 Mattos Eduardo Ilha de 2 0000-0002-5839-8571 Bin Luiz Carlos Pereira 1 2 0000-0003-3130-8211 Boclin Karine de Lima Sírio 3 0000-0002-7016-2043 Roman Marcelo Fialho 1 2 Faculdade Meridional Passo Fundo RS Brazil Faculdade Meridional (IMED), Passo Fundo, RS – Brazil Hospital de Clínicas de Passo Fundo Passo Fundo RS Brazil Hospital de Clínicas de Passo Fundo (HCPF), Passo Fundo, RS – Brazil Universidade Estácio de Sá Rio de Janeiro RJ Brazil Universidade Estácio de Sá (UNESA), Rio de Janeiro, RJ – Brazil Mailing Address: Vítor Boniatti Neves Rua Senador Pinheiro, 304. Postal Code: 99070-220, Passo Fundo, RS – Brazil E-mail: vitor.vbn@gmail.com

Author contributions

Conception and design of the research: Neves VB, Roman MF, Boclin KSL. Acquisition of data: Neves VB, Roman MF, Vendruscolo T, Heineck G, Mattos CAS, Mattos EI, Bin LCP. Statistical analysis: Neves VB, Roman MF. Writing of the manuscript: Neves VB, Roman MF, Roman RM. Conception and design of the research: Neves VB, Roman MF, Boclin KSL.

Potential Conflict of Interest

No potential conflict of interest relevant to this article was reported.

 16 07 2021 Mar-Apr 2022 35 2 174 180 02 06 2020 19 01 2021 07 04 2021 This is an Open Access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Abstract Background

The wide range of clinical presentations of acute coronary syndrome (ACS) makes it indispensible to use tools for risk stratification and for appropriate risks management; thus, the use of prognosis scores is recommended in the immediat clinical decision-making.

 Objective

To validate the Global Registry of Acute Coronary Events (GRACE) score as a predictor of in-hospital and 6-month post-discharge mortality in a population diagnosed with ACS.

 Methods

This is a prospective cohort study of consecutive patients diagnosed with ACS between May and December 2018. GRACE scores were calculated, as well as their predictive value for in-hospital and 6-month post-discharge mortality. The validity of the model was assessed by two techniques: discriminative power using the area under the receiver operating characteristic curve (AUC) and goodness-of-fit, using the Hosmer-Lemeshow (HL) test, at the 5% level of significance.

 Results

A total of 160 patients were included, mean age 64 (±10.9) years; of which 60% were men. The risk model showed to have satisfactory ability to predict both in-hospital mortality, with an area under the curve (AUC) of 0.76 (95% confidence interval [CI], 0.57-0.95; p = 0.014), and 6-month post-discharge mortality, with AUC of 0.78 (95%CI, 0.62-0.94), p = 0.002. The HL test indicated good-fit for both models of the GRACE score.

 Conclusion

In this study, the GRACE risk score for predicting mortality was appropriately validated in patients with ACS, with good discriminative power and goodness-of-fit. The results suggest that the GRACE score is appropriate for clinical use in our setting.

 Acute Coronary Syndrome Mortality Prognosis Introduction

Cardiovascular disease is the leading cause of mortality in Brazil and worldwide, and ischemic heart disease accounts for a large portion of this concerning scenario.1 Among its forms of presentation, acute coronary syndrome (ACS) has a wide range of severity.2 However, the use of validated mathematical models of clinical prediction is essential and recommended in national and international guidelines for the management of patients with ACS.3,4

With this stratification, high-risk patients may receive more aggressive antiplatelet and antithrombotic therapy and early invasive intervention, whereas lower-risk patients may receive less aggressive treatments.3,5

Based on the Global Registry of Acute Coronary Events (GRACE) report, the GRACE score was designed with 8 variables analyzed on patient's admission,6 5 semi-quantitative ones (age range, heart rate, systolic blood pressure, plasma creatinine, and Killip class) and 3 dichotomic ones: cardiac arrest at admission, ST-segment deviation, and elevation of cardiac markers. The final score can range from 0 to 372.6

Therefore, the present study aimed to assess the value of the GRACE score as a predictor of in-hospital prognosis and 6-month post-discharge prognosis in patients with ACS in our setting.

 Methods Sample Selection

All individuals admitted with a diagnosis of ACS at Hospital de Clínicas de Passo Fundo (HCPF), Brazil, from May to December 2018 were selected. Demographic, clinical, and angiographic variables were prospectively collected. The patients were treated according to the criteria of the attending physician, without intervention from researchers. The study was approved by the Research Ethics Committee of Faculdade IMED, in compliance with the Resolution 466/2012 of the National Health Council.

Inclusion criteria were: age 18 years or older, symptoms suggestive of acute coronary ischemia on admission, and presence of at least one of the following characteristics: changes suggestive of ACS on electrocardiogram (ECG), elevation of serum biochemical markers of myocardial necrosis, and/or documented previous coronary artery disease with angiography showing coronary obstruction ≥ 50%. Patients whose ACS was triggered by secondary factors, such as trauma or surgery, were excluded. No patient refused to participate in the study, and all of them provided informed consent to participate in the research. Hospitalization outcomes were obtained by phone interview or outpatient visit 6 months after hospital discharge.

 GRACE Score

The GRACE score was published in 2004 based on the GRACE registry, which was designed to reflect the full of patients with ACS. Data were obtained in 14 countries (Europe, North and South America, Australia, and New Zealand), including 94 hospitals, of which 6 were Brazilian, with a total population of 17142 patients. The aim of the score was to develop a tool to estimate of probability risk of 6-month mortality or myocardial infarction so as to facilitate the stratification of patients with ACS.

Based on the GRACE score, patients were classified into low (< 1%), intermediate (1 to 3%), and high risk (> 3%) for in-hospital mortality. For the 6-month post-discharge prognosis, patients were divided into those with low (< 3%), intermediate (3 to 8%), and high mortality risk (> 8%).3,5

 Clinical outcomes

The primary outcome was defined as in-hospital and 6-month post-discharge mortality. With regard to secondary outcomes, the accuracy of the GRACE score was assessed in the different presentations of ACS.

 Statistical analysis

Categorical data were presented as frequencies (percentages), and continuous variables were presented as mean and standard deviation (SD) or median and interquartile range (IQR). The Kolmogorov-Smirnov test was used to verify of distribution. The level of significance was set at p < 0.05 for all analyses. The discriminative power of the score with regard to in-hospital and 6-month post-discharge was assessed using the C statistics. The area under the receiver operating characteristic curve (AUC) represented the accuracy of the GRACE score in distinguishing survivors from non-survivors. Along with this analysis, cutoff values were identified to define the best prognostic sensitivity and specificity, with their 95% confidence intervals (CI). Goodness-of-fit for the scores was assessed by the Hosmer-Lemeshow test and by the dispersion graph between predicted mortality at each risk decile and the observed mortality. The analysis was performed using the SPSS 20.0, Minitab 16 and MedCalc, version 19.1 software.

 Results Sample characteristics

The sample consisted of 160 patients. Two patients (1.25%) were lost to follow-up, due to absence of outpatient follow-up and telephone contact failure. Demographic and clinical characteristics with regard to the prevalence of cardiovascular risk factors and initial presentation are presented in Table 1.

– Sample characteristics
Variables Distribution
Sample 160
Age (years) 64 (10.9)
Male gender 96 (60%)
BMI (kg/m2) 28.2 (5.1)
Treatment on SUS 142 (88.8%)
Diabetes 112 (70%)
Hypertension 121 (75.6%)
Sedentary 66 (41.25%)
Smoking 50 (31.25%)
Systolic blood pressure (mm Hg) 138 (29.1)
Heart rate (bpm) 78.7 (20.7)
Killip class > 1 16 (10%)
ST-segment depression 43 (26.9%)
Creatinine (mg/L) 1.07 (0.94-1.25)
Positive HS troponin 100 (62.5%)
ACS UA NSTEMI STEMI 60 (37.5%) 58 (36.25) 42 (26.25%)
GRACE score - Intra-hospital 111.5 (94.3-139.5)
GRACE score - 6 months after discharge 95.5 (80.5-117)

Creatinine and GRACE scores were expressed in median and interquartile range. The remaining (continuous) variables were expressed as mean (SD).

BMI: body mass index; SUS: Brazilian Unified Health System (Sistema Único de Saúde); HS: high sensitivity; ACS: acute coronary syndrome; UA: unstable angina; NSTEMI: non-ST segment elevation myocardial infarction; STEMI: ST segment elevation myocardial infarction; and GRACE: Global Registry of Acute Coronary Events.

Primary outcome

In-hospital mortality was 5.1% (8 deaths). Six hospital deaths were caused by cardiogenic shock, and 2 by infectious complications with septic shock. The Hosmer-Lemeshow test for the in-hospital GRACE score yielded a c2 of 7.14 (p = 0.522) and an AUC of 0.76 (95% confidence interval (CI), 0.57-0.95). Six-month post-discharge mortality was 7% (11 deaths). Among the patients who died after hospital discharge, 2 had sudden death, and 1 had a new episode of MI. The results for the 6-month post-discharge GRACE score showed c2 of 4.53 (p = 0.81) and AUC of 0.78 (95%CI, 0.62-0.94). Therefore, both predictions exhibited a good-fit (Figure 1).

– ROC curves for in-hospital and 6-month post-discharge GRACE scores. AUC: Area under the receiver operating characteristic curve (C statistics).

According to the ROC curve, the best cutoff value for the in-hospital GRACE score was 179, with sensitivity of 50% and specificity of 98%. Conversely, the best cutoff value for the 6-month post-discharge GRACE score was 119.5, with sensitivity of 72.7% and specificity of 81.6% (Table 2).

– Predictive accuracy of in-hospital and 6-month post-discharge GRACE scores
Area (AUC) Specificity Sensitivity p-value
In-hospital GRACE score 0.76 (0.57-0.95) 98% 50% 0.014
6-month post-discharge GRACE score 0.78 (0.62-0.94) 81.6% 72.7% 0.002

AUC: area under the receiver operating characteristic curve.

Secondary outcome

The accuracy of the GRACE score in the different forms presentations of ACS was also tested. There was no outcome UA to be analyzed.

With regard to the non-ST segment elevation myocardial infarction (NSTEMI), mortality rate was 5.3%, all of which occurred during hospitalization. The in-hospital GRACE score had a c2 of 5.96 (p = 0.425) and an AUC of 0.64. The cutoff value was 121.5, with sensitivity of 66.7% and specificity of 74.1%. Conversely, the GRACE score 6 months after discharge had a c2 of 5.6 (p = 0.102) and an AUC of 0.59. The cutoff value was 98.5, with sensitivity of 66.7% and specificity of 63% (Figure 2 and Table 3).

– Receiver operating characteristic (ROC) curves for in-hospital and 6-month post-discharge GRACE. AUC: area under ROC curve (C statistics). NSTEMI: non-ST segment myocardial infarction; STEMI: ST segment elevation myocardial infarction.

– Predictive accuracy of in-hospital and 6-month post-discharge GRACE scores in NSTEMI and STEMI subgroups
Area Sensitivity (%) Specificity (%) p-value
In-hospital NSTEMI 0.64 (0.29-0.98) 66.7 74.1 0.432
STEMI 0.78 (0.49-1) 80 91.9 0.043
6-month post-discharge NSTEMI 0.59 (0.25-0.93) 66.7 63 0.592
STEMI 0.77 (0.57-0.98) 62.5 88.2 0.018

Area under the receiver operating characteristic curve for in-hospital and 6-month post-discharge GRACE scores in the NSTEMI and STEMI subgroups, with sensitivity and specificity for their cutoff values.

NSTEMI: Non ST-segment elevation myocardial infarction; STEMI: ST-segment elevation myocardial infarction.

In the ST segment elevation myocardial infarction (STEMI), which had a mortality rate of 11.9%, the in-hospital GRACE score had c2 of 8.8 (p = 0.359) and an AUC of 0.78. The cutoff value was 179, with sensitivity of 80% and specificity of 91.9%. Conversely, the GRACE score 6 months after discharge, when cumulative mortality was 19%, had a c2 of 7.99 (p = 0.435) and an AUC of 0.77. The cutoff value was 135, with sensitivity of 62.5% and specificity of 88.2% (Figure 2 and Table 3).

The hospital mortality rates for patients with predicted low, intermediate, and high risk according to the in-hospital GRACE score were 2.8% (2 deaths), 2% (1 death), and 13.9% (5 deaths) respectively. Conversely, for patients with predicted low, intermediate, and high risk according to the 6-month post-discharge GRACE score was 3% (3 deaths), 0% (0 death) and 22% (8 deaths), respectively (Figure 3 and Table 4).

– Comparison of mortality rates according to risk stratification by in-hospital and 6-month post-discharge GRACE scores.

– Number of deaths according to risk classification
Death Risk
Low Intermediate High
In-hospital 2 (2.8%) 1 (2%) 5 (13.9%)
6-month post-discharge 3 (5%) 0 (0%) 8 (22.8%)

Discussion

The use of score risk for stratification and prognostic is recommended in the clinical practice by the national and international guidelines on NSTEMI and STEMI.3,5 The GRACE score includes quantitative and qualitative variables and has greater discriminative accuracy than other prognostic tools, such as the TIMI risk.3

In our validation study, the GRACE score showed a satisfactory discriminative power. The AUC was 0.76 for the in-hospital GRACE score and 0.78 for the 6-month post-discharge GRACE score. In the classical study of Eagle et al. with 17.142 patients, this score had a discriminative value of 0.81 in patients with ACS.6

With regard to the secondary outcome, the subgroup of patients with STEMI showed an in-hospital AUC of 0.78. Two Brazilian studies were conducted with patients with STEMI, one by Correia et al., who reported an AUC of 0.867 in a sample of 152 patients, and another by Sola et al., who shown an AUC of 0.803 in a cohort of 169 individuals from Salvador, state of Bahia.7,8 In these national studies, it was not possible to compare 6-month post-discharge outcomes, because they were not assessed.

International studies, such as that conducted by Bargos et al. with an Argentinean cohort, found results similar to those of our study for the in-hospital GRACE score, with an AUC of 0.76.9 Furthermore, a similar AUC (0.6) for in-hospital mortality was observed em patients with NSTEMI.

The Spanish study by Abu-Ass et al. validated the 6-month post-discharge GRACE score with an AUC of 0.861.10

In our setting, in-hospital mortality was 5.1%. However, it was 2.8%, 2% and 13.9% for patients with predicted low, moderate, and high mortality risk, respectively. In the Spanish study with 6997 participants conducted by Cordero et al., mortality rate was 5.33% and 0%, 0.6% and 9.6%, respectively.11 Even with a smaller sample, our study found values similar to those obtained in the Spanish study.

The limitation of our study is the fact that there was no exploratory analysis either of the factors related to mortality in our sample or of the impact of the prescribed pharmacological and interventional treatments.

It is the first Brazilian study that showed the validity of the GRACE score beyond in-hospital prognosis.

 Conclusion

The GRACE score was validated to predict in-hospital and 6-month post-discharge mortality in our setting in a non-selected sample of patients with ACS. The discriminative power of the score was found to be satisfactory, ratifying recent guidelines that recommend using the GRACE score in risk stratification and selection of intensive early treatment strategies, as well as in the watchful post-discharge follow-up.

 References 1. World Health Organization. World health statistics 2018: monitoring health for the SDGs, sustainable development goals. Geneva: World Health Organization; 2018. World Health Organization World health statistics 2018: monitoring health for the SDGs, sustainable development goals Geneva World Health Organization 2018 2. Hammami R, Jdidi J, Mroua F, Kallel R, Hentati M, Abid L, et al. Accuracy of the TIMI and GRACE scores in predicting coronary disease in patients with non-ST-elevation acute coronary syndrome. Rev Port Cardiol. 2018;37(1):41-49. doi: 10.1016/j.repc.2017.05.012. Hammami R Jdidi J Mroua F Kallel R Hentati M Abid L et al Accuracy of the TIMI and GRACE scores in predicting coronary disease in patients with non-ST-elevation acute coronary syndrome Rev Port Cardiol 2018 37 1 41 49 10.1016/j.repc.2017.05.012. 3. Ibanez B, James S, Agewall S, Antunes MJ, Bucciarelli-Ducci C, Bueno H, et al. 2017 ESC Guidelines for the management of acute myocardial infarction in patients presenting with ST-segment elevation: The Task Force for the management of acute myocardial infarction in patients presenting with ST-segment elevation of the European Society of Cardiology (ESC). Eur Heart J. 2018;39(2):119-177. doi: 10.1093/eurheartj/ehx3937. Ibanez B James S Agewall S Antunes MJ Bucciarelli-Ducci C Bueno H et al 2017 ESC Guidelines for the management of acute myocardial infarction in patients presenting with ST-segment elevation: The Task Force for the management of acute myocardial infarction in patients presenting with ST-segment elevation of the European Society of Cardiology (ESC) Eur Heart J 2018 39 2 119 177 10.1093/eurheartj/ehx3937. 4. Nicolau JC, Timerman A, Marin-Neto JA, Piegas LS, Barbosa CJ, Franci A, et al. Guidelines of Sociedade Brasileira de Cardiologia for Unstable Angina and Non-ST-Segment Elevation Myocardial Infarction (II Edition, 2007) 2013-2014 Update. Arq Bras Cardiol. 2014;102(3 Suppl 1):1-61. doi: 10.5935/abc.2014S001. Nicolau JC Timerman A Marin-Neto JA Piegas LS Barbosa CJ Franci A et al Guidelines of Sociedade Brasileira de Cardiologia for Unstable Angina and Non-ST-Segment Elevation Myocardial Infarction (II Edition, 2007) 2013-2014 Update Arq Bras Cardiol 2014 102 3 Suppl 1 1 61 10.5935/abc.2014S001. 5. Collet JP, Thiele H, Barbato E, Barthélémy O, Bauersachs J, Bhatt DL, et al. 2020 ESC Guidelines for the management of acute coronary syndromes in patients presenting without persistent ST-segment elevation. Eur Heart J. 2021;42(14):1289-1367. doi: 10.1093/eurheartj/ehaa575. Collet JP Thiele H Barbato E Barthélémy O Bauersachs J Bhatt DL et al 2020 ESC Guidelines for the management of acute coronary syndromes in patients presenting without persistent ST-segment elevation Eur Heart J 2021 42 14 1289 1367 10.1093/eurheartj/ehaa575. 6. Eagle KA, Lim MJ, Dabbous OH, Pieper KS, Goldberg RJ, Van de Werf F, et al. A validated prediction model for all forms of acute coronary syndrome: estimating the risk of 6-month postdischarge death in an international registry. JAMA. 2004;291(22):2727-33. doi: 10.1001/jama.291.22.2727. Eagle KA Lim MJ Dabbous OH Pieper KS Goldberg RJ Van de Werf F et al A validated prediction model for all forms of acute coronary syndrome: estimating the risk of 6-month postdischarge death in an international registry JAMA 2004 291 22 2727 2733 10.1001/jama.291.22.2727. 7. Correia LC, Garcia G, Kalil F, Ferreira F, Carvalhal M, Oliveira R, et al. Prognostic value of TIMI score versus GRACE score in ST-segment elevation myocardial infarction. Arq Bras Cardiol. 2014;103(2):98-106. doi: 10.5935/abc.20140095. Correia LC Garcia G Kalil F Ferreira F Carvalhal M Oliveira R et al Prognostic value of TIMI score versus GRACE score in ST-segment elevation myocardial infarction Arq Bras Cardiol 2014 103 2 98 106 10.5935/abc.20140095. 8. Solla DJF, Novais VON, Carvalho LGMC, Camara SFC, Teixeira LST, Correa LBC, et al. Evaluation of the prognostic value of GRACE score in STEMI in Salvador, Bahia, Brazil: Findings from RESISST. Resuscitation. 2012;83:e65. doi: 10.1016/j.resuscitation.2012.08.166. Solla DJF Novais VON Carvalho LGMC Camara SFC Teixeira LST Correa LBC et al Evaluation of the prognostic value of GRACE score in STEMI in Salvador, Bahia, Brazil: Findings from RESISST Resuscitation 2012 83 e65 10.1016/j.resuscitation.2012.08.166. 9. Burgos ML, Costabel JP, Cigalini I, Godoy CL, Zamora SG, Giordanino EF, et al. Validation of the GRACE score (global registry of acute coronary events) as a predictor of in-hospital mortality in patients with acute myocardial infarction in Argentina. Rev Fed Argent Cardiol. 2017;46(4):228-32. Burgos ML Costabel JP Cigalini I Godoy CL Zamora SG Giordanino EF et al Validation of the GRACE score (global registry of acute coronary events) as a predictor of in-hospital mortality in patients with acute myocardial infarction in Argentina Rev Fed Argent Cardiol 2017 46 4 228 232 10. Abu-Assi E, García-Acuña JM, Peña-Gil C, González-Juanatey JR. Validation of the GRACE risk score for predicting death within 6 months of follow-up in a contemporary cohort of patients with acute coronary syndrome. Rev Esp Cardiol. 2010;63(6):640-8. doi: 10.1016/s1885-5857(10)70138-9. Abu-Assi E García-Acuña JM Peña-Gil C González-Juanatey JR Validation of the GRACE risk score for predicting death within 6 months of follow-up in a contemporary cohort of patients with acute coronary syndrome Rev Esp Cardiol 2010 63 6 640 648 10.1016/s1885-5857(10)70138-9. 11. Cordero A, Rodriguez-Manero M, García-Acuña JM, López-Palop R, Cid B, Carrillo P, et al. Additive value of the CRUSADE score to the GRACE score for mortality risk prediction in patients with acute coronary syndromes. Int J Cardiol. 2017;245:1-5. doi: 10.1016/j.ijcard.2017.07.095. Cordero A Rodriguez-Manero M García-Acuña JM López-Palop R Cid B Carrillo P et al Additive value of the CRUSADE score to the GRACE score for mortality risk prediction in patients with acute coronary syndromes Int J Cardiol 2017 245 1 5 10.1016/j.ijcard.2017.07.095.

Study Association

This study is not associated with any thesis or dissertation work.

Ethics approval and consent to participate

This study was approved by the Ethics Committee of the Faculdade Meridional – IMED under the protocol number 2.531.453. All the procedures in this study were in accordance with the 1975 Helsinki Declaration, updated in 2013. Informed consent was obtained from all participants included in the study.

Sources of Funding: There were no external funding sources for this study.