| Abstract |
Background: Managing critical illness requires quick and accurate identification of the critically ill patients. Several prognostic models (severity scales and scores) have been proposed for this aim – three of the most widely known and broadly applied include the Acute Physiology and Chronic Health Disease Classification System II (APACHE II), the Sequential Organ Failure Assessment (SOFA), and the Simplified Acute Physiology Score II (SAPS II). These prognostic models use clinical and laboratory variables to predict in-hospital mortality and their output is an estimated probability of in-hospital death, which is regarded as an estimate of disease severity. In critical care, such models are mostly utilized for risk stratification (benchmarking) when comparing outcomes and care across settings, for example between hospitals or of one hospital over time. However, these models are lacking wide external validation of predictive accuracy in broad cohorts of critically ill patients, including the setting of the mixed adult Intensive Care Unit (ICU) in Greece. To be clinically useful, a prognostic model should have good predictive performance (discrimination and calibration) when tested in the setting where it will be used and demonstrate usefulness for decision-making.
Objective: This study aimed to perform external validation involving head-to-head comparisons of the predictive performance of APACHE II, SOFA and SAPS II for intra-ICU and in-hospital mortality in an independent cohort of critically ill patients.
Methods: We conducted a retrospective cohort study from March 2018 to February 2020, in a 12-bed mixed medical-surgical ICU at a University Hospital in Greece. To assess the effectiveness of SAPS II, APACHE II, and SOFA to predict intra-ICU and in-hospital mortality we examined discrimination ability through ROC curve analysis, and calibration between predicted probabilities and observed mortality rates via graphical methods and the Hosmer-Lemeshow goodness-of-fit test. Moreover, we examined whether these models may aid decision-making by means of Decision Curve Analysis.
Results: We analyzed data from 778 patients (median age 68 years, 64% male). Mean SAPS II, APACHE II, and SOFA scores were 40.9 ± 13.0, 23.6 ± 7.4, and 9.9 ± 3.5 respectively. Intra-ICU mortality was 21.9% (95% confidence interval [CI] 19.0 – 24.9%) and in-hospital mortality was 34.7% (95%CU 31.4 - 38.2%). For intra-ICU mortality prediction, index scores were found to discriminate acceptably and equally well, with areas under ROC curves (AUC) of 0.75 (95%CI 0.71-0.80), 0.72 (95%CI 0.67-0.76) and 0.71 (95%CI 0.67-0.76) for SAPS II, APACHE II, and SOFA respectively (p = 0.204). SAPS II displayed excellent calibration across the full range of probabilities for intra-ICU death, but APACHE II and SOFA had poor calibration in high-risk patients (the former over-estimated and latter under-estimated risks exceeding 40%). For in-hospital mortality prediction, SAPS II and APACHE II had significantly better discrimination ability than SOFA (AUCs 0.77 (95%CI 0.73-0.80), 0.76 (95%CI 0.72-0.79), and 0.70 (95%CI 0.66-0.74) respectively; p = 0.002), while calibration between observed and predicted probabilities was good for all three scores. Decision curve analysis showed that SAPS II has increased net clinical benefit across a wide range of risk thresholds (from about 15% to 80%) for intra-ICU and in-hospital mortality. APACHE II displayed lower net benefit than SAPS II for intra-ICU mortality but comparable net benefit for in-hospital mortality. SOFA had lower net benefit than SAPS II and APACHE II for both intra-ICU and in-hospital mortality, but nevertheless higher than alternative strategies such as using patient’s age for decision making.
Conclusion: In this study, we observed that SAPS II, APACHE II, and SOFA prognostic scores are adequate tools for intra-ICU and in-hospital mortality prediction in a cohort of patients from a mixed adult ICU in Greece. The SAPS II model was the best performing model for both intra-ICU and in-hospital mortality, but APACHE II could also be helpful for identifying patients at risk of dying in the hospital. All scores, especially SAPS II, exhibited potential to be used as ancillary tools to support decision-making by physicians and families regarding the level of therapeutic management and palliative care.
|
Search
