The objective of this study is to identify the risk factors that influence the surgical and medical Intensive Care Unit (ICU) mortality. We considered data that was collected at Bay State Medical Center in Springfield, Massachusetts.¹ We developed statistical models that identify the risk factors associate with ICU mortality. In order to identify the risk factors without subjective bias, we explored multiple variable selection methods. We explored several methods including what we call manually picked best model, forward selection, backward elimination, and the least absolute shrinkage and selection operator (LASSO). We applied 5-fold cross validation on the final model of manually picked best model, forward selection and backward elimination and applied both validation set approach and 5-fold cross validation on LASSO to create confusion matrices and calculate the error rate of each method. Finally, we recommended the model for predicting ICU mortality with lowest misclassification error rate.

The care and treatment of critically ill patients in special rooms with life-saving technology is the major component of modern medical science. The diagnosis and treatment of the patients in critical conditions is highly dependent on invasive diagnostic as well as therapeutic procedures. However, the main disruption of host defense mechanisms comes from the life support systems.

According to Morandi, Jackson, and Wesley,² there are ICU-acquired infections that is responsible for the high mortality rate of the ICU patients. The researchers’ study offers a useful information concerning the topic. The aim of their study was to determine the epidemiology as well as the risk factors for nosocomial infections and the mortality rate in the ICU. Due to the variations of the study methods, the infection rates from different ICUs are difficult to compare.

Girard, Pandharipande, & Ely³ show that delirium, a fluctuating disturbance of cognition, is a sign of acute brain dysfunction in the patients with critical illnesses in the ICU. The patients with critical illnesses are more likely to have delirium. Also, Morandi, Jackson, & Wesley Ely² show that delirium can be related to the cognitive impairment that persists for an extended period after discharge. Techniques of treating delirium at the ICU has been the subject of investigations in the recent past.

Age is also suggested as one of the predictors of mortality in the ICU. The number of elderly patients who are being admitted to the ICU has been increased, not only in the USA but also internationally (Belayachi et al.⁴). There are few studies that have been conducted to link old age with the ICU mortality rate. The current research includes age as one of the risk factors for ICU mortality.

The ICU mortality rate in any hospital is the highest compared to other units. The United States of America has an approximate 4 million ICU admissions annually and the mortality rate of 500,000 deaths every year. The medical errors occur in any unit of the hospital, but it more likely to occur in the ICU since the ICU patients undergo complex interventions.

A study on the mortality risk factors and validation of severity scoring systems in the critically ill patients with acute renal failure was conducted to identify the determinants for improving patient care. Renal failure has a high prevalence in the ICU and associated with high mortality rates. Identification of the mortality risk factors helps to address intervention to these risk factors and improves patient care (Lima, Zanetta, Castro & Yu).⁵

Iwuafor, et al.⁶ conducted a study sought to determine the prevalence, risk factors, clinical outcome, and the microbiological profile of the hospital-acquired infections in the ICU of a Nigerian hospital. Infections commonly affect critically ill patients and have a high association with mortality. The study identified blood stream infections and the urinary tract infections as a significant risk factors associated with the ICU mortality.

We considered data collected at Bay State Medical Center in Springfield, Massachusetts that can be downloaded from University of Massachusetts, Amherst website.¹ The dataset consists of 200 observations with 20 variables. The response variable (STA), vital status is categorical. The other categorical predictor variables are: Gender, Race, SER (Service at ICU Admission), CAN (Cancer Part of Present Problem), CRN (History of Chronic Renal Failure), INF (Infection Probable at ICU Admission), CPR (CPR Prior to ICU Admission), PRE (Previous Admission to an ICU within 6 Months), TYP (type of admission), FRA (Long Bone, Multiple, Neck, Single Area, or Hip Fracture), PO2 (PO2 from Initial Blood Gases), PH (PH from Initial Blood Gases), PCO (PCO2 from initial Blood Gases), BIC (Bicarbonate from Initial Blood Gases), CRE (Creatinine from Initial Blood Gases) and LOC (Level of Consciousness at ICU Admission). The continuous predictor variables are Age, SYS (Systolic Blood Pressure at ICU Admission) and HRA (Heart Rate at ICU Admission).

Table 1 shows the total number of ICU patients according to vital status. It shows 80% of patients survived and 20% died. Figure 1 shows the graphical representation of these statistics.

Figure 1 Number of ICU patients according to vital status.

The table above shows that, 24 males died (12%) out of 124 total males and 16 females died (8%) out of 76 females at the ICU. From these results, it clearly demonstrates that males are more vulnerable to the ICU mortality as compared to females. The study also sought to determine whether ethnicity is associated with mortality. From the table above, 37 (18.5%) of 175 Whites died, 1 (0.5%) out of 15 Blacks died, and 2 (1%) out of 10 others died. The survival rate for the black patients and other races was less (7% and 4% respectively) compared to that of the white patients (69%). However, there wasn’t sufficient data in the Black and Other category to make decisive comparison. We can see 46.5% of patients were medically treated at ICU compared to 53.5% of patients who were treated surgically. There was no incidence of cancer in 90% of ICU patients. Out of those, 18% died. In addition, 2% of patients who had cancer died. The above results demonstrate that the cancer is a low predictor of ICU mortality for the critically ill patients admitted to the ICU. According to these results, the presence of cancer can lead to the survival of the patient in the ICU. We notice that only 9.5% of patients had chronic renal failure compared to 95.5% who did not have that failure. The results above show that the history of chronic renal disease is not a risk factor for mortality of the critically ill patients admitted to the ICU. For the patients with a history of chronic renal failure, the mortality was 4%, which is four times lower compared to those who did not have a history of chronic renal failure which was 16%. We notice that 12% of patients who had infection at ICU admission died compared to 8% without infection who died. Infection at ICU admission is a useful predictor of mortality in the intensive care unit with those infected having a higher likelihood of death compared to the uninfected ones. Also 6.5% of ICU patients had CPR prior to ICU admission. The percentage of patients with CPR prior to the ICU admission who died is low compared to the percentage of patients without CPR. It can be seen 15% of patients had previous admission to an ICU within 6 months. The previous admission to the intensive care unit is not a predictive factor for the mortality rate in the ICU. The percentage of people with previous admissions to the ICU who died is low compared to those who had not been admitted before. We can see that there are more deaths associated with emergency admission compared to the elective admission. From the above results, we can see that the mortality rate is 19% for the emergency admission as compared to the 1% of elective admission. The only 15 patients had fracture and 3 of them died. In contrast 37 patients out of 185 who did not have fracture died. The majority of patients at ICU had PO2 from initial blood gases greater than 60. The mortality rate for patients whose PO2 is greater than 60 is 17.5%. Patients whose initial blood gases’ PH was higher than 7.25 showed higher mortality rate that was twice that of the patients whose initial blood gases’ PH was below 7.25. In addition, the mortality rate for patients whose PCO2 from initial blood gases less than 45 is higher than those whose PCO2 is greater than 45. The mortality rate of patients whose Bicarbonate from initial blood gases was greater than 18 is seven times the mortality rate of patients whose Bicarbonate from initial blood gases was less than 18. It can be seen the ICU patients whose creatinine level from initial blood gases was greater than 2.0 have a higher mortality rate. The results above show that the mortality rate of ICU patients who had deep stupor or coma was high. Patients who had no coma survived with a probability of more than 85%.

The results in Table 2 demonstrate that age plays a significant role in the admission of patients to the ICU.

Table 3 and Figure 2 show that patients’ age is between 16 – 92 years old and the majority of ICU patients are between 40 – 80 years old. In addition, the boxplot shows the mean age of died patients is 70.

Figure 2 Age and vital status.

Figure 3 shows that the majority of patients were at risk who had systolic blood pressure between 120 to 140. The mean of systolic blood pressure of the patients at ICU is 132.3 mmHg. The boxplot shows a patient Figure 4 who had 256 mmHg died.

Figure 3 Systolic blood pressure and vital status.

Figure 4 Heart rate at ICU admission and vital status.

Since the response variable in our data (vital status) is binary, binary logistic regression is an appropriate model to consider. Logistic regression is a predictive analysis technique used to illustrate the relationship between a binary response variable and the predictors using the regression on the logarithm of the odds of having a response:^7,8

Let $Y_i$ be the binary response variable for $i^{th}$ patient, with
$Y_i=0$ , the patient survived
$Y_i=1$ , the patient died
then, logit $\left({\pi }_i\right)= \log \left(\frac{{\pi }_i}{1- {\pi }_i}\right)$
${\beta }_0+{\beta }_1{x}_{i1}+\dots \dots \dots +{\beta }_k{x}_{ik}$
$\approx \mathcal{X}\beta$
Where,
${\pi }_i=\frac{e^{\mathcal{X}\beta }}{1+ e^{\mathcal{X}\beta }}$

We consider multiple variable selection methods in order to identify risk factors in an objective manner. These are manually picked best model, forward selection, backward elimination and least absolute shrinkage and selection operator (LASSO). We discuss these methods briefly as follows.

Manually picked best model

In order to identify the risk factors for the vital status at ICU, which is a binary response variable, we started with the process of manually picked best model. To correctly leverage this method, we begin with binary logistic regression model. First, we fit a model with all predictor variables. Next we pick the significant variables (in this case risk factors) with the smallest P-value (P <.10) manually and remove all the factors which are insignificant. We then refit the model with all the significant variables in the model.

Forward selection

For forward selection, a null model (a model with no predictors), serves as the starting point. We add one variable at a time to the null model and refitted the model including the added variables. The idea was to keep it if the variable that had been added was significant and then add the next variable. If not, we eliminated it and added the next variable. We refitted the model using the same procedure until the stopping rule was satisfied (all the variables in the model are significant).

Backward selection

Backward elimination method starts with a full model that contains all the predictors in the model. The least significant risk factors; that is, the ones having the largest P value (greater than 10%) are eliminated, and the model is then refitted. Each step removes the least significant variable from the model until the remaining variables have their P values smaller than the specified 0.10.

Least Absolute Shrinkage and Selection Operator (LASSO)

With this method, there is an automatic selection of predictors of the target variable from the large set of potential predictors. By doing so, the method will return the coefficients of the irrelevant variables to zero thereby performing an automatic selection of variables. The LASSO formulates a curve fitting as a quadratic programming problem with the objective function that penalizes the absolute size of the coefficients based on a value of a tuning parameter, say $\lambda$ . The method, therefore, shrinks the size of the nonzero coefficients and ends up with the most useful variables.

 logit $\left({\pi }_i\right)=log\left( \frac{{\pi }_i}{1- {\pi }_i}\right)$
${\beta }_0+{\beta }_1{x}_{i1}+\dots \dots \dots +{\beta }_k{x}_{ik}+LASSOpenalty$
$\approx \mathcal{X}\beta +LASSOpenalty$
$LASSOpenalty=\lambda {\displaystyle \sum }_{j=1}^p\left|{\beta }_j\right|$
Where,
${\pi }_i=\frac{e^{\mathcal{X}\beta +\lambda {{\displaystyle \sum }}_{j=1}^p\left|{\beta }_j\right|}}{1+ e^{\mathcal{X}\beta +\lambda {{\displaystyle \sum }}_{j=1}^p\left|{\beta }_j\right|}}$
${\pi }_i>0.5, {\stackrel{⌢}{y}}_i=1\left(dead\right)$

With the predicted probability of the binary response, ${\pi }_i$  we can predict the response itself using the above cut point.
Thus, if the predicted probability of dying is greater than 0.5, we coded the predicted response as 1 (died) and 0 (survived) otherwise.

Validation and k-fold cross validation approach

Validation/cross validation approach is an objective methodology to select an optimal strategy. To select the best model from the variable selection models selected by four strategies, we implement validation and cross validation. We applied k-fold cross validation approach on the final model of manually picked best model, forward selection and backward elimination to create the confusion matrices and calculate the error rate of each method. Since we have 200 observations, we decided to use 5 folds which split the data into two; training and testing datasets. Training set had 160 observations and testing set had 40 observations (total 5 sets each having 40 observations). k-fold cross-validation approach is applicable where the original sample is partitioned at random into k subsamples and one is left out in every iteration step. Let k parts be $C_1,C_2\dots .,C_K,$ where $C_i$ denotes the indices of the observations in part i. We have the following formula to estimate error rate:

$CV= {\displaystyle \sum }_{i=1}^K\frac{n_i}{n} \left(MS{E}_i\right)$

Where$MS{E}_k={\displaystyle \sum }_{i\in C_i}{\left(y_i- \stackrel{}{{\stackrel{⌢}{y}}_i}\right)}^2/n_i$ , ${\stackrel{⌢}{y}}_i$  is the fit for observation i, and $n_i= \raisebox{1ex}{$n$}\!\left/ \!\raisebox{-1ex}{$K$}\right.$ . For this study, n=200, K= 5. So, there are 200/5 parts of 40. The data is split to two groups of testing and training: testing = 40 and training = 160. We fit logistic models on the training data sets and calculate misclassification error rate on the test data. In addition, we conducted k- fold cross validation and validation set on LASSO and compared the results of each method. Since we have 200 observations, we decided to use validation set which split the data into two; training and testing datasets. Training set had 100 observations and testing set had 100 observations (total 2 sets each having 100 observations).

The validation set error rate is determined using:

$CV= \frac{1}{n} {\displaystyle \sum }_{i=1}^n{\left[\raisebox{1ex}{$e_i$}\!\left/ \!\raisebox{-1ex}{$1-h_i$}\right.\right]}^2$

Where $e_i$ is the residual obtained from fitting a model to all the n observations.

Manually picked best model

In this method, we fit a model with all predictor variables. We pick the significant variables (in this case risk factors) with the smallest P-value (P < 0.10) manually and remove all the factors which are insignificant. We then refit the model with all the significant variables in the model. Table 4 shows the significant and insignificant predictors of the manually picked best model.

We refit the model with the significant predictors (risk factors in this case) and eliminate the insignificant predictors all together.

Table 5 below shows the final model with all significant variables. It shows that the risk factors age, cancer part of present problem, type of admission, PH from Initial blood gases, PCO2 from initial blood gases and Level of consciousness at ICU admission as the risk factors.^9,10

From Table 5, our model equation can be written as:
logit $\left({\pi }_i\right)= \log \left(\frac{{\pi }_i}{1- {\pi }_i}\right)$
$={\beta }_0+{\beta }_1{x}_1+ {\beta }_2{x}_2-{\beta }_3{x}_3+{\beta }_4{x}_4+{\beta }_5{x}_5+{\beta }_6{x}_6\approx \mathcal{X}\beta$
               $$\begin{array}{l}=-6.75128+0.04018AGE+2.14668CAN+2.81592TYP\\ +1.76830PH+\left(-2.13254\right)PCO+2.30890LOC\approx \mathcal{X}\beta \end{array}$$  
${\pi }_i= \frac{e^{\mathcal{X}\beta }}{1+ e^{\mathcal{X}\beta }}$
Considering the median age =63, presence of cancer =1, type of admission was elective =0, had no coma or stupor =0, and keep the rest factors fixed, we found:
${\pi }_i= \frac{e^{\mathcal{X}\beta }}{1+ e^{\mathcal{X}\beta }} = 11.2\%$ chance of mortality.
Considering the same as above except the type of admission was emergency =1, we found: ${\pi }_i= \frac{e^{\mathcal{X}\beta }}{1+ e^{\mathcal{X}\beta }} = 67.8\%$ chance of mortality.

In addition, if type of admission is emergency =1, had coma =2, and keep the remaining factors fixed we found the chance of mortality increased from 67.8% to 99.5%.

Forward selection model

For forward selection, a null model (a model contains no predictors), served as the starting point. We added one variable at a time to the null model and refitted the model including the added variables. The idea was to keep it if the variable that had been added was significant and then add the next variable. If not, we eliminated it and added the next variable. We refitted the model using the same procedure until the stopping rule was satisfied (all the variables in the model are significant meeting the level of 10%).

The table above shows the final model of forward selection method. It shows that the risk factors age, type of admission and level of consciousness at ICU admission statistically significant for the ICU status.
From Table 6, our model equation can be written as:

logit $\left({\pi }_i\right)= \log \left(\frac{{\pi }_i}{1- {\pi }_i}\right)$
$={\beta }_0+{\beta }_1{x}_1+ {\beta }_2{x}_2+{\beta }_3{x}_3$
$\begin{array}{l}=-5.52063+0.03291AGE+2.18842TYP+1.83445LOC\\ \approx \mathcal{X}\beta \end{array}$  
Considering the median age =63, type of admission was elective=0, had no coma or stupor=0, and keep the rest factors fixed, we found:
${\pi }_i= \frac{e^{\mathcal{X}\beta }}{1+ e^{\mathcal{X}\beta }} = 3.09\%$ chance of mortality.
Considering the same as above except the type of admission was emergency =1, we found: ${\pi }_i= \frac{e^{\mathcal{X}\beta }}{1+ e^{\mathcal{X}\beta }} = 22.12\%$ chance of mortality.

Backward elimination model

In this method, we began with the full model which includes all predictors in the model and eliminate variables one at a time. The least significant risk factors; that is, the ones having the largest P value (greater than 10%) are eliminated, and the model is then refitted.

Table 7 shows the final model of backward elimination method. It shows that the risk factors age, cancer part of present problem, systolic blood pressure at ICU admission, type of admission, PH from initial blood gases, PCO2 from initial blood gases, and level of consciousness at ICU admission statistically significant for the ICU status.

From Table 7, backward elimination model equation can be written as:

Least absolute shrinkage and selection operator (LASSO)

We fit LASSO with validation set approach. The results are presented in Table 8.

Table 8 shows the final model of LASSO (validation set). We can see type of admission, and level of consciousness at ICU admission as the risk factors.

We also fitted LASSO with applying 5-fold cross validation approach. Table 9 has the results of this approach.
Table 9 shows the final model of LASSO (5-fold cross validation). Cancer part of present problem, previous admission to an ICU within 6 months, type of admission, PH from initial blood gases, PCO2 from initial blood gases, and level of consciousness at ICU admission are identified as the risk factors by this approach.

Misclassification error rate

The cross-validation approach allows to compute misclassification error rate by calculating the confusion matrix. Tables 10 – 14 present confusion matrices for manually picked best model, forward selection model, backward elimination and LASSO.

From the confusion matrix in Table 10, we calculate the misclassification error rate of manually picked best model as $\left( \frac{7+68}{200}\right)*100=37.5\%$ .

From the confusion matrix in Table 11, we calculate the misclassification error rate of forward selection model as $\left( \frac{4+80}{200}\right)*100=42.0\%$ .

From the confusion matrix in Table 12, we calculate the misclassification error rate of backward elimination model as $\left( \frac{7+68}{200}\right)*100=37.5\%$ .

From the confusion matrix in Table 13, we calculate the misclassification error rate of LASSO under validation set approach as $\left( \frac{30+2}{200}\right)*100=16.0\%$ .

From the confusion matrix in Table 14, we calculate the misclassification error rate of LASSO under 5-fold cross-validation as $\left( \frac{27+2}{200}\right)*100=14.5\%$ .

From the results above we conclude that, LASSO (with applying 5-fold cross validation) is the best model for identifying the risk factors associated with the ICU mortality with the lowest error rate (14.5%). We can see cancer part of present problem, previous admission to an ICU within 6 months, type of admission, PH from initial blood gases, PCO2 from initial blood gases, and level of consciousness at ICU admission as the risk factors.^11,12

The major objective of this study is to identify the risk factors associated with medical and surgical ICU mortality. In order to identify the risk factors without subjective bias, we considered different variable selection methods and recommended the method that had the lowest misclassification error rate. The variable selection methods considered in this study were manually picked best model, forward selection, backward elimination and least absolute shrinkage and selection operator (LASSO). Cross validation and validation set approach are applied to the final model of manually picked best model, forward selection, backward elimination, and conducted both validation set and 5-fold cross validation on LASSO. Validation set and 5-fold cross validation approaches allow us to calculate the misclassification error rates for each method and finalize the decision by choosing the model with the lowest misclassification error rate. The procedure determines a reliable model that would identify the risk factors associated with ICU mortality, in an objective manner.

From the results obtained in this study we recommend LASSO (with applying 5-fold cross validation) as the best model that identifies the risk factors associated with the ICU mortality since it has the lowest error rate (14.5%). The model identified cancer part of present problem, previous admission to an ICU within 6 months, type of admission, PH from initial blood gases, PCO2 from initial blood gases, and level of consciousness at ICU admission as the risk factors. One limitation of this study is that the methodology is applied to a limited publicly available data on ICU mortality from a single hospital. In order to confirm the results of this study an elaborative study on ICU mortality should be performed on a randomly selected hospitals throughout the country.

None.

None.

1. https://www.umass.edu/statdata/statdata/data
2. Morandi A, Jackson JC, Wesley Ely E. Delirium in the intensive care unit. International Review of Psychiatry. 2009;21(1):43‒58.
3. Girard TD, Pandharipande PP, Ely EW. Delirium in the intensive care unit. Critical Care. 2008;12(S3).
4. Belayachi J, Dendane T, Madani N, et al. Factors predicting mortality in elderly patients admitted to a Moroccan medical intensive care unit. Southern African Journal of Critical Care. 2012;28(1):22‒27.
5. Lima EQ, Zanetta DMT, Castro I, et al. Mortality risk factors and validation of severity scoring systems in critically ill patients with acute renal failure. Renal failure. 2005;27(5):547‒556.
6. Iwuafor AA, Ogunsola FT, Oladele RO, et al. Incidence, Clinical Outcome and Risk Factors of Intensive Care Unit Infections in the Lagos University Teaching Hospital (LUTH), Lagos, Nigeria. PLoS One. 2016;11(10):e0165242.
7. Bursac Z, Gauss CH, Williams DK, et al. Purposeful selection of variables in logistic regression.Source code for biology and medicine. 2008;3(1):17.
8. Hosmer DW, Lemeshow S, Sturdivant RX. Applied Logistic Regression: Third Edition. 2013;528.
9. http://www.mayoclinic.org/healthy‒lifestyle/fitness/expert‒answers/heart‒rate/faq‒20057979
10. https://www.cdc.gov/bloodpressure/measure.htm
11. Leong IYO, Tai DYH. Is Increasing Age Associated with Mortality in the Critically iII Elderly. Singapore Med J. 2002;43(1):33‒36.
12. Toufen C, Franca SA, Okamoto VN, et al. Infection as an independent risk factor for mortality in the surgical intensive care unit. Clinics. 2013;68(8):1103‒1108.