10.17843/rpmesp.2020.374.5064
ORIGINAL ARTICLE

Development of a model for predicting major infection following pediatric heart surgery

Alfredo M. Jauregui ¹, Medical doctor
Paula V. Urrunaga ¹, Medical doctor
Juan A. Gonzales ¹, Medical doctor
Luis E. Silva ¹, Medical student
Vinay Pasupuleti ², Medical doctor, Doctor of Epidemiology
Ewout W. Steyerberg ^3,4, doctor en Epidemiología
Adrian V. Hernandez ^1,5, Médico cirujano, doctor en Epidemiología
Eduardo W. Silva ⁶, Médico cirujano, magíster en Gerencia en Salud

¹   Facultad de Medicina, Universidad Peruana de Ciencias Aplicadas (UPC), Lima, Perú.
²   ProEd Communications, Inc., Cleveland, Ohio, Estados Unidos.
³   Departamento de Salud Pública, Erasmus MC, Rotterdam, Países Bajos.
⁴   Departamento de Estadística Médica, Leiden University Medical Center, Leiden, Países Bajos.
⁵   Escuela de Farmacia, University of Connecticut, Storrs, Connecticut, Estados Unidos.
⁶   Departamento de Cirugía, Unidad Postoperatoria Cardiovascular, Instituto Nacional de Salud del Niño (INSN), Lima, Perú.

INTRODUCTION

The aim of the prediction models is to help care personnel during the decision-making process by identifying patients with a higher risk of presenting a certain outcome ⁽¹⁾. Postoperative complications occur in 43% of pediatric patients undergoing heart surgery with extracorporeal circulation ^(2,3). Postoperative infections are associated with pediatric heart surgery, with a reported incidence of 11% to 38% ^(1-10). In this context, there are three particularly important infectious complications: sepsis, mediastinitis, and endocarditis, which together form an entity called major postoperative infection (MPI), which is defined as one or more of the three infections during the established postoperative period from 48 hours after the operation to the date of discharge. The incidence of MPI worldwide varies between 2.8% and 3.5% in the pediatric and adult population ^(11-19). Although rare, MPI is a very severe complication due to the impact on mortality (22% to 25% in patients with MPI vs. 3% to 3.9% in patients without MPI), on length of hospital stay and on institutional costs ^{(11-13,15-20)}.

Several factors are associated with the occurrence of MPI and the probability is higher in those procedures assisted by cardiopulmonary bypass or extracorporeal circulation (ECC) ^(1,12). Associated preoperative factors include younger age, low weight, previous long hospital stay, previous admission to an intensive care unit (ICU), higher functional class of heart failure, higher level of surgery complexity (Risk Adjustment for Congenital Heart Surgery score [RACHS-1]), respiratory support, cyanotic heart disease, genetic disorders, and other comorbidities ^{(1,7,9,10-12,17,18)}. MPI has also been associated with intraoperative factors, such as prolonged surgery time and ECC, volume and number of blood transfusions, intraoperative hypothermia, use of intracardiac prosthesis, and open chest condition upon leaving the operating room (OR) ^{(1,7,10,17,18)}.

Different predictive models have been proposed to calculate the MPI risk after heart surgery. Among the most relevant factors were the following: age under six months, time with open chest, and postoperative stay for more than two days ⁽³⁾. There is a predictive model of MPI from the Society of Thoracic Surgeons (STS) that includes preoperative variables, comorbidities, and cardiac surgical procedures ⁽¹¹⁾; the model showed good performance and good discriminatory performance and was later validated by Kansy in Poland ⁽¹²⁾.

The aim of the research was to develop a risk prediction model for pediatric MPIs and to validate the one of the STS.

MATERIALS AND METHODS

Type of study

A retrospective cohort study was carried out. The target population was all patients under 18 who underwent heart surgery with ECC admitted between 2000 and 2010 to the Post-Operative Cardiovascular Unit (UPOCV) of the Instituto Nacional de Salud del Niño (INSN) in Lima, Peru. Patients who did not use ECC were excluded.

The UPOCV team of intensivists and an INSN pediatric infectious disease specialist made the diagnosis of infection. The diagnostic criteria and definitions established by STS for sepsis, mediastinitis and endocarditis were used ⁽²¹⁾.

Procedure

Independent variables

Risk factors for postoperative pediatric cardiac infections were formally sought in the literature ^{(1,2,7,9-13,17-22)}. The search was conducted on PubMed and Scopus through October 2018. Preoperative variables studied included age, weight, sex, provenance, prematurity, acute malnutrition, length of preoperative stay, previous cardiac surgery, type of heart disease, Risk Adjustment for Congenital Heart Surgery score (RACHS-1), previous mechanical ventilation, pneumonia, pulmonary tuberculosis, tracheostomy, congenital malformations, genetic disorders, cardiac and non-cardiac malformations, hepatitis, sepsis and endocarditis, renal failure, dialysis, cardiogenic shock, arrhythmias, pulmonary hypertension, cyanotic heart disease, Ross-modified functional class, hypothyroidism, history of epilepsy and neurological damage ^{(7,11-13,17,21)}. The intraoperative variables studied were non-elective surgery (versus elective surgery), intraoperative diagnosis, type of surgery, time of ECC, circulatory arrest, ultrafiltration during ECC, use of corticosteroid, deep hypothermia, and open chest.

Data source

The sources of information were the patient’s medical records and medical history. The data contained the same information indicated in the STS file for congenital heart surgery database. Data collection was performed by one of the researchers (EWS). The information collected was exported to the SPSS version 22 program (IBM Corp., Armonk, NY, USA) for statistical analysis.

Development of the model

We used The Chi-square test and Fisher’s exact test to determine differences in population distribution between patients with and without MPIs. Subsequently, we carried out a logistic and binary regression model and selected those with a p < 0.200 ⁽²³⁾. The association of each variable with the primary outcome was reported using odds ratio (OR) and a 95% confidence interval (95% CI).

Internal validation of the prediction model

The model was validated internally using the bootstrapping technique with 200 repetitions. Each repetition created models in samples identical in size to the study sample size, using random sampling with replacement of individuals. The internal validation provided a correction factor to adjust the model coefficients, assuming that the initial model coefficients were inflated for predictions in the new groups.

Evaluation of model performance

The predictive model was evaluated for discriminatory performance and clinical utility. Discrimination refers to how useful the model is in differentiating between patients with presence and absence of MPI. Discrimination is quantified with the c-statistic, which is equivalent to the area under the ROC curve. The c-statistic has values ranging from 0.5 to 1, and the closer it is to 1, the better the model. Clinical utility was evaluated with benefit analysis ⁽²⁴⁾; this method evaluates the benefit of correctly identifying a patient with a high risk of MPIs (true positives) versus mislabeling them as probable MPIs (false positives) in a range of reasonable MPI risk thresholds where the physician feels comfortable deciding (for example, to administer antibiotics for an MPI). A threshold of 20% was considered reasonable, with a range of 5% to 40%.

Finally, the calibration of an internally validated model was evaluated with a calibration graph describing the relationship between the expected and observed MPI risks. Restricted cubic splines were used to visualize the relationship. The calibration of the model was also evaluated using the Hosmer-Lemeshow test.

Score and nomogram

We presented a model as scores for clinical use. For each variable we assigned a score that was proportional to the strength of association with the primary outcome of MPI. Therefore, the variables of the final prediction model with higher OR had higher scores. The final score was obtained by the sum of the scores of each variable in the final model. The scores were created by multiplying the variable’s coefficient β by 10. We also created A nomogram, which allows the direct calculation of the probability of the primary outcome. In the nomogram, each variable has a different scale that is proportional to the weight of its coefficients ^(20,24-27). The equivalence of MPI scores and risks with preoperative variables was also calculated. We carried out all these analyses with the R 3.3.2 program (www.r-project.org).

Validation of STS model

Besides discrimination and internal calibration, the comparison of infection rates was also used for model validation. Our model and STS model were compared using the MPI rates of six heart surgeries comparable to our study (atrial septal defect, ventricular septal defect, atrial ventricular canal, tetralogy of Fallot, Glenn, and double outlet right ventricle) through Fisher’s exact test.

Ethical considerations

The Ethics Committees of Universidad Peruana de Ciencias Aplicadas (CEI/515-03-15) and INSN (Letter No. 01265-CEI- INSN-2015) approved the realization of the study.

RESULTS

Demographic characteristics

A total of 1,025 patients were analyzed, 59 (5.8%) of them had at least one episode of MPI (4.8% sepsis, 1.0% mediastinitis, 0% endocarditis). Three patients had more than one type of infection. Ninety-five percent of the operated patients had postoperative sepsis. There were only ten cases of mediastinitis.

Patients with MPI were younger (2.95 vs. 5.42 years; p < 0.001), and 62.7% were men (Table 1). The average weight of patients with MPI was significantly lower than the rest of the patients (10.9 kg vs. 16.8 kg; p < 0.001) (Table 2). The length of preoperative stay was longer in patients with MPI (median of 40 vs. 15 days; p < 0.001). Most patients with MPI were diagnosed with cyanotic heart disease (69.5%; p < 0.001). In addition, patients with MPI had a higher percentage on the RACHS-1 3-4 scale (54% vs. 27%; p < .001). A similar result was found for Ross-modified functional class IV (higher in patients with MPI, 37% vs. 15% in the total population, p < 0.001). On the other hand, more patients with MPI had previous mechanical ventilation (11.9% vs 0.9%; p < 0.001). There was a greater number of patients with a history of previous infection resolved at the time of surgery in the cases with MPI (7.5% vs 32%; p < 0.001). There were no differences in congenital anomalies, malformations, and malnutrition status.

Table 1. Clinical demographics of pediatric patients who had cardiac surgery with extracorporeal circulation at Instituto Nacional de Salud del Niño, between 2000 and 2010 

* Compared to prematurity.
OR: odds ratio; SD: standard deviation; m: months; y: years, Ref: reference.

Table 2. Preoperative and intraoperative clinical characteristics of pediatric patients who underwent extracorporeal circulation heart surgery at Instituto Nacional de Salud del Niño, between 2000 and 2010.

* By increase in 1 unit. ⱡ vs. elective.
VSD: ventricular septal defect; TF: tetralogy of Fallot; ASD: atrial septal defect; TAPVR: total anomalous pulmonary venous return; RACHS-1: classification adjusted to the risk for congenital heart surgery; SD: standard deviation; IQR: interquartile range; OR: odds ratio; Ref: reference.

We found that patients undergoing non-elective surgery had more MPIs (79.7% vs 44.8%; p < 0.001) (Table 2). There was no difference between the cases of MPI for the year of surgery, the longest time of extracorporeal circulation, deep hypothermia, open chest, and valve prosthesis.

The total mortality rate at 30 days was 13.7% and significantly higher in the group of patients with MPI (mortality 37/59 [63%] vs. 125/966 [13%]; p < 0.001). The duration of postoperative mechanical ventilation (301.6 hours vs. 34.3 hours; p < 0.001), postoperative intensive care unit stay (20.9 days vs. 5.1 days; p < 0.001), and postoperative stay (42.2 days vs. 15.1 days; p < 0.001) were greater in patients with PMI. Nine deaths with MPI had a history of sepsis and two of endocarditis. Excluding these patients, mortality in patients with MPI remained high (58%) and significantly different from patients without MPI (p < 0.001). Of the 32 patients, two died of sepsis.

Bivariate and multivariate regression model

In the bivariate analysis of preoperative variables, a strong association was found between MPI and younger age, lower weight, longer preoperative stay, type of heart disease, cyanotic heart disease, higher RACHS-1 scale and Ross-modified functional class, and previous mechanical ventilation (Table 1 and 2). The non-elective (vs. elective) surgeries also had a statistical association with MPI (Table 2).

We created a preoperative model and a combined model (preoperative and intraoperative). In the first model, shown in Table 3, the variables associated with PMI were prolonged preoperative hospital stay (p = 0.053), diagnosis of cyanotic heart disease (p = 0.020), RACHS-1 (p = 0.013), and Ross-modified functional class IV (p = 0.009). In the second model, the associated variables were cyanotic heart disease (p = 0.020), RACHS-1 (p = 0.020), Ross-modified functional class IV (p = 0.010), and non-elective surgical condition (p = 0.020).

Table 3. Preoperative multivariate analysis

95% CI: 95% confidence interval; RACHS-1: risk-adjusted classification for congenital heart surgery; OR: odds ratio.

Performance of the preoperative model

Internal validation of the model produced a shrinkage factor of 0.84. The discriminatory performance of the model was good, with an adjusted c-statistic of 0.80 (95% CI 0.74-0.86). The combined preoperative and intraoperative model showed no significant improvement in discrimination compared to the preoperative model (P = 0.600), which is shown in the supplemental material (Figure S1). The net benefit analysis showed that both models had a similar performance for MPI ranging from 0% to 40%, and their use is better than not using a predictive model (supplemental material, Figure S2). The calibration of both models was good as expected in an internally validated model.

The preoperative model was presented as scores and a nomogram. The scores per variable included in the final model are shown in Table 4, and the equivalence of MPI scores and risks are shown in Table 5. Scores between 12 and 29 correspond to MPI probabilities of 5% to 95%, respectively. A nomogram is shown in Figure 1. For example, a 5-year-old girl with 20 kg, 30 days of preoperative stay, with cyanotic heart disease and a RACHS-1 scale of 3 will have a 19‑point and 30% risk of MPI.

Table 4. Scores for risk of major postoperative infections after pediatric heart surgery by variable included in the preoperative multivariate model.

*Note: The preoperative stay is divided by 5 to express the risk for increased 5-day hospital stay. For example: a patient with 25 days of hospitalization before surgery has a value of 5 (i.e., obtained dividing 25 by 5) and has 1 point according to the model.

RACHS-1: Risk-adjusted classification for congenital heart surgery.