This study aimed to validate four clinical prediction models (CPMs) for identifying serious infections (SI) in children in ambulatory care. The four CPMs tested were Feverkidstool, Craig model, SBI score, and PAWS. Results showed that Feverkidstool and Craig’s models had a good discriminative ability for predicting SI, while the SBI score and PAWS performed poorly. This study emphasizes the importance of externally validating CPMs before their implementation in clinical practice and highlights the potential of the Feverkidstool and Craig models in identifying children at risk of SI.
The diagnosis of severe infections (SIs) in children is a challenge to ambulatory care providers. Early detection of SIs can help with clinical decision-making and the prompt start of the best course of treatment. Medical professionals can use clinical prediction models (CPMs) to identify children who are at risk of SIs, but they must first go through external validation before being used in clinical settings. In this study, four CPMs developed in ERs were externally validated in a group of ill kids who visited GPs, pediatric outpatient clinics, or ERs in Flanders, Belgium.
Methods:
A prospective cohort of acutely ill children presenting to general practices, outpatient pediatric practices, or emergency departments in Flanders, Belgium, was included in the study. The four CPMs evaluated in this study were Feverkidstool, the Craig model, the SBI score, and PAWS. The Feverkidstool and Craig models are multinomial regression models, while the SBI score and PAWS are risk scores. Feverkidstool and the Craig model’s discriminative power and calibration were analyzed, whereas the diagnostic test precision of PAWS and the SBI score were evaluated. Reestimating coefficients with an overfitting correction allowed for the updating of the Craig and Feverkidstool models.
Results:
A total of 8211 children were included in the study, of whom 498 had SIs and 276 had serious bacterial infections (SBIs). The Feverkidstool model had a C-statistic of 0.80 (95% confidence interval [CI] 0.77–0.84) with good calibration for pneumonia and 0.74 (0.70–0.79) with poor calibration for other SBIs. The Craig model had a C-statistic of 0.80 (0.77–0.83) for pneumonia, 0.75 (0.70–0.80) for complicated urinary tract infections, and 0.63 (0.39–0.88) for bacteremia, with poor calibration. The model update resulted in improved C-statistics for all outcomes and good overall calibration for Feverkidstool and the Craig model. The SBI score and PAWS performed extremely poorly, with sensitivities of 0.12 (0.09–0.15) and 0.32 (0.28–0.37), respectively.
Discussion:
The results of this study suggest that the Feverkidstool and Craig models have a good discriminative ability for predicting SIs and have the potential for early recognition of SIs in a low prevalence setting for SBIs. The model update resulted in improved performance for both models. However, the SBI score and PAWS performed poorly in this cohort, suggesting that they may not be useful in identifying children at risk of SIs in ambulatory care settings. It is important to note that the performance of these models may vary depending on the prevalence of SIs in different settings.
