Secular and modulator-specific drifts in the predictive performance of a rapid lung function decline algorithm: a cystic fibrosis patient registry study

Predicting the onset of pulmonary exacerbation for people with cystic fibrosis (CF) is critical to identify those at high risk in advance and allows timely clinical intervention. The FEV1-indicated exacerbation signal (FIES) has been proposed by the CF Learning Network to identify pulmonary exacerbation events, but temporal predictive performance of this marker is unknown.

To assess the epidemiologic impact of secular trends in CF care and treatments on algorithms aimed at prediction of pulmonary exacerbation, we evaluated prediction model “drift” using a longitudinal retrospective cohort of 29,476 individuals from the U.S. CF Foundation Patient Registry (2011–2018). The FIES marker was defined for each clinical encounter using relative drops in lung function, measured as forced expiratory volume in 1 s of % predicted (FEV1pp). We formed predictive probabilities for each FIES event via a target function for a predictive algorithm of FEV1pp. Models were trained using 2 year data windows with prediction performance tested on subsequent years.

Year-over-year FIES event discrimination remained consistent (area under the receiver-operator characteristic curve, AUC > 80%). Annual drift was < 1% with actual drop in AUC ~ 3% in 6 years. The model performed best in individuals with moderate-to-low baseline FEV1pp (AUC range: 84–85%). For relatively higher FEV1pp levels, discrimination was slightly lower with higher AUC variability.

While model fit improved by accounting for modulator therapy, prediction performance was similar. Most impactful model drifts occurred after 2012 coinciding with increased ivacaftor modulator use. Additional evaluation will be necessary given recent, broad uptake of highly effective modulators.