Description and validation of a spectrum score method to measure antimicrobial de-escalation in healthcare associated pneumonia from electronic medical records data

Karl Madaras-Kelly, Makoto Jones, Richard Remington, Christina Caplinger, Benedikt Huttner, Matthew Samore

Comparison of antimicrobial de-escalation rates between healthcare settings is problematic. To objectively and electronically measure de-escalation a method based upon the spectrum of antimicrobial regimens administered (i.e., spectrum score) was developed. A Delphi process was used to develop applicable concepts. Spectrum scores were created for 27 antimicrobials based upon susceptibility for 19 organisms. National VA susceptibility data was used to estimate microbial spectrum. Susceptibility estimates were converted to ordinal scores, and values for organisms with multi-drug resistance potential were weighted more heavily. Organism scores were summed to create antibiotic-specific spectrum scores and extended mathematically to score multi-antimicrobial regimens. Vignettes were created from antimicrobial regimens administered to 300 patients hospitalized with pneumonia. Daily spectrum scores were calculated for each case. Hospitalization day 4 scores were subtracted from day 2 scores (i.e., spectrum score ∆). A positive spectrum score ∆ defined de-escalation. Experts ranked each pneumonia case on a 7-point Likert scale (Likert >4 indicated de-escalation). Spectrum score ∆s were compared to expert review. Findings were used to identify score deficiencies. Next, 40 pairs of cases were modified to include antimicrobial administration routes. Each pair contained almost similar regimens; however, one contained oral (PO) the other only intravenous (IV) antimicrobials on day 4 of therapy. Experts reviewed cases as described. Spectrum score ∆ credits to account for PO conversion were derived from the mean paired differences in Likert Score. De-escalation status was evaluated in 100 vignettes containing antimicrobial route by different experts and compared to the modified method. Initial sensitivity and specificity of the spectrum score ∆ to detect expert classified de-escalation events was 86.3 and 96.0%, respectively. In paired cases, the mean (± SD) Likert score was 5.0 (1.5) and 4.6 (1.5) for PO and IV (P = 0.002), respectively. To improve de-escalation event detection, two credits were added to spectrum score ∆s based upon the percentage of antimicrobials administered PO on day 4. The final method, exhibited sensitivity and specificity to detect expert classified de-escalation events of 96.2 and 93.6%, respectively. The final spectrum score method exhibited excellent agreement with expert judgments of de-escalation events in pneumonia.