Implementation of an evidence- based algorithm reduces blood culture overuse in an adult emergency department

Bloodstream infections, or bacteremia, continue to be a leading cause of morbidity and mortality in hospitalized patients, particularly in the elderly and in those admitted to the Intensive Care Unit (ICU) [1-4]. Up to 35% of total mortality in ICUs has been attributed to bloodstream infections [1], and the mortality associated with a diagnosis of bacteremia consistently exceeds 20% in the general hospitalized patient population [2,3,5,6]. Moreover, the presence of bacteremia has been shown to affect patients’ longterm outcome and recovery, even after the initial infection has been treated [5-7]. Thus, the timely and accurate diagnosis of bacteremia is essential. Blood cultures are a critical tool in the diagnosis and treatment of bacteremia, showing not only the presence of bacteria in the bloodstream but also identifying the causative pathogen and providing antibiotic sensitivity data that enables providers to optimize antimicrobial therapy to each individual patient. Sets of two to three 20-ml blood cultures have been shown to yield sensitivities of 90% and 98%, respectively, for identifying the causative pathogen in bloodstream infections [8].


Introduction
Bloodstream infections, or bacteremia, continue to be a leading cause of morbidity and mortality in hospitalized patients, particularly in the elderly and in those admitted to the Intensive Care Unit (ICU) [1][2][3][4]. Up to 35% of total mortality in ICUs has been attributed to bloodstream infections [1], and the mortality associated with a diagnosis of bacteremia consistently exceeds 20% in the general hospitalized patient population [2,3,5,6]. Moreover, the presence of bacteremia has been shown to affect patients' long-term outcome and recovery, even after the initial infection has been treated [5][6][7]. Thus, the timely and accurate diagnosis of bacteremia is essential. Blood cultures are a critical tool in the diagnosis and treatment of bacteremia, showing not only the presence of bacteria in the bloodstream but also identifying the causative pathogen and providing antibiotic sensitivity data that enables providers to optimize antimicrobial therapy to each individual patient. Sets of two to three 20-ml blood cultures have been shown to yield sensitivities of 90% and 98%, respectively, for identifying the causative pathogen in bloodstream infections [8].
However, inappropriate ordering of blood cultures in patients with a low pretest probability of bacteremia has the potential to cause patient harm and substantially increase costs of care through false positive results. Bates, Goldman, and Lee [9] determined that the presence of blood culture contaminants was found to be associated with a 20% increase in subsequent laboratory costs, a 39% increase in the cost of intravenous antibiotics, and a 4.5 day increase in median length of hospital stay. Although criteria exist to aid in differentiating between contaminated cultures and true positives, many physicians still continue the use of broad-spectrum intravenous antibiotics to treat patients whose cultures have been determined to be contaminated [10].
Two clinical decision rules have been developed to help risk stratify patients for bacteremia and potentially prevent the overuse of blood cultures in low risk patients. Jones and Lowes [11] proposed using the presence of Systemic Inflammatory Response Syndrome (SIRS) criteria as a guideline, finding that patients who did not fulfill SIRS criteria had a low probability of bacteremia and likely did not need blood cultures. In the original study population, SIRS was over 95% sensitive for bacteremia. Subsequent studies have yielded slightly lower numbers with Laupland et al. [12] finding SIRS to be 93% sensitive for bacteremia in hospitalized ICU patients, and Bossink, Groeneveld, and Thijs [13] estimating the sensitivity of SIRS in medical ward patients at 95%. In an attempt to improve sensitivity, Shapiro et al. [14] developed a more complex clinical decision rule that combined variables from distributed to all AED and inpatient providers and incorporated into the electronic medical record, which provided a reminder each time a blood culture order was placed. All AED nurses were also educated on the algorithm and encouraged to use it proactively in patient care. This study aimed to assess the effects of the implementation of this evidencebased algorithm in the AED on the rate of blood culture collection and contamination. Our secondary goal was to describe the most common reasons for which blood cultures were being ordered in our AED to aid in future quality improvement measures.

Study design
This study was designed as a retrospective observational study, performed at the University of Florida Health (UF Health) Adult Emergency Department (AED) with approval from the University of Florida Institutional Review Board (IRB #201500455). The AED at UF Health is a regional referral center and has an annual census of over 65,000. The AED is staffed with 24-hr coverage of Board-Certified Emergency Physicians as well as advanced practice providers, and emergency medicine residents.
The quantitative arm of this study assessed the impact of the implementation of a blood culture ordering algorithm ( Figure 1) on the patient's history, vital signs, and basic laboratory testing to assign patients into low or high-risk groups. Their low-risk criteria were found to be 98% and 97% sensitive in the original derivation and validation, respectively, though a recent retrospective validation study in an Emergency Department (ED) population found a lower sensitivity of 94% for bacteremia [15]. However, despite initially promising results, providers have been reluctant to adopt these rules for clinical use.
At our institution, blood cultures were being ordered on a large number of patients who had a low pretest probability of bacteremia. In response to this, a departmental multi-disciplinary Continuing Quality Improvement (CQI) initiative, which included administrators, nurses, physicians, and pharmacists, was created and tasked with decreasing the rate of unnecessary blood culture ordering in the AED. After a literature review and discussion with both AED and inpatient providers, we developed an algorithm to standardize blood culture ordering in the AED. The algorithm used a combination of the clinical decision rule developed by Shapiro et al. and recommendations from the Infectious Diseases Society of America (IDSA) to identify patients in whom blood cultures could be safely omitted with minimal risk of missing clinically significant bacteremia. The Shapiro rule was chosen as the base for this algorithm due to its higher reported sensitivity, though it did make the algorithm more complex. Patients were excluded from the algorithm if they had a known immune compromising condition. The algorithm was blood culture ordering rates and blood culture contamination rates in the AED. These data were provided to us on a monthly basis by our hospital laboratory as part of routine quality assurance operations and included results from all blood cultures sent from the AED. All blood cultures obtained in the AED were included in this analysis. Baseline (pre-intervention) blood culture ordering data were collected between January 1 st , 2014 and May 31 st , 2014. Post-intervention data were collected between July 1 st , 2014 and May 31 st , 2015. We chose to exclude the month of June 2014 from our analysis as this was a transition month during which the blood culture algorithm was being developed and education was being rolled out to all AED providers.
The secondary aim of this study is to describe the indication providers cite for blood cultures collected in the AED. Four months after the initial implementation of the blood culture algorithm, an electronic clinical decision tool was incorporated into the electronic medical record (EMR) system which prompted providers to the algorithm and allowed them to indicate why cultures were being ordered. These data were extracted from the EMR as part of the CQI project.

Data and statistical analysis
For the quantitative arm of the study, we compared baseline blood culture data collected with post-intervention data using a nonparametric permutation statistical test based on 10000 permutations. Separate analyses were conducted for monthly blood culture ordering rates and for blood culture contamination rates. The data obtained from the EMR decision support tool were analyzed using descriptive statistics. The analysis was conducted by using the open source statistical software package R version 3.1.3.

Results
A total of 16,887 blood cultures were ordered in our Emergency Department during the study period from January 1 st , 2014 to May 31 st , 2015. During the baseline data collection period from January 1 st , 2014 and May 31 st , 2014, a mean of 1279 blood cultures were ordered per month, with a standard deviation of 7.12. During the post-intervention data collection period from July 1 st , 2014 and May 31 st , 2015, a mean of 851 cultures were ordered per month, with a standard deviation of 89.4. This represents a 33.5% decrease in the mean monthly blood culture orders placed in our ED. The empirical p-value of our statistical analysis resulted highly significant (p<0.001), suggesting that on average the monthly blood culture ordering rate is significantly lower after the implementation of the blood culture ordering algorithm. Over this time period, we observed no significant change in our monthly ED census.
With respect to contamination rates, the mean contamination rate was 2.33% (SD=0.11) for the baseline data collection period, compared to 2.15% (SD=0.31) during the post-intervention period. The empirical p-value for this analysis was not statistically significant (p=0.217) suggesting that no significant difference in blood culture contamination rates exists before and after implementation of the blood culture ordering algorithm. Overall results are summarized in the Table 1

below, and graphed in Figures 2 and 3.
In the time frame after implementation of the electronic clinical decision tool, a total of 5166 blood culture orders were placed. Of these, providers documented responses to the decision aid prompts in 756 culture orders, for an overall response rate of 14.6%. In 53.9% of the responses, the provider selected the presence of an immune compromising condition as the indication for ordering a blood culture. For the purposes of this study, "immune compromising condition" was defined as patients taking active chemotherapy or immunosuppressant medications; patients with sickle cell disease, asplenia, acquired immune deficiency syndrome (AIDS); transplant recipients, or another known immunodeficiency condition. Only 12.87% of responders indicated that the culture was performed per provider discretion without meeting the ordering criteria. A full summary of the survey results is listed in Table 2.

Discussion
The results of this study found a significant decrease in the number of blood cultures ordered from our Emergency Department following implementation of an evidence-based blood culture ordering algorithm, with a 33.5% decrease in monthly blood cultures compared to the baseline study period. Shortly after the algorithm was introduced and integrated into the EMR, we observed a progressive decline in monthly blood culture orders over the subsequent four to five months, followed by a sustained stabilization in the ordering rate. Since that time, no additional interventions were needed to sustain this change with regards to the protocol or the decision support tool, though we did perform data checks every few months to confirm that the decrease was being maintained. We considered whether the decrease in blood culture ordering was influenced by seasonal differences, since the baseline data period included mainly winter months, but we feel that this is unlikely since there has been minimal seasonal variation over the 11-month post-intervention period. Furthermore, though not  included in this study, our ongoing quality assurance data has shown no significant increase in blood cultures through the end of December 2015. Based on this consistent finding, it appears that the blood culture algorithm and the discussion it generated among providers were the major factors in reducing blood culture overuse in our AED.
Since the introduction of the algorithm, we have also observed a change in culture among AED staff with respect to the blood cultures.
We have found that nurses feel more empowered to ask why blood cultures are being ordered when patients did not fit the criteria and draw cultures proactively when they do. We believe that this has served to improve communication among members of the care team, in addition to contributing substantially to this project's sustainability.
These results translate directly into reduced health care costs and an improvement in the quality of patient care. The reduction in blood culture ordering has saved an estimated $36,000 per month in health care costs, taking into account costs of the blood cultures alone. This does not include the reduction in treatment costs and unnecessary hospital stays attributed to false positive results. Such costs are more difficult to quantify, but have been shown to be substantial at other institutions [16].
Although blood culture contamination rates were also tracked as part of this study, the algorithm was not designed to decrease blood culture contamination rates, as it did not address collection techniques. It is therefore not surprising that there was no observed significant change in contamination rates in the post-intervention period. Nevertheless, since contamination remained relatively constant and within acceptable limits throughout the study, the total number of false positive cultures decreased due to a reduction in the absolute number of cultures being collected.
Regarding the qualitative arm of the study addressing the reasons for blood culture ordering, we found it interesting that the most commonly cited reason for ordering a blood culture was the presence of an immune compromising condition, accounting for over half of all responses. This result may be biased by the patient population seen in our tertiary, academic emergency department, which receives patients with significant comorbidities from a wide catchment area. However, it is quite possible that when deciding whether or not to order a blood culture, emergency physicians and advance practice providers primarily focus on a patient's comorbidities as a predictor of bacteremia rather than the signs and symptoms of their acute presentation. Future data from other institutions could help elucidate this further. Apart from this finding, the low response rate makes it difficult to draw other significant conclusions from the data collected by the electronic clinical-decision rule. It is worth noting, however, that very few cultures were performed due to chills or vomiting, both of which are thought to be common symptoms in septic patients. It is uncertain whether providers are overlooking these symptoms, which have been shown to increase likelihood of bacteremia.
The main limitations of this study include the use of retrospective data and inclusion of data from only one Emergency Department. The quantitative outcome data were provided in aggregate on a monthly basis from the clinical laboratory as part of a quality improvement initiative and thus did not include any patient specific data. While this facilitated comprehensive data collection and insured that none were omitted from the analysis, it limited the available data about each individual culture and patient. The aggregate data was de-identified and we were unable to collect demographic information on our patient population or perform any subgroup analyses. Due to the aggregate data, we were also unable to exclude duplicate culture orders.
Regarding the qualitative arm of the study, our data were mainly limited by an inconsistent documentation of indications for obtaining blood cultures. Utilizing the electronic clinical-decision tool when ordering blood cultures was left optional, and responses were only filled out for 14.6% of culture orders during the data collection period. While we initially considered making these responses a mandatory hard-stop when placing a blood culture order, which would improve response rate, we decided against it due to concerns that in some cases it could cause a delay in order entry and potentially compromise the medical care of the patient. Blood cultures are often ordered in critically ill patients, and we did not want to risk discouraging the timely ordering of cultures when appropriately indicated. We must also note that the accuracy of the survey responses could not be secondarily verified due to limitations in our database.
Further research involving other institutions is needed to evaluate the generalizability of our results. For other institutions seeking to introduce similar protocols, our experience with this project suggests that the use of an interdisciplinary approach and the incorporation of reminders into electronic medical record systems may help increase initial acceptance and ensure long-term sustainability of ED protocols.