Take a look at the Recent articles

The common pitfall in studies of biomarkers for diagnosing infections in patients with systemic lupus erythematosus

Stamatis Karakonstantis

Second Department of Internal Medicine, General Hospital of Heraklion �Venizeleio-Pananeio�, Leoforos Knossou, Heraklion, Greece

E-mail : aa

Dimitra Kalemaki

Department of General Medicine, University Hospital of Heraklion, Greece

DOI: 10.15761/GIMCI.1000153

Article
Article Info
Author Info
Figures & Data

Abstract

Infections in patients with systemic lupus erythematosus may be difficult to differentiate from lupus flares without associated infections. Several authors have searched for biomarkers to allow the detection of infection in patients with lupus. Here, we discuss a common pitfall of the design of these studies. Including patients with obvious infections or patients with no reason to suspect an infection may overestimate the diagnostic value of biomarkers. Therefore, although such studies are a useful first step for the identification of promising biomarkers, prospective studies with an appropriate study population are needed to confirm the value of novel biomarkers.

Key words

lupus, biomarkers, pitfalls, study design, infection, flares

Introduction

Infections in patients with systemic lupus erythematosus (SLE) may result in symptoms like those during SLE flares without infection rendering the distinction between infection and SLE flares difficult in some cases especially in patients presenting with fever [1, 2]. This distinction is clinically important, as mistakenly treating an infection with immunosuppression or a flare with antibiotics can have serious consequences [1]. Several authors have studied biomarkers to aid with the differentiation of lupus flares associated with infections from lupus flares without underlying infection [1-7]. However, all these studies share a common pitfall in design that could complicate the application of their results in clinical practice.

The clinically useful question is whether assessment of a biomarker would alter the evaluation or management of SLE patients that truly pose a diagnostic dilemma (flare versus infection). Therefore, the ideal population for such a study would be patients with unclear diagnosis at presentation. Studies that include patients with either obvious infections, or patients with flares but with no reason to suspect an infection, are likely to overestimate the diagnostic value of biomarkers, and results of such studies would likely not be applicable to clinical practice. For example, checking any biomarker would likely not alter the management of an SLE patient presenting with high fever, new productive cough and a new pulmonary consolidation on the chest x-ray.

However, reviewing previous publications [1-7] the study populations were far from ideal in most of these studies. [Table 1] summarizes some prior studies. An unclear diagnosis at presentation was not a requirement for inclusion in any of these studies and it seems that many of the patients included had either obvious non-infectious flares or obvious infections, thus obviating the need for assessment of any biomarker. Such studies may be useful as a first step for the identification of novel biomarkers. However, promising biomarkers identified by these studies should be re-evaluated in prospective studies with an appropriate study population, i.e., patients with unclear diagnosis at presentation that truly pose diagnostic/therapeutic dilemmas. Of note is that for the same biomarkers different studies result in different cut-offs (e.g. CRP≥5-6mg/dl [6], CRP≥0.71mg/dl [4], CRP≥1.28-1.35mg/dl [3, 5]) and different sensitivities/specificities (see the example of CPR and procalcitonin in [Table 1], illustrating the need for validation of biomarkers in a defined and clinically meaningful study population. Evaluation of biomarkers in different subgroups of patients according to clinical presentation may also be important. This is highlighted by the example of CRP, which can be markedly elevated in certain subsets of SLE patients, e.g. patients with serositis or synovitis [8], in the absence of infection.

Table 1: Summary of selected prior studies

Study

Study population

Hussein et al 2010 [7]

Biomarker: CD64 mean fluorescent intensity

Study design: Case-control

Final sample: 20 healthy subjects and 43 patients with autoimmune diseases (including 19 with SLE) categorized in 3 groups (inactive disease, active disease, and infection)

Cut of for infection: ≥43.5 mean fluorescent intesnsity, 94.4% sensitivity, 88.9% specificity.

Notes: Athough the reported sensitivity and specificity are high, comparing patients with infection to a group including healthy patients and patients with inactive disease is not clinically helpful. Although CD64 was significantly higher in patients with infection (mean 49, range 13-205) there was some overlap with patients in the active disease group (mean 36.15, range 12-133)

Firooz et al 2011[6]

Biomarker: hsCRP and ESR

Study design: retrospective

Inclusion criteria: Patients admitted to the hospital with a diagnosis of SLE and a hsCRP and ESR measurement within the same week. The following patients were excluded: patients without both ESR and CRP measurement (n=338), SLE patients admitted due to unrelated conditions (n=51), uncorfirmed diagnosis (n=198), insufficient data (n=4).

Final sample: SLE flare n=24, active infection n=22 (n=12 culture-proven), both flare and active infection (n=8).

Cut offs for infection: CRP≥5mg/dl sensitivity 59%, sensitivity 80%, CRP≥6mg/dl sensitivity 55%, specificity 84%.

Notes: Patients with both flare and active infection were very few, and the CRP in those patients was only slightly higher compared to the CRP of patients with SLE flares without infection (mean 3.7mg/dl versus 2.7mg/dl). Of note is also the number of patients excluded due to an uncorfirmed diagnosis, i.e., the group of patients that trully pose diagnostic/therapeutic dillemas, and the group that would benefit from identification of a useful biomarker.

Kim et al 2012 [3]

Biomarker: CRP, PTC, S100A8/A9

Study design: prospective

Inclusion criteria: SLE patients admitted due to confirmed flare or infection. Patients with fever of unclear origin were excluded. ‘Flare was defined arbitrarily as any clinical event directly attributable to disease activity leading to an SLEDAI ≥ 6 that would require an escalation of treatment without evidence of infection.

Final sample: SLE flare without infection n=39. SLE patients with infection n=34: pneumonia n=7, acute pyelonephritis n=3, gastrointestinal infection n=1, acute cholangitis n=1, gynecological infection n=2, acute pharyngitis n=5, parotid gland abscess n=1, soft tissue infection n=2, tuberculosis n=2, sepsis n=10.

Cut-offs for infection: CRP≥1.38mg/dl sensitivity 100%, specificity 90%, PCT≥0.025ng/ml (with a reported detection limit for PCT of 0.05ng/ml) sensitivity 38.2%, specificity 93.3%.

Notes:2021 Copyright OAT. All rights reservatients trully posed a diagnostic dilemma. E.g. it seems that many of the patients in the infection group had clinically obvious infectious.

Yu et al 2014 [4]

Biomarker: PCT and CRP

Study design: retrospective

Inclusion criteria: Patients with active SLE admitted to the hospital for any condition.

Exclusion criteria: SLEDAI score=0

Final sample: 47 cases with bacterial infections (pneumonia n=36, urinary tract infection n=9, abdominal cavity infection n=4, catheter-related bloodstream infection n=3, cholecystitis n=1, pyothorax n=1), 67 cases without bacterial infections. About 50% in each group of the patients had high disease activity defined as SLEDAI score>10. 43 patients in the infection group were febrile, and 40 patients in the group without infection.

Cut offs for infection: PCT≥ 0.38 ng/ml, sensitivity 74.5%, specificity 95.5%. CRP ≥ 0.71mg/dl sensitivity 70.2%, specificity 62.7%.

Notes: Although PCT was significantly higher in those with bacterial infection (including the subgroup of patients with fever and those with high SLE activity), and had a high specificity, it is unclear how this would alter patient management in actual clinical practice given the study population (SLE patients admitted due to any condition). Nevertheless the subgroup analysis in patients with fever and patients with high disease activity compensated for this.

Beca et al 2015 [2]

Biomarker: Risk calulator based on the duration of fever, anti-dsDNA and CRP

Study Design: Retrospective

Inclusion criteria: SLE patients admitted because of fever associated with flare or infection.

Final sample: SLE flares n=94, infection n=101 (urinary tract infections n=26, community-acquired pneumonia n=23, acute gastroenteritis n=19, acute bronchitis n=15, cutaneous abscess n=4, cellulitis n=3, upper airway infection n=3, herpes zoster n=3, abdominal abcsess n=2, meningitis n=2), both flare and infection n=15.

Notes: SLE patients presenting with fever is a clinically relevant study population. However, similar to other studies, many included patients seem to have had obvious infectious diagnoses. Of note is also that the AUC of the proposed algorithm was much lower in the validation cohort than in the derivation cohort (0.72 versus 0.92).

Kim et al 2017 [5]

Biomarker: Neutrophil to lymphocyte ratio (NLR), monocyte to lymphocyte ratio, CRP

Study design: Prospective

Inclusion criteria: As in Kim et al 2012 (see above).

Final sample: 60 cases with flare without infection. 60 cases with infection: pneumonia n=9 (unknown pathogen n=3), acute pyelonephritis n=6, gastrointestinal infection n=4 (unknown pathogen n=4), acute cholangitis n=1 (unknown pathogen n=1), appendicitis n=1, gynaecological infection n=3 (unknown pathogen n=3), acute pharyngitis n=8 (unknown pathogen n=8), pyogenic arthritis n=1, parotid gland abscess n=3 (unknown pathogen n=3), soft tissue infection n=4, tuberculosis n=2, sepsis n=11, herpes zoster n=4, influenza n=3

Cut-offs for infection: CRP≥ 1.28 mg/dL sensitivity 93.3% specificity 98.3%, NLR≥5.7 sensitivity 90%, specificity 75%

Notes: Despite the high sensitivity and specificity of both NLR and CRP the study population is not clinically relevant. Many of the patients included seem to have had an obvious infectious diagnosis obviating the need for evaluating any biomarker. Furthermore, 90% of the patients with infection had fever compared to only 35% of patients with flares.

Abbreviations: AUC= area under the curve, ESR= erythrocyte sedimentation rate, hsCRP= high sensitivity C-reactive protein, PCT= procalcitonin, SLE= systemic lupus erythematosus.

Clinical significance

Detecting an infection in patients with lupus flares is clinically important to guide appropriate treatment (antibiotics versus immunosuppressive therapy). Biomarkers to differentiate flares from infection should be evaluated in a defined and clinically meaningful population. Including patients with obvious diagnoses, e.g. patients with obvious infections or patients with no reason to suspect an infection, might overestimate the diagnostic value of some biomarkers.

Funding

The authors declare that no funding was received for this manuscript.

Disclosure of interest

The authors declare that there is no conflict of interest.

References:

  1. Ospina FE, Echeverri A, Zambrano D (2017) Distinguishing infections vs flares in patients with systemic lupus erythematosus. Rheumatology (Oxford) 56: i46-i54.
  2. Beca S, Rodriguez-Pinto I, Alba MA, Cervera R, Espinosa G (2015) Development and validation of a risk calculator to differentiate flares from infections in systemic lupus erythematosus patients with fever. Autoimmun Rev 14: 586-593.
  3. Kim HA, Jeon JY, An JM, Koh BR, Suh CH (2012) C-reactive protein is a more sensitive and specific marker for diagnosing bacterial infections in systemic lupus erythematosus compared to S100A8/A9 and procalcitonin. J Rheumatol 39: 728-734.
  4. Yu J, Xu B, Huang Y (2014) Serum procalcitonin and C-reactive protein for differentiating bacterial infection from disease activity in patients with systemic lupus erythematosus. Mod Rheumatol 24: 457-463.
  5. Kim HA, Jung JY, Suh CH (2017) Usefulness of neutrophil-to-lymphocyte ratio as a biomarker for diagnosing infections in patients with systemic lupus erythematosus. Clin Rheumatol 36: 2479-2485.
  6. Firooz N, Albert DA, Wallace DJ (2011) High-sensitivity C-reactive protein and erythrocyte sedimentation rate in systemic lupus erythematosus. Lupus 20: 588-597.
  7. Hussein OA, El-Toukhy MA, El-Rahman HS (2010) Neutrophil CD64 expression in inflammatory autoimmune diseases: its value in distinguishing infection from disease flare. Immunol Invest 39: 699-712.
  8. Gaitonde S, Samols D, Kushner I (2008) C-reactive protein and systemic lupus erythematosus. Arthritis Rheum 59: 1814-1820.

Editorial Information

Editor-in-Chief

Dario Marchetti
Baylor College of Medicine

Article Type

Commentary

Publication history

Received date: December 27, 2017
Accepted date: January 27, 2018
Published date: February 02, 2018

Copyright

©2018 Karakonstantis S. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Citation

Karakonstantis S, Kalemaki D (2018) The common pitfall in studies of biomarkers for diagnosing infections in patients with systemic lupus erythematosus. Gen Int Med Clin Innov 3: doi: 10.15761/GIMCI.1000153.

Corresponding author

Stamatis Karakonstanti

Second Department of Internal Medicine, General Hospital of Heraklion 'Venizeleio-Pananeio', Leoforos Knossou, Heraklion, Greece, 71409, Tel: +30-6940100298

Table 1: Summary of selected prior studies

Study

Study population

Hussein et al 2010 [7]

Biomarker: CD64 mean fluorescent intensity

Study design: Case-control

Final sample: 20 healthy subjects and 43 patients with autoimmune diseases (including 19 with SLE) categorized in 3 groups (inactive disease, active disease, and infection)

Cut of for infection: ≥43.5 mean fluorescent intesnsity, 94.4% sensitivity, 88.9% specificity.

Notes: Athough the reported sensitivity and specificity are high, comparing patients with infection to a group including healthy patients and patients with inactive disease is not clinically helpful. Although CD64 was significantly higher in patients with infection (mean 49, range 13-205) there was some overlap with patients in the active disease group (mean 36.15, range 12-133)

Firooz et al 2011[6]

Biomarker: hsCRP and ESR

Study design: retrospective

Inclusion criteria: Patients admitted to the hospital with a diagnosis of SLE and a hsCRP and ESR measurement within the same week. The following patients were excluded: patients without both ESR and CRP measurement (n=338), SLE patients admitted due to unrelated conditions (n=51), uncorfirmed diagnosis (n=198), insufficient data (n=4).

Final sample: SLE flare n=24, active infection n=22 (n=12 culture-proven), both flare and active infection (n=8).

Cut offs for infection: CRP≥5mg/dl sensitivity 59%, sensitivity 80%, CRP≥6mg/dl sensitivity 55%, specificity 84%.

Notes: Patients with both flare and active infection were very few, and the CRP in those patients was only slightly higher compared to the CRP of patients with SLE flares without infection (mean 3.7mg/dl versus 2.7mg/dl). Of note is also the number of patients excluded due to an uncorfirmed diagnosis, i.e., the group of patients that trully pose diagnostic/therapeutic dillemas, and the group that would benefit from identification of a useful biomarker.

Kim et al 2012 [3]

Biomarker: CRP, PTC, S100A8/A9

Study design: prospective

Inclusion criteria: SLE patients admitted due to confirmed flare or infection. Patients with fever of unclear origin were excluded. ‘Flare was defined arbitrarily as any clinical event directly attributable to disease activity leading to an SLEDAI ≥ 6 that would require an escalation of treatment without evidence of infection.

Final sample: SLE flare without infection n=39. SLE patients with infection n=34: pneumonia n=7, acute pyelonephritis n=3, gastrointestinal infection n=1, acute cholangitis n=1, gynecological infection n=2, acute pharyngitis n=5, parotid gland abscess n=1, soft tissue infection n=2, tuberculosis n=2, sepsis n=10.

Cut-offs for infection: CRP≥1.38mg/dl sensitivity 100%, specificity 90%, PCT≥0.025ng/ml (with a reported detection limit for PCT of 0.05ng/ml) sensitivity 38.2%, specificity 93.3%.

Notes: It is unclear if included patients trully posed a diagnostic dilemma. E.g. it seems that many of the patients in the infection group had clinically obvious infectious.

Yu et al 2014 [4]

Biomarker: PCT and CRP

Study design: retrospective

Inclusion criteria: Patients with active SLE admitted to the hospital for any condition.

Exclusion criteria: SLEDAI score=0

Final sample: 47 cases with bacterial infections (pneumonia n=36, urinary tract infection n=9, abdominal cavity infection n=4, catheter-related bloodstream infection n=3, cholecystitis n=1, pyothorax n=1), 67 cases without bacterial infections. About 50% in each group of the patients had high disease activity defined as SLEDAI score>10. 43 patients in the infection group were febrile, and 40 patients in the group without infection.

Cut offs for infection: PCT≥ 0.38 ng/ml, sensitivity 74.5%, specificity 95.5%. CRP ≥ 0.71mg/dl sensitivity 70.2%, specificity 62.7%.

Notes: Although PCT was significantly higher in those with bacterial infection (including the subgroup of patients with fever and those with high SLE activity), and had a high specificity, it is unclear how this would alter patient management in actual clinical practice given the study population (SLE patients admitted due to any condition). Nevertheless the subgroup analysis in patients with fever and patients with high disease activity compensated for this.

Beca et al 2015 [2]

Biomarker: Risk calulator based on the duration of fever, anti-dsDNA and CRP

Study Design: Retrospective

Inclusion criteria: SLE patients admitted because of fever associated with flare or infection.

Final sample: SLE flares n=94, infection n=101 (urinary tract infections n=26, community-acquired pneumonia n=23, acute gastroenteritis n=19, acute bronchitis n=15, cutaneous abscess n=4, cellulitis n=3, upper airway infection n=3, herpes zoster n=3, abdominal abcsess n=2, meningitis n=2), both flare and infection n=15.

Notes: SLE patients presenting with fever is a clinically relevant study population. However, similar to other studies, many included patients seem to have had obvious infectious diagnoses. Of note is also that the AUC of the proposed algorithm was much lower in the validation cohort than in the derivation cohort (0.72 versus 0.92).

Kim et al 2017 [5]

Biomarker: Neutrophil to lymphocyte ratio (NLR), monocyte to lymphocyte ratio, CRP

Study design: Prospective

Inclusion criteria: As in Kim et al 2012 (see above).

Final sample: 60 cases with flare without infection. 60 cases with infection: pneumonia n=9 (unknown pathogen n=3), acute pyelonephritis n=6, gastrointestinal infection n=4 (unknown pathogen n=4), acute cholangitis n=1 (unknown pathogen n=1), appendicitis n=1, gynaecological infection n=3 (unknown pathogen n=3), acute pharyngitis n=8 (unknown pathogen n=8), pyogenic arthritis n=1, parotid gland abscess n=3 (unknown pathogen n=3), soft tissue infection n=4, tuberculosis n=2, sepsis n=11, herpes zoster n=4, influenza n=3

Cut-offs for infection: CRP≥ 1.28 mg/dL sensitivity 93.3% specificity 98.3%, NLR≥5.7 sensitivity 90%, specificity 75%

Notes: Despite the high sensitivity and specificity of both NLR and CRP the study population is not clinically relevant. Many of the patients included seem to have had an obvious infectious diagnosis obviating the need for evaluating any biomarker. Furthermore, 90% of the patients with infection had fever compared to only 35% of patients with flares.

Abbreviations: AUC= area under the curve, ESR= erythrocyte sedimentation rate, hsCRP= high sensitivity C-reactive protein, PCT= procalcitonin, SLE= systemic lupus erythematosus.