The Dutch national ERCP quality registration using self-assessment does not correlate with clinical ERCP outcome

Background: Quality measurement of endoscopic procedures is an actual and important subject. In the Netherlands endoscopists are obligated to register their Endoscopic Retrograde Cholangiopancreatography (ERCP) performances in a National ERCP quality registration (NEQR). NEQR is a self-assessment that evaluates ERCP quality and volume per centre and endoscopist, but complications and outcome of the procedure are both not registered in NEQR. The aim of this study is to examine whether NEQR correlates with objective post-ERCP clinical outcome in a tertiary referral centre. Material and methods: A retrospective cohort study of all ERCPs performed at a tertiary referral centre in 2016. Correlations between NEQR self-assessments and post-ERCP complications and mortality were measured. Results: 333 procedures were performed, of which 254 were registered in the NEQR (76.3%). No significant correlations were found between the subjective self- assessment and objective post-ERCP outcomes. Of all 333 procedures the total post-ERCP complication rate was 13.5% and the ERCP-related 30-day mortality rate was 0.6%. Conclusions: The currently used NEQR does not correlate with post-procedural clinical outcomes. Clinical outcome measures in a 30-day post-ERCP period, including complications, should be implemented to improve the usefulness of this specific quality registration.


Introduction
Quality measurement of endoscopic procedures is of broad and current interests. Self-assessment tools have been developed for endoscopic procedures eg. colonoscopy and ERCP. A few studies evaluated these self-assessments for colonoscopy, but outcomes were neither unequivocal nor correlated with the post endoscopy complications [1][2][3]. A recent study tried to objectify the performance of ERCP by using an algorithm, which showed variation in value among the endoscopists with a regression to the mean [4]. The American Society of Gastrointestinal Endoscopy (ASGE) and European Society of Gastrointestinal Endoscopy (ESGE) have published several guidelines with quality indicators for endoscopic procedures, the newest quality indicators published by the ESGE are from 2018 [5,6]. These publications recommend to focus mostly on quality indicators related to outcome measures and the outcome measures themselves, such as cannulation rate, success of stone extraction and stent placement and rates of post-ERCP pancreatitis and adequate antibiotic prophylaxis before ERCP. According to the ASGE publication other indicators that monitor post-ERCP complications, such as perforation, bleeding, cholangitis, cardiopulmonary events, repeat ERCP and ERCP-related mortality should also be registered.
The Dutch Association of Gastroenterologists monitors current quality of ERCPs performed in the Netherlands. In near future the quality and/or volume criteria per centre as well as the individual endoscopist will determine whether ERCP may be performed [7,8]. In 2012 an ERCP registration pilot started to investigate the quality using the Rotterdam Assessment Form for ERCP (RAF-E). Endoscopists participated in this registration on a voluntary basis. Ekkelenkamp et The Dutch national ERCP quality registration using selfassessment does not correlate with clinical ERCP outcome

Study design
This retrospective cohort study was conducted with approval of the Medical Ethical Board of the Catharina Hospital Eindhoven. Data for this study were collected retrospectively from the national ERCP quality registration (NEQR) and medical records from the hospital electronic medical system. The study population consisted of all the patients that underwent ERCP between January 1. 2016 and January 1. 2017 in this tertiary referral centre for ERCP in the Netherlands.
The ERCPs were performed in the angiography room of the radiology department with endoscopic facilities by one of 5 expert endoscopists and 1 trainee guided by one of the experts present in the room. All endoscopists performed over 750 ERCP's in their career. Two nurses assisted the endoscopists. Patients were in a semi-prone position. A sedation specialist administered propofol sedation during regular hours. The propofol dosage was based on the weight of the patient, induction was one milligram per kilogram and maintenance pump dosage was half of the weight of the patient in milligram per hour. In case of pain during ERCP, rapifen 0.25 milligram or ketanest 5 to 10 mg was added. Outside regular hours traditional sedation with midazolam 5 milligram i.v. and/or fentanyl 50 to 100 microgram i.v. was administered and regulated by the endoscopist. In case of disturbing peristalsis intravenous boluses of 10 milligram buscopan were used to mute bowel peristalsis, if allowed by hemodynamics. All patients received rectal diclofenac 100 mg pre-procedural (pancreatitisprophylaxis), unless there was a contra-indication for the use of diclofenac such as allergy or renal dysfunction.
To collect data, all patients were retrospectively traced in the hospital computer system with the unique procedural code that is used for ERCP.
Digital medical records were reviewed, including patient characteristics, ERCP report, outcome, complications, and mortality. From the nationwide ERCP quality registration, pre-and intraprocedural quality indicators were collected.
A complication was defined as an adverse event occurring during the procedure or in the 30 day period after ERCP, if this required unplanned hospital admission or prolongation of a planned admission [12]. Complications such as pancreatitis, bleeding, infection and perforation were classified according to standardised criteria proposed by Cotton et al. [13]. Cholecystitis was defined as an emergency cholecystectomy after ERCP with no prior clinical signs of cholecystitis [14]. Cardiopulmonary complications included hypercapnia, hypoxia, apnoea, arrhythmias, blood pressure changes, silent myocardial ischemia and respiratory failure or acute respiratory distress syndrome from aspiration pneumonia or as a late sequel of complicated pancreatitis [15]. Mortality in 30 days was evaluated and graded as ERCP related or not. The results of the self-assessment for common bile duct cannulation, pancreatic duct cannulation, sphincterotomy, precut papillotomy, stone extraction, stent placement, and pancreatic duct intervention in the quality registration ERCP were correlated with complications and mortality, registered in the medical file of the patients.

Statistical analysis
Data analysis was performed using SPSS Statistics 21 and 24 (IBM Corporation, Armonk, New York, USA). Data were presented as frequencies of categorical variables, and as mean ± standard deviation for continuous variables. Normal distribution was tested for continuous variables. For correlations Spearman's rank correlation coefficient was used to compare variables. A p value of <0.05 was considered significant.

Results
In 2016, 333 ERCPs were performed in our centre, of which 254 were registered in the national quality registration (76.3%). These were performed by 5 expert endoscopists and 1 trainee guided by an expert, with a median of 61 ERCP's per endoscopist/year (range 13 -108), the results of the trainee were registered by the supervisor.
Patient characteristics are presented in Table 2. Mean patient age was 66.8 years, 55.6% were women. The ASA score distribution for grade 1, 2, 3, and 4 was respectively 32.4%, 48.6%, 18.3%, and 0.6%. The Schutz' degree of ERCP difficulty grade 1 was the most common (68.9%), followed by grade 2 (17.7%) and grade 3 (13.4%). The main intention of the procedure was complete stone extraction from the common bile duct (n=202, 60.7%) followed by metal stent placement for stenosis in the common bile duct (n=56, 16.8%) and endoprosthesis placement for stenosis in the common bile duct (n=22, 6.6%). Patients were sedated with propofol in 96,4%, with midazolam and/or fentanyl in 2.4% and had general anaesthesia in 1.2%. In 61.6% of the procedures a sphincterotomy had not been performed earlier (virginal papilla).

Correlations of the self-assessment with complications and mortality
The Spearman's correlation was run for the 254 patients registered in the quality registration to determine the relationship between the self-assessment of the quality registration and the post-ERCP complications, all complications together and separately, as well as mortality. For some interventions e.g. pancreatic duct cannulation and pancreatic duct intervention, it was not possible to measure the correlation because of the lack of power.   A statistically significant negative correlation was found between self-assessment of the stent placement and the total complications post ERCP (r=-0.315; p=0.018). The only complication that showed a correlation with self-assessment was post-ERCP cholangitis: a weak, negative correlation (r= -0.308; p=0.021). Other interventions showed neither a correlation with complications nor with mortality. Two patients died because of post-ERCP complications during this time period. One of these was registered in the NEQR. This 68-year-old patient underwent common bile duct cannulation, sphincterotomy and stone extraction, which were rated 99%, 98% and 96% respectively by the endoscopist. Nevertheless, this patient appeared to have a post-ERCP perforation and died due to multi organ failure caused by abdominal sepsis. The other non-registered patient was 92-year-old, ASA 3 and already septic before ERCP. ERCP was performed with midazolam and fentanyl sedation. Cannulation of the common bile duct was possible with a guidewire, but stone-removal was difficult, and a plastic stent was inserted in the common bile duct. At the end of the procedure the patient developed hypoxia and bradycardia resulting in an admission to the intensive care and eventually leading to death by cardiac complications.

Discussion
This study demonstrates that subjective scoring of ERCP quality only by self-assessment of the endoscopist does not correlate with objective parameters of quality such as complications and mortality.
The examined patient population of this study was of sufficient power to draw clinically relevant conclusions and characteristics were comparable to previous studies on this subject [9,21,24,25]. Only a difficult stent insertion as assessed by the endoscopist showed a correlation with overall complications. Our total complication rate (13.5 percent) was a little higher than in the rates reported in the literature. An explanation might be the higher rate of cholangitis (4.8%) in our study, maybe caused by a relatively low precut success (79.0%). The mortality rate (0.6%) was comparable to earlier reports [14,[17][18][19]24]. Self-assessment scores have shown to be useful in enhancing the learning curves in endoscopy trainees [1,11,32]. The authors also believe that self-assessment helps improving quality of endoscopic skills. To use self-assessment as the main ingredient for quality measurements of experienced endoscopists, however, seems debatable and possible pitfalls need to be addressed. First, the subjective NEQR does not rate some important objective clinical ERCP outcomes such as 30-day success, post-ERCP complications and mortality. The parameters used in the RAF-E are correlated with the direct procedural technical outcome, but these are in fact surrogate markers [9]. The markers give information about the procedural success, but do not predict post-ERCP success. In addition, the RAF-E does not include the number of papilla cannulation attempts or the cannulation time. Difficult cannulation of the common bile duct defined as >5 minutes or >5 cannulation attempts has proven to be a risk factor for post-ERCP pancreatitis [33,34]. ESGE guidelines in 2018 and ASGE guidelines in 2015 advised to document these cannulation attempts and post-ERCP complications, such as pancreatitis, cholangitis, perforation and bleeding [5,6].
Secondly, when political decisions about where ERCP are allowed to be performed and by whom are based on a subjective self-assessment, this scoring method is susceptible to overrating of endoscopy skills.
In the Netherlands the NEQR is a quality parameter that is followed up by the national Ministry of Health. It is expected that in the near future NEQR will determine whether departments and/or individual endoscopists may perform ERCP, therefore an objective registration is pivotal.
The RAF-E subjectivity is also demonstrated in this study by the case of a perforation leading to death. A positive NEQR filled in by the endoscopist does not guarantee foreclosure of complications or even death, as has been described earlier.
Although the reported patient characteristics and ERCP outcomes are comparable to other reports some restrictions of this study should be mentioned. It was designed as a retrospective single center study, included only one-year registration, which was performed in the beginning of the Dutch registration of RAF-E. The latter could be an explanation for 24% missing self-assessments. Other reasons for not filling in the NEQR completely or not at all were: interfering with clinical work, too time consuming, need to login on a separate website, because there is no link between ERCP report and NEQR-website. Notable is that the complication rates of the 79 missing procedures in the NEQR are mostly comparable with the rates of the complete registration as demonstrated in Table 2. Therefore, no registration bias seems to be caused by this loss of registration. Another point if criticism may be the large difference in ERCP numbers per endoscopist (range 13 to 100), making it statistically difficult to compare the results of individual endoscopists.
Based on our findings we suggest that an ERCP quality registration should be determined mostly by procedural outcome: i.e. objective clinical parameters (pancreatitis, cholangitis, bleeding, perforation, cardiopulmonary complications and mortality), measured in a 30day post-ERCP time period. Additionally, the number of papillary cannulation attempts and cannulation time should be taken into account. Subjective self-assessment scoring should only play a minor role and mainly be used to enhance the learning curve of endoscopy trainees or junior consultants. We believe that objective and harder clinical outcome measures will enhance representativeness of the mostly subjective NEQR results, improving its usefulness in future political decisions on quality, volume criteria and centralization of ERCP.
To conclude, a quality registration based on subjective selfassessment of ERCP does not correlate with objective clinical outcomes. To improve the usefulness of endoscopy quality registrations, clinical outcome measures in a 30-day post-ERCP period should also be taken into account.