Objective: The World Health Organisation Surgical Safety Checklist improves surgical outcomes in resource poor settings but the best method of implementing the checklist is unknown. We aimed to evaluate three different methods of training in Guinea and evaluated the outcome at 3–6 months.
Methods: A total of 13 individuals (4 surgeons, 7 anaesthetists and 2 nurses) from 6 different hospitals underwent 3 methods of training (hospital team training, hospital individual training and classroom only training). None had previous knowledge the checklist. Effectiveness of training was evaluated by hospital visits and structured interview at 3-6 months. Corroborating evidence was obtained from interviews with the Hospital Directors and other staff.
Findings: Team training was the most successful. All the hospitals who received team training reported improvements in teamwork, anaesthesia, and infection control. No hospital managed to implement the checklist in its entirety. Anaesthetists who received individual training were unable to implement any changes in their own hospitals.
Conclusion: Team training is more effective than individual training in ensuring more of the key steps of the checklist are followed. Our results question the effectiveness of running a one day classroom training for implementation of the WHO checklist for single groups of professionals such as anaesthetists in the absence of support from other members of the operating team.
anesthesia, surgical safety checklist, patient safety, teamwork
The World Health Organisation (WHO) Surgical Safety Checklist significantly improves surgical outcomes in resource poor settings [1,2]. The key question is no longer ‘does the checklist work?’ but ‘how can we make the checklist work?’ We aimed to conduct a descriptive observational study to evaluate three different methods of checklist training by assessing change in behaviour at 3-6 months post-training.
Guinea is one of the poorest countries in the world, ranked 182nd out of 188 countries in the United Nations, Human Development Index . In December 2012, at the invitation of the Ministry of Health, Mercy Ships evaluated Guinea’s healthcare capacity and structure. None of the government hospitals used the WHO Surgical Safety checklist although one mission hospital was registered as a participating hospital but was not yet actively using the checklist. There are 7 main government hospitals in Guinea, all of which lack reliable water and electricity supply; spinals and ketamine are the mainstay of anaesthesia. 5 out of 7 hospitals have no oxygen or pulse oximetry and patients are never intubated. In the other 2 hospitals, oxygen and pulse oximetry is occasionally available and used when general anaesthesia with halothane is administered.
During a ten month period (2012-2013), we undertook three methods of training in the use of the WHO checklist for Guinean surgeons, anaesthetists and ward nurses onboard the hospital ship the Africa Mercy. Evaluation of the training was assessed by structured interview at 3-6 months during visits to participants in their own hospitals. Corroborating evidence was obtained from interviews with the Hospital Directors and other staff.
Materials and methods
The Africa Mercy, operated by the global charity Mercy Ships is a surgical hospital ship with 5 operating rooms, 5 wards, and a standard of care equivalent to most UK hospitals. Mercy Ships visits countries at the invitation of their Government / President to deliver surgical services and train healthcare providers on board the ship. This educational observational study was reviewed by the Mercy Ships Institutional Review Board and declared exempt from further review or need for written informed consent (M/S 18606). However, all the Mercy Ships surgical services and training projects were approved by the Ministry of Health, Guinea; and all participants completed written applications for training, and gave permission for follow-up and evaluation.
From November 2012 to June 2013 we compared three methods of checklist training:
- Team training in the Operating Room (OR) and Classroom (surgeon PLUS anaesthesia provider or nurse from same institution).
- Individual training in the OR and Classroom (single surgeon or anaesthesia provider).
- Individual training in the Classroom only (anesthesia provider).
Training in the OR and Classroom comprised lectures, watching the checklist performed during routine surgery and then taking part in the checklist during routine surgery. Classroom training consisted of lectures and role-play.
A total of 4 surgeons, 7 anaesthetists and 2 ward nurses from 6 different hospitals participated. All the surgeons were taking part in Obstetric Fistula Surgical Training, as were the ward nurses. Two out of 7 of the Anaesthesia providers were doctors, and the others were nurses.
Effectiveness of training was evaluated by hospital visits and structured interview at 3-6 months. The interview was divided into three parts:
Part 1: The interview focussed on the 3 of the 4 areas of patient care covered by the WHO Safe Surgery Saves Lives initiative: Teamwork, Anaesthesia, Infection Control. The fourth area, Measurement of Surgical Services was not addressed. Participants were asked open questions regarding what they learnt in these 3 areas, what they had been able to implement in their own hospital setting and what were the barriers to implementation.
Part 2: Participants were specifically questioned if training had made a difference especially in relation to the 6 essential safety steps that form the basis for the checklist:
- Confirmation of identity of patient and surgical procedure
- Assessment of risk of placing breathing tube
- Assessment of risk of major blood loss
- Antibiotics given within 1 hour of the start of surgery
- Use of pulse oximetry
- Counting of sponges and instruments
Part 3: Where possible the hospital director; and other surgeons, anaesthesia providers and ward nurses were interviewed to corroborate findings.
Conversations were recorded with pen and paper by the interviewer and two listeners.
In Guinean culture, the correct answer to a question is often thought to be the answer which least offends the person answering the question. This can lead to responder bias. We tried to control for this by having most of the participants interviews (part 1 and part 2) conducted initially by an independent observer not directly involved in the checklist training process. If clarification was required, then the trainer was used. Part 3 interviews were conducted either by the trainer or administrator, and we interviewed hospital directors and other medical staff to attempt to corroborate the answers given by the participants and identify responder bias.
The hospitals, specialty of participants, and training received are shown in Table 1. Corroborative interviews were held with 4 out of 6 Hospital Directors (Hospitals C,D,E,F); Staff who had not participated in the training were interviewed at 3 out of 6 hospitals (Hospitals C,E,F).
All those who received ‘team training’ felt this improved teamwork; anaesthesia organisation and safety; and infection control in their hospitals.
The individual participant’s perception of the effect of training on the 3 key areas of Save Surgery Saves Lives initiative are shown in Table 2. All the anaesthetists said they felt the Surgical Safety checklist helped improve safety and organisation in anaesthesia. The only participant who said training had not impact on anaesthesia safety was a surgeon from hospital E, who underwent individual training. The surgeon, from hospital F, who did team training with his ward nurses reported that he felt the checklist did improve safety in anaesthesia even though his anaesthetist did not receive training. This was because he was able to teach his anaesthetist the principles and they had now started discussing issues such as haemoglobin concentration and risk of blood loss prior to surgery. The hospital director from this hospital and other surgeons corroborated this on independent interview. All participants who trained on board, but none who trained in a classroom said the checklist improved infection control. Examples cited were, timing of antibiotics, not picking up and re-using instruments that fell on the floor during surgery; and correct gowning and gloving techniques.
Examples of improvements in teamwork; anaesthesia organization and safety; and infection control were as follows:
- ‘To change our bad habits to good habits' (in relation to organization and preparation)
- 'When you work with others you learn new things. Before, you think you know best.'
- It is important for doctor and anaesthetist to be there together because ‘if only one person comes back with new ideas, then people don't believe them and this makes it hard to implement change’. They also said two people were better because they can learn different things. ‘It is too much for the doctor to do alone’.
- Ward nurses presenting a summary of the patient’s condition and medical history to the surgeon in the OR before surgery.
- Use of pulse oximetry in the OR and in the recovery ward after surgery, was noted to be valuable but they said they lacked the equipment to do this.
- To be kind to patients and talk to them while they are having a procedure under spinal anaesthesia. ‘Explaining procedures such as spinal anaesthesia, to the patients beforehand actually makes the spinal easier to perform because the patient knows what to do.’
- Improvements in hygiene and infection control such as
- decontamination and washing surgical instruments
- how to use bleach solution
- not picking up instruments from the floor if they fall onto the floor
- cleaning up blood as soon as it is split on the floor, i.e. regular floor washing
- Observation of the patient in the recovery ward after surgery. Now they give one-to-one care.
- Writing and verbally communication of the postoperative plan
Post-operative care is addressed by the ‘sign out’ phase of the checklist and the implications of this were discussed and witnessed in theatre and recovery on board ship.
Specifically in relation to the 6 essential safety steps: all participants said they already confirmed the identity of the patient but most did not use identification bands. Two of six hospitals (who had undergone team training) had initiated using patient identifiers. Four out of 6 hospitals did not intubate patients, so they consider the risk assessment for placement of a tracheal tube unnecessary. However after direct questioning about treatment of a high spinal, they agreed it would be important. Only those who had undergone team training now initiated discussions about risk of major blood loss. All participants said they already gave antibiotics but they agreed on the value of stopping to check this before commencement of surgery.
Four out of 6 hospitals did not have pulse oximetry available, and the other two only had it available occasionally. All the surgeons saw the value of counting of sponges and instruments and had been able to implement this practice in their hospitals but anaesthetists who received training without the surgeon were not able to do this.
Corroborative interviews backed up the answers given by the participants. One hospital director had organized for the surgeon who had been trained to run a morning session on what he had learnt. This was attended by two trainee surgeons and the ward nurses. In most the hospitals the ward nurses do not sub-specialise into ward and OR nurses. In another hospital, on visiting the wards, there was good evidence of organization with names, dates of surgery and procedure displayed above the patient’s bed which they described as having learnt from the training on board ship. The value of teamwork and communicating with patients, for example, explaining that a spinal anaesthetic would involve sitting bent over and having an injection in your back and using the ward nurses to help the anaesthetist, was also noted on corroborative evidence. Counting sponges at the end of the procedure and not picking up instruments after they fell on the floor were also commonly cited themes in corroborative interviews.
Only participants who had undergone team training were able to implement parts of the checklist in their own environment. Implementing the checklist in its entirety was not achieved.
Team training on board ship was more successful than individual or classroom training. The biggest impact was when a surgeon plus another healthcare provider (anaesthetists or nurses) received training together, discussing, watching and taking part in the WHO checklist. Anaesthetists who trained alone were unable to implement any changes in their home environment.
This could be explained by a very surgeon-led culture in Guinean hospitals. There is no formal training for anaesthetists in Guinea, nurses or even non-nurses can be trained by ‘on the job’ in the local hospital. Other low income countries do offer a formal 2-3 year training program for nurses to become an anaesthetist but this is not the case in Guinea. Therefore, in Guinea, the anaesthetist is generally not respected as someone who can change culture or improve safety. One of the anaesthetists who received individual training on board was a doctor, who trained in Russia, and was the Chief of Anaesthesia in a large government hospital. Even though he recognized the value of the checklist, he said he was powerless to make others use it in his hospital. This underlies the importance of understanding the culture and directing training to those with the most power to effect change.
However, the surgeon from hospital E who trained alone was also unable to effect much change in his own hospital except the counting of sponges at the end of the procedure. In this case the hospital director appeared disinterested during our interview and his attitude may have made it more difficult for the surgeon at hospital E. Two hospitals commented that is was better to have more than just the surgeon receiving training because ‘if only one person comes back with new ideas, then people don't believe them and this makes it hard to implement change’. They also said two people were better because they can learn different things. ‘It is too much for the doctor to do alone’. Our results support this, as the greatest ability to apply the training and effect change in the local environment was when training was done in teams. In this regard it is important to consider the question ‘who is the team?’ in this specific cultural context. In many low income countries in West Africa, it is generally the ward nurses who come to the OR with the patient and then scrub and assist the surgeon. Therefore these are an important group who might otherwise be overlooked in the team training process if a ‘western’ approach was taken whereby ward nurses are not considered part of the OR team.
All the hospitals who received team training reported were improvements in 3 out of the 4 keys areas of care outlined in WHO Safe Surgery Saves Lives framework: Teamwork, Anaesthesia, Infection Control. And those who received individual or classroom training reported improvement in at least one of these areas. Therefore, although individual training was not as effective, it did have some value. The fourth key area, Measurement of Surgical Services was not addressed in our training or evaluation.
In high income countries it is known that successful implementation requires effective and decisive leadership, encouraging active staff participation and ongoing education [4,5]. Neily and colleagues described teamwork training and use of the checklist which involved 2 months of preparation, 1 day training for operating room teams and 3 monthly coaching interviews over a one year period [6,7]. Yet in low income settings there is an increasing number of well-meaning individuals/non-governmental organizations running short one-off courses for single groups of individuals, usually anesthetists, attempting to implement the WHO Surgical Safety checklist. Our results question the effectiveness of one day courses for single groups of professionals to create lasting change at 3–6 months follow-up. We did not examine if one-day classroom training in teams would effective. Neither did we examine the effect of training in the local hospital versus role-model training on board ship. These are areas of further study.
Our study is descriptive and contains small numbers making statistical analysis impossible. It is also limited to data obtained by interview rather than actually watching the teams performing in theatre and therefore it is open to responder bias. We tried to control for this by interviewing both the participants and also other staff and hospital directors. However, despite these limitations, our results highlight the difficulties encountered in trying to institute changes in working practice that affect more than one individual and professional group. Running a one-day course on the WHO Surgical Safety checklist was insufficient to create lasting change in behavior at 3 - 6 months follow-up. Team training that involved watching and participating in the checklist seemed the most effective.
In conclusion, team training was more effective than training a single professional. Cutural awareness and understanding hospital heirachy also seems important in maximizing impact. The main areas of improved practice were: (i) teamwork and communication (ii) organization of anaesthesia; (iii) infection control; (iv) counting sponges and instruments.
Authorship and contributorship
M White designed and implemented the study, and analysed the data and wrote the manuscript and acts as guarantor.
F Millimouno helped with implementation and analysis of data, gave input to the manuscript, approved the final version and agrees to act as guarantor.
Oscar Borjesson and Krissy Close, Hospital Projects Managers, Mercy Ships for help with logistics and translation during the structured interviews.
Funded in part by UK National Institute of Academic Anaesthesia, Stanley Rowbotham Grant, 2012. Awarded to Dr M C White.
The authors have no competing interests to declare.
- Haynes AB, Weiser TG, Berry WR, Lipsitz SR, Breizat AH, Dellinger EP, et al. (2009) A Surgical Safety Checklist to Reduce Morbidity and Mortality in a Global Population. N Engl J Med 360: 491-499. [Crossref]
- Kwok AC, Funk LM, Baltaga R, Lipsitz SR, Merry AF, Dziekan G, et al. (2013) Implementation of the World Health Organisation Surgical Safety Checklist, including introduction of pulse oximetry in a resource limited setting. Ann Surg 257: 633-639. [Crossref]
- UNHDI (2015) Human Development Report: Work for Human Development 2015.
- Lepanluoma M, Takala R, Kotkansalo A, Rahi M, Ikonen S (2014) Surgical Safety Checklist is associated with improved operating room safety culture, reduced wound complications, and unplanned readmissions in pilot study in neurosurgery. Scand J Surg 103: 66-72. [Crossref]
- Tang R, Ranmuthugala G, Cunningham F (2014) Surgical safety checklists: a review. ANZ J Surg 84: 148–154. [Crossref]
- Neily J, Mills P, Young-Xu Y, Carney BT, West P, Berger DH, et al. (2010) Association between implementation of a medical team training programme and surgical mortality. JAMA 304: 1693-1700. [Crossref]
- Young-Xu Y, Neily J, Mills PD, Carney BT, West P, Berger DH, et al. (2011) Association between implementation of a medical team training programme and surgical morbidity. Arch Surg 146: 1368-1373. [Crossref]