Epilepsy is the most common serious chronic childhood neurological disorder. It affects children across the world, with 80% of people living with epilepsy in low- and middle-income countries (LMICs).¹ Globally, over 320 000 children (1–14 years old) died in relation to epilepsy between 2007 and 2017; over 10 million years of life are estimated to have been lost because of epilepsy in the same period.^{2, 3} Convulsive status epilepticus is one of the most common childhood neurological emergencies worldwide, requiring urgent medical intervention to avoid death or disability.⁴ Hospital mortality rates from convulsive status epilepticus in LMICs are between 15% and 38%.^5-8 This is markedly higher than the mortality rates (2.7%–5.2%) reported by high-income countries (HICs).^{7, 9}

A strategic objective of the Intersectional Global Action Plan on Epilepsy and Other Neurological Disorders is to ‘provide effective, timely and responsive diagnosis, treatment and care’. It highlighted the need for health care worker training to achieve this.¹⁰ The concept of the epilepsy ‘treatment gap’ is well recognized.¹¹ Since 2005, paediatric epilepsy training (PET) courses have been taught by the British Paediatric Neurology Association (BPNA), a charitable organization of neurologists, paediatricians, and allied health care professionals. The aim of the courses is to improve the diagnosis and management of children with epilepsy. The courses have been taught internationally since 2012 in 17 countries across five continents. From 2005 and up to September 2021, more than 14 000 participants attended PET courses.

PET1 is an entry level course for all health care professionals who look after children with suspected epilepsy. It is included in the World Health Organization technical brief as a health care workforce case study.¹² Course participants are primarily paediatricians, medical officers, or specialist nurses. It is a 1-day course with a standardized curriculum that teaches evidence-based best practice. Delivering important epilepsy care guidelines to key providers responsible for the main entry point to clinical care for affected patients is critical for their widespread application. The intended learning outcomes include recognition, investigation, management, and treatment of children with epilepsy (Appendix S1). The curriculum is developed collaboratively, referenced against international, peer-reviewed, evidence-based guidelines,^13-15 and revised regularly. At times of revision, course leaders from participating countries come from low-, middle-, and high-income settings to capture variations in the availability of medical resources, including investigations and treatment, and surveyed parental views. The structure is based around established models of adult learning, with lectures and small-group workshop teaching, incorporating simulated case scenarios. The courses are delivered by an unpaid, volunteer faculty of trained health care professionals. The volunteers must participate in a 1-day instructor training course and then perform regular teaching on PET courses, initially mentored, as trained faculty.

Course evaluation has been an integral component of PET1. First, we wish to ensure that the course is fit for purpose and meets the needs of the participants, and that it improves knowledge and changes in clinical practice. Second, the evaluation is used to inform a rolling programme of curriculum updates, thus promoting the longer-term sustainability of the educational intervention.

Ours was a cohort study that gathered data from participants immediately after the course and then 6 months later. The aim of the study was to document a series of evaluations performed on participants' feedback about the course, improvements in knowledge, and self-reported changes in practice. We aimed to establish whether these outcomes differed depending on country resource settings and skill mixes of participants by comparing improvements in self-reported clinical practice between HICs and LMICs. We present the findings from cohorts of over 7000 participants across 252 courses, 17 countries, and five continents over 15 years (2005–2020).

METHOD

The BPNA uses the Kirkpatrick evaluation model,¹⁶ a well-established tool, to measure course effectiveness through outcome evaluation. The evaluation uses four domains: reaction, learning, behaviour, and results. This model has been used previously to evaluate other international medical teaching courses.^{17, 18}

The methods of course evaluation evolved over time. Initially, participants completed a course reaction survey; later, a knowledge quiz before and after the course was introduced, followed by an after-course survey of behavioural change. A total of 7528 participants completed a participant reaction form. Subsets of these participants also undertook the knowledge quiz or the ‘change in behaviour’ survey. Findings were compared between LMICs (n = 12) and HICs (n = 5). All data were collected anonymously.

Level one: reaction

A course reaction form was given to every participant immediately on completion of the course. The form was given to participants of all 252 PET1 courses held between January 2005 and March 2020 in a total of 17 countries (Figure 1).

The form captured information on the participants' reaction (e.g. satisfaction) to the course. It collected information on the whole course, individual lectures, and workshops, as well as other aspects of the course, including venue, facilities, and booking procedures. The form used both Likert scales and open questions. These data also informed the regular updates of the courses every 3 years.

RESULTS

Level one: reaction

A total of 7528 participant reaction forms were received (response rate 86% [7528 of 8757]). Over time, and after course curriculum revisions in 2010, 2013, 2015, and 2018, the proportion of participants rating the course as excellent or good significantly improved from 95% (1691 of 1784) to 99% (1690 of 1703) (p < 0.001; OR = 7.1, 95% CI = 4.0–14.0; Figure 2).

The proportion of participants rating the course as excellent or good was higher among LMICs compared to HICs (99% [2365 of 2388] vs 97% [4852/5007]; p ≤ 0.001; OR = 3.3, 95% CI = 2.1–5.4). Direct quotations from participant feedback are found in Appendix S1.

DISCUSSION

Professional medical training courses are a key educational tool in disseminating evidence-based practice to clinicians to ensure optimal management of, in this case, children with epilepsy. However, evaluations of whether such courses are effective and help support patient care are limited. We have been able to undertake this study because, in large part, our evaluation framework is pragmatic and inexpensive to implement.

Our study demonstrates a universally highly positive response among health care professionals (residing in high-, middle-, and low-income settings) to the PET course. Participants conveyed high course satisfaction, demonstrated a significant improvement in knowledge after the course, and reported positive changes in clinical practice 6 months later. The course narrowed the gap in epilepsy knowledge between participants from HICs and LMICs.

To our knowledge, this study represents the largest evaluation of a medical professional epilepsy training course, both in terms of geographical reach and participant numbers; it is the first standardized multilevel evaluation of epilepsy training across multiple countries. Mental health training (which includes an epilepsy module) as part of the World Health Organization Mental Health Gap Action Programme was reviewed.¹⁵ The review identified 15 published training evaluations from nine countries. Fourteen evaluated a single country and one study evaluated two countries. The authors used different assessments across each study.²⁰ A single-country evaluation of a paediatric epilepsy course from Zambia was published.²¹ In 2012, the World Health Organization initiated an epilepsy training programme for non-specialist health care providers, but with only single-country evaluations published.^{22, 23} A study of paediatric resuscitation training reported high participant numbers (n > 13 000). Again, evaluation occurred in only one country.²⁴

Whether knowledge gained on medical training courses translates into practice is a key issue. The fact that the overwhelming majority of survey respondents (98%) reported changes in their clinical practice 6 months after the course indicates that the courses may positively influence clinical care. Our study identified several examples of translation from knowledge gain to change in practice, including the use of an ECG for suspected epilepsy (to identify a potentially life-threatening cardiac syncope) and in prolonged seizure management. Participants later reported incorporating ECG testing into their practice. In turn, this stimulates appropriate cardiac management and has clear potential to save lives.²⁵ Improved knowledge on prolonged seizure management was also demonstrated. Participants have since reported introducing standard guidelines for managing prolonged seizures into their local hospitals (Table 4). The management of prolonged seizures is linked to high mortality, particularly in LMIC settings.⁷ Again, empowerment of local hospitals to use standardized acute seizure management guidelines will encourage sustainable improvements in childhood epilepsy management among staff and help save more lives. The higher levels of change in practice reported by participants from LMICs after the PET course in part probably reflects the fact that PET has been running in the UK since 2005 but internationally only from 2012. This correlates with the lower reported ‘new concepts’ in HICs as participants from the earlier UK courses may already have experienced some influence on epilepsy care in their local setting.

Our study has several limitations. First, the response rate for the ‘change in behaviour’ survey was low (21%). Similar low response rates have been reported by other medical training evaluations.^{18, 26, 27} Thus, our results are at risk of selection bias. The low response rate for the ‘change in behaviour’ survey was probably partly due to the survey being carried out 6 months after course attendance; however, it was important to evaluate after this interval to allow participants the opportunity to make changes in their clinical practice. There was also no active follow-up to ensure that the survey was completed; more responses may have been obtained had this been in place.

Second, there are no direct measures of the results from the Kirkpatrick evaluation model in these data and our methods are prioritizing a pragmatic, inexpensive, widespread evaluation of process outcomes in LMICs rather than a rigorous methodology, such as a cluster randomized controlled trial. More generally, in a critique of the Kirkpatrick model, some suggested that measuring impact or outcome should not be the criterion standard for programme evaluation, but rather to better understand how and why the intervention changes the outcomes.²⁸ Notwithstanding, it is notoriously difficult to measure the impact of a single educational initiative in a health care setting, particularly so in LMIC settings.

Nonetheless, it is plausible to suggest that an educational intervention to improve care, particularly in relation to convulsive status epilepticus, will have an impact on the risk of death. Epilepsy-related mortality data are the most easily collatable data across a range of health care settings, although this still has considerable challenges in the lowest-resourced countries. In those countries where this has been measured, epilepsy-related mortality has fallen. Epilepsy-related mortality has shown a steep decline in the UK since the 1950s; similar and more recent declines have occurred in other countries.¹⁷ In an age–period–cohort analysis, O'Callaghan et al. described a strong birth cohort effect that has probably contributed to this decline.²⁹ It is highly likely that declines in epilepsy mortality are a result of multiple, interlinking factors, of which professionals' epilepsy education may be only one.

In future evaluations, we will actively follow up the behaviour survey data to ensure we obtain responses from most participants. This will be done through multiple routes, from e-mail reminders through to social messaging. Results will be compared to the current data to check for selection bias in these latter responses. The BPNA is working with behavioural change specialists to develop more in-depth qualitative interviewing of participants. This will help to identify barriers to improvements in practice, some of which will be open to mitigation. We will also undertake an audit of key clinical practices before and after the courses at local health care facilities in countries where PET1 courses have not yet been established. These audits will assess whether self-reported changes correlate with objective changes in clinical practice.

Access to specialists in LMICs is lacking. Clinical care, especially for newly presenting children with seizures, is delivered by non-specialists, typically at the primary health care level. With the roll out of PET courses in LMICs, the principal attendees are primary health care workers, nurses, and general practitioners. This group would have had minimal access to training in the area of children with epilepsy or seizures. As such, it was not surprising that the 1-day teaching course, with content that was viewed as understandable and adaptable to local practice, had such an impact in LMICs and more so than in HICs where access to training support was more established. The regions of the world carrying the greatest burden for neurological diseases have the lowest ratio of paediatric neurologists per head of the population.^{30, 31} The specialist gap will take many years to address; in these settings, upskilling of existing health care workers is essential to empower them to be able to deliver effective care using teaching models that are acceptable and effective.

Complementing primary health care epilepsy training programmes and in line with the recent acknowledgement of PET courses by both the World Health Organization and the Education Council of the ILAE,³² we believe that the PET1 course fills an important training gap for health care professionals in childhood epilepsy management.¹² This evaluation lends further support to the effectiveness of this training. The PET curriculum is being used in the planning of other epilepsy interventions in LMICs.³³ A network of collaborative teachers, using aligned, consistent, and evidence-based content, is growing as a result of course participants training to become PET faculty members in their own countries. In addition, most participants reported improving their own local epilepsy teaching since attending the course.

In conclusion, since 2005, BPNA PET courses have substantially grown in number across multiple countries. Evaluation of these courses demonstrates that they contribute to improvements in knowledge and behaviour change among health care professionals and support a global reduction in the epilepsy ‘treatment gap’.

CONFLICT OF INTEREST STATEMENT

MJG has research supported by Fast Track Diagnostics and Siemens Healthineers, payment for educational events by Siemens Healthineers, and patents pending or approved for a diagnostic for bacterial infection. PR was the Executive Director of the BPNA up to February 2022. JW has received payment for educational lectures on epilepsy from Sanofi, sits on the national advisory board for safe practice of valproate in women of childbearing age and cotreatment of antiseizure medications and antiretroviral agents, is the Chair of the ILAE African Commission and President of the International Child Neurology Association, and receives honoraria for editorial work for the Epilepsia and Frontiers in Neurology journals. JB receives a salary from an ILAE grant to the BPNA to support PET activities. CD is paid via the BPNA within his NHS job plan for his role as the PET Content Development Manager and via the Royal College of Paediatrics and Child Health for his role as the Epilepsy12 Clinical Lead. AM receives payment from UCB and GW Pharma for educational events and from GW Pharma for sitting on their advisory board, and holds an unpaid position as a trustee for the BPNA and Epilepsy Scotland. FO is the immediate past-President of the BPNA and has received honoraria from Novartis and GW Pharma/Jazz Pharmaceuticals for participation in advisory boards and speaking at educational events. MK is a trustee for international education within the BPNA. The authors are willing to share relevant data upon reasonable request.