2.1 IRB approval

This study was approved by the Institutional Review Board at St. Jude Children's Research Hospital as category 2 exempted research including educational tests and survey procedures [SJ-EPECPEDS (FWA00004775)]. Participation in the course resulted in implied consent.

2.2 Course design

The EPEC-Pediatrics Eurasia course was developed based on findings from the Eurasian ADAPT study supplemented with input from the EurADO supportive care working group including regional experts in pediatric oncology, hospice and palliative medicine, pain medicine, and medical education.^{28, 29} The specific course learning objectives were to (1) understand the elements of palliative care and recognize its importance in childhood cancer care, (2) become proficient in practical applications of pain and symptom management, (3) gain skills in effective communication with patients and families, and (4) identify tools for self-care. The EurADO working group evaluated ADAPT data and EPEC-Pediatrics content to identify 10 modules as high-yield topics for the region (Table S1). Additionally, these 10 modules were adjusted to account for the unique cultural context of the region by regional palliative care experts. For example, some medications were added or omitted based on availability in the region, or sensitivity to cultural views of discussion of prognosis with the child was considered in developing the material.

2.3 Course delivery

The 10 EPEC-Pediatrics modules were recorded as didactic lectures by three palliative care experts from St. Jude (MM, JB, and KW); slides and voice-over audio were then translated into Russian (TY and VM). The course was composed of 3 weeks of online content (15 total hours, March 1–March 19, 2021) via Cure4Kids, a free web-based distance-learning platform with an integrated learning management system.³⁰ A subsequent 5-day synchronous virtual workshop (10 total hours, March 22–26, 2021) using the videoconferencing platform Zoom (See Table S2).³¹ Each workshop day was composed of 1 h of interactive didactic lecture led by a palliative care expert with simultaneous translation followed by the second hour of small group practical sessions of skill-development activities led by EPEC-Pediatrics-trained Eurasian facilitators in Russian or English. An English-speaking EPEC-Pediatrics expert co-facilitator assisted the small group with simultaneous translation to allow for bidirectional discussion. Examples of the skill-development activities included a case-based experience teaching the principles of dyspnea, role play for communication, and wellness exercises during the self-care small group sessions.

2.4 Course participants

Course participants were physicians, nurses, and psychologists who care for children with cancer recruited from Eurasian institutions that collaborate with the St. Jude Global Eurasian Regional Program²⁵ or the World Health Organization Global Initiative in Childhood Cancer (WHO GICC).³² Letters of invitation were sent to collaborating regional leaders who distributed the invitation for participation in the course to their colleagues.

2.5 Survey development

We used a pre–postassessment design to evaluate the participant experience .³³ Assessment items were developed based on the ADAPT survey.^{28, 29} The precourse assessment included demographics and questions assessing alignment with WHO guidance^1.through 15 statements that agreed or disagreed with WHO guidelines.²⁸ Additional sections assessed understanding of standard accepted components of palliative care, ideal versus actual timing of palliative care consultation, and comfort in providing palliative care to their patients (see Figure S1). The assessment was developed in English, translated to Russian, reviewed by regional experts, and back-translated to ensure construct consistency.³⁴ The postcourse assessment included similar questions to the precourse assessment with the addition of questions evaluating participant confidence in delivering components of palliative care after the course and an overall course evaluation (see Figure S2). Both assessments were mostly quantitative with multiple-choice or Likert-scale options, supplemented by several free-text questions (See Table S3 for Course Conceptual Framework). The assessments were distributed using the Qualtrics electronic platform,³⁵ with the precourse assessment in February 2021 and the postcourse assessment in April 2021.

2.6 Statistical analysis

Descriptive statistics were used to summarize participant demographics. Differences in pre- and postassessment samples were identified using McNemar's test and Symmetry test. As in the ADAPT study, the 5-point Likert scale was collapsed into two categories (disagree and agree) to compare the pre- and postresults of the 15 statements assessing alignment to WHO guidance, with the neutral response categorized as not in alignment.²⁸ Each participant's percent alignment to WHO guidance was then calculated as the number of statements in alignment out of 15 total, and the mean percent alignment was calculated for the pre- and postcourse assessments. McNemar's test was used for multiple-choice questions assessing the timing of palliative care consultation. The symmetry test was used to compare comfort levels in providing palliative care pre- and posttest; delta change was calculated by the difference between posttest and pretest means. SAS software, version 9.4 was used for all quantitative analyses³⁶ and a p-value of <0.05 was considered statistically significant. The free-text responses were translated to English from Russian by a bilingual native speaker (BE). Due to the concise nature of free-text responses, thematic content analysis was conducted by one author (MJM). Responses were organized according to the presence of specific concepts that were subsequently grouped into themes for content analysis.

3.1 Participant demographics

Seventy clinicians from 23 institutions in 14 countries applied to the course, with 58 ultimately enrolling and 47 submitting all course components (see Table S4 for demographics of participants who did not complete the course). Three participants had incomplete pre- or postcourse assessments so 44 (75.9%) participants were used for final analysis (Figure 1). Most were pediatric hematologists/oncologists (63.6%); few reported previous palliative care training (15.9%) and most did not have access to PPC consultation at their institution (63.6%). Most participants (86.4%) had at least one patient die, with over a third (36.4%) having more than five patients die in their care within the previous year (Table 1). Almost half (45.4%) of participants felt burdened by their inability to control the suffering of children at the end of life, 20.5% felt burned out by their work, and 15.9% felt more callous toward people since starting their current work (Table 1).

3.2 Alignment with WHO

The mean participant alignment with WHO guidance prior to the workshop was 74.5% (range 25.0%–97.7%, standard deviation 22.1%) which improved postcourse to a mean alignment of 90.3% (range 52.3%–100.0%, standard deviation [SD] of 12.1%, p < 0.01). Of the 15 WHO statements, 8 had a statistically significant improvement between the pre- and postcourse assessments; the remaining 7 had precourse WHO alignment of over 89.6% (Table 2). The individual median improvement for alignment was 15.9 percentage points (range: 2.2–40.9%). The four most common precourse misconceptions were (1) it is difficult to know when a patient with cancer would most benefit from meeting the palliative care team, (2) palliative care is synonymous with “end-of-life” care, (3) palliative care for children with cancer can be delivered by healthcare workers of all disciplines, not only by palliative care specialists and (4) early consultation with palliative care causes increased parental burden and anxiety. Participant scores improved by at least a 20-percentage points in the postcourse evaluation.

3.3 Components and timing of palliative care

Participants also broadened their description of the components of palliative care for children with cancer pre- and postcourse (Figure 2A). Approximately 50% or less of respondents identified certain components as a part of palliative care in the precourse assessments, which were significantly higher after the course, including spiritual support to the patient and family (50.0% vs. 88.6%, p < 0.01), aid in family decision-making around treatment options (47.7% vs. 77.3%, p < 0.01), and help clarify goals of care for the patient and family (45.5% vs. 70.5%, p < 0.01).

When asked about the ideal timing of palliative care consultation for a child with cancer assuming unlimited resources, more participants felt palliative care should be involved from diagnosis (79.5%) in the postcourse compared with the precourse assessment (34.1%, p < 0.01, Figure 2B). Additionally, in the postcourse assessment fewer participants stated that the ideal timing for consultation was at the time of complex or high symptoms compared with the precourse assessment (70.5% vs. 47.7%, p < 0.01).

When asked to provide examples of something they will do differently in the future, many participants emphasized the importance of implementing palliative care earlier in the patient's care: “For me, the statement that the involvement of a palliative team is possible at the time of diagnosis was a discovery. I agree that talking with parents about palliative care early will facilitate communication at different stages of the disease. I am going to inform doctors about this, to spread the necessary information about palliative for parents. It's not easy, because the mentality and attitude towards the very word ‘palliative’ is not very simple in our country, and the ‘doctor-patient’ communication is very upsetting” (Ukraine) (Table S5).

3.4 Comfort and confidence in providing palliative care to patients

In the precourse assessment, 59.1% of participants reported feeling comfortable assessing and treating the physical needs of pediatric patients with serious incurable illness, which increased to 75.0% after the course (p = 0.03) (Table S6). While not statistically significant, there were improvements in the postcourse comfort in addressing the emotional needs of patients (72.7% pretest, 86.4% posttest, p = 0.2) and in addressing the grief and bereavement of families (43.2% pretest, 68.2% posttest, p = 0.7).

After participating in the course, most participants felt more confident in their ability to control the suffering of children at the end-of-life (93.2%) and to prescribe opioids and manage pain (90.9%); they also better understood possible solutions to providing palliative care for children with cancer (97.7%) and how to hold difficult conversations with patients and families (97.7%) (Table 3). In free-text responses, many participants described the importance of self-care and the prevention of burnout as key takeaways from the course; “I have been working in oncology for 28 years. I keep in touch with many already cured patients, as well as with the parents of children who have died. I value them, but I understand that I need to learn to love myself and devote more time to my family.” (Belarus) (Table S5).

3.5 Course assessment

All participants reported the course provided them with new skills relevant to their clinical practice, and 97.7% stated that they will change their clinical practice based on knowledge gained in the course. Finally, 95.5% of participants stated that the course met or exceeded expectations (Table 3).

In free-text responses, several participants described plans to apply the pain and symptom management, psychosocial support, and communication skills gained during the course (Table S5). When asked what they enjoyed most about the course, some participants described specific lectures or small group skill-development sessions. The most common theme, however, focused on the unique structure of the course and the benefit of the small-group skill-development activities where participants were able to learn from content experts and from each other: “I liked the possibility of hearing all the voices, with less or more experience, and the fact that there was no bad answer” (Romania).

Suggestions to improve the course included using more case examples, increasing time for discussion, and allowing more interaction with other participants. Other participants were hopeful that this course could be held in-person in the future.