2.1 Eligibility criteria

Predefined inclusion criteria were phase III randomized controlled trials of patients with stage III–IV cutaneous melanoma undergoing novel systemic treatments (KRAS- or MEK-targeted inhibitors, immune CPIs, interferon-α, or combinations). The selected studies used HRQOL as a primary or secondary or exploratory endpoint and must be published in English Oncology journals.

Predefined exclusion criteria were any RCTs that evaluated nonantitumor treatments (psychological education, exercise intervention, sun protection, or nutritional support) and investigated traditional therapies alone (observation, chemotherapy, radiotherapy, or surgery). Any publications that were not randomized controlled trials (review, modeling study, statistical analysis, and cost-effective analysis). Studies were ruled out if the HRQOL reports lacked or questionnaires were not completed by patients themselves.

2.2 Search strategy

We searched for all phase III randomized controlled trials using HRQOL as an endpoint and evaluating systemic therapies in patients with stage III–IV cutaneous melanoma, published between January 1, 2015 and May 1, 2021 regardless of their start or finishing date. The last search was completed on June 1, 2022.

We searched PubMed with the terms (“quality of life”[MeSH Terms] OR “quality of life”[All Fields] OR “qol”[All Fields] OR “health related quality of life”[All Fields] OR “hrqol”[All Fields] OR “patient reported outcome measures”[MeSH Terms] OR “patient reported”[All Fields] OR “patient reported outcomes”[All Fields] OR “pro”[All Fields]) AND (“melanoma”[MeSH Terms] OR “melanoma”[All Fields] OR “melanomas”[All Fields] OR “melanocarcinoma”[All Fields] OR “melanocarcinomas”[All Fields]) AND (“randomized”[All Fields] OR “randomized”[All Fields] OR “randomization”[All Fields] OR “randomization”[All Fields] OR “randomly”[All Fields]) AND (“2015/01/01”[Date - Publication]: “2021/05/01”[Date - Publication]). We limited the article search type to “Randomized controlled Trial.”

2.3 Selection process

Two independent reviewers (CC and ZH W) screened titles and abstracts, followed by scrutinizing full texts. And a senior researcher (YR Q) made the final decision in case of disagreement. One reviewer (CC) extracted data from identified trials and the others (ZH W and YR Q) checked.

2.4 Data extraction

2.5 Study risk of bias assessment

The Centre for Reviews and Dissemination guidance (CRD)²⁴ was employed to check systematic deviations. The responses are limited to Yes/No/Partially/Not stated/Not applicable. The overall score for each study could be calculated as one “Yes” equals one point. The risk of bias was assessed by three independent reviewers (CC, ZH W, and YR Q).

2.6 Data syntheses

For the primary objective, the adequacy of HRQOL reporting was appraised with a checklist referred to as CONSORT-PRO,²⁰ including whether the authors explained the rationale for the choice of a specific instrument, proposed a prior hypothesis concerning HRQOL, described baseline assessment and instrument administration; whether they adequately covered all domains of HRQOL; and which manners they choose to present HRQOL data. The responses for each item include “Yes”/“No”/“Limited.” There is a total of seven assessment items. Each “Yes” is equal to one point. The total number of “Yes” adds up to the score of each study. The final score ranges from 0 to 7. We would appraise the reporting of HRQOL using this checklist.

For the secondary objective, we extracted the efficacy (progression-free survival [PFS]/recurrence-free survival [RFS]), toxicity, and HRQOL data from identified studies. When analyzing the magnitude of HRQOL differences between treatment modalities, both statistical significance and clinical relevance would be considered. The p value of less than 0.05 was traditionally taken to be the cutoff to conclude statistical significance. HRQOL changes exceeding the boundary of the prespecified threshold were defined as minimum clinically important differences (MCIDs). We would explore the associations among efficacy, toxicity, and HRQOL data sets.

3.1 Study selection

Our search strategy totally identified 55 publications, but only 13 (23.6%) were fully eligible for this systematic review. The other 42 (76.4%) were precluded based on the criteria detailed above. The PRISMA 2020 flow diagram (Figure 2) indicated the selection process.

3.2 Key characteristics of identified studies

Table 1 provided an overview of all included publications.

All 13 studies are international, multicenter, phase III RCTs involving a total of 8646 patients. The trial size ranged from 418 to 1019 patients. Of the 13 studies' interventions, 7 (53.8%) evaluated immune CPI, 4 (30.8%) evaluated small-molecular-targeted therapy, and 2 (15.4%) evaluated interferon. Seven (53.8%) studies focused on patients with resected, high-risk stage III melanoma; 6 (46.2%) studies assessed patients with unresectable stage III or metastatic stage IV melanoma. HRQOL was a primary endpoint in only one trial (7.7%), a secondary endpoint in 8 (61.5%), and an exploratory endpoint in the remaining four (30.8%). All 13 articles were published in high-impact journals.

3.3 Quality of evidence of all 13 RCTs

Table 2 outlined the critical appraisal of all involved studies.

All studies reported sufficient randomization information and clearly stated the number of participants in each arm. Ten of 13 studies (77%) reported adequate concealment of allocation. Twelve (92%) presented baseline comparability of participants. Eleven (85%) reported the balance of baseline characteristics between arms. Nine (69%) were double-blinded trials, and the remaining four trials (31%) were reported as “open.” Blinging of assessors was not stated in all 13 studies. Ten (77%) reported baseline compliance of over 80% of participants. Nine studies (69%) reported missing data, but only 4 (31%) stated reasons.

The quality of evidence varied based on CRD (2009).²⁴ The overall score of eligible studies ranged from 6 to 12 out of a maximum of 14. Six studies got more than 10 points. The quality of evidence is robust enough to support our systematic review.

3.4 The adequacy of health-related quality-of-life reporting

Table 3 summarized all HRQOL instruments employed.

HRQOL measurement was conducted by a similar methodology but with diverse QOL instruments. Of all publications we determined, in order of the frequency of QOL questionnaire used, they were respectively EORTC QLQ-C30, EQ-5D, FACT-M, and a self-defined tool. Particularly, 5 of 13 studies (38.5%) employed more than one HRQOL instrument. Of these 5 trials selected both EORTC QLQ-C30 and EQ-5D questionnaires, of which one study additionally employed a melanoma-specific (FACT-M) instrument. Altogether, 12 of 13 studies (92%) used culturally validated instruments to measure melanoma patients' HRQOL.

Table 4 evaluated the adequacy of HRQOL reporting in 13 studies. We defined validity and reliability as to whether QOL instruments have robust psychometric properties and could be applicable in melanoma patients, which is essential for participants from different cultures attending clinical trials for new drug approval. For 11 reports (84.6%), the validity and reliability of the tools were referenced with corresponding validation studies. Of the remaining 2 (15.4%), there was no information regarding the validity and reliability, although one study²⁶ selected an internationally validated questionnaire—EORTC QLQ-C30. However, the other one³⁶ used an unvalidated tool—physical condition and QOL questionnaires that were originally developed, which seemed to be less robust to reflect patient-reported health status.

The rationale for instrument selection was evaluated whether the authors justified the choice process. We defined it as introducing some basic characteristics of an instrument or applicative reasons. Eleven (84.6%) studies provided relevant details. The HRQOL domains covered by selected questionnaires were adequate enough, including both symptom and function subscales. Nevertheless, only 23.1% put forward a pretrial research hypothesis for the assumed differences in HRQOL between arms. One study—the DECOG trial,³⁷ gave a hypothesis about efficacy but not relevant to QOL. Two studies—COMBI-V³⁴ and COMBI-D³⁵—explained the major reason for no hypotheses regarding QOL. The data lacked enough statistical power so the results should be considered exploratory and verified in future research.

The process, describing how the HRQOL was assessed, was generally reported with few details. But some explicit descriptions were noted. For example, 12 studies (92.3%) described the timing of assessment; 77% of studies reported baseline HRQOL assessment, and all of them demonstrated favorable compliance. The instrument administration was assessed if the authors specified who managed the QOL questionnaire or in which clinical setting the HRQOL instrument was administered. Only 4 (30.8%) reported this process in the methodology.

HRQOL measurement was adequate in 8 (61.5%) studies, reporting all scales consistent with the intended analysis described in the introduction or methodology. Conversely, we defined it to be limited if the publication did not provide detailed QOL score change of each domain or did not interpret the meaning or implication of data. Some trials focused on the global health quality-of-life (GHQ) or symptoms without reporting any functions. For instance, one research³⁶ introduced a new instrument that only reflected the general health status without detailed scores of any subscale.

All studies reported HRQOL data in figures and texts, of which 9 reports offered tables. EORTC 1325-MG²⁵ reported HRQOL data in three manners, whereas CheckMate 238³⁰ only provided texts and figures. The former tends to show more details than the latter one.

Overall, the number of Yes in each study is counted as the overall score, ranging from 0 to 7 points. Consequently, 9 studies (69.2%) scored at least 4 points. HRQOL data were accurately measured in most of the identified RCTs.

3.5 Clinical impacts of novel agents on HRQOL

Table 5 presented efficacy, toxicity, and HRQOL comparisons between different treatments.

Novel interventions responsible for prolonged survival (OS, PFS, and RFS) are also associated with treatment-related toxicity, and a longer survival undoubtedly leads to persistent exposure to antitumor agents, all of which might influence HRQOL.

4.1 Quality appraisal of HRQOL reporting

Our review highlights that HRQOL was comprehensively, reliably, and precisely reported in a most international phase III melanoma RCTs between 2015 and 2021, even if we also observe problematic reporting in several critical criteria.

Over half of the reports employed generic instruments—EORTC QLQ-C30 or EQ-5D, but only one study chose a melanoma-specific tool—FACT-M. Although generic tools indeed provide applicable overall QOL assessments, allowing for comparability between melanoma patients and the general population, it might be less sensitive to detect specific symptoms or functions associated with melanoma, such as sun avoidance affecting social functioning. Compared to patients with other malignancies, such as gastrointestinal tumors or lung cancer, melanoma patients are likely to undergo fewer restrictions on their physical functioning but be much more influenced mentally.³⁸ The content of the FACT-M questionnaire is especially applicable for QOL issues associated with melanoma. Another EORTC-modified melanoma-specific module—EORTC-MEL38, was developed but not validated.³⁹

Recognizing the discrepancies between generic and specific questionnaires, Luckett et al.⁴⁰ stated an algorithm for selecting an appropriate one between them, depending upon which aspects of HRQOL (symptoms, functions, or emotional status) a given trial concentrates on. Collectively, the choice of a suitable instrument needs a tradeoff among the sample population, research objectives, and psychological properties of the questionnaire.³⁸

Our review highlights that 84.6% of studies stated the rationale for employing a particular questionnaire. As the choice of the questionnaire affects the data collection, analysis, reporting, and interpretation, it requires accurate justification to test its availability. This demand might be more pronounced when applying melanoma-specific modules, like FACT-M and EORTC QLQ-MEL 38. Since such supplementary modules bring extra psychological burdens to patients, their application needs another round of validation before practice.

Frustratingly, the research hypotheses that refer to the HRQOL were rarely proposed (only 23.1%). The design of the HRQOL endpoint depends upon the research questions to be answered. Although the primary endpoint usually drives the sample size and stratification to conduct analyses, other study characteristics (selection of instruments, method of administration, and data collection) should be tailored to a pretrial hypothesis. Moreover, an upfront hypothesis deserves a clear illustration to develop a statistical plan for each trial. Nonpredefined hypotheses are likely to result in misleading QOL reports, such as overemphasizing statistical significance than clinical relevance; or complicating our evaluation to judge whether the scales they selected were appropriate and justified. An initial research hypothesis is becoming an indispensable need for a good trial design, which determines the generalizability and credibility of findings.⁴¹

The inclusion of baseline assessment should be mandatory in all clinical trials. Its absence could result in worse compliance, and therefore imperil the validity of findings. Most studies (92.3%) reported the timing of assessment, but few (only 30.8%) described instrument administration. If investigators did not report who and how to manage assessment tools, we were uncertain whether all studies were administrated reliably and whether all data were comparable. All trials should be required to inform instrument administration and rigorously comply with this strategy.

The idea of checking whether texts, graphs, or tables are reported is to examine whether data are presented in a straightforward, clear, and complete way or not. The study with tables and graphs presented more details compared with the one that only provided texts. Explicitly, tables tend to offer summarized information, while graphs are often used to demonstrate dynamic changes of QOL in each domain.

Selective reporting bias should be emphasized. A research⁴² revealed patients might not report pain if they considered that it did not originate from melanoma. This phenomenon might be more distinct in unblinded trials. Researchers should cautiously analyze comparisons between open-label and double-blinding studies. Moreover, in comparison with Germans, Norwegian adults reported a slightly higher GHQ score of EORTC QLQ-C30⁴³ but substantially more symptoms of constipation and diarrhea.⁴⁴ Geography may be another obstacle affecting the reliability of findings.

Some designs of the HRQOL measurement are flawed. Due to sun avoidance behaviors, the changing lifestyles, fear of relapse, and increased psychosocial burdens are likely to constitute important domains of HRQOL analysis that are generally not fully measured in any existing instrument.

The heterogeneity of these trials restricted cross-trial comparisons from an HRQOL perspective. These inconsistencies may be attributed to the absence of standardized questionnaires and statistical analysis. The facets of HRQOL assessed the marking system applied, and the outcomes reported undoubtedly vary in specific ways. Not all outcomes were comparable since not all studies were presented in the same way. Despite cross-trial comparisons bringing interesting clinical information, the interpretation of HRQOL outcomes is challenging and should be done carefully.

Evidently, HRQOL can provide information about the experience of melanoma patients in the trial, such as symptom control and function improvement caused by systemic therapy. Immunotherapy and small-molecular-targeted therapy generally bring better efficacy and different toxicity profiles in contrast to chemotherapy. These self-reported HRQOL data could offer valuable information regarding the effect of novel agents on advanced melanoma patients. Medical consensus depends on publications to provide a bunch of evidence that will support the future clinical decisions. Consequently, inadequate or inaccurate HRQOL reporting would seriously affect the reliability and credibility of these outcomes, which can hinder the use of HRQOL data for clinical decision-making.

4.2 Clinical issues of new drugs on HRQOL

Our findings also manifest that, so far, there was less evidence to support one treatment option over another producing significantly improved HRQOL scores. A large percentage of studies showed no clinical differences between arms, and most novel therapies could maintain baseline HRQOL levels to provide long-term survival and manageable toxicity. Although absolute recommendations on medical practice are hard to establish from our current review, it is likely to infer that anti-PD-1 inhibitors and a combination of BRAF/MEK inhibitors are hopeful candidates for wide use in melanoma treatment.

These novel treatment modalities tend to provide potent efficacy. However, our review might not support the positive correlation between efficacy and HRQOL. On the one hand, the sample size is too small to support regression analysis. On the other hand, we observed inconsistent outcomes of these two endpoints reported in 3 studies. That is to say, not all systemic drugs with potent efficacy also showed pleasant HRQOL. Meanwhile, as few studies focused on the relationship between PFS and HRQOL, PFS serving as a potential surrogate for HRQOL remains controversial and uncertain.⁴⁵ There are unmet needs to determine the association between efficacy and HRQOL. Specific regression analyses are needed in future study.

As health status is frequently impaired by AEs, especially for immune-related AEs, the association between toxicity and HRQOL is complicated. Less toxicity could lead to better health status, but it does not mean more AEs would necessarily cause a decrease in HRQOL score. Although HRQOL deterioration occurred less frequently in a nivolumab-plus-ipilimumab group than the ipilimumab group, a higher rate of AE was observed in the combination arm over time. It seems to contradict the idea that toxicity negatively affects HRQOL.¹⁸ Thereby, novel agents might lead to more AEs, but not necessarily a poorer HRQOL. Also, a longer follow-up HRQOL assessment³³ showed the incidence of toxicity decreased with sustained therapy, as the early onset of these AEs could be resolved by clinical management.⁴⁶ Noteworthily, in most studies, patients undergoing higher-grade AEs were not required to finish the HRQOL questionnaire before the withdrawal. By doing so, PRO data would not adequately reflect patients' AEs.³³ To address these concerns, researchers need to involve all participants who discontinued treatment owing to AEs in the final HRQOL analysis, which would be essential steps for reporting precise data and investigating possible correlations.

4.3 Limitations of the review process

Our systematic review has some limitations. We only focused on RCTs. Instead, we did not review protocols, statistical analysis plans, or other types of research. It restricted us to comment on the entire study. We only assessed articles published in English. However, as important studies are generally reported in English journals, our review could cover the most meaningful findings. Another limitation is some subjective criteria were employed to judge the quality of HRQOL reporting. But each criterion was appraised by two independent reviewers under the supervision of a senior researcher. We believe our review is fairly consistent. Furthermore, we might mistakenly exclude some studies that only measured HRQOL subscales or that did not mention the keyword of HRQOL and its abbreviations. We only evaluated HRQOLs that have been published. There is no indication of the number of unreported studies and no explanations for the absence, such as journal-driven or researcher-driven limitations.

4.4 Implications from this review

Applying HRQOL data to guide precision medicine, the precise, adequate, and unified approaches to report HRQOL are fairly important, which need strict conduction with standardized methodology, thoughtful study design, active participation of patients from beginning to end, high-quality data collection, and transparent trial reporting. With more advances in reporting HRQOL, the inclusion of HRQOL in large clinical trials of advanced melanoma will contribute to clinical decision-making.