Cancer Epidemiology

Evaluation of the transferability of survival calculators for stage II/III colon cancer across healthcare systems

Robert N. Jorissen

Systems Biology and Personalised Medicine Division, The Walter and Eliza Hall Institute of Medical Research, Parkville, VIC, Australia

Department of Medical Biology, The University of Melbourne, Parkville, VIC, Australia

Search for more papers by this author

Matthew Croxford,

Matthew Croxford

Department of Surgery, Western Health, Footscray, VIC, Australia

Search for more papers by this author

Ian T. Jones,

Ian T. Jones

Department of Surgery, Royal Melbourne Hospital, Parkville, VIC, Australia

Search for more papers by this author

Robyn L. Ward,

Robyn L. Ward

Faculty of Medicine and Health, The University of Sydney, Sydney, NSW, Australia

Search for more papers by this author

Nicholas J. Hawkins,

Nicholas J. Hawkins

Faculty of Medicine, The University of Queensland, Brisbane, QLD, Australia

Search for more papers by this author

Peter Gibbs,

Peter Gibbs

Systems Biology and Personalised Medicine Division, The Walter and Eliza Hall Institute of Medical Research, Parkville, VIC, Australia

Department of Medical Biology, The University of Melbourne, Parkville, VIC, Australia

Department of Medical Oncology, Royal Melbourne Hospital, Parkville, VIC, Australia

Department of Medical Oncology, Western Health, Footscray, VIC, Australia

Search for more papers by this author

Oliver M. Sieber,

Corresponding Author

Oliver M. Sieber

[email protected]

orcid.org/0000-0001-9480-0786

Systems Biology and Personalised Medicine Division, The Walter and Eliza Hall Institute of Medical Research, Parkville, VIC, Australia

Department of Medical Biology, The University of Melbourne, Parkville, VIC, Australia

Department of Surgery, The University of Melbourne, Parkville, VIC, Australia

Department of Biochemistry and Molecular Biology, Monash University, Clayton, VIC, Australia

Correspondence to: Oliver Sieber, Systems Biology and Personalised Medicine Division, The Walter and Eliza Hall Institute of Medical Research,1G Royal Parade, Parkville, VIC 3052, Australia, Tel.: +61 (0) 3 9345 2885, E-mail: [email protected]Search for more papers by this author

Robert N. Jorissen,

Robert N. Jorissen

Systems Biology and Personalised Medicine Division, The Walter and Eliza Hall Institute of Medical Research, Parkville, VIC, Australia

Department of Medical Biology, The University of Melbourne, Parkville, VIC, Australia

Search for more papers by this author

Matthew Croxford,

Matthew Croxford

Department of Surgery, Western Health, Footscray, VIC, Australia

Search for more papers by this author

Ian T. Jones,

Ian T. Jones

Department of Surgery, Royal Melbourne Hospital, Parkville, VIC, Australia

Search for more papers by this author

Robyn L. Ward,

Robyn L. Ward

Faculty of Medicine and Health, The University of Sydney, Sydney, NSW, Australia

Search for more papers by this author

Nicholas J. Hawkins,

Nicholas J. Hawkins

Faculty of Medicine, The University of Queensland, Brisbane, QLD, Australia

Search for more papers by this author

Peter Gibbs,

Peter Gibbs

Systems Biology and Personalised Medicine Division, The Walter and Eliza Hall Institute of Medical Research, Parkville, VIC, Australia

Department of Medical Biology, The University of Melbourne, Parkville, VIC, Australia

Department of Medical Oncology, Royal Melbourne Hospital, Parkville, VIC, Australia

Department of Medical Oncology, Western Health, Footscray, VIC, Australia

Search for more papers by this author

Oliver M. Sieber,

Corresponding Author

Oliver M. Sieber

[email protected]

orcid.org/0000-0001-9480-0786

Systems Biology and Personalised Medicine Division, The Walter and Eliza Hall Institute of Medical Research, Parkville, VIC, Australia

Department of Medical Biology, The University of Melbourne, Parkville, VIC, Australia

Department of Surgery, The University of Melbourne, Parkville, VIC, Australia

Department of Biochemistry and Molecular Biology, Monash University, Clayton, VIC, Australia

First published: 08 January 2019

https://doi.org/10.1002/ijc.32100

Citations: 1

Conflict of interest: The authors declare no conflict of interest.

Share a link

Email
Wechat
Bluesky

Abstract

Adjuvant! Online Inc (A!O), the Memorial Sloan Kettering Cancer Center (MSKCC), MD Anderson (MDA) and Mayo Clinic (MC) provide calculators to predict survival probabilities for patients with resected early-stage colon cancer, trained on data from United States (US) patient cohorts or patients enrolled in international clinical trials. Limited data exist on the transferability of calculators across healthcare systems. Calculator transferability to Australian community practice was evaluated for 1,401 stage II/III patients. Calibration and discrimination were assessed for overall (OS), cancer-specific (CSS) or recurrence-free survival (RFS). The US patient cohort-based calculators, A!O, MSKCC and MDA, significantly overestimated risks of recurrence and death in Australian patients, with 5-year OS, CSS and RFS prediction differences of −6.5% to −9.9%, −9.1% to −14.4% and − 3.8% to −6.8%, respectively (p < 0.001). Significant heterogeneity in calibration was observed for subgroups by tumor stage and treatment, age, gender, tumor location, ECOG and ASA score. Calibration appeared acceptable for the clinical trial patient-based MC calculator, but restricted tool applicability (stage III patients, ≥12 examined lymph nodes, receiving adjuvant treatment) limited the sample size. Compared to AJCC 7th edition tumor staging, calculators showed improved discrimination for OS, but no improvement for CSS and RFS. In conclusion, deficiencies in calibration limited transferability of US patient cohort-based survival calculators for early-stage colon cancer to the setting of Australian community practice. Our results demonstrate the utility for multi-feature survival calculators to improve OS predictions but highlight the importance for performance assessment of tools prior to implementation in an external health care setting.

Abstract

What's new?

Survival calculators for colon cancer integrating tumor stage and patient variables are emerging as important tools to assist clinical decision-making. However, so far limited data exist on the transferability of calculators across healthcare systems. The present assessment of five calculators trained on data from either US or clinical trial patients in an Australian community cohort identified deficiencies in calibration for four tools. The findings demonstrate the utility for multi-feature survival calculators to improve overall survival predictions but highlight the need for tailoring of cancer survival calculators prior to adoption across healthcare settings.

Supporting Information

Filename

Description

ijc32100-sup-0001-supinfo.docxWord 2007 document , 2 MB

Table S1 Clinical, demographic and tumor pathological characteristics of 1,401 patients diagnosed with colon cancer from 1995 to 2011 (inclusive). CEA, carcinoembryonic antigen; ASA, American Society of Anesthesiologists score; ECOG, Eastern Cooperative Oncology Group Performance Status.

Table S2. Web addresses for colon cancer survival calculators. OS, overall survival; RFS, recurrence-free survival.

Table S3. Clinical, demographic and tumor pathological characteristics of 611Stage I-III colon cancer patients from the MCO dataset with known perineural invasion status diagnosed with colon cancer. These data were used for the imputation perineural invasion status in the ACCORD dataset.

Table S4. Clinical, demographic and tumor pathological characteristics of 587 Stage II/III colon cancer patients from the MCO dataset with known perineural invasion status diagnosed with colon cancer. These data were used to assess the calibration for applicable colon cancer prognosis calculators (Supplementary Table 5).

Table S5. Comparison of colon cancer nomograms for prediction of outcome for 587 patients from the MCO database with stage II/III colon cancer. Note that 576 of 587 patients had usable CSS data. The difference (Diff.) is calculated as predicted (Pred.) minus observed (Obs.). Predictions were not available for all patients due to missing data on some features required for various predictions. The percentage of predictions that could be made is tabulated. * P < 0.05.

Table S6. Calibration of survival calculators in patients with stage III colon cancer with ≥12 examined lymph nodes and receiving adjuvant treatment. The calibration difference (Diff.) is calculated as predicted (Pred.) minus observed (Obs.) survival percentages; 95% confidence intervals (95% CI) are shown. Calibration is for 3-year survival for the Mayo Clinic recurrence-free survival calculator, and for 5-year survival for all other calculators. OS, overall survival; RFS, recurrence-free survival; CSS, cancer-specific survival; A!O (1), Adjuvant! Online model 1; A!O (2), Adjuvant! Online model 2;MSKCC, Memorial Sloan Kettering Cancer Center; MDA, MD Anderson Cancer Center; MC, Mayo Clinic; *P < 0.05; **Percentage of patients for whom predictions could be made.

Table S7. Calibration of survival calculators in an Australian community cohort of patients with stage II/III colon cancer for subgroups by age, ASA score, ECOG status, gender and tumor location. The calibration difference (Diff.) is calculated as predicted (Pred.) minus observed (Obs.) survival percentages. Calibration is for 3-year survival for the Mayo Clinic recurrence-free survival calculator, and for 5-year survival for all other calculators. OS, overall survival; RFS, recurrence-free survival; CSS, cancer-specific survival; A!O (1), Adjuvant! Online model 1; A!O (2), Adjuvant! Online model 2;MSKCC, Memorial Sloan Kettering Cancer Center; MDA, MD Anderson Cancer Center; MC, Mayo Clinic; *P < 0.05; **Percentage of patients for whom predictions could be made.

Table S8. Discrimination of survival calculators in an Australian community cohort of patients with stage II/III colon cancer, compared to the AJCC 7th edition tumor staging system (AJCC7). Point estimates and 95% confidence intervals (95% CI) are shown for C-indices (C). OS, overall survival; RFS, recurrence-free survival; CSS, cancer-specific survival; A!O (1), Adjuvant! Online model 1; A!O (2), Adjuvant! Online model 2; MSKCC, Memorial Sloan Kettering Cancer Center; MDA, MD Anderson Cancer Center; MC, Mayo Clinic; *P < 0.05.

Table S9. Discrimination of survival calculators in patients with stage III colon cancer with ≥12 examined lymph nodes and receiving adjuvant treatment, compared to the AJCC 7th edition tumor staging system (AJCC7). Point estimates and 95% confidence intervals (95% CI) are shown for C-indices (C). OS, overall survival; RFS, recurrence-free survival; CSS, cancer-specific survival; A!O (1), Adjuvant! Online model 1; A!O (2), Adjuvant! Online model 2;MSKCC, Memorial Sloan Kettering Cancer Center; MDA, MD Anderson Cancer Center; MC, Mayo Clinic.

Figure S1.Mean observed versus predicted 5-year survival by quintiles of predicted outcome for (A) stage II colon cancer patients and (B) stage III colon cancer patients. Vertical bars are 95% confidence intervals of observed survival. The gray dashed line is the line of equality for observed versus predicted survival percentages. The solid, purple line indicates a least squares fit of observed versus predicted survival percentages. The P values (P_slope) shown in the bottom right of each panel are for the differences between the slopes of the fitted lines and a line with slope = 1. The P values (P_N-DA) shown in the upper left of each panel are from the Nam-D'Agostino test which compares the observed quintiles of survival against the corresponding predictions.

Figure S2. Mean observed versus predicted survival by quintiles of predicted outcome for patients with stage III colon cancer with ≥12 examined lymph nodes and receiving adjuvant treatment. Vertical bars are 95% confidence intervals of observed survival. Data are for 3-year survival for the Mayo Clinic's recurrence-free survival calculator, and for 5-year survival for all other calculators. The gray dashed line is the line of equality for observed versus predicted survival percentage. The solid, purple line indicates a least squares fit of observed versus predicted survival percentages. The P values (P_slope) shown in the bottom right of each panel are for the differences between the slopes of the fitted lines and a line with slope = 1. The P values (P_N-DA) shown in the upper left of each panel are from the Nam-D'Agostino test which compares the observed quintiles of survival against the corresponding predictions.

Please note: The publisher is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.

References

1Engstrom PF, Arnoletti JP, Benson AB 3rd, et al. National Comprehensive Cancer N. NCCN clinical practice guidelines in oncology: colon cancer. J Natl Compr Canc Netw 2009; 7: 778–831.
10.6004/jnccn.2009.0056
PubMed Web of Science® Google Scholar
2 SB Edge, DR Byrd, CC Compton, et al., eds. AJCC cancer staging manual, 7th edn. New York: Springer, 2010.
10.1007/978-0-387-88441-7
CAS PubMed Google Scholar
3 MB Amin, S Edge, F Greene, et al., eds. AJCC cancer staging manual, 8th edn. New York: Springer, 2017.
10.1007/978-3-319-40618-3
Google Scholar
4Weiser MR, Landmann RG, Kattan MW, et al. Individualized prediction of colon cancer recurrence using a nomogram. J Clin Oncol 2008; 26: 380–5.
10.1200/JCO.2007.14.1291
PubMed Web of Science® Google Scholar
5Weiser MR, Gonen M, Chou JF, et al. Predicting survival after curative colectomy for cancer: individualizing colon cancer staging. J Clin Oncol 2011; 29: 4796–802.
10.1200/JCO.2011.36.5080
PubMed Web of Science® Google Scholar
6Chang GJ, Hu CY, Eng C, et al. Practical application of a calculator for conditional survival in colon cancer. J Clin Oncol 2009; 27: 5938–43.
10.1200/JCO.2009.23.1860
PubMed Web of Science® Google Scholar
7Renfro LA, Grothey A, Xue Y, et al. Adjuvant colon cancer endpoints G. ACCENT-based web calculators to predict recurrence and overall survival in stage III colon cancer. J Natl Cancer Inst 2014; 106: dju333.
10.1093/jnci/dju333
PubMed Web of Science® Google Scholar
8Liu M, Qu H, Bu Z, et al. Validation of the Memorial Sloan-Kettering Cancer Center nomogram to predict overall survival after curative colectomy in a Chinese colon cancer population. Ann Surg Oncol 2015; 22: 3881–7.
10.1245/s10434-015-4495-2
PubMed Web of Science® Google Scholar
9Gill S, Loprinzi C, Kennecke H, et al. Prognostic web-based models for stage II and III colon cancer: a population and clinical trials-based validation of numeracy and adjuvant! Online. Cancer 2011; 117: 4155–65.
10.1002/cncr.26003
PubMed Web of Science® Google Scholar
10Kazem MA, Khan AU, Selvasekar CR. Validation of nomogram for disease free survival for colon cancer in UK population: a prospective cohort study. Int J Surg 2016; 27: 58–65.
10.1016/j.ijsu.2015.12.069
CAS PubMed Web of Science® Google Scholar
11Collins IM, Kelleher F, Stuart C, et al. Clinical decision aids in colon cancer: a comparison of two predictive nomograms. Clin Colorectal Cancer 2012; 11: 138–42.
10.1016/j.clcc.2011.07.001
PubMed Web of Science® Google Scholar
12D'Agostino RB, Nam BH. Evaluation of the performance of survival analysis models: discrimination and calibration measures. In: N Balakrishnan, CR Rao, eds Advances in survival analysis. Amsterdam: Elsevier North-Holland, 2004. 1–25.
Google Scholar
13Harrell FE Jr, Califf RM, Pryor DB, et al. Evaluating the yield of medical tests. JAMA 1982; 247: 2543–6.
10.1001/jama.1982.03320430047030
PubMed Web of Science® Google Scholar
14Papamichael D, Renfro LA, Matthaiou C, et al. Adjuvant colon cancer endpoints G. Validity of adjuvant! Online in older patients with stage III colon cancer based on 2967 patients from the ACCENT database. J Geriatr Oncol 2016; 7: 422–9.
10.1016/j.jgo.2016.07.002
PubMed Web of Science® Google Scholar
15 Central Intelligence Agency. The world factbook. Washington, DC: Central Intelligence Agency, 2017. https://www.cia.gov/library/publications/resources/the-world-factbook/index.html.
Google Scholar
16Doubeni CA, Field TS, Buist DS, et al. Racial differences in tumor stage and survival for colorectal cancer in an insured population. Cancer 2007; 109: 612–20.
10.1002/cncr.22437
PubMed Web of Science® Google Scholar
17Phatak UR, Kao LS, Millas SG, et al. Interaction between age and race alters predicted survival in colorectal cancer. Ann Surg Oncol 2013; 20: 3363–9.
10.1245/s10434-013-3045-z
PubMed Web of Science® Google Scholar
18 World Health Organization. World health statistics 2018: monitoring health for the SDGs, sustainable development goals. Geneva: World Health Organization, 2018.
Google Scholar
19 OECD. Health at a Glance 2015. Paris: OECD Publishing, 2015. https://doi.org/10.1787/health_glance-2015-en.
10.1787/health_glance-2015-en
Google Scholar
20Warren JL, Harlan LC, Fahey A, et al. Utility of the SEER-Medicare data to identify chemotherapy use. Med Care 2002; 40: IV-55-61.
10.1097/00005650-200208001-00008
PubMed Web of Science® Google Scholar
21Sinicrope FA, Shi Q, Smyrk TC, et al. Molecular markers identify subtypes of stage III colon cancer associated with patient outcomes. Gastroenterology 2015; 148: 88–99.
10.1053/j.gastro.2014.09.041
CAS PubMed Web of Science® Google Scholar
22Mouradov D, Domingo E, Gibbs P, et al. Survival in stage II/III colorectal cancer is independently predicted by chromosomal and microsatellite instability, but not by specific driver mutations. Am J Gastroenterol 2013; 108: 1785–93.
10.1038/ajg.2013.292
CAS PubMed Web of Science® Google Scholar
23Williams DS, Mouradov D, Jorissen RN, et al. Lymphocytic response to tumour and deficient DNA mismatch repair identify subtypes of stage II/III colorectal cancer associated with patient outcomes. Gut 2018. DOI:10.1136/gutjnl-2017-315664.
10.1136/gutjnl-2017-315664
Web of Science® Google Scholar
24Hutchins GGA, Treanor D, Wright A, et al. Intratumoral stromal morphometry predicts disease recurrence but not response to 5-fluorouracil-results from the QUASAR trial of colorectal cancer. Histopathology 2018; 72: 391–404.
10.1111/his.13326
PubMed Web of Science® Google Scholar
25Pagès F, Mlecnik B, Marliot F, et al. International validation of the consensus Immunoscore for the classification of colon cancer: a prognostic and accuracy study. Lancet 2018; 391: 2128–39.
10.1016/S0140-6736(18)30789-X
PubMed Web of Science® Google Scholar

Citing Literature

Volume145, Issue1

1 July 2019

Pages 132-142

Evaluation of the transferability of survival calculators for stage II/III colon cancer across healthcare systems

Abstract

Abstract

Supporting Information

References

Citing Literature

References

Information

About Wiley Online Library

Help & Support

Opportunities

Connect with Wiley

Evaluation of the transferability of survival calculators for stage II/III colon cancer across healthcare systems

Abstract

Abstract

Supporting Information

References

Citing Literature

References

Related

Information