A systematic review of outcome reporting in colorectal cancer surgery
Abstract
Aim
Evaluation of surgery for colorectal cancer (CRC) is necessary to inform clinical decision-making and healthcare policy. The standards of outcome reporting after CRC surgery have not previously been considered.
Method
Systematic literature searches identified randomized and nonrandomized prospective studies reporting clinical outcomes of CRC surgery. Outcomes were listed verbatim, categorized into broad groups (outcome domains) and examined for a definition (an appropriate textual explanation or a supporting citation). Outcome reporting was considered inconsistent if results of the outcome specified in the methods were not reported. Outcome reporting was compared between randomized and nonrandomized studies.
Results
Of 5644 abstracts, 194 articles (34 randomized and 160 nonrandomized studies) were included reporting 766 different clinical outcomes, categorized into seven domains. A mean of 14 ± 8 individual outcomes were reported per study. ‘Anastomotic leak’, ‘overall survival’ and ‘wound infection’ were the three most frequently reported outcomes in 72, 60 and 44 (37.1%, 30.9% and 22.7%) studies, respectively, and no single outcome was reported in every publication. Outcome definitions were significantly more often provided in randomized studies than in nonrandomized studies (19.0% vs 14.9%, P = 0.015). One-hundred and twenty-seven (65.5%) papers reported results of all outcomes specified in the methods (randomized studies, n = 21, 61.5%; nonrandomized studies, n = 106, 66.2%; P = 0.617).
Conclusion
Outcome reporting in CRC surgery lacks consistency and method. Improved standards of outcome measurement are recommended to permit data synthesis and transparent cross-study comparisons.
Introduction
There have been major changes in the surgical management of colorectal cancer (CRC) over the last 30 years. This is partly the result of enhanced perioperative care and the introduction of novel surgical techniques, including minimal access surgery and total mesorectal excision 1-3. Such innovations require robust technology appraisal or evaluations in randomized controlled trials (RCTs) to ensure that they are safe and effective, and one aspect of this is reporting valid and relevant outcomes. Recent work has, however, highlighted problems with outcome reporting in many areas and has demonstrated the need for consistent use of outcomes and for better-designed studies 4. Addressing these issues in surgical research will facilitate data synthesis in meta-analyses, allow transparent cross-study comparisons and ultimately should positively influence clinical decision-making.
Outcome assessment in CRC surgery is particularly important because the information is used to inform patients before surgery of the possible benefits and harms of treatment. In the perioperative period there is a risk of major complications, including death, anastomotic leakage and reoperation. During follow up, important outcomes include overall survival, tumour recurrence and also functional issues related to stoma formation and bowel, bladder and sexual dysfunction. A number of systematic reviews have considered these outcomes but data synthesis in meta-analyses has often been prevented by variable outcome measurement and definitions 5-7. Inconsistent reporting of outcomes on the basis of their ‘statistical significance’ may also lead to bias when data sets are pooled 8. Standardization of outcome assessment may help to overcome these problems but it is first necessary to determine which outcomes should be measured. Therefore, the aim of this study was to summarize and undertake an in-depth analysis of outcome reporting in CRC surgery.
Method
Data sources and searches
This systematic review adhered to a predefined protocol (available on request from the authors). The OVID SP versions of MEDLINE (1950 to present) and Embase (1980 to present) and the Cochrane Central Register of Controlled Trials (Issue 3, 2011) were searched using validated terms for ‘surgery’, ‘colorectal cancer’ and ‘randomized controlled trials or prospective studies’ separated by the Boolean operator ‘AND’ (Table 1). The search strategy was designed by two researchers (RNW and AM) and performed by RNW. The search was limited to studies of humans, 18 years of age or older, published in English between January 2009 and December 2010 inclusive. The length of the review period was restricted to 2 years to identify current clinical outcomes and because scoping work showed that this would identify over 100 studies (considered to be sufficient to capture the most important outcomes). Abstracts and Conference Proceedings were excluded because of the high probability of incomplete data. The websites of relevant medical journals and the reference lists of included articles were hand-searched to identify additional prospective studies. Citations were collated with Reference Manager 12 (Thomson Reuters, New York, NY, USA) and duplicates were removed.
| Search criteria | Search terms |
|---|---|
| Colorectal cancer |
1. exp Colonic Neoplasms/ 2. exp Rectal Neoplasms/ 3. ((colorect or colon or colonic or rect$) adj3 (cancer$ or tumo?r$ or neoplasm$ or carcinoma$ or adenocarcinoma$ or malignan$)).tw. 4. or/1–3 |
| Surgery |
1. exp Specialties, Surgical/ 2. surg$.tw. 3. operat$.tw. 4. intervention$.tw. 5. procedur$.tw. 6. resect$.tw. 7. or/1–6 |
| Randomized controlled trials/prospective studies |
1. randomized controlled trial.pt. 2. controlled clinical trial.pt. 3. randomized controlled trials.sh. 4. random allocation.sh. 5. double blind method.sh. 6. single-blind method.sh. 7. or/1–6 8. exp animals/ not human/ 9. 7 not 8 10. clinical trial.pt. 11. exp clinical trials/ 12. (clin$ adj25 trial$).ti,ab. 13. ((singl$ or doubl$ or trebl$ or tripl$) adj25 (blind$ or mask$)).ti,ab. 14. placebos.sh. 15. placebo$.ti,ab. 16. random$.ti,ab. 17. research design.sh. 18. or/10–17 19. 18 not 8 20. 19 not 9 21. comparative study.sh. 22. exp evaluation studies/ 23. follow up studies.sh. 24. prospective studies.sh. 25. (control$ or 29prospective$ or volunteer$).ti,ab. 26. or/21–25 27. 26 not 8 28. 27 not (9 or 20) 29. 9 or 20 or 28 |
- Adapted versions of the strategy were used in Embase and the Cochrane Central Register for Controlled Trials (data not shown). Results for ‘colorectal cancer’ were combined using ‘AND’ with search terms for ‘surgery’ ‘AND’ ‘randomized/prospective studies’.
Study selection
Titles and abstracts of identified publications were screened. Included were prospective studies reporting clinical outcomes of CRC surgery, with or without neoadjuvant or adjuvant chemotherapy or radiotherapy. Prospective studies were defined as reports from prospectively maintained national or regional cancer registries, or where there was an explicit statement that data were collected prospectively. Excluded were studies of (i) nonbiomedical interventions; (ii) noncurative, nonsurgical treatments; (iii) treatment of colorectal metastases; (iv) molecular and genetic prognostic studies; and (v) screening studies. Studies of more than one cancer site were permitted only if the outcomes for CRC were presented separately from those of the other cancer site. Full papers of potentially eligible abstracts were screened and only eligible articles were included. Primary articles were identified for studies with multiple citations.
Data extraction
A dedicated data-extraction form (designed by RNW and JMB) was used to record details from each paper of: (i) participant demographics; (ii) treatment intent; (iii) study design; (iv) interventions, comparators and co-interventions; (v) names of individual clinical outcomes reported and whether these were specified as primary or secondary end-points; and (vi) definitions provided of clinical outcomes. Clinical outcomes were defined as the exact terms used in CRC publications for end-points measured by clinicians or researchers 9 with the exclusion of any physiological, biochemical or haematological measurements. The latter were considered to be efficacy end-points and more relevant to early Phase I studies, whereas this review was primarily concerned with clinical effectiveness studies and outcomes relevant to patients. Patient-reported outcomes, defined as end-points provided by patients themselves 9, and healthcare economic outcomes were recorded but will be presented elsewhere. Surrogate outcomes, defined as measurements used as a substitute for clinically meaningful end-points (such as disease-free survival) 10, were permitted provided that they were not physiological, biochemical or haematological in nature. Demographics included the number, age and gender of participants. Surgical treatment intent was classified as aimed at cure, palliation, combined cure and palliation, or unclear. Included studies were grouped as RCTs or as nonrandomized prospective studies (case–control studies, cohort studies, prospective case series and cross-sectional studies). For the RCTs we recorded if any attempt at blinding of personnel was made and this was classified as satisfactory, unsatisfactory or unclear. Data extraction was checked by a second reviewer (ROF) for a sample of included articles (n = 50) and any discrepancies were discussed with the senior author (JMB) and resolved through consensus.
Outcome reporting and quality assessment
Where authors presented outcomes as ‘primary’ this was recorded and if no primary outcome was stated this was considered to be the outcome on which the sample-size calculation was based. All other outcomes were identified and recorded verbatim and categorized as secondary outcomes. Outcomes were considered as defined if text was provided of their meaning or if they were supported by a citation. Where no explanation or reference was present, outcomes were classified as not defined. All data were entered onto a Microsoft Access (Microsoft, Redmond, Washington, USA) database to facilitate data management and analyses. For the purpose of presenting the results, outcomes were grouped into seven broad categories (outcome domains) by two researchers (RNW and JMB). These categories were ‘mortality’, ‘complications’, ‘cancer-related outcomes’, ‘pathological outcomes’, ‘observer-rated symptoms’, ‘perioperative technical outcomes’ and ‘treatment pathway outcomes’. Discrepancies were discussed with a third member of the research team (DGJ or AMP). Full definitions of these outcome domains are provided in Table 2.
| Outcome domain | Definition | No. of individual outcomes in each domain |
|---|---|---|
| Mortality | Outcomes related to short- and long-term survival/death rates and cause of death | 84 |
| Complications | Forms of short- and long-term postoperative morbidity | 326 |
| Cancer-related outcomes | Measures of disease recurrence or disease progression | 66 |
| Pathological outcomes | Outcomes related to the reporting of the histological specimens from the surgical intervention | 50 |
| Observer-rated symptoms | Signs and symptoms of disease reported by clinicians and not those reported by patients themselves | 101 |
| Perioperative technical outcomes | Outcomes recorded directly in the operating theatre (e.g. operation time, blood loss) | 69 |
| Treatment pathway outcomes | Outcomes related to the flow of patients through the healthcare system (e.g. hospital stay, readmission) | 70 |
| Total | – | 766 |
Data analyses
Data analyses were conducted by RNW. The total number of individual outcomes was calculated and the frequency that each was reported was examined. Outcome definitions were extracted and compared. Inconsistent outcome reporting was considered to be present when publications failed to report the results of all the clinical outcomes specified in the methods. For each of the outcome domains a synopsis of outcome reporting is presented in the Results. In order to examine outcome reporting in more detail, the number of defined outcomes and the consistency of outcome reporting were compared between studies of different design (RCTs vs prospective nonrandomized studies). Data analyses were performed using Stata version 11 (StataCorp, College Station, Texas, USA).
Results
Titles and abstracts of 5644 papers were identified, 3517 abstracts were reviewed and 194 full papers of 556 839 patients were included (Fig. 1). The studies included are summarized in Table 3. There were 34 RCTs and 160 other prospective study designs. Attempts at blinding of study personnel were made in 16 of 34 RCTs. In eight of these the blinding was satisfactory, in eight it was unsatisfactory and in 18 it was unclear. Most publications were of single-centre studies, reporting outcomes from patients with predominantly rectal cancer. Less than 50% of papers reported obtaining Research Ethics Committee approval.
| All studies (n = 194) | Randomized trials (n = 34) | Nonrandomized studies (n = 160) | |
|---|---|---|---|
| Number of participants | 556 839 | 18 460 | 538 379 |
| Age range of participants (years) | 17–105 | 17–89 | 17–105 |
| Number of participating centres | |||
| Single | 123 (63.4) | 22 (64.7) | 101 (63.1) |
| Multiple | 71 (36.6) | 12 (35.3) | 59 (36.9) |
| Country | |||
| United Kingdom | 21 (10.8) | 4 (11.8) | 17 (10.6) |
| United States of America | 23 (11.9) | 2 (5.9) | 21 (13.1) |
| European Economic Area (excluding the UK) | 68 (35.1) | 14 (41.2) | 54 (33.8) |
| Other | 82 (42.3) | 14 (41.2) | 68 (42.5) |
| IRB or ethical approval reporteda | 94 (48.5) | 23 (67.6) | 71 (44.4) |
| Primary tumour site | |||
| Colon | 31 (16.0) | 3 (8.8) | 28 (17.5) |
| Rectum | 98 (50.5) | 19 (55.9) | 79 (49.4) |
| Mixed colon and rectum | 65 (33.5) | 12 (35.3) | 53 (33.1) |
| Treatment intent | |||
| Curative | 183 (94.3) | 34 (100.0) | 149 (93.1) |
| Mixed curative and palliative | 11 (5.7) | 0 (0.0) | 11 (6.9) |
| Surgical approach | |||
| Minimal access | 20 (10.3) | 4 (11.8) | 16 (10.0) |
| Hand-assisted minimal access | 1 (0.5) | 1 (2.9) | 0 (0.0) |
| Open | 11 (5.7) | 3 (8.8) | 8 (5.0) |
| Mixed | 34 (17.5) | 6 (17.6) | 28 (17.5) |
| Not reported or incomplete information reported | 128 (66.0) | 20 (58.8) | 108 (67.5) |
| Neoadjuvant treatmentb | |||
| Radiotherapy alone | 29 (14.9) | 11 (32.4) | 18 (11.3) |
| Chemotherapy alone | 2 (1.0) | 1 (2.9) | 1 (0.6) |
| Chemoradiotherapy | 63 (32.5) | 5 (14.7) | 58 (36.3) |
| None | 16 (8.2) | 3 (8.8) | 13 (8.1) |
| Not reported or incomplete information reported | 84 (43.3) | 14 (41.2) | 70 (43.8) |
| Adjuvant treatmentb | |||
| Chemotherapy or chemoradiotherapy | 77 (39.7) | 10 (29.4) | 67 (41.9) |
| None | 3 (1.5) | 0 (0.0) | 3 (1.9) |
| Not reported or incomplete information reported | 114 (58.8) | 24 (70.6) | 90 (56.3) |
- Values are presented as n (%) unless indicated otherwise.
- a IRB, Institutional Review Board.
- b Some studies included patients with or without neoadjuvant therapy, and some patients were undergoing different neoadjuvant or adjuvant treatments within the same study.
Outcome definitions and consistency of outcome reporting
There were 766 different clinical outcomes reported. These were categorized into the seven domains and the number of individual outcomes in each domain is summarized in Table 2. Overall, 15.8% of outcomes were defined in either the methods and/or the results of included publications, and outcomes were more frequently defined in RCTs than in other prospective studies (19.0% vs 14.9%, P = 0.015). Sixty-seven (34.5%) studies did not report all outcomes in the results that were prespecified in the methods. Inconsistent outcome reporting occurred at similar rates for RCTs (n = 13; 38.5%) and other prospective studies (n = 54; 33.8%; P = 0.617).
Mortality
There were 84 different individual clinical outcomes relating to mortality after surgery for CRC in 118 (60.8%) studies. Of these, 41 were related to in-hospital mortality, 42 concerned long-term survival and one was related to cause of death (Table 4). The most frequently recorded mortality outcome was ‘overall survival’, which was measured in 60 (30.9%) studies. However, ‘overall survival’ was defined in only 20 studies with six different definitions 11-30. ‘Disease-free survival’ was reported in 43 (22.2%) articles and defined in 27 with 16 unique definitions 13-15, 17-23, 26, 28, 30-44. The most commonly used measures of short-term operative mortality were ‘30-day mortality’ and ‘postoperative mortality’, which were reported in 16 (8.2%) and 14 (7.2%) of the included publications, respectively. ‘Cause of death’ was recorded in 22 (11.3%) articles.
| Individual mortality outcome | Frequency reported (n = 194) | Frequency defined | No. of different definitions |
|---|---|---|---|
| Short-term mortality terms | |||
| Death | 27 | 1 | 1 |
| Mortality | 17 | 8 | 7 |
| 30-day mortality | 16 | 2 | 2 |
| Postoperative mortality | 14 | 8 | 5 |
| Postoperative death | 6 | 3 | 2 |
| In-hospital mortality | 5 | 0 | 0 |
| Treatment-related death | 4 | 0 | 0 |
| Death within 30 days | 4 | 0 | 0 |
| 30-day postoperative mortality | 3 | 0 | 0 |
| In-hospital death | 3 | 0 | 0 |
| Long-term mortality terms | |||
| Overall survival | 60 | 20 | 6 |
| Disease-free survival | 43 | 27 | 16 |
| Survival | 20 | 4 | 3 |
| Cancer-specific survival | 13 | 6 | 6 |
| Relative survival | 7 | 6 | 4 |
| Recurrence-free survival | 4 | 3 | 3 |
| Local recurrence-free survival | 3 | 0 | 0 |
| Progression-free survival | 2 | 0 | 0 |
| Time to death | 2 | 0 | 0 |
| Tumour-free survival | 2 | 0 | 0 |
Complications
There were 326 individual clinical outcomes categorized in the complication domain. Some 292 were system-specific and 34 were generic terms. Of the system-specific complications, 98 (30.1%) were abdominal (including morbidity related to the anastomosis and stoma), 30 (9.2%) were wound-related, 28 (8.6%) were cardiovascular, 25 (7.7%) were haematological (including haemorrhage and thromboembolism), 22 (6.7%) were respiratory, 21 (6.4%) were urogenital, 20 (6.1%) were neurological, 18 (5.5%) were sepsis-related, 15 (4.6%) were dermatological and 15 (4.6%) were miscellaneous. The most frequently reported individual morbidities were ‘anastomotic leak’ (recorded in 72 studies and defined in 14, with each defined differently) 36, 45-57 (Table 5) and ‘wound infection’ (recorded in 40 studies but defined in none). Generic terms that were used were ‘postoperative complications’ (recorded in 27 studies and defined in eight, in seven different ways) and ‘complications’ (recorded in 20 studies and defined in five, in five different ways).
| Author (reference number) | Definition from text |
|---|---|
| Akiyoshi et al. 49 | Diagnosed by the presence of any of the following: gas or faecal discharge from the incisional wound, vagina or drain tract; faecal peritonitis; or intra-abdominal abscess or peritonitis along with an anastomotic defect verified by image study. Intraperitoneal abscess near the anastomotic site without an obvious faecal fistula was also diagnosed |
| Bertelsen et al. 46 | Peritonitis and a defect in the anastomosis, or discharge of pus from the rectum, or rectovaginal fistula, or the passage of faeces or gas from an abdominal drain |
| Biondo et al. 56 | Peritonitis caused by leakage from any staple line, pelvic abscess with or without radiologically proven leakage confirmed by clinical (digital palpation, inspection of drain contents) and/or radiological (rectal contrast study, CT scan) investigations, if suspected |
| Choi et al. 50 | (A) Pus or faecal discharge from the drain or (B) increased temperature (> 38°C) or leucocyte count or peritoneal irritation sign on physical examination or (C) rectovaginal fistula or abscess in the pelvic cavity |
| den Dulk et al. 36 | Clinically apparent leakage such as faecal discharge from a pelvic drain or abdominal wound, or radiologically, endoscopically or surgically proven anastomotic leakage in symptomatic patients such as those with peritonitis |
| Frye et al. 55 | Categorized as either major clinical leaks with signs of generalized peritonitis requiring emergency abdominal reoperation or minor leaks diagnosed on clinical signs, generally confirmed radiologically and managed expectantly without abdominal reoperation |
| Hyman et al. 51 | Defined clinically on consensus. Any postoperative fistula with the anastomosis was included as a leak. Postoperative abscess associated with extravasation of enteric contrast on an imaging study, those with significant peri-anastomotic air, or those shown to be in communication with the anastomosis after radiological drainage were recorded |
| Kube et al. [48.] | An anastomotic leak was recorded if any of the following criteria were met: (i) surgical procedure (re-laparotomy performed); (ii) radiological results (CT or X-ray contrast medium); or (iii) evidence from an autopsy |
| Lujan et al. 53 | Considered to be present when dehiscence was detected by digital examination or endoscopy and the patient had peritonitis, leakage of gas, faecal drainage or pelvic abscess |
| Matthiessen et al. [45.] | Peritonitis caused by leakage, pelvic abscess or discharge of faeces from the pelvic drain, and included from all stapler lines of the anastomosis without time limit. Rectovaginal fistula (anastomotic vaginal fistula) and pelvic abscess without a proven leakage mechanism were also included. Radiologically demonstrated leakage without symptoms was not included |
| Pigazzi et al. 54 | Anastomotic dehiscence verified by contrast enema or operative examination in patients showing a change in drainage material or signs of sepsis |
| Szynglarewicz et al. 52 | Present if any of the following features were noted: peritonitis caused by anastomotic dehiscence; feculent substances or gas from the pelvic drain; or pelvic abscess with demonstration of leakage by transrectal examination, endoscopy or imaging tests |
| Telem et al. 47 | Gross anastomotic dehiscence conferring faeculent or purulent peritonitis or evidence of communication between the anastomotic site and an intra-abdominal abscess, wound or fistula tract. Diagnosis of anastomotic leak was confirmed by intra-operative findings or extravasation of enteric contrast from the anastomotic site or radiographically |
| Ugolini et al. 57 | Clinical diagnosis without operation or reoperation for peritonitis |
Cancer-related outcomes
There were 66 individual outcomes, of which 22 (33.3%) described recurrence at a distant site, 13 (19.7%) described overall recurrence and 10 (15.2%) described local or locoregional recurrence. A further 13 (19.7%) individual outcomes concerned the time to recurrence, six (9.1%) concerned disease progression and two (3.0%) reported second primary malignancies. The two most frequently reported cancer-related individual outcomes were ‘local recurrence’ (recorded in 44 studies and defined in 19, in 15 different ways) 12, 13, 17, 22, 27, 33, 38, 58-69 and ‘distant recurrence’ (recorded in 23 studies and defined in four, in four different ways) 13, 18, 22, 61.
Pathological outcomes
There were 50 different individual pathological outcomes of which 25 (50.0%) described involvement of the resection margin, 11 (22.0%) were related to the pathological response to neoadjuvant chemotherapy or radiotherapy, nine (18.0%) documented the number of lymph nodes harvested, three (6.0%) described the success of sentinel node mapping and two (4.0%) reported the length of the resected specimen. The pathological outcomes reported in most publications were ‘number of retrieved lymph nodes’ (recorded in 40 studies but defined in none) and ‘circumferential resection margin’ (recorded in 32 studies and defined in eight, in four different ways) 36, 38, 40, 56, 70-73.
Observer-rated symptoms
Some 101 different clinical outcomes were categorized as observer-rated symptoms. These included symptoms related to gastrointestinal dysfunction (e.g. faecal incontinence, urgency, frequency, bloating or nausea and vomiting; n = 38; 37.6%), pain (n = 15; 14.9%), sexual dysfunction (n = 11; 10.9%), problems with passing urine (n = 10; 9.9%), anxiety/depression (n = 4; 4.0%), physical function (n = 4; 4.0%), fatigue (n = 4; 4.0%) and change in weight (n = 3; 3.0%). There were an additional 12 (11.9%) miscellaneous outcomes classified as observer-rated symptoms. ‘Diarrhoea’ was the most frequently reported observer-rated symptom and was reported in 23 studies and defined in one 74. ‘Nausea’ was the second most commonly reported observer-rated symptom (recorded in 14 studies but defined in none).
Perioperative technical outcomes
Sixty-nine individual outcomes described surgical data from the operating theatre. These included iatrogenic visceral injury (n = 22; 31.9%), stoma formation (n = 6; 8.7%), operative duration (n = 6; 8.7%), pelvic nerve preservation or injury (n = 5; 7.2%), conversion rate from minimal access to open surgery (n = 4; 5.8%), blood transfusion (n = 4; 5.8%), incision size (n = 3; 4.3%), sphincter-preservation rate (n = 2; 2.9%) and intra-operative blood loss (n = 1; 1.4%). There were several miscellaneous perioperative technical outcomes (n = 16; 23.2%). ‘Operation time’ (recorded in 44 studies and defined in five, in two ways) 29, 54, 75-77 and ‘blood loss’ (recorded in 32 studies and defined in one) 54 were the individual clinical outcomes most frequently reported.
Treatment pathway outcomes
Seventy individual outcomes were related to the flow of patients through the healthcare system. These included outcomes describing the duration of hospital stay (n = 10; 14.3%), time to return of bowel function (n = 9; 12.9%), time to oral intake (n = 8; 11.4%), failure to complete proposed treatment (n = 7; 10.0%), the need for postoperative nutritional support (n = 6; 8.6%), postoperative analgesic usage (n = 5; 7.1%), the destination to which patients were discharged or whether additional care in the community was required (n = 5; 7.1%), unplanned readmission to hospital (n = 3; 4.3%), delays to proposed treatment (n = 3; 4.3%), unscheduled reoperation (n = 2; 2.9%), stay on the intensive care unit (n = 2; 2.9%) and a further 10 (14.3%) miscellaneous outcomes. The most frequently reported of these clinical outcomes were ‘reoperation’ (recorded in 30 studies and defined in six, in five ways) 57, 78-82, ‘hospital stay’ (recorded in 25 studies and defined in one) 83 and ‘readmission’ (recorded in 19 studies and defined in 10, in six ways) 83-92. The time frame during which ‘reoperation’ was reported to occur was specified in just two studies (30 days in both). A range of time frames was reported for ‘readmission’, including 7 days (n = 1), 28 days (n = 1), 30 days (n = 6), 6 weeks (n = 1) and 90 days (n = 1).
Discussion
This systematic review has identified inconsistency in the selection, definition and reporting of clinical outcomes of CRC surgery. Some 766 different individual outcomes were reported in the included studies, with no single outcome reported in all. Just 15.8% of the outcomes were adequately defined and the same individual outcomes were often defined differently. Provision of outcome definitions was significantly better in RCTs than in nonrandomized prospective studies. Over one third of studies did not report all outcomes in the results that were prespecified in the methods. It is therefore recommended that a scientific approach to outcome reporting in studies of CRC surgery is developed and adopted nationally and internationally.
Previous work has considered the standard of outcome reporting in surgical oncology. A systematic review in 2012 of 122 articles reporting short-term outcomes of oesophagectomy identified 10 different measures of operative mortality and 210 unique complications 93. These outcomes were defined infrequently and outcomes with the same names often had different definitions. Another systematic review, in 2011, of breast reconstruction following mastectomy for cancer (n = 134 publications) identified 950 unique complications, of which just 19.3% were defined 94. This review also highlighted the impact of poor methodological rigor, including inconsistent outcome reporting and inadequate blinding of outcome assessors (leading to potential measurement bias), on the standard of outcome reporting. To date, no systematic review has considered the quality of reporting of all clinical outcomes of CRC surgery. However, heterogeneity in definitions of certain outcomes has been described previously, with one review identifying 56 different definitions of anastomotic leak 95. Wound infection is another frequent complication of CRC surgery and in the present review was not defined in a single study. This is despite an early review of 90 prospective studies that found 41 unique definitions of wound infection after general surgical procedures 96. Other authors have also highlighted inconsistency in the reporting of long-term outcomes of CRC surgery. Punt et al. 97 investigated survival measurement after CRC surgery and found nine different survival terms in common use. In this study, disease-free survival was defined in three different ways and an expert panel was used to reach consensus on the most appropriate definition. However, evidence from the present review suggests that this recommendation has not been widely adopted.
The number of different outcomes identified in this review, together with other papers that considered the measurement of surgical complications described above, all underline the need for more agreement about what should be measured in the evaluation of CRC surgery. One method for dealing with the large number of different outcomes identified is to develop a core outcome set and to mandate its use in randomized and nonrandomized studies, and in audits of CRC surgery. Additional outcomes may also be reported at researchers’ discretion so long as the core set is uniformly presented. Standardizing outcome reporting through the use of Core Outcome Set (COS) may improve data synthesis in meta-analyses, allow cross-study comparisons and ultimately facilitate clinical decision-making. However, COSs only go part way to addressing the problems surrounding outcome reporting. COS development resolves the issue of ‘which’ outcomes to measure in a trial, but does not define precisely ‘how’ outcomes should be assessed (e.g. which outcome definitions should be used and what measurement tools should be adopted). As such, future work is necessary to build on the development of COSs and to standardize ‘how’ outcomes are measured and reported.
The methodology of COS development has been pioneered by the Outcome Measures in Rheumatology (OMERACT) collaboration in arthritis 98, 99 and extended to a range of different diseases 100-102. The Core Outcome Measures in Effectiveness Trials (COMET) initiative is now championing the development and promotion of COS for all diseases 4. Although there is no consensus on best methodological practice in COS development, it is appropriate first of all to identify an exhaustive list of outcomes using systematic reviews and then to refine this into the COS using a validated method of achieving consensus (e.g. nominal groups, consensus conferences or the Delphi process). Patient involvement is also key to COS development and there is evidence that inclusion of patients in the consensus process results in the inclusion of different outcomes in the core set 103-105. A COS for use in studies of CRC surgery is recommended and this review comprises the first phase in its development. Future work will identify an exhaustive list of patient-reported outcomes and these will subsequently be amalgamated with the clinical outcomes. Delphi voting methodology will then be used to survey the views of key stakeholders (patients, clinicians and nurse specialists) to identify the outcomes of most importance to report in all studies of CRC surgery.
This study is not without limitations and should be interpreted in light of these. Our initial search was limited to studies published between January 2009 and December 2010. A more exhaustive search of the literature over a longer time period or inclusion of non-English publications or unpublished data may have revealed further clinical outcomes. However, the time frame was chosen to ensure that only current outcomes were identified and to exclude older, redundant, end-points. In addition, widening the search period and including non-English publications or unpublished data would have probably only identified further rare outcomes.
This study has shown that outcome reporting after surgery for CRC is inconsistent and lacks method. We recommend that a core outcome set should be developed for use in studies of CRC surgery. Future work will define the COS by reaching consensus on the most important outcomes amongst key stakeholders.
Acknowledgements
We would like to thank George Smith for his help and guidance with the design and conduct of this study.
Disclosure
This report is independent research arising from a Clinical Fellowship supported by the National Institute for Health Research. The views expressed in this publication are those of the authors and not necessarily those of the NHS, the National Institute for Health Research or the Department of Health.
Author contributions
RNW, ROF, AGKM, STB, KNLA and JMB (i) provided substantial contributions to the conception and design, acquisition of data, or analysis and interpretation of data; (ii) drafted the article; and (iii) gave final approval of the version to be published. AMP, DGJ, JEJ, JB, MGC, SJD, RH, RoH, RHK, DM, AO, AR and MGT (i) provided substantial contributions to the conception and design, acquisition of data, or analysis and interpretation of data; (ii) revised the manuscript critically for important intellectual content; and (iii) gave final approval of the version to be published.
