Quality and educational content of Douyin and TikTok short videos on early screening of rectal cancer
Lian-Shuo Li and Xiao-Juan Luo contributed equally to this work.
Declaration of conflict of interest: The authors declare no conflicts of interest.
Author contribution: Lian-Shuo Li and Xin-Peng Shu conducted the video searches and data extraction. Xiao-Juan Luo, Zi-Wei Li, Fei Liu, Xu-Rui Liu, and Yue Tong performed quality assessment and content scoring. Quan Lv, Xiao-Yu Liu, Wei Zhang, and Dong Peng analyzed the data. Lian-Shuo Li, Xin-Peng Shu, and Xiao-Juan Luo wrote the original draft. All authors commented on and edited the original manuscript and read and approved the final version.
Financial support: This study was supported by the CQMU Program for Youth Innovation in Future Medicine (W0190).
Abstract
Background and Aim
The aim of this study was to assess the quality and content of videos on Douyin and TikTok for their educational role on early screening of rectal cancer (RC).
Methods
We conducted a search for videos related to RC on the Douyin and TikTok applications on 20 April 2023. The search was conducted in Chinese on Douyin and in English and Japanese on TikTok. A sample of the first 100 videos recommended by the software was selected for each language group. The content of the videos was evaluated using a content scorecard, while the quality of the videos was assessed using DISCERN. Subsequently, we conducted two partial correlations: one between the DISCERN score and the number of likes, and another between the video content score and the number of likes.
Results
This study encompassed a total of 89 Chinese, 54 English, and 51 Japanese videos. After selection, 78 Chinese, 38 English, and 25 Japanese videos were identified to contain content related to early screening for RC, prompting further quality assessment. Notably, videos in the Chinese language showed the highest DISCERN score (P < 0.05). In terms of partial correlation analysis, it was observed that both the content score and DISCERN score did not show a significant correlation with the number of likes (P > 0.05).
Conclusion
In terms of quality score and content score, the Chinese videos on Douyin show superiority over the English and Japanese videos on TikTok. However, there is potential for improving the overall appeal of the Chinese videos.
Introduction
Global incidence and mortality of colorectal cancer (CRC) ranked third and second, respectively. More than 1.9 million new CRC cases and 935 000 deaths were estimated to occur in 2020.1 For rectal cancer (RC), the etiology is multifactorial, with age as well as genetic and environmental factors playing a major role in the development of the disease.2, 3 As in the case of low-risk T1 tumors, the current standard treatment for RC remains total mesorectal excision. Whether to combine neoadjuvant (chemotherapy) radiotherapy depends on the specific stage of the tumor.4-6
With the popularity of smartphones, social media software is beginning to have an unprecedented impact on people's lives. Short videos, a new form of social media, are popular worldwide because of their short duration and diverse content. TikTok, the world's largest short video platform, had over 1 billion monthly active users, of which 150 million were in the United States.7 TikTok is not allowed to be used in China because of the internet regulation. Douyin, which is also owned by ByteDance, was available as a Chinese version of TikTok. Douyin had 420 million daily active users, and the per capita daily use of Douyin was 110 min.8
Because of the shortage of medical resources and the cumbersome and expensive medical treatment process, more and more people are chosing to solve their health problems through the Internet. Therefore, short videos have tremendous potential for public health education and popularization because of their large user base and rapid dissemination. Several studies have investigated the effects of TikTok on health popularization in different diseases.9-12 Early screening and diagnosis of CRC has led to sustained reductions in CRC incidence and mortality over the past 20 years in people over 50 years of age because of the relatively well-defined precancerous lesions.13-15 The American Cancer Society also recommends that adults aged 45 years and older with an average risk of CRC undergo regular screening with either a high-sensitivity, stool-based test or a structural (visual) examination.16 This also illustrates the need for CRC screening education. Douyin and TikTok could be used as tools to popularize the importance of early screening for CRC. However, the characteristics of Douyin and TikTok videos about the type of early screening for RC are still unclear. It is also unclear whether Douyin and TikTok could play a role in the promotion of early screening for RC. The aim of this study was to assess the quality and content of short videos on Douyin and TikTok for their educational role in early screening of RC.
Methods
Search of short videos
We searched for RC-related short videos on the Douyin and TikTok mobile apps on 20 April 2023. The following keywords related to RC were used for the search: “直肠癌” on Douyin (Chinese), “rectum cancer” on TikTok (English), and “直腸ガン” on TikTok (Japanese). Searches were performed on Douyin in Chinese and on TikTok in English and Japanese. We selected the first 100 short videos recommended by the mobile phone software as samples for each group.
Inclusion and exclusion criteria
To select suitable videos, the inclusion criteria were as follows: (i) the content of the video was related to RC; (ii) the medical knowledge about RC in the video was correct; (iii) the language of the videos was not limited to Chinese, English, and Japanese. The exclusion criterion was that there should be no duplication of the content.
Elimination and assessment short videos
All short videos were viewed and evaluated by the same two authors. Both authors are specialists in gastrointestinal surgery. And disagreements were resolved by a third senior author. We judged the accuracy of the content based on National Comprehensive Cancer Network Guidelines.17
First, we excluded other languages and unrelated short videos according to the inclusion and exclusion criteria. Then we collected relevant information about the remaining videos, including the source of the video, length of the video, the number of times the video was accessed on the Internet, the number of likes, the number of comments, and the number of favorites.
To date, there are no high-quality tools to evaluate the content of short videos on RC. We designed a content scorecard for RC short videos. The details are shown in Table 1. We divided the video content into seven sections and scored each section according to the volume of the content. The seven sections were definition, epidemiology, etiology, symptoms, screening and diagnosis, treatment, and outcome. For every section, the scoring was as follows: 0 point for no content, 1 point for little content, 2 points for part content, and 3 points for comprehensive content. Finally, the content score of each part of each video was obtained, and then the average score of different parts of the video in different languages was calculated.
| Content | Description | Score |
|---|---|---|
| Definition | — | No content: 0 points Little content: 1 point Partly content: 2 points Comprehensive content: 3 points |
| Epidemiology | Incidence Common onset age group |
|
| Etiology | Heredity Polyps Dietary habits Habits of life |
|
| Symptoms | Changes in defecation habits Blood in the stool Weight loss Abdominal pain Anemia |
|
| Screening and diagnosis | Bloody stool test FIT-DNA test Colonoscopy Abdominal CT |
|
| Treatment | Surgery Chemotherapy Radiotherapy Immunotherapy |
|
| Outcomes | Tumor staging Complications |
- CT, computed tomography; FIT, fecal immunochemical test.
We divided the videos into useful and useless categories based on whether they had content related to early screening of RC. Then, we evaluated the quality of useful videos.
We used the DISCERN instrument18 to assess the quality of the content of short videos. This instrument is increasingly used as a simple and effective tool for assessing health content on the Internet.19-22 The DISCERN consisted of 16 questions, which are divided into three sections, and the score for each question is 1–5 points. Questions 1–8 are on the reliability of the content, questions 9 to 15 are on the quality of treatment selection, and question 16 is on the overall value of the content. Then we calculated the mean and SD of DISCERN scores for the videos in different languages.
Finally, we performed two partial correlations: one between the DISCERN score and the number of likes, and the other between the video content integrity score and the number of likes.
Statistical analysis
Frequencies and percentages were calculated for categorical variables, and means or medians (SDs or ranges) were calculated for continuous variables. One-way analysis of variance was used to compare differences between groups. Partial correlation coefficients were used to compare correlations between datasets. A P-value <0.05 was considered statistically significant. SPSS version 27 was used for data analysis in this study.
Results
Selection of short videos
A total of 300 videos were collected from Doyin and TikTok apps. After viewing, 106 videos were ruled out for unrelated content or being in other languages. Finally, 194 short videos were included in this study (Fig. 1).
Characteristics of short videos
Of the 106 videos, 79 (88.8%) Chinese, 20 (37.0%) English, and 7 (13.7%) Japanese videos came from health professionals or health organizations. And 6 (6.7%) Chinese, 33 (61.1%) English, and 43 (84.3%) Japanese videos were uploaded by private users. Only 4 (4.5%) Chinese, 1 (1.9%) English, and 1 (2.0%) Japanese videos were uploaded by news media. The median length of the videos was 63 s (4–260 s) for Chinese, 56.5 s (7–237 s) for English, and 15 s (9–542 s) for Japanese. The median time on the Internet of the videos was 355 days (1–1729 days) for Chinese, 152.5 days (9–993 days) for English, and 115 days (13–964 days) for Japanese. Median likes on Chinese, English, and Japanese videos were 2624 (2–795K), 357.5 (2–337.6K), and 85 (10–4751), respectively. For the median number of comments, 347 (0–58K), 36.5 (0–11.5K), and 7 (0–268) were in Chinese, English, and Japanese, respectively. The median number of favorites of videos was 282 (0–102K) Chinese, 31.5 (0–8254) English, 2 (0–154) Japanese. Finally, 78 Chinese, 38 English, and 25 Japanese videos were considered useful. The details are given in Table 2.
| Contents | Category | Douyin (China) (n = 89) | TikTok (America) (n = 54) | TikTok (Japan) (n = 51) |
|---|---|---|---|---|
| Source | Health professionals or health organizations | 79 (88.8%) | 20 (37.0%) | 7 (13.7%) |
| Private users | 6 (6.7%) | 33 (61.1%) | 43 (84.3%) | |
| News media | 4 (4.5%) | 1 (1.9%) | 1 (2.0%) | |
| Characteristics | Video length (s) | 63 (4–260) | 56.5 (7–237) | 15 (9–542) |
| Time on the internet (days) | 355 (1–1729) | 152.5 (9–993) | 115 (13–964) | |
| Number of likes | 2624 (2–795K) | 357.5 (2–337.6K) | 85 (10–4751) | |
| Number of comments | 347 (0–58K) | 36.5 (0–11.5K) | 7 (0–268) | |
| Number of favorites | 282 (0–102K) | 31.5 (0–8254) | 2 (0–154) | |
| Utility | Useful | 78 (87.6%) | 38 (70.4%) | 25 (49.0%) |
| Useless | 11 (12.4%) | 16 (29.6%) | 26 (51.0%) |
- Values are shown as cases (percentage) or median(range).
Content of short videos
Each useful video was scored according to the content scorecard. The mean score for definition was 0.103 for Chinese, 0.162 for English, and 0.040 for Japanese videos. As for epidemiology, the mean scores was 0.244 for Chinese, 0.108 for English, and 0.020 for Japanese. The mean score for etiology was 0.487 for Chinese, 0.297 for English, and 0.080 for Japanese. The mean score for symptoms for Chinese, English, and Japanese videos was 0.910, 1.000, and 0.400, respectively. For screening and diagnosis, the mean score was 0.538 for Chinese, 0.432 for English, and 0.080 for Japanese. And the mean score for treatment was 0.295 for Chinese, 0.351 for English, and 0.640 for Japanese. The mean score for outcomes was 0.179 for Chinese, 0.010 for English, and 0.200 for Japanese. The radar map of the content score is shown in Figure 2.
DISCERN score and correlation analysis
The DISCERN scores of Chinese, English, and Japanese videos were 34.91 ± 4.59, 31.51 ± 4.05, and 28.24 ± 5.09, respectively. Chinese videos had the highest DISCERN score (P < 0.05). English videos had the second highest DISCERN score, but there was no significant difference between English and Japanese videos (P = 0.19). There was no significant difference in the partial correlation between the DISCERN score and the number of likes, excluding the influence of the number of days online. No significant difference also was found in the partial correlation between the video content score and the number of likes, excluding the influence of the number of days online. The details are shown in Table 3.
| Douyin (China) | TikTok (USA) | TikTok (Japan) | |
|---|---|---|---|
| DISCERN† | 34.91 ± 4.59 | 31.51 ± 4.05‡ | 28.24 ± 5.09§ |
| r1 | −0.28 (P = 0.88) | 0.007 (P = 0.97) | 0.27 (P = 0.20) |
| r2 | 0.54 (P = 0.64) | −0.68 (P = 0.69) | 0.24 (P = 0.26) |
- † Values are shown as mean ± SD.
- ‡ Discern scores between USA and China showed statistical difference.
- § Discern scores between Japan and China showed statistical difference.
- r1 is the partial correlation between the DISCERN score and the number of likes, excluding the influence of the number of days online. r2 is the partial correlation between the video content score and the number of likes, excluding the influence of the number of days online.
Discussion
A total of 194 short videos were included in this study, including 89 Chinese, 54 English, and 51 Japanese. After scoring the video content, we found that videos in both English and Chinese scored highest in the symptoms section and the screening and diagnosis section. As for videos in Japanese, the symptoms and treatment sections scored the highest. This suggested that both the Chinese and English videos focused on the symptoms, screening, and diagnosis of RC. However, the Japanese videos focused on the symptoms and treatment of RC. And a total of 78 Chinese, 37 English, and 25 Japanese videos were considered useful. Videos in Chinese had the highest DISCERN score. As for partial correlation analysis, we found that, after excluding the influence of time on video presence, both the content score and the DISCERN score did not correlate strongly with the number of likes.
Although the annual rate of death from CRC in the United States has declined by 1.8% from 2012 to 2017, the mortality rate remains high, that is, 13.9 per 100 000,23 which is not a satisfactory outcome. Fortunately, as Islami et al.24 have pointed out, more than one-half of the cases of CRC were attributable to modifiable risk factors such as smoking, unhealthy diet, high alcohol consumption, physical inactivity, and excess body weight, which are preventable. Miller et al.25 found that the 5-year survival rate of patients with RC was slightly higher than that of patients with colon cancer, reflecting a higher proportion of patients with RC with earlier tumor stages at diagnosis. The importance of early screening for RC is self-evident.
Short videos, a new form of social media, are rapidly gaining popularity worldwide due to their short duration and strong entertainment properties. TikTok and Douyin, the world's largest short-video platforms, showed the potential value of health education. Zhang et al.26 found that the quality and reliability of the cosmetic surgery videos on Douyin were satisfactory. However, Lan Yao et al.27 found that the validity and quality of short videos on chronic kidney disease on Douyin were still unsatisfactory. Hu et al.28 showed that the videos on gastric cancer in English and Japanese on TikTok were unsatisfactory, but videos in Chinese on Douyin were the opposite. Currently, there are no studies focusing on short videos on RC. Thus, we aimed to investigate the quality and educational role of short videos on early screening for RC in Douyin and TikTok in three languages.
In our study, we found that videos in different languages differed in several ways. Most of the videos in Chinese were uploaded by medical professionals or organizations. Most were on RC symptoms and screening, and the DISCERN score was also the highest. This is due in large part to the release by Douyin in March 2021 of the “Statements for the Management of Douyin Medical Content”.29 It specified that only those who provided documentary evidence would be certified as medical professionals or institutions. Only certified individuals or organizations could publish health-related content, and certifications would be revoked if any violations were found. This really helped improve the quality and reliability of the videos. The scores for videos in Chinese were unbalanced in terms of content, which were the same for English and Japanese videos. Videos in Chinese often explained symptoms and screening for RC in a formal and serious way, with high quality but little attention. Some videos aimed to attract attention by telling awful patient experiences, which were theoretically useful for RC prevention and early screening but often counterproductive. Too much negative content will not get very much attention or good publicity. So, the number of likes did not correlate with the content score or DISCERN score.
For English and Japanese videos on TikTok, they were mainly uploaded by private users, which resulted in less useful and lower quality videos. Some studies had also found that the quality of videos on TikTok for other diseases was not satisfactory.30-33 Although some studies34, 35 pointed to an increasing number of medical professionals using social media, there is still considerable scope for medical professionals in the United States and Japan to use social media for health education. We found that English videos were more often about symptoms reported by patients with RC, whereas Japanese videos were more often about shared treatment processes. Such videos are often unprofessional and unobjective, but they were also more amiable and sympathetic when presented by patients. These videos usually got more likes, and this might explain the negative correlation between DISCERN scores and the number of likes in English videos.
Short videos often fail to explain all aspects of RC because of their short duration, and this is why the content scores were unbalanced. In addition, short videos that receive more likes can usually get more pushes and even became popular. Popular videos on Douyin were viewed even as many as 400 billion times a month,36 which would lead to better health education results. As entertainment software, it is not easy to conduct health education on Douyin and TikTok. A good health education video first needs to ensure quality and reliability and then to balance the content and engagement within a limited duration.
To the best of our knowledge, this is the first study to access the quality and education role of short videos on early screening for RC in Douyin and TikTok in three languages. However, this study also has some limitations. First, we used only content and DISCERN scores to assess the videos, which may have led to insufficient analysis of the videos. Second, we chose only Douyin and TikTok apps to assess the videos on RC, whereas content on other social media software may lead to different results and conclusions. Third, future uploading of new short videos and deleting of original ones may lead to different conclusions.
Conclusions
In terms of quality score and content score, Chinese videos on Douyin showed superiority over the English and Japanese videos on TikTok. However, there is potential for enhancing the overall appeal of Chinese videos.
Acknowledgments
The authors are grateful to Dr. Kang Tang, who graduated from Chiba University, for his help with the translation and evaluation of Japanese videos.
Open Research
Data availability statement
The data was accessed from the database.
