RETRACTED: Treatment of diabetic foot ulcers with external application of Chinese herbal medicine: An overview of overlapping systematic reviews
Abstract
This overview of systematic reviews (SRs) and meta-analysis (MAs) aimed to systematically collate, appraise and synthesize evidence for the treatment of diabetic foot ulcers (DFUs) with the external application of Chinese herbal medicine (CHM). SRs/MAs of external application of CHM for DFUs were collected by searching Cochrane Library, Web of science, CNKI, PubMed, VIP, Embase and Wanfang. Two independent reviewers carried out the literature selection and data extraction. Subsequently, AMSTAR-2 tool, PRISMA, and GRADE system were applied by two reviewers independently to evaluate the methodological quality, reporting quality, and evidence quality of the included studies, respectively. Eight SRs/MAs met the eligibility criteria and were included. According to AMSTAR-2, a very low methodological quality assessment was given to the included SRs/MAs due to the flaws of items 2, 4 and 7. The PRISMA system identified protocol and registration weaknesses, as well as search method weaknesses. With GRADE, no high-quality evidence was identified to support the role of external application of CHM for DFUs, and the quality of evidence for the vast majority of outcomes was rated as low or moderate. In conclusion, low- to moderate-quality evidence supports the promise of external application of CHM for the treatment of DFUs. Due to the limitations of the evidence supporting external application of CHM for DFUs, rigorously designed and larger samples of high-quality studies are needed going forward before broad recommendations can be made.
1 BACKGROUND
Diabetic foot ulcers (DFUs) are a major contributor to both rising healthcare costs and worldwide mortality rates among diabetic individuals.1 DFUs frequently occur as a precursor to infection and even amputation since they are typically chronic and persistent in their course.2 Society bears a large burden of 80.80%, 69.01% and 28.94% survival rates among patients with DFUs over 1, 2 and 5 years.3 Globally, improving the management of DFUs is a top concern.4 Restoring blood flow, increasing metabolism, lowering stress, preventing infection and providing health education are all part of the standard wound treatment (SWT) for DFUs.5 Nevertheless, these approaches continue to be insufficiently effective in accelerating wound healing.
DFUs has traditionally been treated in China with Chinese herbal medicine (CHM).6 External treatment, in which drugs are applied directly to the lesion, are considered to be more likely to act on the lesion and possess safer characteristics compared with oral administration.7, 8 The number of overlapping systematic reviews (SRs)/meta-analyses (MAs) published on DFUs treated with external application of CHM.9-16 However, their results were inconsistent and vary in quality, which poses a challenge for the external application of CHM as a clinical treatment for DFUs. Therefore, this overview aimed to systematically collate, appraise and synthesize evidence from SRs/MAs on the management of DFUs with external application of CHM.
2 METHODS
2.1 Inclusion criteria
Studies were screened for inclusion using the following criteria: (a) only SRs/MAs were included; (b) patients diagnosed with DFUs; (c) the experimental intervention was external application of CHM plus SWT, with SWT alone for the control intervention; (d) outcomes included effective rate, healing rate, healing time and wound size reduction.
Studies were screened for exclusion using the following criteria: (a) duplicating published literature; (b) studies with incomplete data; (c) animal experiments, letters or commentaries.
2.2 Search strategy
A systematic search was carried out in Cochrane Library, PubMed, Web of science, CNKI, VIP, Embase and Wanfang from database establishment to 20 November 2023. Systematic review, diabetic foot, diabetes mellitus and herbal medicine were applied as search terms. Table 1 shows the search strategy for PubMed.
| Query | Search term |
|---|---|
| # 1 | Diabetic foot [Mesh] |
| # 2 | Diabetic foot [Title/Abstract] OR foot ulcer [Title/Abstract] OR plantar ulcer [Title/Abstract] OR diabetic feet [Title/Abstract] |
| # 3 | #1 OR #2 |
| # 4 | Diabetes mellitus [Mesh] |
| # 5 | Diabetes mellitus [Title/Abstract] OR diabet* [Title/Abstract] |
| # 6 | #4 OR #5 |
| # 7 | Herbal medicine [Mesh] |
| # 8 | Herbal medicine [Title/Abstract] OR traditional medicine [Title/Abstract] OR traditional Chinese medicine [Title/Abstract] OR Chinese medicine [Title/Abstract] |
| # 9 | #7 OR #8 |
| # 10 | Meta-analysis as Topic [Mesh] |
| # 11 | Meta-analyses [Title/Abstract] OR meta analyses [Title/Abstract] OR meta-analysis OR systematic review [Title/Abstract] |
| # 12 | #10 OR #11 |
| # 13 | #3 AND #6 AND #9 AND #12 |
2.3 Data collection and extraction
Two reviewers independently carried out data collection and extraction. The abstract and title of the literature were first read for inclusion, followed by a reading of the full text of potentially eligible reviews to judge the eligibility of reviews. For the included reviews, year of publication, the first author, country, sample size, quality of included trials, interventions, methods of quality assessment and pooled results were extracted.
2.4 Quality assessment
The methodological quality was evaluated by two reviewers independently with AMSTAR-2 tool.17 AMSTAR-2 contains 16 entries, and 7 of which are key items. Each item can be rated on three levels, which are ‘yes’, ‘partially yes’ and ‘no’. Methodological quality is very low when more than 1 critical item is not met, low when 1 critical item is not met, moderate when more than 1 non-critical item is not met and high when no or only 1 non-critical item is not met.17
The reporting evidence was evaluated by two independent reviewers with PRISMA checklists.18 Based on the completeness of the reported information, each checklist can be rated as yes (fully reported), partially yes (partially reported) or no (not reported).
The quality of evidence was evaluated by two independent reviewers with GRADE system.19 Evidence could be downgraded owing to imprecision, publication bias, risk of bias, indirectness or inconsistency. Quality of evidence can be rated on three levels, which are ‘very low’, ‘low’, ‘moderate or ‘high’.19
3 RESULTS
3.1 Literature selection
The initial literature search obtained 154 articles (Figure 1), and after removing duplicates, 113 articles remained. Subsequently, 99 articles were excluded by screening their titles and abstracts, and six articles were excluded by reading their full text. Eight studies finally met the inclusion criteria for this overview.9-16
3.2 Study characteristics
Between 2013 and 2023, the included studies were all carried out in China and published during this period. Sample sizes for SRs/MAs varied widely (630 to 3758). The control group was treated by SWT, and the treatment group added external application of CHM to the control. More details of the included reviews are presented in Table 2.
| Author, year | Country | Sample size | Treatment intervention | Control intervention | Quality assessment | Conclusion summary |
|---|---|---|---|---|---|---|
| Sun, 2013 | China | 15 (945) | SWT + external application of CHM | STW | Cochrane criteria | CHM dressings were superior to the routine treatment in the total effective rate and wound closure time. More high-quality trails are required to verify this conclusion due to the low quality and small scale of the included studies. |
| Huang, 2015 | China | 19 (1425) | SWT + external application of CHM | STW | Jadad | The local application of CHM in treating DFUs show better effect than the control group, so it is worth of further study and summary. |
| Wang, 2022 | China | 16 (1404) | SWT + external application of CHM | STW | Cochrane criteria | CHM soaking method is an effective and safe method for the treatment of DFUs; however, there is publication bias as well as no high-quality research, so high-quality large-sample multicentre randomized controlled trials should be carried out. |
| Zhou, 2022 | China | 24 (2017) | SWT + external application of CHM | STW | Cochrane criteria | Compared with SWT, therapy of encircling lesion with CHM is more efficient and safer in DFUs. Further large-sample and high-quality clinical trials are required to provide reliable evidence due to the low quality and small samples included in the systemic review. |
| Yang, 2023 | China | 34 (3758) | SWT + external application of CHM | STW | Cochrane criteria | Our results revealed that common CHM applied externally could significantly improve the clinical efficacy on DFUs comparing with the basic treatment. |
| Wu, 2023 | China | 38 (3316) | SWT + external application of CHM | STW | Cochrane criteria | CHM foot bath can improve the overall effect on DFUs, it can promote wound healing and has good safety. |
| Dong, 2023 | China | 16 (1325) | SWT + external application of CHM | STW | Cochrane criteria | The external medicinal liquid application of CHM can effectively improve the effective rate and healing rate of DFUs, and shorten the wound healing time and reduce inflammation. |
| Liu, 2023 | China | 9 (630) | SWT + external application of CHM | STW | Cochrane criteria | The treatment of DFUs with CHM dressing can improve the overall clinical efficiency, significantly reduce the ulcer area and promote the healing of DFUs wounds with good safety. However, due to the small amount of included studies, the above findings need to be verified by more high-quality studies. |
3.3 Methodological quality
The AMSTAR-2 assessment of the methodological quality is shown in Table 3. Deficiencies that undermine the quality of the methodology included items 2 (lack of the protocol being registered), 4 (inadequate search strategies for each database) and 7 (absence of a list of trials that were excluded). Therefore, all included studies were rated as critically low methodological quality.
| Author, Year | AMSTAR-2 | Quality | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Q1 | Q2 | Q3 | Q4 | Q5 | Q6 | Q7 | Q8 | Q9 | Q10 | Q11 | Q12 | Q13 | Q14 | Q15 | Q16 | ||
| Sun, 2013 | Y | N | Y | PY | Y | Y | N | Y | Y | Y | Y | Y | Y | Y | Y | Y | CL |
| Huang, 2015 | Y | N | Y | PY | Y | Y | N | Y | Y | Y | Y | Y | Y | Y | Y | Y | CL |
| Wang, 2022 | Y | N | Y | PY | Y | Y | N | Y | Y | Y | Y | Y | Y | Y | Y | Y | CL |
| Zhou, 2022 | Y | N | Y | PY | Y | Y | N | Y | Y | Y | Y | Y | Y | Y | Y | Y | CL |
| Yang, 2023 | Y | N | Y | Y | Y | Y | N | Y | Y | Y | Y | Y | Y | Y | Y | Y | CL |
| Wu, 2023 | Y | N | PY | PY | Y | Y | N | Y | Y | Y | Y | Y | Y | Y | Y | Y | CL |
| Dong, 2023 | Y | N | Y | PY | Y | Y | N | Y | Y | Y | Y | Y | Y | Y | Y | Y | CL |
| Liu, 2023 | Y | N | Y | PY | Y | Y | N | Y | Y | Y | Y | Y | Y | Y | Y | Y | CL |
- Abbreviations: CL, critically low; N, No; PY, partial Yes; Y, Yes.
3.4 Methodological quality
A PRISMA compliance rate of 0% was reported for Q5, 12.50% for Q8, and the other checklists were relatively comprehensive (Table 4). As a whole, the included SRs/MAs were fairly completely described in their Titles, Abstracts, Introductions, Results, Discussions and Funding sections. Although the report was complete, unreported details in Method hampered its quality.
| Section/topic | Items | Sun, 2013 | Hang, 2015 | Wang, 2022 | Zhou, 2022 | Yang, 2023 | Wu, 2023 | Dong, 2023 | Liu, 2023 | Compliance (%) |
|---|---|---|---|---|---|---|---|---|---|---|
| Title | Q1. Title | Y | Y | Y | Y | Y | Y | Y | Y | 100% |
| Abstract | Q2. Structured summary | Y | Y | Y | Y | Y | Y | Y | Y | 100% |
| Introduction | Q3. Rationale | Y | Y | Y | Y | Y | Y | Y | Y | 100% |
| Q4. Objectives | Y | Y | Y | Y | Y | Y | Y | Y | 100% | |
| Methods | Q5. Protocol and registration | N | N | N | N | N | N | N | N | 0% |
| Q6. Eligibility criteria | Y | Y | Y | Y | Y | Y | Y | Y | 100% | |
| Q7. Information sources | Y | Y | Y | Y | Y | Y | Y | Y | 100% | |
| Q8. Search | PY | PY | PY | PY | Y | PY | PY | PY | 12.5% | |
| Q9. Study selection | Y | Y | Y | Y | Y | Y | Y | Y | 100% | |
| Q10. Data collection process | Y | Y | Y | Y | Y | Y | Y | Y | 100% | |
| Q11. Data items | Y | Y | Y | Y | Y | Y | Y | Y | 100% | |
| Q12. Risk of bias in individual studies | Y | Y | Y | Y | Y | Y | Y | Y | 100% | |
| Q13. Summary measures | Y | Y | Y | Y | Y | Y | Y | Y | 100% | |
| Q14. Synthesis of results | Y | Y | Y | Y | Y | Y | Y | Y | 100% | |
| Q15. Risk of bias across studies | Y | Y | Y | Y | Y | Y | Y | Y | 100% | |
| Q16. Additional analyses | Y | Y | Y | Y | Y | Y | Y | Y | 100% | |
| Results | Q17. Study selection | Y | Y | Y | Y | Y | Y | Y | Y | 100% |
| Q18. Study characteristics | Y | Y | Y | Y | Y | Y | Y | Y | 100% | |
| Q19. Risk of bias within studies | Y | Y | Y | Y | Y | Y | Y | Y | 100% | |
| Q20. Results of individual studies | Y | Y | Y | Y | Y | Y | Y | Y | 100% | |
| Q21. Synthesis of results | Y | Y | Y | Y | Y | Y | Y | Y | 100% | |
| Q22. Risk of bias across studies | Y | Y | Y | Y | Y | Y | Y | Y | 100% | |
| Q23. Additional analysis | Y | Y | Y | Y | Y | Y | Y | Y | 100% | |
| Discussion | Q24. Summary of evidence | Y | Y | Y | Y | Y | Y | Y | Y | 100% |
| Q25. Limitations | Y | Y | Y | Y | Y | Y | Y | Y | 100% | |
| Q26. Conclusions | Y | Y | Y | Y | Y | Y | Y | Y | 100% | |
| Funding | Q27. Funding | Y | Y | Y | Y | Y | Y | Y | Y | 100% |
- Abbreviations: N, no; PY, partial yes; Y, yes.
3.5 Evidence quality
Evidence quality of 19 outcome indicators from included reviews was summarized by the GRADE system (Table 5). Overall, the quality of evidence was generally moderate to very low, with 4 moderate 12 low and 3 very low, respectively. Risk of bias was the first element that leads to downgrading the quality of evidence, followed by inconsistency and imprecision.
| Author; year | Outcomes | No of trails | Certainty assessment | Sample | Relative effect (95% CI) | Quality | |||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Limitations | Inconsistency | Indirectness | Imprecision | Publication bias | Experimental | Control | |||||
| Sun, 2013 | Healing rate | 15 | −1 | 0 | 0 | 0 | 0 | 509 | 436 | OR 3.82 (2.79, 5.23) | M |
| Healing time | 3 | −1 | 0 | 0 | −1 | 0 | 107 | 71 | MD −8.12 (−16.04, −0.19) | L | |
| Huang, 2015 | Effective rate | 19 | −1 | 0 | 0 | 0 | 0 | 808 | 617 | OR 4.96 (3.54, 6.96) | M |
| Wang, 2022 | Effective rate | 16 | −1 | −1 | 0 | 0 | 0 | 716 | 688 | RR 1.22 (1.12, 1.33) | M |
| Healing time | 3 | −1 | −1 | 0 | 0 | 0 | 95 | 95 | MD −9.47 (−13.65, −5.28) | L | |
| Wound size reduction | 4 | −1 | −1 | 0 | 0 | 0 | 164 | 164 | MD −2.41 (−3.87, −0.94) | L | |
| Zhou, 2022 | Effective rate | 8 | −1 | 0 | 0 | 0 | 0 | 315 | 225 | RR 1.14 (1.07, 1.22) | M |
| Yang, 2023 | Effective rate | 21 | −1 | −1 | 0 | 0 | 0 | 1590 | 1223 | RR 1.31 (1.20, 1.42) | L |
| Healing rate | 29 | −1 | −1 | 0 | −1 | 0 | 1914 | 1537 | RR 1.84 (1.56, 2.17) | CL | |
| Healing time | 9 | −1 | −1 | 0 | 0 | 0 | 302 | 305 | SMD −2.51 (−3.39, −1.63) | L | |
| Wu, 2023 | Effective rate | 23 | −1 | 0 | 0 | 0 | −1 | 1394 | 1077 | OR 4.04 (3.25, 5.03) | L |
| Healing rate | 23 | −1 | 0 | 0 | −1 | 0 | 1394 | 1077 | OR 2.63 (2.15, 3.21) | L | |
| Healing time | 8 | −1 | −1 | 0 | 0 | 0 | 276 | 276 | MD −5.46 (−6.79, −4.13) | L | |
| Dong, 2023 | Effective rate | 16 | −1 | 0 | 0 | 0 | −1 | 673 | 652 | OR 4.27 (2.94, 6.22) | L |
| Healing rate | 16 | −1 | 0 | 0 | −1 | 0 | 673 | 652 | OR 0.25 (0.19, 0.31) | L | |
| Healing time | 4 | −1 | −1 | 0 | −1 | 0 | 158 | 140 | MD −8.22 (−13.88, −2.56) | CL | |
| Liu, 2023 | Effective rate | 8 | −1 | 0 | 0 | 0 | −1 | 285 | 285 | RR 1.15 (1.08, 1.22) | L |
| Wound size reduction | 5 | −1 | −1 | 0 | −1 | 0 | 195 | 195 | MD −2.61 (−3.72, −1.51) | CL | |
| Healing time | 3 | −1 | 0 | 0 | −1 | 0 | 94 | 94 | MD −10.91 (−15.34, −6.48) | L | |
- Abbreviations: CL, critically low; L, low; M, moderate.
4 DISCUSSION
It is well known that SRs/MAs are considered by evidence-based medicine as a source of the highest level of evidence.20, 21 However, not all medical evidence derived from SRs/MAs is convincing, as SRs/MAs may introduce various risks of bias in the production of evidence that undermine the reliability of their results. Therefore, a systematic evaluation of the methodological quality of published SRs/MAs and the quality of the evidence they produce is a must before using evidence.22 Experts in evidence-based medicine have responded to this issue by proposing the method of overview of SRs/MAs, with the goals of evaluating and synthesizing the available data on the same subject.23 The number of overlapping SRs/MAs published on DFUs treated with external application of CHM is increasing. However, their results were inconsistent and vary in quality. Therefore, we carried out this overview aimed to systematically collate, appraise and synthesize current evidence on the management of DFUs with external application of CHM.
4.1 Main findings
This study is the first overview to critically evaluate the evidence-based medical evidence for external application of CHM in the treatment of DFUs. AMSTAR-2, PRISMA and GRADE were the primary evaluation tools used in this study. The AMSTAR-2 assessment results showed that all included studies had significant deficiencies both in critical and non-critical items. It was also these deficiencies that led to all methodological quality being judged as very low. The items with obvious methodological flaws included item 2, item 4 and item 7. PRISMA identified similarly weakness in reporting. The GRADE assessment showed that all included outcome indicators were at varying degrees of risk of bias, and the presence of these risks of bias resulted in an outcome indicator of high evidence quality of 0, moderate of 16, low of 23 and very low of 3. Risk of bias was the first element that leads to downgrading the quality of evidence, followed by inconsistency, publication bias and imprecision. The results of the descriptive analysis of the outcome indicator effect sizes showed that external application of CHM is an overall effective method for DFUs, but inconsistencies exist. Therefore, external application of CHM may be effective for DFUs based on the above findings, but this conclusion should be viewed with caution in light of the limitations of the available evidence.
4.2 Methodological quality and evidence certainty of SRs/MAs
AMSTAR-2 was officially promulgated in 2017 to evaluate the methodological quality of SRs/MAs and to promote their standardization to ensure the reliability of the evidence generated. We evaluated the methodological quality of the SRs in this study using AMSTAR-2, and disappointingly, all included studies had more than one critical flaw in their methodology and were therefore rated as low methodological quality. For item 2, five studies did not register a protocol, which may lead to the possibility of significant risk of bias. Pre-registration protocols are essential to improve the overall methodological quality of SRs/MAs due to the increased transparency of the research process.24 For item 4, only three studies provided specific search strategies for each database, which may not be convincing enough for the adequacy of the search and reduce the reliability of the findings. The lack of a specific search strategy would make the search results irreducible and therefore would not guarantee the scientific validity of the included and excluded studies. For item 7, all reviews did not provide a list of excluded trials, which likely resulted in the non-inclusion of eligible trials, posing a risk of publication bias. Providing a list of excluded studies is considered necessary for high-quality SRs, and the lack of this process will not guarantee whether some usable literature has been excluded, increasing the potential for risk of bias. Thus, in terms of methodology, the current SRs/MAs still possess considerable room for improvement. Targeted solutions to the deficiencies identified in this study are needed to focus on future high-quality SRs.
Surprisingly, most of the authors of the included studies lacked confidence in their SRs/MAs and were therefore reluctant to draw definitive conclusions about external application of CHM for DFUs. We used GRADE to rate the certainty of the evidence and found that the certainty of most of the evidence was not satisfactory. Such results explain the authors' concern, as low-quality evidence tends to suggest that results from SRs/MAs may differ from the true results. Further analysis of the GRADE results found that low-quality original RCTs were the most direct cause of low evidence. There is still much room for improving the current published RCTs on external application of CHM for DFUs with respect to shortcomings in randomization, allocation concealment or blinded bias. As evidence-based medicine continues to evolve, clinical decision making is increasingly dependent on high-quality evidence. As a source of evidence generation, rigorously designed and implemented trails are increasingly being advocated, as only high-quality trails are the gold standard for reducing the risk of bias to evaluate the effectiveness of interventions.25
4.3 Strength and limitations
This overview critically evaluated and summarized the current evidence on external application of CHM for DFUs, and the results may facilitate clinical decision making for complementary therapies. In addition, identification of limitations of included SRs/MAs may be beneficial in guiding future high-quality studies. However, this study was primarily based on subjective evaluation tools, and the process of subjective evaluation may not have the same results for different evaluators and cannot guarantee the accuracy of the assessment.
5 CONCLUSION
In conclusion, low- to moderate-quality evidence supports the promise of external application of CHM for the treatment of DFUs. Due to the limitations of the evidence supporting external application of CHM for DFUs, rigorously designed and larger samples of high-quality studies are needed going forward before broad recommendations can be made.
FUNDING INFORMATION
This work was funded by the Science and Technology Innovation Projects of Shenzhen (No. JCYJ20220530144212026), Futian Healthcare Research Project (No. FTWS2022012), Sanming Project of Medicine in Shenzhen (No. SZZYSM202202012) and Scientific Research Project of Guangdong Provincial Bureau of Traditional Chinese Medicine (NO. 20231061).
CONFLICT OF INTEREST STATEMENT
The authors declare that they have no conflicts of interest.
Open Research
DATA AVAILABILITY STATEMENT
No further data is necessary because all analyses were founded on already published studies.
