Identification and description of controlled clinical trials in Spanish language dental journals

Introduction

Controlled clinical trials (CCTs) are considered the best approach to assess the effects, benefits and harms of a therapeutic intervention, drug, device or technique in human beings (Bonfill et al., 2013; Cañedo Andalia, Arencibia, Perezleo Solorzano, Conill González & Araújo Ruiz, 2004; Dickersin, Scherer & Lefebvre, 1994; Manríquez, Valdivia, Rada & Letelier, 2005). CCTs are also the foundation of systematic reviews and other evidence synthesis documents (Garcia-Alamino et al., 2006). However, identifying all CCTs on a specific intervention is complex (Hopewell, Clarke, Lefebvre & Scherer, 2007) due to issues associated with bias in the reporting of CCTs.

Reporting bias arises from a tendency of investigators to report research results depending on the strength, direction or statistical significance of results (McGauran et al., 2010; Meerpohl et al., 2015). Reporting bias takes several forms, including delays in the reporting of research results (time-lag bias), reporting in journals depending on its impact factor or level of indexation on bibliographic databases (location bias), reporting in English as opposed to other languages (language bias), differential reporting of outcomes that had been pre-defined in study protocols (selective outcome reporting), or reluctance to disseminate research via scientific articles or other methods (dissemination bias), all in function of the nature of the obtained results (McGauran et al., 2010).

Reporting bias has serious ethical and economical implications. It undermines the contribution of participants in CCTs, who often invest time and money without the assurance of benefit from an intervention (or no intervention). They accept the possibility of potential adverse events as well as of uncertainty derived from blinding to treatment interventions and other methodological practices (Jones et al., 2013; McGauran et al., 2010). Reporting bias also results often in redundant work, since unpublished research may be revisited by other investigators, with the consequent waste in economic and human resources (Moher et al., 2016) Not surprisingly, reporting bias has been targeted as one the main challenges of the evidence-based medicine movement for the upcoming years (Djulbegovic & Guyatt, 2017).

The Cochrane Handbook provides several approaches for addressing the possibility of reporting bias during the development of systemic reviews. The most commonly used is the need of developing properly designed search strategies that are applied to different databases and repositories (McGauran et al., 2010). However, using exclusively electronic databases (e.g. MEDLINE, EMBASE) has shown to be suboptimal (Iberoamericano, 2012) due to several reasons. First, the term CCT was not indexed as such until 1990 and incorporated into MEDLINE and EMBASE in 1991 and 1994, respectively. Therefore, CCTs prior to this date were classified using broader categories. Second, data suggest that the descriptors available in databases have been used inconsistently by those responsible for their codification and classification (Dickersin et al., 1994). Third, in some occasions, authors do not report their research methods clearly enough, which hinders the CCTs indexing process (Koletsi, Pandis, Polychronopoulou & Eliades, 2012).

Regarding literature in Spanish, even though there are several databases, directories and registries that allow searching for journals in Spanish language (e.g. LILACS, SCIELO, PERIODICA, IMBIOMED, LATINDEX and PUBMED), none contain all existing journals. Moreover, some databases can provide outdated and/or incomplete information (Armstrong, Jackson, Doyle & Waters, 2005; Betrán, Say, Gülmezoglu, Allen & Hampson, 2005; Hopewell, Clarke, Lusher, Lefebvre & Westby, 2002), including inactive journals that are marked as active, with obsolete contact information, or inaccurate data about the publishers. This supports the notion that it is necessary to broaden searches for dental journals to libraries, catalogues, publishers and specialised registries. Last but not least, electronic searches do not retrieve studies from journals that are not regularly indexed in the classic databases. The large increase in the number of studies published in Spanish language during the last decade makes it imperative to overcome this problem (Reveiz et al., 2006).

One of the strategies proposed to overcome the shortcomings of electronic searches is the implementation of a rigorous and thorough handsearching approach (Dickersin et al., 1994; García-Alamino, Parera, Ollé & Bonfill, 2007; Martí, Bonfill, Urrutia, Lacalle & Bravo, 1999). The superiority of handsearching over electronic searching has been reported in studies that tried to identify the maximum possible number of CCTs on Anesthesiology (García-Alamino et al., 2007; Suarez-Almazor, Belseck, Homik, Dorgan & Ramos-Remus, 2000), General and Internal Medicine (Martí et al., 1999), Patient Safety (Barajas-Nava, Calvache, López-Alcalde, Sola & Bonfill, 2013), among others.

The main objective of this study was to conduct a comprehensive search to identify CCTs published in Spanish language dental journals from Spain and Latin America. We also aimed to provide researchers with access to this body of evidence in a single source, namely CENTRAL, the Cochrane Collaboration repository of CCTs, to facilitate the inclusion of these studies in future systematic reviews.

Methods

We manually and retrospectively identified CCTs published up to 2014 in dental journals published in Spanish language in Spain and the 18 Spanish-speaking countries in Latin America: Argentina, Bolivia, Chile, Colombia, Costa Rica, Cuba, Dominican Republic, Ecuador, El Salvador, Guatemala, Honduras, Mexico, Nicaragua, Panama, Paraguay, Peru, Uruguay and Venezuela.

Handsearching was conducted retrospectively, backwards from the last year of publication. If CCTs were not found in any of six consecutive years, the handsearch for the journal in question was stopped.

Results

Identification of CCTs

A total of 17 051 articles were reviewed in eligible journals (Table 2), 244 of which were CCTs (1.43%). Of these, 104 were published in journals from Spain (42.6%), 53 from Chile (21.7%), 29 from Colombia (11.9%), 20 from Venezuela (8.2%), 16 from Mexico (6.5%), 16 from Peru (6.5%) and six from Argentina (2.5%) (Table 2). No CCTs were found in journals from Bolivia, Costa Rica, Cuba, Ecuador or Uruguay.

Chile (2.7%) and Peru (2.2%) had the highest number of CCTs per number of articles reviewed (1.4%) (Table 2). When assessing the number of CCTs published over time, we found a sustained increment, with 115 CCTs published in the 2009–2014 period alone. (Figure 1).

Identified CCTs focused on ten dental specialties, with periodontics (70, 29.0%) and oral and maxillofacial surgery (66, 27.0%) prevailing over others (Table 3). Most CCTs examined interventions related to pharmacological interventions (112, 46.0%), restorative techniques (16, 7.0%), prosthetic rehabilitation (15, 6.0%) or educational interventions (13, 5.0%) (Table 4). Over 153 studies (62.7%) used random treatment allocation, 51 (20.9%) were classified as quasi-randomised, and 40 (16.4%) did not specify the method of allocation of patients in the compared groups.

Table 3. Distribution of controlled clinical trials by dental specialty

Specialty	Number of CCTs identified	Proportion of total CCTs (%)
Periodontics	70	29
Oral and Maxillofacial Surgery	66	27
Oral Rehabilitation	25	10
Restorative Dentistry	23	9
Pediatric Dentistry	13	5
Orthodontics	12	5
Oral Pathology	10	4
Implant Dentistry	8	3
Occlusion and Temporomandibular Disorders	8	3
Not classifiable	6	2
Endodontics	3	1
Total	244	100

• CCT, controlled clinical trial.

Table 4. Distribution of controlled clinical trials by intervention

Intervention	Number of CCTs identified	Proportion of total CCTs (%)
Pharmacological Interventions	112	46
Restorative approaches	16	7
Prosthetic rehabilitation (removable partial dentures, overdentures, etc.)	15	6
Educational intervention	13	5
Periodontal approach	12	5
Surgical approach	10	4
Plaque control techniques	8	3
Bone regeneration techniques	7	3
Dental materials (dental porcelain, dental cements, bolt, etc.)	7	3
Oral hygiene materials (toothbrushes, dental floss, etc.)	7	3
Types of interocclusal appliances	6	2
Types of dental implants	5	2
Laser treatment	4	2
Hygiene techniques	4	2
Other interventions	18	7
Total	244	100

• CCT, controlled clinical trial.

Discussion

In this study, we identified 107 dental journals publishing original clinical in Spanish language. We found that less than a third of these journals are indexed in more than one bibliographic database. Moreover, the low proportion of journals indexed in MEDLINE (5.6%) and EMBASE (0.9%) is alarming given the relevance and recognition associated with these databases by the international scientific community (Suarez-Almazor et al., 2000). In addition, only one publication (0.9%) had an impact factor in 2014 (1.171). It is possible that this situation discourages many Spanish-speaking authors from publishing in local journals. Given the fact that the impact of their publications has a direct effect on the evaluation of their research and professional achievements, they may be motivated to publish their work elsewhere, in international, English language journals that are indexed or have an impact factor (Armstrong et al., 2005; Crumley, Wiebe, Cramer, Klassen & Hartling, 2005; Hopewell et al., 2002).

The term ‘randomised or controlled clinical trial’ has become more common in the titles of articles published in dental journals over time. However, a large number of these articles do not meet the requirements to be classified as such. A recent study (Koletsi et al., 2012), published in 2012, observed articles published in dental journals with highest impact factor, representing five dental specialties and assessed the consistency between the title reporting the study as a randomised clinical trial and whether they were indeed RCTs (19). Between 1979 and 2011, a total of 222 trials were detected, of which 88 (39.6%) were considered RCTs, 107 (48.2%) were deemed ‘not clear’, and 27 (12.2%) were not considered RCTs (Koletsi et al., 2012).

Our study does not have this limitation as we not only screened the title and abstract of the included reports but also verified in the ‘Materials and Methods’ sections that each eligible study was truly a CCT. Only 27 articles contained the term ‘clinical trial’ in the title, which could indicate that both authors and dental journal editors publishing in Spanish lack awareness of the CONSORT statement (Schulz, Altman & Moher, 2010), at least since it was published and accepted by the scientific community. Similar trends have been reported by other authors assessing the quality of reporting in Orthodontics randomised controlled trials (Fleming, Buckley, Seehra, Polychronopoulou & Pandis, 2012). Our search also showed an increase in the number of published CCTs starting in the nineties, which is in line with the findings of Martí et al. (Martí et al., 1999) in General and Internal Medicine journals, and of other similar studies (Chung & Lee, 2013; Hopewell et al., 2002; Manríquez et al., 2005).

In our opinion, the low number of published CCTs reflects the insufficient development of this important line of clinical research in dental journals published in Spanish. This phenomenon could also be explained by a preference of Spanish and Latin American authors to publish in foreign journals with a greater impact factor.

Despite these shortcomings, we believe it is important to increase visibility of the CCTs published in local and Spanish language journals to control the effects of reporting bias, specifically regarding language and location bias, during the development of systematic reviews and other documents of synthesis. The Cochrane Collaboration, through the Methods Expectations for Cochrane Intervention Reviews (MECIR Standards) (Higgins, Lasserson, Chandler, Tovey & Churchill, 2016), recommends searching for CCTs in regional databases, such as LILACS, as well as handsearching the reference sections of eligible studies and other sources, to complement the search results of electronic strategies. Through the work we have conducted here, especially by submitting the identified CCTs to CENTRAL, we hope to decrease the burden inherent to these tasks. In addition, the process of identifying CCTs in eligible dental journals from Spain and Latin America is ongoing. The Iberoamerican Cochrane Centre coordinates an effort for regularly contacting editors of eligible journals that were found to publish CCTs requesting new CCTs published after the search period of this study (Bonfill et al., 2017; Pardo-Hernandez et al., 2017). Newly identified CCTs will also be submitted to CENTRAL.

The main strengths of our study are the comprehensive handsearching approach we took for identifying eligible CCTs. The handsearching process was conducted in duplicate, which minimises the risk of including ineligible CCTs or leaving out real CCTs. We looked at a high volume of journals, all of which, at least in theory, publish original research, and were able to review over 17 000 references.

Our work also has some limitations. We focused only on dental journals, which may have left out CCTs on this discipline published in General Medicine, Surgery or other types of Spanish language journals. In addition, as mentioned throughout the article, we did not review journals in English language or from countries outside Spain and Latin America. Therefore, our work does not include CCTs, presumably of higher quality, that Spanish and Latin American authors may have published in those journals.

Future research projects should attempt to identify studies by Spanish and Latin American authors published in international, English language journals. Additionally, it would be useful to assess in depth the risk of bias in the CCTs identified in this study. This would further improve their usefulness for inclusion in systematic reviews and other evidence summaries. Lastly, the handsearching process should and will be conducted prospectively from the point where it was stopped in this article. New CCTs identified in handsearched dental journals will be uploaded into BADERI and submitted to Central regularly.

To our knowledge, this is the first study that retrieves and describes CCTs published in dental journals in Spanish language. The methodological measures we took to complete this study, including the collaborative approach among researchers in different countries to identify CCTs that are not retrievable using electronic searches, as well as the rigour with which the handsearching of eligible journals was completed, may be informative to the readership of Health Information and Libraries Journal for designing future research projects. Likewise, the identification of these studies, as well as the list of journals and websites (Bonfill et al., 2015), brings to the research and clinical communities the opportunity to access additional evidence that may have been ignored by systematic reviewers in the past. Moreover, our work helps to increase the visibility of CCTs published in dental journals that are not indexed in any database. References to all CCTs identified are available and accessible via CENTRAL, the Cochrane Collaboration CCT register, for potential inclusion in systematic reviews and other evidence synthesis documents. The inclusion of these articles in ongoing and future systematic reviews may increase the strength of the conclusions or change the direction of relevant findings.

Conclusions

There are a considerable number of dental journals from Spain and Latin America publishing CCTs. However, they are not indexed in the most used electronic databases, which limits their visibility for potential inclusion in systematic reviews and meta-analysis. In addition, the ratio of CCTs relative to all the studies published in these journals is almost negligible (1.4%). We therefore believe it is necessary to promote research and the development of CCTs in this field in Spain and Latin America. Finally, to ensure proper reporting of CCTs, journals should require adherence to CONSORT as requirement for publication.

Conflict of interests

The authors have no conflict of interests to declare.

Funding

There was no funding available for this study.

Ethics approval

No ethical approval was required for this study.