Unsung Heroes of Research Integrity

In the November 2018 issue of JBMR, we published an Editorial outlining a new program and several new policies related to research integrity and our commitment to rectifying any errors identified in our journal.⁽¹⁾ In the ensuing months, the JBMR editorial leadership rolled out new author and reviewer resources to help with transparency and rigor in data presentation, and in early 2019 the ASBMR Publications Committee announced the Research Integrity Panel, whose charter was to independently assess reader concerns and complicated issues with published science that were brought to the attention of the editorial office. The Research Integrity Panel is a group of three ASBMR members who serve for 3 years on a staggered, yearly rotation basis. They are appointed by the Publications Committee based upon professional stature and integrity, experience with scientific publishing, and service to ASBMR. The Panel has been working vigorously since its inception to support editors with rigorous self-correction of problems identified in the content of ASBMR publications.

Since early 2018, JBMR has received 55 reader concerns regarding articles published in the journal. The majority of these were reported during the 2018 calendar year, and then the flow began to ease, especially in 2020 and 2021 (the pandemic may have played a role, but enhanced attention to content that is currently being accepted for publication certainly contributed). Most of these concerns and allegations were submitted anonymously or under screen names. The editorial office has always followed the Committee on Publication Ethics (COPE) process for resolving allegations. However, each investigation requires dedicated staff and editor time and effort. Because of the high number of readers' concerns submitted and the complexity of some cases, the editorial team felt that the length of time before a resolution could be achieved in each instance was becoming unacceptably long. The Research Integrity Panel has been empowered to evaluate such concerns, request explanations from the authors and, if necessary, from their institutions, and provide a recommendation to the Editor-in-Chief for final decision. COPE guidelines represent the guiding principles of the Panel's activity and actions. The final decision and accountability to publish an Erratum or Expression of Concern or to retract an article remain with the Editor-in-Chief, in coordination with the Society and the Publisher. This process allows addressing all allegations and concerns in a systematic and focused fashion, eventually reducing the time from initial report to resolution.

Of the 55 reports sent to the JBMR inbox, nearly half were concerns about duplicated images either within the same article or images replicated over more than one article without attribution (five of these were categorized as image manipulation and four as questionable results, among others). Over the last 3 years, the journal has published 24 Errata and one Expression of Concern. We have retracted three articles; two were initiated by the authors after concerns were brought to their attention, and one was retracted after a long and contentious investigation. Three institutions were notified about Panel findings, and we turned over all of our investigative materials to them for any further action.

In five cases, the investigation concluded that the concerns were not actionable, either because they were without merit or because the author provided a satisfactory explanation. In some cases, the final decision has been debated by the initial complainant, sometimes for a long time. As for any editorial decisions, consensus and mutual agreement is not always possible, but every effort is made to be thorough in our investigations. However, there is more work to do to better clarify outcomes in these difficult cases. The Panel and the journal are still considering the best ways to notify readers of problems with an article that cannot be corrected with an Erratum (for instance, when the concern reported does not change the article's findings, but the authors are not responsive, and it is not clear what should be corrected).

In addition to concerns reported about published articles dating back more than 20 years, the Panel also investigated seven reports of plagiarism, data manipulation, or duplications in submitted and then rejected manuscripts. These problems were identified during the peer-review process by the editors or reviewers. The editorial office continues to focus on efforts to identify problematic data presentations well before publication to prevent mistakes or fraudulent or misleading data from reaching the readership and entering the scientific literature. A pilot program on image analysis of submitted manuscripts is underway, and the journal is still looking into automated image checks.

The journal worked extensively on revamping the author guidelines to help authors improve data presentation in ways that are both more transparent and reduce the likelihood of breaches of integrity. For instance, the adoption of the boxplot and mandatory reporting of all data points for small sample sizes reduce the chances of data manipulation, disguising distributions and trends, or simple mistakes. Greater emphasis has been put on assessing the rigor of study design and appropriate statistical analysis in the peer-review process, and we published an Editorial to help authors and readers navigate through the new guidelines and improve transparency and rigor in their manuscripts.⁽²⁾ The journal continuously revises these guidelines based on feedback from authors, editors, editorial board members, and Publications Committee. We welcome additional feedback from readers.

The JBMR editorial leadership is fully vested in creating a culture where transparency, rigor, and integrity are embedded in the scientific endeavor. All editors are paying increased attention to possible image manipulation and text overlap. Keeping up with the evolving research ethics space and sharing experiences as learning opportunities among editors have increased awareness of possible breaches of research integrity and honed our ability to detect them. Likewise, we have dedicated resources to educate our Editorial Board members, who represent the bulk of our reviewers, as well as younger colleagues with less experience as reviewers, on research ethics and on how to detect possible problems when reviewing a manuscript. Introduction of transparent peer review in 2019, whereby peer review documents of accepted papers are published online, has helped reviewers be more constructive and thorough in their analysis and more vigilant on data integrity. As proposed by some of our Editorial Board members in a recent Editorial, we would like to see reviewers as shepherds rather than gatekeepers.⁽³⁾

Increased scrutiny of research integrity and of scientific publishing as a whole have been the major focus of publishers and more mainstream media in recent months, especially in the face of rapid COVID-19 preprint publishing, a fair share of which have proven unreliable or wrong. A recent article in The Atlantic called out an “irreparably flawed body of literature” dating back decades, arguing that science may not be as self-correcting as believed.⁽⁴⁾ In response to a rising cry for more action on errors, misrepresentations, and duplications found post-publication, such as those spotted by data sleuths and watchdog groups, many publishers have established groups and committees similar or much broader in scale to our Research Integrity Panel.

SpringerNature established a Springer Nature Editorial Advisory Group, which works closely with the Springer Nature Research Integrity Team. This team supports editors with addressing and resolving errors and publication ethics issues following COPE guidelines. eLife recently established an Ethics Committee, with the purpose of advising and developing “policy focused on establishing and maintaining the highest standards of research and publication practices across the scope of the journal.” The STM Working Group on Image Alteration and Duplication Detection, whose members include Springer, Elsevier, Wiley, Taylor & Francis, among others, have developed best practice recommendations for handling image integrity issues and a three-tier classification for different types of image and data aberrations commonly detected.

These new initiatives are welcome in the scholarly publishing industry and will advance the transparency of scientific communication, while increasing the public trust in published science. However, questions remain on both large and small scales. While readers' concerns provide an opportunity for “cleaning up” problematic data and strengthening our processes for ensuring integrity, should individual journals be more proactive and implement procedures to systematically scan their own published record? On a case-by-case basis, how far should the action of the ASBMR Research Integrity Panel and editors go with their investigations? A full-scale forensic analysis of a publication with dubious or challengeable data would be desirable, but it would take time, resources, and expert oversight—activities under the original obligation of peer reviewers and editors—that a journal may not be able to undertake. While the Research Integrity Panel is charged with addressing potential breaches in research integrity, it is not resourced or empowered to re-create studies or rerun analyses. ASBMR journals are fully committed to ensure the integrity of published content, but there are some limits on the investigations journals can perform, particularly when authors and institutions are not responsive and peer review data or original submissions and revisions are unavailable. Complicated reanalysis and full-scale investigations on possible research misconduct are better managed by the institution with whom the authors are affiliated.

Should the allegations that do not result in a public correction or action be made public if the outcome is not straightforward or remains contentious? Traditionally, in a process of Letters to the Editor, the author gets the final word to answer any queries or concerns and all of the correspondence is published. Because the journal values transparency and openness that would benefit readers, we continue to consider the merits and risks of making anonymous allegations and any resolution public, even if no actions such as an Erratum or Retraction are taken. Furthermore, what recourse do the complainants have to make their opinion public if they do not agree with the outcome of the investigation? In cases where a substantial length of time has passed since the original article appeared or when the correspondence is lengthy and contentious, the Letter to the Editor system is not a good mechanism. As a matter of policy, journals could respond to all PubPeer and Retraction Watch mentions, but should each journal also have its own mechanism to publish such debates to best reach its readers? All these open questions are currently being debated among the editorial leadership and across the ASBMR publication enterprise, and it is likely that further initiatives will follow. The current model of article retraction should also be interrogated. Most retracted articles continue to be cited, although there are new mechanisms, such as Scite Reference Check, for scanning a reference list for retractions upon and before publication. It may also be possible in the future to “un-cite” an article and display (or link to) an editor's or author's note when a retracted article appears in a reference list. Indeed, popular reference managing products, EndNote, Papers and Zotero have all introduced alerts and notifications of retracted publications in user databases. These are welcome steps that will alert authors citing articles in their manuscripts and improve transparency for the readers.

Stop being dismayed by an idea and reject it only because it is new; weigh it based on your best judgment; and if you find it to be true, embrace it; or, if it is false, go against it.

T Lucretius Caro, De Rerum Natura, II, 1040–1043.