Journal editors have various tasks and responsibilities: from defining submission, peer review and disseminating processes, selecting articles to be published, contributing editorial pieces, dealing with scientific misconduct, providing leadership and editorial team management, and ultimately raising standards in methodological rigor, scientific reporting and public knowledge. They also face numerous challenges, particularly in ensuring the integrity of the published records, such as publication misconduct. Some of these issues have been addressed by the policies and guidelines of editorial organizations such as the World Association of Medical Editors (WAME), International Committee of Medical Journal Editors (ICMJE), Committee on Publication Ethics (COPE) and Council of Science Editors (CSE). Yet, despite an estimated number of more than twenty thousand journals (and thus many more editors) in the world, research into the work of editors and the publication process in biomedical and health care research is quite rare.
On the 12th and 13th December 2012 in Barcelona, an invitation-only Esteve Foundation Discussion Group was organized by the Esteve Foundation, a non-profit scientific institution whose primary goal is to foster progress in pharmacotherapy by means of scientific communication and discussion. The goal of the discussion group was to examine current editorial research in biomedicine and make recommendations for its future.
The two day discussion started with a presentation by Ana Marušić, editor in chief of the Journal of Global Health, vice president of the European Association of Science Editors (EASE) and former president of WAME and CSE. She elaborated on terms indexed by the National Library of Medicine (NLM) through the Medical Subject Headings (MeSH) thesaurus. These terms present concepts in medicine, which are then used to find evidence for research and practical questions in biomedicine (1,2). Mapping of key concepts in editorial research, such as journal, impact factor, publishing, peer review, and authorship, is rather incomplete and often only a recent addition to the MeSH. Most of these concepts are categorized within information sciences, but some do not match their current use in health research. A good example is the definition of authorship. Whereas MeSH defines the concept of authorship as: “The profession of writing. Also the identity of the writer as the creator of a literary production.”; the current ICMJE guidelines specifically address four qualifying criteria related to research, writing, approval of the manuscript and accountability for the work (3). Prof. Marušić also pointed out that several general and specialty journals publish articles on editorial research (often called ‘peer review’ or ‘publication research’) but, to date, no journal fully specializes in this research field. Likewise, the funding opportunities for editorial research are scarce.
Annette Flanagin, executive managing editor of JAMA (Journal of the American Medical Association) and coordinator of the International Congress on Peer Review and Biomedical Publication, reported on some of the groundbreaking research presented at the past six Peer Review Congresses and described its origins. Drummond Rennie, JAMA editor and director of the Peer Review Congresses, has initiated and led the effort to put peer review and other methods of assessment in biomedical research under the same level of scrutiny that journals and editors demand of science itself. Since 1989, the quadrennial Peer Review Congresses have encouraged and provided a forum for research into all aspects of the biomedical research dissemination enterprise, including research into management and reporting of bias, transparency of research reporting, authorship and contributorship, peer review, editorial evaluation, and dissemination of biomedical information (4).
Xavier Bosch, consultant internist at the Hospital Clinic of Barcelona and researcher in peer review, reflected on the scale of scientific misconduct in biomedicine, means of dealing with it and the journals’ role in promoting research integrity. In a survey of 41 journals which published retractions or corrections due to scientific misconduct from 1992 to 1999 only 14% addressed scientific misconduct in their instructions to authors (5). Of 50 high-profile biomedical journals surveyed in 2006, only 7 (14%) had written and publicly available misconduct policies (6). In 2009, a survey of 197 editors of randomly chosen journals showed that 94 (48%) journals had a written policy, of which 53 (56%) were policies of the publishers (7). Additionally, of 399 top-ranked journals, 140 (35%) had explicit definition of misconduct, 143 (36%) used policies of the publishers, and another 124 (31%) adopted them from policy-producing bodies. A total of 112 (28%) journals used plagiarism-checking services (8).
Erik von Elm, senior epidemiologist at the Institute of Social and Preventive Medicine in Lausanne, Switzerland, co-director of Cochrane Switzerland, and Academic Editor of PLoS ONE presented evidence on journals’ endorsement of recommendations aimed at improving publication practice. Several surveys of paediatric, haematology and urology journals showed that about 60% of journals direct their submitting authors to the Recommendations for the Conduct, Reporting, Editing and Publication of Scholarly Work in Medical Journals (formerly, Uniform Requirements), published by the International Committee of Medical Journal Editors (ICMJE) (3). About 35% required that trials reported in submitted manuscripts had been previously registered. 70% of journals had conflict of interest policies but only 24% recommend use of any reporting guidelines (9-13). These numbers demonstrate that more efforts are needed to improve endorsement of such recommendations. In turn, more accurate and complete reporting of primary studies would enable their inclusion in systematic reviews and, thus, potentially contribute to improvements in health care.
Peter Gøtzsche, professor of Clinical Research Design and Analysis, and Director of The Nordic Cochrane Centre in Copenhagen, Denmark, reflected on the accountability, transparency and honesty in the medical sciences. He called for a revolution in scientific research and reporting, and emphasised that too many published papers serve no one but the authors or sponsors, with a large number of papers being biased or superfluous and therefore unusable for furthering of public knowledge. He also stressed that anonymised raw patient-level data, alongside research protocols and amendments, needs to be made publicly available for independent research and scrutiny, and this practice enforced by legislations (14).
David L. Schriger, professor of emergency medicine at the University of California, Los Angeles, USA, and deputy editor of Annals of Emergency Medicine, discussed current peer review practices. While peer reviewers are typically given specific instructions on the format of their review, they are often not told the main purpose of their task. Most likely, editors also lack a clear view on whether reviewers are being asked to sort the true from the non-true, or if their primary job is to ensure that the paper contains sufficient information to allow readers to decide whether they agree with the authors’ conclusion, a state Ziman termed consensibility (15). The current focus on methodological rigor may make reviewers believe that their primary task is to determine validity but this can be a dangerous practice, e.g. in the case of Helicobacter discovery, which would later be awarded the Nobel Prize in Medicine, editors of Lancet were unable to find reviewers who believed the study findings (16). An argument can therefore be made that the main focus of peer review is to ensure that the work is complete and clear so that the scientific community can judge its validity. Similarly, in keeping with the focus on methodological rigor, most reporting guidelines emphasize the reporting of methods, giving less attention to the means of reporting results. Measures such as enhanced access to datasets, use of improved graphics and liberal use of electronic supplementary data tables and figures, can provide readers with a more comprehensive view of published research findings.
Harvey Marcovitch, former chairman of COPE, associate editor of the British Medical Journal, and a former Board Director of CSE, reported on retractions of publications and pressures faced by editors. From 2000 to 2010 there has been an increase in the number of retracted publications in biomedicine (17). Research on 2047 retracted articles indexed by PubMed showed that most (67%) were retracted due to misconduct (18). There is also evidence from individual cases that editors ignore suspicions of fraud and fail to properly investigate matters, even after receiving support by organizations such as COPE. As membership in COPE is optional, legislative or other measures are needed to deal with and prevent scientific misconduct.
Following each of the plenary lectures, many questions were raised, discussed and some solutions for the existing problems proposed. In the final session and through input provided one week after the meeting, several key topics and recommendations emerged.
There is considerable heterogeneity in funding, experience and tradition of large and small journals, as well as those in different countries and fields. This requires broader editorial guidelines that would cover the whole spectrum of editorial endeavour, and include sections on means to obtain indexation of journals in bibliographic and citation databases, as well as methods of screening for duplicate or plagiarized research. As many editors of academic and scholarly journals are researchers themselves and may publish in journals they edit (19), clear guidelines and editorial processes need to be established and made transparent to tackle this specific conflict of interest. Likewise, existing guidelines for editors need to be made more visible and their effectiveness appraised.
Depending on the available resources, contract agreements, publisher dependence, and level of interest, editors differ in the amount of time they dedicate to editorial research. On a larger scale, many aspects of editorial work thus remain unknown, including more precise estimates of key editorial practices such as rejection rates, proportion of manuscripts sent for review and changes that occur following peer review. In order to encourage more editors and independent researchers to become involved in editorial research, and especially in multi-journal collaborations, it is critical that publishers and funders recognize the importance of this research field and establish funding schemes that would both sustain and increase editorial research competitiveness, as well as its quality. Likewise, additions and re-definitions of key concepts of editorial research in MeSH are needed to increase the field’s visibility.
Initiatives of individual journals and editorial groups alone are probably insufficient to bring about necessary changes in reporting, availability and accessibility of scientific research and data. Legislative actions, similar to those of the Research Councils’ Policy on Open Access and Supporting Guidance in the UK (20), and the proposed Fair Access to Science and Technology Research Act in the USA (21), would not only secure open accessibility of all research involving humans, but also establish, promote and monitor quality standards of scientific reporting. Ethical and institutional boards need to play a more proactive role in monitoring research they approve by requiring project updates or milestone reports. Study protocols of all types of research should be made available either through institutional boards or publicly accessible registries. These databases should allow for submission of raw data, within a fixed timeframe following the studies completion, and in that way prevent instances of non-publication or partial publication and unavailability of data from publicly or industry funded human research. Educational courses on research methodology, statistics and scientific reporting, complemented by courses on scientific integrity, should be mandatory in graduate curriculums. And lastly, a part of a research grant should be reserved exclusively for publication purposes, depositing raw data in an easily analyzable format, and possibly for adherence to reporting guidelines.
Knowledge, experience and practice from editorial research in biomedicine and health research should be disseminated to other fields and vice versa. To reduce wasteful use of resources, peer review reports of manuscripts submitted but not accepted for publication could be made available through journal websites or specific databases to help and enable editors of other journals to make better informed decisions. This might also prevent instances in which authors submit unfinished manuscripts and abuse a journal’s peer review process as a means of assistance with writing (or polishing) their papers.
Letters to the editor and post publication peer review
A crucial component of scholarly publishing, and science in general, is debate. However, many journals impose stringent word limits and short submission periods for letters to the editors in response to published articles (22). To provide opportunities for debate, electronic means for commenting and reviewing published articles have been introduced by some journals (e.g. BMJ or Pediatrics), but smaller journals may not have the resources to do so. For online post-publication peer review to be useful, effective models of its moderation and linkage to literature databases need to be developed.
Editors today should reaffirm their focus on the integrity and completeness of the data they publish. Several initiatives and instruments exist to help them with this endeavour, but additional efforts are needed to make them known, in particular to editors of regional and newly established journals. Initiatives of individual journals, e.g. publication of submitted manuscripts alongside peer review comments and final versions, demonstrate that innovative procedures are feasible, at least at small scale. However, without changes in policy, either state or inter-journal, these initiatives remain only perks of individual journals. More research is needed to better understand critical questions such as why many journals are reluctant to use reporting guidelines or checklists, or to react adamantly to improperly published papers (23). Furthermore, better identification of actors and stakeholders of editorial research is needed, including identification of obstacles editors face when engaging in editorial research, and incentives that are or should be in place in order for them to do so. In the fast changing world of biomedical publishing, editorial research could play an essential role in identifying the changes that are needed and viable if editors are to remain gatekeepers of scientific knowledge.
Potential conflict of interest
1. Wieland LS, Robinson KA, Dickersin K. Understanding why evidence from randomised clinical trials may not be retrieved from Medline: comparison of indexed and non-indexed records. BMJ 2012;344:d7501.
2. Djebbari A, Karamycheva S, Howe E, Quackenbush J. MeSHer: identifying biological concepts in microarray assays based on PubMed references and MeSH terms. Bioinformatics 2005;21:3324–26.
3. Recommendations for the conduct, reporting, editing, and publication of scholarly work in medical journals: Roles and responsibilities of authors, contributors, reviewers, editors, publishers, and owners: defining the role of authors and contributors. [Internet]. International Committee of Medical Journal Editors. Available at: http://www.icmje.org/about_a.html. Accessed November 24, 2013.
4. Malički M, von Elm E, Marušić A. Study design, publication outcome, and funding of research presented at international congresses on peer review and biomedical publication. JAMA 2014;311:1065-7.
5. Scheetz MD. Instructions to the Author: An integrity issue. Investigating Research Integrity. Proceedings of the First ORI Research Conference on Research Integrity; Nov 19-20, 2000. Bethesda, Maryland: Office of Research Integrity, US Department of Health & Human Services 2001;285-290.
6. Redman BK, Merz JF. Research misconduct policies of high impact biomedical journals. Account Res 2006;13:247-58.
7. Resnik DB, Peddada S, Brunson W Jr. Research misconduct policies of scientific journals. Account Res 2009;16:254-67.
8. Bosch X, Hernández C, Pericas JM, Doti P, Marušić A. Misconduct policies in high-impact biomedical journals. PLoS One 2012;7:e51928.
9. Meerpohl JJ, Wolff RF, Niemeyer CM, Antes G, von Elm E. Editorial policies of pediatric journals: survey of instructions for authors. Arch Pediatr Adolesc Med 2010;164:268-72.
10. Meerpohl JJ, Wolff RF, Antes G, von Elm E. Are pediatric Open Access journals promoting good publication practice? An analysis of author instructions. BMC Pediatr 2011;11:27.
11. Meerpohl JJ, Wolff RF, Grambauer N, Antes G, von Elm E. Editorial policies of hematology and oncology journals: An analysis of author instructions. Proceedings of the 19th Cochrane Colloquium. 2011 Oct 19-22. Madrid, Spain.
12. Kunath F, Grobe HR, Keck B, Rücker G, Wullich B, Antes G, Meerpohl JJ. Do urology journals enforce trial registration? A cross-sectional study of published trials. BMJ Open 2011;1:e000430.
13. Kunath F, Grobe HR, Rücker G, Engehausen D, Antes G, Wullich B, Meerpohl JJ. Do journals publishing in the field of urology endorse reporting guidelines? A survey of author instructions. Urol Int 2012;88:54-59.
14. Gøtzsche PC. We need access to all data from all clinical trials. Cochrane Database Syst Rev 2011;12:ED000035.
15. Ziman JM.Public knowledge: an essay concerning the social dimension of science. Cambridge, University Press; 1968.
16. Warren JR. Helicobacter: the ease and difficulty of a new discovery (Nobel lecture). ChemMedChem 2006;1:672-85.
17. Steen RG. Retractions in the scientific literature: is the incidence of research fraud increasing? J Med Ethics 2011;37:249-53.
18. Fang FC, Steen RG, Casadevall A. Misconduct accounts for the majority of retracted scientific publications. Proc Natl Acad Sci USA 2012;109:17028-33.
19. Bošnjak L, Puljak L, Vukojević K, Marušić A. Analysis of a number and type of publications that editors publish in their own journals: case study of scholarly journals in Croatia. Scientometrics 2011;86:227-33.
20. Research Councils UK. RCUK Policy on open access. Available at: http://www.rcuk.ac.uk/research/outputs/. Accessed May 5th 2014.
21. Fair Access to Science and Technology Research Act of 2013. A proposed law (S.350, 2013) introduced in Senate on February 13th 2013. Available at: http://beta.congress.gov/bill/113th-congress/senate-bill/350/text?q=s350. Accessed November 24, 2013.
22. von Elm E, Wandel S, Juni P. The role of correspondence sections in post-publication peer review: A bibliometric study of general and internal medicine journals. Scientometrics 2009;81:747-55.
23. Steen RG, Casedevall A, Fang FC. Why has the number of scientific retractions increased? PLoS ONE 2013;8:e68397.