| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
1 Department of Medicine, University of Texas Health Center at Tyler, Tyler, Texas
Correspondence and requests for reprints should be addressed to David Coultas, M.D., Department of Medicine, The University of Texas Health Center at Tyler, 11937 U.S. Highway 271, Tyler, TX 75708-3154. E-mail: david.coultas{at}uthct.edu
ABSTRACT
The ethical interpretation and communication of research results is essential to ensure the validity, timeliness, and accessibility of new knowledge for patients, physicians, and regulatory agencies. Failure to adhere to ethical principles may cause adverse outcomes for patients because of overestimation of benefit, underestimation of harm, and lack of timely awareness of benefit or harm. Although fabrication, falsification, and plagiarism are the traditional criteria for research misconduct, other more subtle behaviors may cause greater threats to public safety and trust in the research enterprise. Growing awareness of research misconduct has led to a number of initiatives worldwide during the past decade in an attempt to control the problem at various stages of the research process through the funding agencies, research institutions, and editorial oversight. The objective of this article is to raise awareness among the pulmonary research community of the broad range of ethical issues that arise during manuscript preparation, review, publication, and dissemination of research results, and efforts that are in progress to minimize misconduct.
Key Words: ethics • research misconduct • clinical trial • bias • publications • data interpretation
The ethical interpretation and communication of research results are essential to ensure the validity, timeliness, and accessibility of new knowledge for patients, physicians, and regulatory agencies. Failure to adhere to ethical principles may cause adverse outcomes for patients because of overestimation of benefit, underestimation of harm, and lack of timely awareness of benefit or harm.
Recent reports of withholding data on adverse drug events in clinical trials highlight the threat of unethical communication of research results to public safety and trust in research (1–3). Although these recent events have prompted debate and a call for action to prevent similar episodes, the ethical principles and research misconduct are not new and have been reviewed elsewhere (4–6). However, these issues have received little attention in the pulmonary research literature.
The objective of this article is to raise awareness among the pulmonary research community of the broad range of ethical issues that arise during manuscript preparation, review, publication, and dissemination of research results, and efforts that are in progress to prevent misconduct. Moreover, institutional and governmental policies relevant to research misconduct are considered where appropriate.
DEFINITION OF MISCONDUCT IN INTERPRETATION AND COMMUNICATION
The definition of research misconduct traditionally is defined as fabrication, falsification, or plagiarism (6). However, some experts in clinical trials consider more subtle issues as representing misconduct, and with new knowledge, the definitions and standards of ethical interpretation and communication of research results will surely evolve. Furthermore, "mundane ‘regular’ misbehaviors present greater threats to the scientific enterprise than those caused by high-profile misconduct such as fraud" (7). In a recent survey of 26 experts, Al-Marzouki and coworkers used a consensus method to determine opinions on the types of misconduct that would most likely distort the results of a clinical trial and the likelihood of occurrence (8). Overall, 60 items in four categories (design, conduct, analysis, and reporting) were identified by the majority (> 50%) as likely or very likely to distort research results. Of these 60 items, 13 were also identified as likely or very likely to occur (Table 1), and all of these were problems with analysis and reporting (8).
|
Little is known about the prevalence of research misconduct; selected sources of data include the following: the Office of Research Integrity (ORI; http://ori.dhhs.gov/), the Committee on Publication Ethics (COPE; http://www.publicationethics.org.uk), and surveys of investigators (7, 9). The absolute number of reported cases by institutions to the ORI and by journal editors to COPE is small. Between 1993 and 2004, a total of 1,305 allegations (mean = 109/yr) and 876 new cases (mean = 73/yr) were reported to ORI by institutions nationwide (10). However, a minority of these cases involved clinical trials. Of 136 episodes of scientific misconduct reported by the ORI during the period 1992 through 2002, 13% were in clinical trials (11). In the 2005 annual report of COPE, a total of 212 cases of misconduct were reported by editors during the period 1997 through 2004, and of these, evidence of misconduct was found for 163 cases (12). The small number of misconduct reports from institutions and editors substantially underestimates the occurrence of research misconduct.
The best available evidence on the magnitude of research misconduct is available from self-reports of misconduct from investigators (7, 9). Gardner and coworkers surveyed 549 authors of pharmaceutical clinical trials published between 1998 and 2001, and, overall, they found that 4.7% reported at least one problem concerning research integrity that they personally participated in during the past 10 years (9). These problems included seriously misleading interpretation of results (1.6%), deceptive or misleading report of data (1.2%), fabrication or falsification (1.2%), deletion of data in an unjustified way (0.9%), and deceptive or misleading report of design (0.9%). The prevalence of at least one problem was much higher (17.4%) for studies in which the author knew about but did not personally participate, including misleading interpretation of results (6.5%), deceptive or misleading report of data (7.2%), fabrication or falsification (7.8%), deletion of data in an unjustified way (6.2%), and deceptive or misleading report of design (7.2%) (9). Martinson and coworkers surveyed 3,247 National Institutes of Health–funded investigators and found a wide range of self-reported misbehaviors during the previous 3 years relevant to the interpretation and communication of research results, including the following: falsifying or "cooking" research data (0.3%); not properly disclosing conflicts of interest (0.3%); failing to present data that contradict one's own previous research (6.0%); overlooking others' use of flawed data or questionable interpretation of data (12.5%); changing the design, methodology, or results of a study in response to pressure from a funding source (15.5%); publishing the same data or results in two or more publications (4.7%); inappropriately assigning authorship credit (10.0%); withholding details of methodology or results in papers or proposals (10.8%); and dropping observations or data points from analyses based on a gut feeling that they were inaccurate (15.3%) (7).
These self-reports of misconduct probably underestimate the magnitude of the problems. Chan and coworkers found that, in a survey of 122 published reports from 102 randomized trials, 50% of efficacy and 65% of harm outcomes per trial were incompletely reported, yet 86% of authors denied underreporting outcomes (13). Moreover, statistically significant outcomes were more likely to be reported compared with nonsignificant findings. Similar findings were reported by the same authors on a review of 68 publications from 48 trials funded by the Canadian government and from 519 trials found in PubMed (14, 15).
In addition to problems within published reports, failure to publish negative results, known as publication bias, raises concerns about the validity of the larger body of scientific evidence (5). Various estimates have been reported on the magnitude of publication bias, with less than 50% of trials reported (5). For example, Camacho and coworkers examined the publication rates of phase I oncology trials submitted for presentation at a national scientific meeting (16). Of 275 abstracts submitted, 54% were selected for presentation, and 72% of selected and 62% of nonselected abstracts were subsequently published. Although a number of factors may contribute to failure to publish research results, limited evidence suggests that author, and not editorial decisions, predominate (17).
PREVENTION OF UNETHICAL INTERPRETATION AND COMMUNICATION
The prevention of unethical interpretation and communication of research results requires a comprehensive approach, consisting of individual-, institutional-, and external-level controls (Figure 1). Growing awareness of research misconduct has led to a number of initiatives worldwide during the past decade in an attempt to control the problem at various stages of the research process through the funding agencies, research institutions, and peer review and editorial oversight. The methods have included researcher education and mentoring, reporting and monitoring for misconduct, interinstitutional agreements before beginning research, research registries, publication guidelines, and open-access publishing.
|
Institutional Environment
An institutional environment that supports the researcher in the ethical conduct of research is critical. A comprehensive review of the regulations and other methods that contribute to an ethical research environment is beyond the scope of this article, but they are briefly outlined here. Procedures for institutional oversight and reporting of research misconduct are required for institutions that receive federal funding for research by the ORI in the Department of Health and Human Services (6). The Association of American Medical Colleges has developed institutional guidelines for managing individual conflicts of interest in conducting clinical research (20). Increasingly, clinical trial agreements between academic medical centers and industry are used to set ethical standards (21). For example, in a survey of 107 institutions, 85% prohibit sponsors from making decisions about publication of results or in revising manuscripts. However, there is substantial variability among institutions on issues of responsibility and authority for analysis and interpretation of results.
National and International Initiatives
A number of national and international initiatives are in progress to improve the quality of interpretation and communication of research results, including clinical trial registries, publication guidelines, and open-access publishing. All of these efforts are relatively new and results on their effectiveness are limited.
Clinical trial registries.
Although clinical trial registration was proposed in the 1970s, only in recent years has there been growing demand for comprehensive registration of clinical trials, as a response to the problems of publication bias and underreporting of trial results (22–25). In 2000, as a result of earlier federal legislation, a web-based registry, ClinicalTrials.gov, became operational (www.ClinicalTrials.gov) (24, 26). Beginning in 2005, the International Committee of Medical Journal Editors required registration of trials in ClinicalTrials.gov or another qualified public registry to be published (27). There are a growing number of public and private registries—for example, the Cochrane Central Register of Controlled Trials (http://www.mrw.interscience.wiley.com/cochrane/cochrane_clcentral_articles_fs.html) and the World Health Organization (http://www.who.int/ictrp/en/). (25) In a recent review of trial registration in ClinicalTrials.gov between May and October 2005, Zarin and coworkers found that the number of clinical trials being registered substantially increased, from 13,153 to 22,714 (26). However, they found many gaps in the reporting. Although clinical trial registration will likely increase transparency in trial conduct and reporting, its effectiveness for preventing misconduct and publication bias remains to be determined.
To address ongoing trial underregistration by the pharmaceutical industry, legislation is under consideration (28). The Fair Access to Clinical Trials (FACT) legislation, which was first introduced in 2005 and was reintroduced in 2006 (http://olpa.od.nih.gov/tracking/house_bills/session2/hr-5252.asp), would require drug makers to register clinical trials of drugs and devices and make the results available, regardless of outcome, to the scientific community, health care practitioners, and members of the public (29).
Publication standards.
Editorial oversight, through publication standards and independent investigations, is a key method for controlling misconduct in the interpretation and communication of research results. Recognizing the need for quality improvement of clinical trial reports, two groups published proposals in the mid-1990s for improved and more structured reporting of clinical trial results (30, 31). Subsequently, these guidelines were consolidated into Consolidated Standards of Reporting Trials (CONSORT) in 1996, and updated in 2001; these are now available online at www.consort-statement.org (19, 32). The current CONSORT statement consists of a 22-item checklist (Table 2) and a diagram template for patient flow in a clinical trial. In addition to CONSORT, other groups have developed publication standards for clinical trials, although they are less widely disseminated, including the Good Publication Practice for Pharmaceutical Companies (www.gpp-guidelines.org). A brief overview of selected topics discussed in the CONSORT statement follows.
|
Over- and underestimation of benefit and underestimation of harm are ethical concerns identified by clinical trial experts (Table 1), and these factors are addressed in the CONSORT checklist (Table 2). Sufficient sample size is critical to have power to detect statistically significant differences. Moreover, statistical significance is not equivalent to clinical significance or relevance. Clinical relevance is based on patient-centered outcomes and the number needed to treat (NNT). The NNT is the cornerstone for evaluating clinical relevance, and is the reciprocal of the absolute risk reduction between the treatment and control groups. However, the relevance of the NNT is subject to interpretation and should be part of the discussion of any clinical trial publication. Similarly, the number needed to harm (NNH) is calculated as the reciprocal of the absolute risk increase for adverse outcomes between the intervention and control group. In small studies with relatively few subjects, the process of randomization and control may not be sufficient to assess safety, such that ongoing postmarketing, observational studies are needed (4).
The issue of external validity refers to the study results' generalizability "relevant to a definable group of patients in a particular clinical setting" (33). Rothwell (33) identifies six factors that affect external validity, including the type of health care system in which the clinical study is conducted (e.g., primary vs. secondary or tertiary care, private [United States] vs. nationalized [Canadian or United Kingdom] health care systems), to variations in intensity of trial safety procedures (which can lead to misleading conclusions about an intervention's safety in routine clinical practice). Lack of consideration of external validity in clinical trial design may have detrimental effects on health care delivery, resulting in clinical inertia—the underuse of efficacious treatments (34). A 2001 report from the Institute of Medicine stated that it takes 17 years, on average, for the incorporation of new knowledge from randomized, controlled trials into clinical practice, and the reasons for this inertia include lack of time, poor access to or recall of study results, and beliefs about external validity (35).
In addition to publication standards, statistical methods have been used to detect data fabrication (36). However, this approach requires access to the research data, which would be facilitated with clinical trial registration.
Open access publishing.
Another approach to overcome publication bias and to improve access to the tremendous growth in research output is a growing movement to open-access publishing (5, 37). Because of the massive volume of published research results, it is increasingly difficult for physicians and the public to access and use important new knowledge from clinical trials. In recognition of this problem, a major recommendation in the 2001 Institute of Medicine report "Crossing the Quality Chasm" was that the "The Secretary of DHHS should be given the responsibility and necessary resources to establish and maintain a comprehensive program aimed at making scientific evidence more useful and accessible to clinicians and patients" (35). Although there is yet to be a national initiative addressing this recommendation, there are examples of online publishers attempting to increase access to research results, including BioMed Central (www.biomedcentral.com) and the Public Library of Science (www.plos.org); the Journal of Negative Results in Biomedicine (www.jnrbm.com) was launched to "receive papers on all aspects of unexpected, controversial, provocative and/or negative results/conclusions in the context of current tenets, providing scientists and physicians with responsible and balanced information to support informed experimental and clinical decisions." The cost of open-access publishing may limit the effectiveness of this approach to overcoming publication bias (37).
CONCLUSIONS
At the individual level, little is known about the factors that contribute to unethical interpretation and communication of research results. Many individual and institutional factors may have a role, including individual attitudes, knowledge, beliefs, and motivations, as well as institutional policies and expectations. Although not directly related to interpretation and communication, limited evidence suggests that institution-related factors, such as institutional review board decisions, may partly contribute to researcher frustration and unethical behavior in research conduct (38). It is not known to what extent similar experiences with peer review, editorial decision-making, and promotion process may contribute to unethical behavior in the interpretation and communication of research results. Better understanding of the many factors that may lead to unethical behavior offers the potential to target specific issues to prevent research misconduct.
The goal in clinical trial reporting is to conduct unbiased data analysis and communication of results. Furthermore, interpretation and communication of research results should neither provide an overly optimistic estimate of benefit nor underestimate the risks. To maximize access to new knowledge about benefits and risk, and minimize bias in the interpretation and dissemination of results, all trial results should be available to the scientific and regulatory communities, as well as to the general public, especially those who volunteer to participate as subjects.
DISCUSSION HIGHLIGHTS
Registries
DR. ROBERT R. FENICHEL (Independent Consultant, Washington, DC): Several of the major journals have said they're not going to publish papers whose protocols were not frozen in the registry before recruitment. That may be inadequately appreciated by most researchers. This pledge by journals was originally designed to address publication bias, but even if the trials are not that interesting to journals, one should be able to go back to the data and do formal or informal meta-analyses and see what really happened.
I think the most important effect of the registries is going to be this process of creating snapshots of the protocol before the results are known. At the FDA, where I had access to original protocols, I often found that when a trial was finally described in a publication, the primary endpoint was reported as having been different from what the protocol had said it was going to be. Registries will make that sort of chicanery harder to do.
DR. ROBERT A. WISE (Johns Hopkins University School of Medicine, Baltimore, MD): The registries are a good idea, because they address this perception that researchers are burying negative or imperfect results. However, I think that the journal editors were being somewhat disingenuous when they say, "We're not going to publish anything that hasn't been put into the registry." They are the ones that have not accepted the studies with negative results, so they are somewhat culpable too, particularly the editors of the leading journals.
DR. FENICHEL: There are two sorts of negative results. First, there are the negative results from large trials that are powered properly, so the negative results mean that the effects, if any, are very small). Those results get published fairly often. Second, there are the negative results of small trials that, in hindsight, should have been better powered, and that should never have been expected to find any convincing effects of the study interventions. Those trials are not published.
DR. WISE: The registries don't take the results and actually don't have very much detail about the protocol. They just indicate the study topic.
DR. COULTAS: The registry is simply that, it's a registry. Disclosing the trial results is a separate issue. However, with any compound, including the ones that don't make it to market, there may be 100 to 200 experiments (including preclinical data in vitro and in animals), which is all submitted to the FDA.
Most of the U.S. pharmaceutical companies have now agreed to post all their results. There are some companies that have been mandated—due to poor past behavior—by attorneys general to do this. But posting results is relative because clinical trial reports are just a stack of information with results. Some top-line results get there but not everything gets there, so it's a question of how much disclosure—sometimes the small details are important.
Also, this is only regarding drugs that are already approved. Pharmaceutical companies don't have to report on preapproved drugs; that is much more voluntary.
DR. FENICHEL: There is the problem of sharing too much information prematurely. Every once in awhile, a trial has been stopped because of apparent inefficacy (or even harm), but concomitant larger trials of the same treatment have convincingly shown that the first trial's results were just bad luck. If the first trial's discontinuation had been broadcast, the other studies might have been aborted too. You can imagine a cascade effect, leading to abandonment of parallel compounds too. There isn't a simple answer to the sharing of information.
DR. COULTAS: I haven't seen specific examples in the literature on respiratory disorders. Generally, though, finding clinically relevant outcomes for chronic disease (e.g., COPD) is an active area of discussion. Is it really meaningful to the patient? Not only from the individual patient, but then also from the societal perspective in terms of cost-effectiveness, and more of that kind of work.
FOOTNOTES
Supported by Boehringer-Ingelheim and Pfizer, Inc.
Conflict of Interest Statement: D.C. received $12,000 in 2005 for speaking at conferences sponsored by Pfizer, and $2,000 for speaking at this conference on research ethics, sponsored by Boehringer-Ingelheim and Pfizer, which formed the basis for his article. R.R.F. has served as a consultant to Aveo Pharmaceuticals, Abbott Laboratories, BioCryst Pharmaceuticals, Boehringer-Ingelheim Pharmaceuticals, Bracco Diagnostics, Bristol-Myers Squibb, CV Therapeutics, CoTherix, Corgentech, Daiichi Sankyo Pharma, Eli Lilly & Company, Eyetech Pharmaceuticals, Forest, H. Lundbeck A/S, Hope Medical Enterprises, Idun Pharmaceuticals, Janssen Pharmaceuticals, Johnson & Johnson, KAI Pharmaceuticals, King Pharmaceuticals, Laboratoires Fournier, Lifescape BioSciences, Luitpold Pharmaceuticals, LungRx, MDS Pharma, MedImmune, Medtronic, Merck, Mylan-Bertek Pharmaceuticals, Navitas Pharma, Neurochem, NicOx, NovaCardia, Orion Pharma, Otsuka America Pharma, Prestwick Pharmaceuticals, Reliant Pharmaceuticals, Roxane Laboratories, Sanofi-Aventis, Schwarz BioSciences, Scios, Servier Canada, Shire Development, Solvay, TAP Pharmaceutical Products, Teijin America, Vanda Pharmaceuticals, Watson Pharma, Wyeth-Ayerst, and Yamanouchi Pharma America. R.A.W. has served as a paid consultant to Boehringer-Ingelheim and Pfizer and received payment for the preparation of his article and attendance at this symposium, at which it was presented. He also receives research funding from Boehringer-Ingelheim, which provided support for this symposium, which generated his article.
(Received in original form January 3, 2007; accepted in final form February 13, 2007)
REFERENCES
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
