Where's the Bias?
Annals of Internal Medicine 15 June 1997. 126:986-988.
Related Letters
acponline.org
In July 1996, Annals published the results of a double-blind, randomized, controlled trial describing the effects of zinc lozenges on the symptoms of the common cold (1). The lozenges, produced by the Quigley Corporation of Doylestown, Pennsylvania, seemed to reduce the duration of symptoms by about 40% compared with placebo, confirming the results of several previous studies but contrasting with the findings of others. As noted in the published article, the study was supported by funds from the researchers' institution; the Quigley Corporation supplied the active medication and the placebo.
In the cover letter that accompanied the manuscript, the authors told us that none of them had held any financial interest in Quigley before designing and conducting the study but that after the randomization code was broken and the results analyzed two of the authors, Dr. Macknin and Ms. Medendorp, had acquired stock in Quigley through private sale. Annals was therefore duly informed about a potential conflict of interest. In our judgment, the potential for bias from commercial influence was minimal because the financial link between the authors and the company occurred after the fact of the research; we therefore chose not to publish the authors' disclosure.
When the study appeared, neither the press nor the public took much notice of the results, although the value of Quigley stock increased slightly. With November, however, came a rapid drop in outdoor temperatures and an even faster increase in the price of Quigley stock, which triggered extensive coverage in the general press of both the conduct and publication of the study itself and of subsequent business developments (2-7). Most of this coverage was accurate; some of it was misleading (8). Despite the substantial "eyebrow raising" over the financial decision of Dr. Macknin and Ms. Medendorp, the scientific quality of the study has received little criticism (9-12); in fact, it has received very little attention at all, which is curious in light of the inference in most of the commentary that conflict of interest automatically makes for bad science.
The essential task of science is to create and test hypotheses about how the world works, leading ultimately to the development of theory that explains and predicts. Hypotheses, by their very nature, reflect points of view; they thus remain "hypothetical" until their validity is tested by experimentation, the details of which must be open for others to examine and corroborate. Anything that distorts hypothesis testing is considered to be a form of bias and therefore antithetical to the fundamental nature of science. A correspondingly large proportion of the effort and skill in science, including medical research, goes into minimizing bias. However, because bias can never be completely eliminated, a great deal of additional effort goes into detecting, disclosing, and correcting for the biases that inevitably remain.
The list of potential biases in the conduct of research is long: statistical biases, such as the play of chance (13); biases associated with study design, such as lack of controls (14); and cognitive (psychological) biases, such as representativeness (15). Potentially important biases from other sources are also at work in academic settings: for example, the pressure to obtain grants, the need to publish for promotion, and an institution's interest in producing groundbreaking research results. Scientists also confront a wide range of personal biases, including everything from family relationships to academic loyalties and scholarly competition; these last two are sometimes referred to collectively as "intellectual passion."
To those outside of the arcane world of science, however, bias involving money is in a class by itself; to many, it appears, financial bias is the only form of bias t hat matters, almost the only one that exists. Most academic researchers are aware that financial links between medical scientists inside academia and the commercial sector outside it can create powerful biases. But to most researchers, struggling daily with the demons of the scientific method, it appears that a financial relationship, if present, is simply one factor among many with the potential for creating bias, and rarely the most important one at that.
As if these complexities weren't daunting enough, the link between commerce and medical research is not monolithic but comes in three distinct forms: industry-project, industry-scientist, and industry-clinician relationships. In the first, a company provides financial or technical support for a specific research project. In the second, a company and an individual scientist establish a relationship. Sometimes extending over many years, these relationships involve professional activities of many kinds (for example, industry-sponsored lectures and paid consulting) and often have little or nothing to do with a particular research project. In the third, a company provides goods and services, such as books, reprints, lectures, and detailing, to clinicians, who are the ultimate consumers of most medical research results. Each type of link creates its own special tensions. Industry-clinician relationships, for example, tend primarily to blur the distinction between marketing and education rather than raising concerns for scientific publishing.
The science in many industry-sponsored studies is superb. Nonetheless, it is not unreasonable to assume a priori that the potential for bias is inherent in the conduct and reporting of industry-supported research, given the direct and specific nature of the link between company and project; this assumption is supported by recent accounts (16). For this reason, since January 1995, Annals has required the authors of industry-supported studies to describe the role of the sponsoring company (if any) in the collection, analysis, and interpretation of the data, activities that, from an editor's point of view, are issues of authorship. Perhaps most important, we require information on whether the company controlled the decision to publish the paper-the ultimate prerogative of authorship. Because, as noted, we view the bias introduced by such commercial links as a priori no different, qualitatively or quantitatively, from other project-related methodologic biases (17), we ask authors to include such information in the Methods sections of their papers (18) rather than in a footnote or appendix outside the text.
When it comes to relationships between industry and individual scientists, however, things get trickier. How much of a connection is too much? On the one hand, to be sure, we ignore substantive relationships at our peril; on the other, it would be a mistake to become obsessed with relationships that are trivial (the expression De minibus non curat lex recognizes a similar principle in the law). Does giving lectures on a disease that can be treated with a sponsor's product create meaningful conflict? Does serving as a consultant or research advisor to a company? Does owning, say, $100 of a company's stock create meaningful bias? What about $500? Does it matter that an author was a consultant for a company 2 years ago? Ten years? If an author holds an equity interest in a company before beginning a study, few would doubt the potential for conflict of interest. But suppose an author obtains an equity interest after completing a study, rather than before or during it? What then?
Impressions to the contrary (8), Annals, like many other medical journals, has for years asked authors to disclose the nature and extent of their links with industry so that editors, reviewers, and readers can judge the potential for bias from those links. Such disclosure is entirely analogous to the mechanism that has long served as the touchstone for dealing with methodologic bias generally: the requirement for thorough, accurate reporting of study methods. Given the enormous variability of individual-industry links, however, it does not seem appropriate to assume a priori that a link between an individual author and industry automatically creates a meaningful conflict of interest; this is in contrast to the situation of industry support for research projects. Because of this variability and because the potential for meaningful bias depends on both the quality and quantity of the link, we use two different expressions, "dual commitment" and "conflict of interest," in Annals Information for Authors (18) to distinguish between those relationships that are not likely to bias studies in a meaningful way and those that are.
The wide variation in author-industry relationships is reflected in the enormous range of editorial policies of biomedical journals. Most of these journals are willing to publish original scientific studies whose authors disclose the existence o f substantial financial relationships with industry (or are industry employees), arguing that the rationale, methods, and data are in the published work for all to see. But whereas some journals publish all such disclosures, others, including Annals, view the publication of disclosures as a matter of editorial judgment. And some of the same journals that accept original research by authors with declared ties to industry will not even consider, much less publish, review articles and editorials by authors with industry ties, arguing that these "secondary" formats provide too many opportunities for the introduction of bias, conscious or unconscious (19). At the other extreme are journals that see the potential for individual conflict of interest as irrelevant, or worse. Nature, for example, states that it has never required authors to declare such links "because the reasons proposed by other are less than compelling" (20). Further, it asserts that nowadays "virtually every good paper with a conceivable biotechnological relevance . . . has at least one author with a financial interest&emdash;but what of it?" The editor of The Lancet has recently spelled out a number of major fallacies that underlie the case for full disclosure (21). Kenneth Rothman, the editor of Epidemiology, has even suggested that the automatic assumption of conflict of interest for any scientist who works with industry "borders on the unethical" (22). In short, editors differ widely in their views on the potential significance of authors' relationships with industry, and editors can and do make judgments all the time about the meaning of specific relationships.
What, then, of the disclosure that two authors of the zinc lozenge study obtained equity interests in the product after they finished their study? All scientists have the opportunity to invest after the fact in discoveries that they have helped to make. Profiting from finished medical work may not be uncommon, although we have n o idea how often it happens because editors generally don't ask about after-the-fact investments and authors don't tell. In this respect, Dr. Macknin and Ms. Medendorp were unusually forthcoming. It could be argued that at the time they designed and conducted the study, these authors might have been thinking about acquiring future equity interest if the lozenges turned out to be effective and that such thoughts might have biased the trial. The truly zealous might, on that basis, argue that editors should require authors to disclose all relevant financial hopes and dreams, realized or unrealized, particularly any that occurred to them before they planned and carried out their research. Fortunately, medical journals have not yet become thought police.
Does the study in question prove the efficacy of zinc in controlling the symptoms of the common cold? Of course not. But, as editors, we agreed with our outside reviewers and statistical consultant that the work was well designed and carefully executed&emdash;in a word, that it was good science. Like most well-conducted studies, it adds a small but credible bit of evidence to the existing data on the subject, in this case increasing our confidence, however slightly, in the likelihood of a true effect (23). Should we have published the information about the authors' post hoc financial interests along with their paper? Perfect hindsight suggests that doing so might have been useful. However, we believed at the time of publication and still believe now that meaningful bias is more likely to be found in the drab intricacies of scientific methodology than in the drama of the marketplace. Clearly, constant vigilance is required to prevent conflict of interest from distorting research. We agree, however, that excessive concern with financial conflict diverts readers' attention from the scientific argument and, perhaps more importantly, that our current obsession with conflict of interest runs a substantial risk for harming free discussion in science (21).
Frank Davidoff, MD
Editor
Ann Intern Med. 1997;126:986-988. Annals of Internal Medicine is published twice monthly and copyrighted © 1997 by the American College of Physicians.
References
1. Mossad SB, Macknin ML, Medendorp SV, Mason P. Zinc gluconate lozenges for treating the common cold. A randomized, double-blind, placebo-controlled study. Ann Intern Med. 1996;125:81-8.
2. Gelles J. Hope for a cure for colds sets off rush for lozenges. Philadelphia Inquirer. 19 December 1996:A1.
3. Alpert B. Cure for the common cold? As Quigley shares soar, questions arise about medical studies, shady associates. Barron's 1997 January 13:18-9.
4. Hamilton K. Reaching for the rock. Zinc is the hot cold remedy, and sales are nothing to sneeze at. Newsweek. 20 January 1997:74.
5. Day K. Cold researcher made profit on Quigley shares. Washington Post. 31 January 1997:G1.
6. Hilts P. Researcher made profit after study. New York Times. 1 February 1997:6.
7. Shaw D. Drug researchers' investments are debated. Philadelphia Inquirer. 4 February 1997:C1.
8. Who's minding the lab? Tufts researcher puts scientific bankbooks under the microscope. Tufts University Health and Nutrition Letter. 1997;15:4-5.
9. Farr BM, Hayden FG, Gwaltney J Jr. Zinc gluconate lozenges for treating the common cold [Letter]. Ann Intern Med. 1997;126:738.
10. Shumate MJ. Zinc gluconate lozenges for treating the common cold [Letter]. Ann Intern Med. 1997;126:738.
11. Barton JC, Bertoli LF. Zinc gluconate lozenges for treating the common cold [Letter]. Ann Intern Med. 1997;126:738-9.
12. Macknin ML, Mossad SB, Medendorp SV. Zinc gluconate lozenges for treating the common cold [Letter]. Ann Intern Med. 1997;126:739.
13. Ingelfinger JA, Mosteller F, Thibodeau LA, Ware JA. Biostatistics in Clinical Medicine. 2d ed. New York: Macmillan; 1987.
14. Sackett DL, Haynes RB, Guyatt GH, Tugwell P. Clinical Epidemiology: A Basic Science for Clinical Medicine. 2d ed. Boston: Little, Brown; 1991.
15. Kahneman D, Slovic P, Tversky A. Judgment Under Uncertainty: Heuristics and Biases. New York: Cambridge Univ Pr; 1982.
16. Rennie D. Thyroid storm [Editorial]. JAMA. 1997;277:1238-43.
17. Davidoff F. Annals now and then [Editorial]. Ann Intern Med. 1996;124:67-8.
18. Information for authors. Ann Intern Med. 1997;126(1):I11-7.
19. Angell M, Kassirer JP. Editorials and conflicts of interest [Editorial]. N Engl J Med. 1996;335:1055-6.
20. Avoid financial "correctness" [Editorial]. Nature. 1997;385:469.
21. Horton R. Conflicts of interest in clinical research: opprobrium or obsession? Lancet. 1997;349:1112-3.
22. Medicine and the Media: A Changing Relationship: October 13-14, 1994, Cantigny, Wheaton Illinois. Chicago: Robert R. McCormick Tribune Foundation; 1995:81. (Ethiel N, ed; The Cantigny Conference Series).
23. Brophy JM, Joseph L. Placing trials in context using Bayesian analysis. GUSTO revisited by Reverend Bayes. JAMA. 1995;273:871-5. |
