SkepDic.com

Click to order from Amazon

file-drawer effect

The file-drawer effect refers to the practice of researchers filing away studies with negative outcomes. Negative outcome refers to finding nothing of statistical significance or causal consequence, not to finding that something affects us negatively. Negative outcome may also refer to finding something that is contrary to one's earlier research or to what one expects.

The practice of reporting and publishing only positive-outcome research creates a misrepresentation of the subject under investigation, especially if a meta-analysis is done.

One criticism of parapsychology has been that its researchers have ignored studies with negative outcomes. In 1975, the American Parapsychological Association established an official policy against the selective reporting of only positive results.

Little research seems to have done on the extent of the practice of scientific researchers to file away studies with negative outcomes. Brian Martinson, an investigator with the HealthPartners Research Foundation, led a study for the scientific journal Nature (published in June 2005). Martinson and his colleagues sent a survey to thousands of scientists funded by the National Institutes of Health (which commissioned the study) and received 1,768 responses with usable data from the 3,600 surveys mailed to mid-career scientists. They received 1,479 responses with usable data from 4,160 surveys sent to early-career scientists (Martinson et al. 2005). The respondents were allowed to remain anonymous. Of the scientists who responded, 6.0 percent admitted  to having "tossed out data because the information contradicted their previous research." And more than 15 percent admitted they had ignored observations because they had a "gut feeling" they were inaccurate.* Martinson et al. write:

our approach certainly leaves room for potential non-response bias; misbehaving scientists may have been less likely than others to respond to our survey, perhaps for fear of discovery and potential sanction. This, combined with the fact that there is probably some under-reporting of misbehaviours among respondents, would suggest that our estimates of misbehaviour are conservative.

See also meta-analysis.

further reading

• Filed under F (for forgotten) by Dan Vergano, May 16 2001
• Stenger, Victor J. (2002). "Meta-Analysis and the File-Drawer Effect." Skeptical Briefs.
• Scientists who do not publish trial results are "unethical" by Gavin Yamey, BMJ
• Bias against negative studies in newspaper reports of medical research by G. Koren and N. Klein
• Scientists behaving badly by Brian C. Martinson, Melissa S. Anderson, and Raymond de Vries.  Nature 435, 737-738. June 9, 2005.
• Publication Bias: The “File-Drawer” Problem in Scientific Inference by Jeffrey D. Scargle

fetish

	Web skepdic.com

firewalking