Epidemiologists and biostatisticians have evolved numerous ways of reducing bias in clinical trials. Randomization of patients, and masking them to their treatment allocation are two. Another is masking clinicians who assess their outcomes.Why are these simple measures so rarely used in preclinical animal studies? And do animal studies show exaggerated effects as a consequence of poor methodology?
The March 2008 issue of Stroke reports a "meta-meta-analysis" of 13 studies comprising over fifteen thousand animals. Perhaps surprisingly, the study did not show a relationship between the use of randomization or masked outcome assessment and the size of treatment effect. It did, however, show a positive relationship between size of treatment effect and failure to mask investigators during treatment allocation.
This is probably the largest analysis of its kind. It isn't perfect: publication bias is very likely to skew the analysis. For example, size of treatment effect is likely to strongly influence whether a study gets published. If so, effects of methodological bias could be obscured; preclinical researchers might simply be stuffing their methodologically rigorous studies in their filing cabinets because no effect was observed.
The conclusion I draw? Preclinical researchers should randomize and mask anyway. There is some evidence it matters. Moreover, the logical rationale is overwhelming, and the inconvenience for investigators seems more than manageable. (photocredit: Chiara Marra 2007)
2 comments:
Hello,
I just came across your post and could not agree more strongly, both with you and with Crossley et al. Coicidentally, I had a letter published in April (7 months after submitting!) about the same topic as it applies to failed trials in neurodegenerative diseases. Also, check out an article by M.D. Lindner in Pharmacol. Ther. 7/2007...
Thanks, Noam
Thanks, Noam, for your comments. I had caught Lindner's article, but not your excellent letter in Neurology.
There is a virtual cottage industry of articles on methodological quality in preclinical neurological studies-- and to some extent, groups like STAIR seem to be taking steps to directly address the issue.
Strangely, however, there is comparably little written about the cancer preclinical literature-- or virtually any other preclinical research area. I've often wondered why that is...
Post a Comment