The analogy is not mine, it is used by many epidemiologists/statisticians while explaining how to design/interpret clinical trials.
The "beyond a reasonable doubt" is a metaphor even in criminal trials.
The analogy is very clear, to prove guilty (effective), not to prove innocent (non-effective)which does not matter.
The null hypohtesis is precisely that: reject if there is no effect. Why don't they go with an "effect" hypothesis right immediately if that is what they are looking for? cause it must dispelled doubt!
Why 1 in 20 chances? for a drug that saves lives it should be 1 in 10, 1 in 5, even less should be fine 1 in 2, if the effect is consistent why not go for 1 in 2? Cause the burden of standard proof is 1 in 20 quite BEYOND reasonable doubt.
If We are so "pro-saving lives" We should accept less than 1 in 20 (a big burden). And single "tail" test (not double tails), and lower Power (60% to 70%, not the usual 80% to 90%).
Besides, there are the safeguards of POWER determining the size of the sample to accept the 1 in 20, for some big double digit % difference a sample of 100 would be fine, but for single digit % difference a sample must be in the several 1000s range.
Add to that the confidence interval to assure the effect need to fall within certain expected minimal limits.
And even more complex for multivariate and subgroup analysis.
This "safeguards" are tough to satisfy the "beyond reasonable doubt" clause.
There is big debate among clinicians/statisticians (FDA included)/drug industry to accept not only small differences as "proof" like in sepsis trials attempt to decrease mortality (cause of the tons of failures, and the consistent good quality results with 3% to 4% differences) and the use of 1 in 10 as an acceptable compromise.
Yes, costs and hope are factors to be considered, cause "1 in 20" and big % differences, and large number of subjects trials are very costly and takes longer times affecting hope AND actual lives (delays), and there is a reasonable big movement (and many non reasonable ones) to lesser the burden of proof necessary to declare a drug "guilty (effective), beyond a reasonable doubt".
Erbitux test conditions: 1. at least 15% of responding subjects, 2. each response must be at least a 50% decrease in tumor mass. It is very stringent (I agree with the rigid requirement) but if 10% of subjects had a consistent proven 40% decrease is that "unreasonable" (No, it is very impressive) but the burden of proof is tough:
...BEYOND a reasonable doubt, which is not the same as to say "beyond reason" !!! |
