[R] acceptable p-level for scientific studies

Frank E Harrell Jr fharrell at virginia.edu
Wed Dec 18 15:20:14 CET 2002


On Wed, 18 Dec 2002 14:44:05 +0200
Kyriakos Kachrimanis <kgk at pharm.auth.gr> wrote:

> Dear list members,
> 
> I have a statistical question, that doesn't belong to this list, and I
> apologise for that in advance but I would appreciate your help very much.
> Is there some convention for selecting the a level for significance testing
> in scientific (e.g. chemical processes) studies? Most people use the 0.05
> level but I could not find a reference to justify this. Why not 0.01 or 0.1?
> Montgomery in his book "Design and Analysis of Experiments" disagrees with
> setting a priori acceptable levels at all. Is it necessary to set a limit
> for significance testing since R can provide exact probability levels for
> the significance of each effect?
> 
> Thanks in advance.
> 
> Kyriakos Kachrimanis.
> 

Want to open up the floodgates?  Some personal opinions:

- Alpha=0.05 is arbitrary, silly, and boring
- Reporting P and letting the reader decide is a bit better
- Bayesian posterior probabilities are still better although
  more thinking is required
- Confidence limits can be good compromise solutions (some journals are
  almost disallowing P-values in favor of CLs)
- P-values are dangerous, especially large, small, and in-between ones.
  See http://hesweb1.med.virginia.edu/biostat/teaching/bayes.short.course.pdf for a full sermon.
-- 
Frank E Harrell Jr              Prof. of Biostatistics & Statistics
Div. of Biostatistics & Epidem. Dept. of Health Evaluation Sciences
U. Virginia School of Medicine  http://hesweb1.med.virginia.edu/biostat




More information about the R-help mailing list