[R] Power of t-test in R vs. S-PLUS

[Peter Dalgaard BSA]

Prof Brian D Ripley <ripley at stats.ox.ac.uk> writes:


> BTW, the t-test that power.t.test computes the power of is not the default
> t-test in R, as I understand the code.  (?power.t.test is silent on which
> two-sample t-test but the calculations look right for the one that uses the
> pooled variance.  Not that it necessarily matters.)

Exactly. For equal sample sizes the equal and non-equal variance
t-tests are essentially the same unless the variance difference is
huge (the t statistic is identical in that case, although the degrees
of freedom can be different). The important cases for deciding between
the Welch test and the equal-variance one are when you're comparing a
small group with a large variance to a large group with a small
variance. 

The Welch test relies on an asymptotic expansion, and I wouldn't know
how it behaves in very small sample cases, so it did seem best to
stick with the plain "classical theory" t-test where all the
distributions can be worked out exactly.

The help page does seem to be improvable (said the author...)

> 
> This is only evident because n1 and n2 are so small: at those sample sizes
> you are relying critically on normality of the samples, and for
> power.t.test on equal variances.  For n1 = n2 = 25, the difference is much
> smaller (0.193 vs 0.200).

Yes. This is mostly to do with the degrees of freedom issue though.
Neither of the procedures are any good at correcting for
non-normality. The central limit theorem does that to some extent, and
for equal sample sizes skewness tends to cancel out (in the two sample
case, of course).

Anyways, in my line of business it is quite common for people to
design studies with single-digit numbers per group, and the
traditional normal approximations can be 10-20% off target on the
sample sizes relative to the exact power calculation using the
noncentral t, so I though it would be a good thing to include that
correction, even though multiple other factors could bias the
calculations.


-- 
   O__  ---- Peter Dalgaard             Blegdamsvej 3  
  c/ /'_ --- Dept. of Biostatistics     2200 Cph. N   
 (*) \(*) -- University of Copenhagen   Denmark      Ph: (+45) 35327918
~~~~~~~~~~ - (p.dalgaard at biostat.ku.dk)             FAX: (+45) 35327907
-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._