[R] Power of t-test in R vs. S-PLUS

[Thomas Lumley]

On Fri, 1 Mar 2002, Alan T. Arnholt wrote:

> Could someone shed some light on the statement from Thomas
> that "(with sampling uncertainties of about +/- 2% in each
> number)".  What exactly is being said about the accuracy of
> the simulation and how is that number +/-2% being
> determined.
>

The power is estimated by doing 10000 simulated experiments and finding
that 7977 of them give a p-value below 0.01.

The number of p-values less than 0.01 is random (these are simulations),
and has a Binomial(10000, P) distribution, where P is the power.  It is
possible to construct an exact 95% confidence interval for P based on the
observed results of the simulations. The binom.test function does this and
says that a 95% ci for the power is (0.790,0.806).  Similarly for the
delta specified by S-PLUS the 95% ci for the power is (0.537,0.557).  The
sampling error is actually about +/- 1% (I was incorrectly rounding in my
previous email).

If you had good parameter values to use in a power calculation and wanted
to be as precise in your claims as possible you might do a simulation like
this and quote a suitable lower confidence limit for the power. You can
get very tight limits if you want. For example, a lower 99.99% confidence
limit for the power based on this simulation is 0.782.

It matters more when the problem is more complicated and you can't readily
do 10000 replicates in the simulation.

	-thomas

-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._