[R] t-test bootstrap versus permutation question

Torsten Hothorn Torsten.Hothorn at rzmail.uni-erlangen.de
Fri Dec 13 09:49:03 CET 2002


> Hi,
> 
> I have a little problem that puzzles me about contradictory results returned
> by a bootstraped t-test (as in MASS 3rd ed p. 146) versus a permutation
> t-test (as in MASS 3rd ed, p 147).
> 
> Data are measurements done on 100 cells in 9 slides of fish blood. With one
> method, cells are randomly sampled, and with the other method, the operator
> selects cells arbitrarily (in a way it is done usually with this test). We
> want to determine wheither the methods yield same results or not. Since we
> are interested by the mean measurement for 100 cells, we take the average
> for each slide and each method. We compare then the nine paired samples
> (that is, for the nine slides) with a paired t-test. However, since we
> cannot make the hypothesis that both distributions are normal, we prefer to
> use a bootstraped test.
> 

that is: you have 9 differences and you hypothesis is "the underlying
distribution is symmetric about zero", i.e. the two methods do not
differ in this sense, right?


> We do:
> (1) 1000 simple bootstraps with:
> boot(B-A, function(x,i), mean(x[i]), R=1000)
> and then:
> boot.ci(...)
> and check wheter the CI includes 0 (no significant difference between
> methods) or not.

so you are computing a bootstrap estimate of the standard error of the
mean for later use in a confidence interval for the mean? But you can
calculate this directly (Section 5.2 in Efron/Tibshirani: Intro to the
Boostrap) and the confidence set per t.test, if I'm not completely
misguided.

> 
> (2) a permutation test with the perm.t.test() function of MASS p. 147
> and calculate a bootstraped p-value corresponding to the fraction of values
                  ^^^^^^^^^^^

I just can't find my MASS3 at the moment, but I suspect perm.t.test
computes the statistic for all possible permutations, so no bootstrap
here.

> larger or equal to the observed one. If this p-value is > 5%, we consider
> there is no significant difference between both methods.
> 
> Is this correct?
> 
> The problem is that, in our particular case, both test give opposite
> results: the bootstrap test indicates significant differences at 5%, while
> the permutation test gives p-value = 0.35-0.45, thus no differences between
> methods. I think I probably miss something here! Does somebody could help
> me?

maybe you should post the 9 measurements ;-) 
For the shoes data, one could do the following, which looks consitent to
me:

R> 
R> library(MASS)
R> library(exactRankTests)
R> data(shoes)
R> attach(shoes)
R> t.test(A,B, paired=TRUE, conf.int=TRUE)

	Paired t-test

data:  A and B 
t = -3.3489, df = 9, p-value = 0.008539
alternative hypothesis: true difference in means is not equal to 0 
95 percent confidence interval:
 -0.6869539 -0.1330461 
sample estimates:
mean of the differences 
                  -0.41 

R> wilcox.exact(A,B, paired=TRUE, conf.int=TRUE)

	Exact Wilcoxon signed rank test

data:  A and B 
V = 3, p-value = 0.007812
alternative hypothesis: true mu is not equal to 0 
95 percent confidence interval:
 -0.7 -0.1 
sample estimates:
(pseudo)median 
          -0.4 

R> wilcox.test(A,B, paired=TRUE, exact=FALSE)

	Wilcoxon signed rank test with continuity correction

data:  A and B 
V = 3, p-value = 0.01431
alternative hypothesis: true mu is not equal to 0 

R> perm.test(A*10,B*10, paired=TRUE) # map into integers

	1-sample Permutation Test

data:  A * 10 and B * 10 
T = 3, p-value = 0.01367
alternative hypothesis: true mu is not equal to 0 


best, 

Torsten

> 
> Best,
> 
> Philippe Grosjean
> 
> ...........]<(({°<...............<°}))><...............................
> ( ( ( ( (
>  ) ) ) ) )      Philippe Grosjean
> ( ( ( ( (
>  ) ) ) ) )      IFREMER Nantes - DEL/AO
> ( ( ( ( (       rue de l'Ile d'Yeu, BP 21105, 44311 Nantes Cedex 3
>  ) ) ) ) )      tel: (33) 02.40.37.42.29, fax: (33) 02.40.37.42.41
> ( ( ( ( (	e-mail: philippe.grosjean at ifremer.fr
>  ) ) ) ) )
> ( ( ( ( (      "I'm 100% confident that p is between 0 and 1"
>  ) ) ) ) )                                L. Gonick & W. Smith (1993)
> .......................................................................
> 
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> http://www.stat.math.ethz.ch/mailman/listinfo/r-help
>




More information about the R-help mailing list