[R] t-test bootstrap versus permutation question

[kjetil halvorsen]

Philippe Grosjean wrote:
> 
> Hi,
> 
> I have a little problem that puzzles me about contradictory results returned
> by a bootstraped t-test (as in MASS 3rd ed p. 146) versus a permutation
> t-test (as in MASS 3rd ed, p 147).
> 
> Data are measurements done on 100 cells in 9 slides of fish blood. With one
> method, cells are randomly sampled, and with the other method, the operator
> selects cells arbitrarily (in a way it is done usually with this test). We
> want to determine wheither the methods yield same results or not. Since we
> are interested by the mean measurement for 100 cells, we take the average
> for each slide and each method. We compare then the nine paired samples
> (that is, for the nine slides) with a paired t-test. However, since we
> cannot make the hypothesis that both distributions are normal, we prefer to
> use a bootstraped test.

But with a paired t-test you doesn't need to assume that both
distributions are normal, it holds that the difference is? can you
assume that? The differen can be large, as if both distributions are
equal, 
the difference certainly is symmetric.

Kjetil Halvorsen

> 
> We do:
> (1) 1000 simple bootstraps with:
> boot(B-A, function(x,i), mean(x[i]), R=1000)
> and then:
> boot.ci(...)
> and check wheter the CI includes 0 (no significant difference between
> methods) or not.
> 
> (2) a permutation test with the perm.t.test() function of MASS p. 147
> and calculate a bootstraped p-value corresponding to the fraction of values
> larger or equal to the observed one. If this p-value is > 5%, we consider
> there is no significant difference between both methods.
> 
> Is this correct?
> 
> The problem is that, in our particular case, both test give opposite
> results: the bootstrap test indicates significant differences at 5%, while
> the permutation test gives p-value = 0.35-0.45, thus no differences between
> methods. I think I probably miss something here! Does somebody could help
> me?
> 
> Best,
> 
> Philippe Grosjean
> 
> ...........]<(({°<...............<°}))><...............................
> ( ( ( ( (
>  ) ) ) ) )      Philippe Grosjean
> ( ( ( ( (
>  ) ) ) ) )      IFREMER Nantes - DEL/AO
> ( ( ( ( (       rue de l'Ile d'Yeu, BP 21105, 44311 Nantes Cedex 3
>  ) ) ) ) )      tel: (33) 02.40.37.42.29, fax: (33) 02.40.37.42.41
> ( ( ( ( (       e-mail: philippe.grosjean at ifremer.fr
>  ) ) ) ) )
> ( ( ( ( (      "I'm 100% confident that p is between 0 and 1"
>  ) ) ) ) )                                L. Gonick & W. Smith (1993)
> .......................................................................
> 
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> http://www.stat.math.ethz.ch/mailman/listinfo/r-help