[BioC] adjustment for dependent hypotheses when pairwise testing
smyth at wehi.edu.au
Mon Jan 16 04:33:15 CET 2006
A good question. The book on microarray expression data by McLachlan,
Do and Ambroise contains a review of FDR methods in Chapter 5, but a
more detailed review is much needed.
BTW, Yoav Benjamini has shown me that one can substitute a particular
first order approximation to pi0 into the BH method without harming
the ability of the method to work for small numbers of genes. So far,
I haven't been motivated enough to do that because the improvement is
usually relatively small for microarray data sets.
At 01:13 PM 16/01/2006, Naomi Altman wrote:
>This discussion convinced me to read through my collection of FDR
>papers, and I am finding it heavy going. Is there a review paper
>somewhere that goes through these methods + Storey et al (and maybe
>the Simes and Bonferroni FWER methods) and explains their
>applicability to the types of dependence structures typically found
>in e.g. factorial ANOVA where we may be testing main effects and
>interactions, or 1-way ANOVA with all pairwise comparisons, or
>eBayes tests like limma, where dependence is induced by the
>At 04:32 PM 1/14/2006, Gordon K Smyth wrote:
>>Actually I don't think the number of genes is a problem. Neither
>>BH nor BY use an estimate of pi0
>>(the true proportion of hull hypotheses). Whenever pi0 would
>>appear as a multiplier in the
>>formulae, they use 1 instead. The methods are therefore somewhat
>>conservative if pi0 is not
>>large, but they continue to control the FDR at below the requested level.
>>This means that BY gives rigorous control of expected FDR for any
>>dependence structure and any
>>number of genes, given only that the p-value distribution is valid
>>under the null hypothesis. BH
>>is the same except that it is theoretically valid only for certain
>>dependence structures (positive
>>I think that Rebecca's question is about the particular (negative)
>>dependence structure which is
>>found between the set of pairwise contrasts for a particular
>>gene. Yes, I think this does
>>theoretically invalidate BH. However in practical situations
>>you'll actually find the main
>>problem to be conserativism rather than the other way around
>>because there are more pairwise
>>comparisons than there are independent comparisons to be made. So,
>>personally, this wouldn't make
>>me move to BY.
>>Hope this helps
>> > Date: Fri, 13 Jan 2006 16:57:28 -0500
>> > From: Naomi Altman <naomi at stat.psu.edu>
>> > Subject: Re: [BioC] adjustment for dependent hypotheses when pairwise
>> > testing
>> > To: pmt1rew at leeds.ac.uk, "bioconductor at stat.math.ethz.ch"
>> > <bioconductor at stat.math.ethz.ch>
>> > I do not think you will be able to use these methods with such a
>> > small number of genes. A large number of genes are required to
>> estimate pi0.
>> > --Naomi
>> > At 07:52 AM 1/13/2006, pmt1rew at leeds.ac.uk wrote:
>> >>Dear all,
>> >>I data for 13 genes with the samples from 4 different tissue
>> >>types. Would like
>> >>to compare the normal means of the different tissue types using pairwise
>> >>testing with an adjustment for fdr.
>> >>Having considered the Benjamini and Yekutieli reference on the
>> p.adjust help
>> >>page I am unsure as to whether my dependency structure satisfies
>> the criteria
>> >>to just use the regular "BH" adjustment or if the "BY"
>> adjustment would be
>> >>more appropriate?
>> >>Any suggestions for me please?
>> >>Many thanks
>> >>Rebecca Walls
>> > Naomi S. Altman 814-865-3791 (voice)
>> > Associate Professor
>> > Dept. of Statistics 814-863-7114 (fax)
>> > Penn State University 814-865-1348 (Statistics)
>> > University Park, PA 16802-2111
>Naomi S. Altman 814-865-3791 (voice)
>Dept. of Statistics 814-863-7114 (fax)
>Penn State University 814-865-1348 (Statistics)
>University Park, PA 16802-2111
More information about the Bioconductor