[BioC] Multiple testing correction on 2-Way ANOVA

[Adaikalavan Ramasamy]

I think I asked this question on the list before but regarding one-way
ANOVA and pairwise comparison. And I am no expert in multiple comparison
either.

In the following paper, there are two main effect group and time. If I
remember correctly the authors argue that interaction term is most
important (otherwise one-way ANOVA would suffice) followed by groups
effect.

Statistical tests for identifying differentially expressed genes in
time-course microarray experiments.
Park T., Yi S.G., Lee S., Lee S.Y., Yoo D.H., Ahn J.I., Lee Y.S.
Bioinformatics 2003; 19(6):694-703
12691981

(Don't you just hate p-values ?)


On Tue, 2004-07-27 at 21:38, Eric wrote:
> Hi Andy,
> 
> Thanks for the reply. My reasoning here is a little Byzantine so bear with me.
> 
> If the significant results are relatively evenly distributed across the 
> main effects and the interaction (about the same number of genes found in 
> each), then using the omnibus test will not make much of a difference. 
> However, say one of the two main effects is much stronger than the other, 
> then I have a case where the overall test will pick up all of those changes 
> from the 'powerful' treatment (or most of them). Because of that, multiple 
> testing correction at the overall level will allow genes with larger 
> p-values from the second main effect through the filter compared to the 
> list of genes that would make it through a multiple testing correction 
> applied at the level of the second main effect.
> 
> Contrast this with the case where multiple testing is applied separately to 
> each of the three outputs. Here the first main effect is relatively 
> unaffected, but the second main effect is nuked (if the second main effect 
> has no more genes than would be expected by chance). IMHO it doesn't matter 
> what the original question was, the two multiple testing corrections change 
> the list of genes and the experimental question does not address which of 
> these procedures should be used. It would be disingenuous to say "Well, 
> we're mainly interested in main effect 2 (the weak one), so we'll use the 
> overall correction and at least see a list of genes" or "We wanted to 
> disagree with previous work about main effect two's importance to research 
> so we used individual correction to show the world that main effect two is 
> not doing anything". Perhaps the proportion of genes assigned an 
> interaction significance could be used to gauge the dependence of the two 
> main effects; the more dependent they are, the more applicable the overall 
> testing correction. While the smaller the proportion of genes showing an 
> interaction term, the more appropriate independent correction for each main 
> effect would be.
> 
> 
> At 03:05 PM 7/27/2004, you wrote:
> >I am absolutely no expert in multiple comparison / multiple testing / gene
> >expression data analysis, so take the following with appropriate dose of
> >salt:
> >
> >It really depends on what you are looking to get out of the data.  Just
> >because you have multi-factor data with > 2 levels and thousands of
> >responses, it doesn't automatically mean that the usual multiple comparison
> >procedures are appropriate.  You design the experiment to answer some
> >specific questions (hopefully).  How you analyze the data depends greatly on
> >what those questions are, and (hopefully, therefore) how the experiment is
> >designed.
> >
> >Best,
> >Andy
> >
> > > From: Eric
> > >
> > > Hi,
> > >
> > > I apologize for this being off-topic- it's really a
> > > statistical question
> > > but I'd be interested in the community's input. If I run a
> > > 'per gene' 2-way
> > > ANOVA on single channel microarray data (i.e., each gene is tested
> > > separately by 2-Way ANOVA), should I run multiple testing
> > > correction for
> > > each factor and interaction separately? Alternatively, should
> > > I use an
> > > overall (omnibus) F-test, correct that for multiple testing,
> > > and treat the
> > > main effects and interaction results as post-hoc to the overall test?
> > >
> > > Thanks,
> > > -E
> > >
> > > Eric Blalock, PhD
> > > Dept Pharmacology, UKMC
> > > 859 323-8033
> > >
> > > STATEMENT OF CONFIDENTIALITY\ \ The contents of this e-mail
> > > ...{{dropped}}
> > >
> > > _______________________________________________
> > > Bioconductor mailing list
> > > Bioconductor at stat.math.ethz.ch
> > > https://www.stat.math.ethz.ch/mailman/listinfo/bioconductor
> > >
> > >
> >
> >
> >------------------------------------------------------------------------------
> >Notice:  This e-mail message, together with any attachments, contains 
> >information of Merck & Co., Inc. (One Merck Drive, Whitehouse Station, New 
> >Jersey, USA 08889), and/or its affiliates (which may be known outside the 
> >United States as Merck Frosst, Merck Sharp & Dohme or MSD and in Japan, as 
> >Banyu) that may be confidential, proprietary copyrighted and/or legally 
> >privileged. It is intended solely for the use of the individual or entity 
> >named on this message.  If you are not the intended recipient, and have 
> >received this message in error, please notify us immediately by reply 
> >e-mail and then delete it from your system.
> >------------------------------------------------------------------------------
> 
> Eric Blalock, PhD
> Dept Pharmacology, UKMC
> 859 323-8033
> 
> STATEMENT OF CONFIDENTIALITY\ \ The contents of this e-mail ...{{dropped}}
> 
> _______________________________________________
> Bioconductor mailing list
> Bioconductor at stat.math.ethz.ch
> https://www.stat.math.ethz.ch/mailman/listinfo/bioconductor
>