[R] aov() and anova() making faulty F-tests

[Rolf Turner]

Your subject line is patent nonsense.  The aov() and anova() functions
have been around for decades.  If they were doing something wrong
it would have been noticed long since.

You should realize that the fault is in your understanding, not in these
functions.

I cannot really follow your convoluted and messy code, but it would
appear that you want to consider "M" and "I" to be random effects.

Where have you informed aov() as to the presence of these
random effects?

     cheers,

         Rolf Turner

On 03/06/2013 03:36 PM, PatGauthier wrote:
> Dear useRs,
>
> I've just encountered a serious problem involving the F-test being carried
> out in aov() and anova(). In the provided example, aov() is not making the
> correct F-test for an hypothesis involving the expected mean square (EMS) of
> a factor divided by the EMS of another factor (i.e., instead of the error
> EMS).
>
> Here is the example:
>
>
>                    Expected Mean Square            df
> Mi                     σ2+18σ2M                              1
> Ij                  σ2+6σ2MI+12Ф(I)                      2
> MIij                   σ2+6σ2MI                              2
> ε(ijk)l                        σ2                                   30
>
> The clear test for Ij is EMS(I) / EMS(MI) -  F(2,2)
>
> However, observe the following example carried out in R,
>
> M <- rep(c("M1", "M2"), each = 18)
> I <- as.ordered(rep(rep(c(5,10,15), each = 6), 2))
> y <-
> c(44,39,48,40,43,41,27,20,25,21,28,22,35,30,29,34,31,38,12,7,6,11,7,12,15,10,12,17,11,13,22,15,27,22,21,19)
> dat <- data.frame(M, I, y)
> summary(aov(y~M*I, data = dat))
>                         Df    Sum Sq       Mean Sq         F value
> Pr(>F)
> m                     1     3136.0           3136.0            295.85      <
> 2e-16 ***
> i                        2      513.7              256.9              24.23
> 5.45e-07 ***
> m:i                   2      969.5              484.7              45.73
> 7.77e-10 ***
> Residuals   30       318.0                10.6
> ---
>
> In this example aov has taken the F-ratio of MS(I) / MS(ε) -  F(2,30) =
> 24.23 with F-crit = qf(0.95,2,3) = 9.55 -- significant
>
> However, as stated above,  the correct F-ratio is MS(I) / MS(MI) -  F(2,2) =
> 0.53 with F-crit = qf(0.95,2,2) = 19 -- non-significant
>
> Why is aov() miscalculating the F-ratio, and is there a way to fix this
> without prior knowledge of the appropriate test (e.g., EMS(I)/EMS(MI)?
>
> Thanks for your help,