[R] Proving (instead of rejecting) that two groups are actually equal

[Marc Schwartz]

> On Aug 10, 2016, at 5:22 AM, Dominik Marti <dom at inik.ch> wrote:
> 
> Hej R helpers
> 
> The standard in statistical hypothesis testing is to reject the null hypothesis that there is a difference between groups, i.e. to "prove" the alternative. However, failing to reject the null hypothesis does not prove it; its rejection just fails.
> 
> Now, as stated in the article "Unicorns do exist: a tutorial on "proving" the null hypothesis." by David L Streiner (Canadian Journal of Psychiatry, 48(11) 2003), we can define the null hypothesis to be that there IS a difference (exceeding a certain value, delta), the alternative hypothesis being that there is none (or it is at least smaller than delta). If the data now manages to reject the null hypothesis (of there being a difference exceeding delta), we can say with a certain probability that there is none.
> 
> Can I do this test in R? And if yes, any leads?
> 
> (In my actual dataset I deal with paired data.)
> 
> Best
>   Dominik


Bear in mind that we are not "proving" anything with statistics. There is still a level of uncertainty in everything we do.

In the scenario above, you are, in essence, reversing the normal approach to testing a null versus alternative hypothesis. The null, in this case, is that there is a difference and the alternative being that there is none, within some pre-defined, acceptable, margin. 

In clinical studies, these are called "equivalence" studies or "bioequivalence" studies, a subset of which are called "non-inferiority" studies, which are one-sided versions. This is typically done, for example, when testing a generic version of a drug versus the pre-existing "brand name" version of the drug to demonstrate that they have equivalent efficacy and safety profiles, within a clinically acceptable range.

There is at least one R package that is relevant, conveniently called "equivalence":

  https://cran.r-project.org/web/packages/equivalence/

that addresses these scenarios.

Regards,

Marc Schwartz