[R] Query regarding stats/p.adjust package (base) - specifically 'Hochberg' function

[Martin Maechler]

>>>>> Bert Gunter 
>>>>>     on Tue, 24 Aug 2021 10:50:50 -0700 writes:

    > 1. No Excel attachments made it through. Binary
    > attachments are generally stripped by the list server for
    > security reasons.

    > 2. As you may have already learned, this is the wrong
    > forum for statistics or package specific questions. Read
    > *and follow* the posting guide linked below to post on
    > r-help appropriately. In particular, for questions about
    > specific non-standard packages, contact package
    > maintainers (found through e.g. ?maintainers)

    > 3. Statistics issues generally don't belong here. Try
    > stats.stackexchange.com instead perhaps.

    > 4. We are not *R Core development,* and you probably
    > should not be contacting them either.  See here for
    > general guidelines for R lists:
    > https://www.r-project.org/mail.html


    > Bert Gunter

    > "The trouble with having an open mind is that people keep
    > coming along and sticking things into it."  -- Opus (aka
    > Berkeley Breathed in his "Bloom County" comic strip )

Well, this was a bit harsh of an answer, Bert.

p.adjust() is a standard R  function  (package 'stats') -- as
David Swanepoel did even mention.

I think he's okay asking here if the algorithms used in such a
standard R functions are  "ok"  and how/why they seemlingly
differ from other implementations ...

Martin

    > On Tue, Aug 24, 2021 at 10:39 AM David Swanepoel
    > <davidswanepoel using hotmail.com> wrote:
    >> 
    >> Dear R Core Dev Team, I hope all is well your side!  My
    >> apologies if this is not the correct point of contact to
    >> use to address this. If not, kindly advise or forward my
    >> request to the relevant team/persons.
    >> 
    >> I have a query regarding the 'Hochberg' method of the
    >> stats/p.adjust R package and hope you can assist me
    >> please. I have attached the data I used in Excel, which
    >> are lists of p-values for two different tests (Hardy
    >> Weinberg Equilibrium and Linkage Disequilibrium) for four
    >> population groups.
    >> 
    >> The basis of my concern is a discrepancy specifically
    >> between the Hochberg correction applied by four different
    >> R packages and the results of the Hochberg correction by
    >> the online tool,
    >> MultipleTesting.com<http://www.multipletesting.com/>.
    >> 
    >> Using the below R packages/functions, I ran multiple test
    >> correction (MTC) adjustments for the p-values listed in
    >> my dataset. All R packages below agreed with each other
    >> regarding the 'significance' of the p-values for the
    >> Hochberg adjustment.
    >> 
    >> 
    >> * stats/p.adjust (method: Hochberg) * mutoss/hochberg *
    >> multtest/mt.rawp2adjp (procedure: Hochberg) * elitism/mtp
    >> (method: Hochberg)
    >> 
    >> In checking the same values on the MultipleTesting.com,
    >> more p-values were flagged as significant for both the
    >> HWE and LD results across all four populations. I show
    >> these differences in the Excel sheet attached.
    >> Essentially, using the R packages, only the first HWE
    >> p-value of Pop2 is significant at an alpha of 0.05. Using
    >> the MT.com tool, however, multiple p-values are shown to
    >> be significant across both tests with the Hochberg
    >> correction (the highlighted cells in the Excel sheet).
    >> 
    >> 
    >> I asked the authors of MT.com about this, and they gave
    >> the following response:
    >> 
    >> "we have checked the issue, and we believe the
    >> computation by our page is correct (I cannot give opinion
    >> about the other packages).  When we look on the original
    >> Hochberg paper, and we only use the very first (smallest)
    >> p value, then m"=1, thus, according to the equation in
    >> the Hochberg 1988 paper, in this case practically there
    >> is no further correction necessary.  In other words, in
    >> case the *smallest* p value is smaller than alpha, then
    >> the *smallest* p value will remain significant
    >> irrespective of the other p values when we make the
    >> Hochberg correction."
    >> 
    >> I have attached the Hochberg paper here but,
    >> unfortunately, I don't understand enough of the stats to
    >> verify this. I have applied their logic on the same Excel
    >> sheet under the section "MT.com explanation", which shows
    >> why they consider the highlighted values significant.
    >> 
    >> I have also attached the 2 R files that I used to do the
    >> MTC runs and they can be run as is. They are just quite
    >> long as they contain many of the other MTC methods in the
    >> different packages too.
    >> 
    >> Kindly provide your thoughts as to whether you agree with
    >> this interpretation of the Hochberg paper or not? I would
    >> like to see concordance between the MT.com tool and the
    >> different R packages above (or understand why they are
    >> different), so that I can be more confident in the
    >> explanations of my own results as a stats layman.
    >> 
    >> I hope this makes sense. Please let me know if I need to
    >> clarify anything.
    >> 
    >> 
    >> Many thanks and kind regards, David
    >> ______________________________________________
    >> R-help using r-project.org mailing list -- To UNSUBSCRIBE and
    >> more, see https://stat.ethz.ch/mailman/listinfo/r-help
    >> PLEASE do read the posting guide
    >> http://www.R-project.org/posting-guide.html and provide
    >> commented, minimal, self-contained, reproducible code.

    > ______________________________________________
    > R-help using r-project.org mailing list -- To UNSUBSCRIBE and
    > more, see https://stat.ethz.ch/mailman/listinfo/r-help
    > PLEASE do read the posting guide
    > http://www.R-project.org/posting-guide.html and provide
    > commented, minimal, self-contained, reproducible code.