[Rd] choose incorrect for fractional and some negative integer values (PR#10766)

[Jerry.Lewis at biogenidec.com]

Full_Name: Jerry W. Lewis
Version: 2.6.2
OS: Windows XP Professional
Submission from: (NULL) (198.180.131.16)


choose() returns incorrect values for all fractional arguments, regardless of
sign.  It returns 0 when both arguments are negative integers, which is not
always correct (as in some formulations of the negative hypergeometric).

Examples (correct answers from Maple's binomial function):

choose( 4.75,2.75)  # should be 8.90625
choose(-0.75,-0.5)  # should be 1.6692536833481464
choose(-1.75,-0.5)  # should be 0.55641789444938212
choose(-2.75,-1.5)  # should be 0.15897654127125204
choose(-5,   -7)    # should be 15

The correct answers for first two can be verified easily since choose(n,k)
should equal
  1/beta(n-k+1,k+1)/(n+1)
but does not for fractional arguments.  The latter three are instances where
beta incorrectly returns NaN, as noted in PR#10763