[Rd] Competing with one's own work

Fri Dec 3 20:18:32 CET 2010

On 03/12/2010 12:01 PM, Ravi Varadhan wrote:
> Dear Duncan,
>
> What constitutes a convincing argument for making significant changes?

I don't think there's any answer to that other than "an argument that 
convinces someone to make the changes".  What would convince you to work 
on a problem?   Your answer is very different from mine, and mine is 
different from that of anyone else in the core group.

> Taking the example of optimization algorithms (say, for smooth objective
> functions), how does one make a convincing argument that a particular class
> of algorithms are "better" than another class? This can be a difficult task,
> but quite doable with good benchmarking practices.

I don't see how that's relevant.  That's an argument to make to users, 
not to the core group.   A user wants to use the best optimizer for 
his/her own problem.  The core group wants functions in base R that we 
will maintain.

> Supposing for the moment that such a convincing argument has been made, is
> that sufficient to get the R-core to act upon it?

By definition, yes.

> Are there compelling
> factors other than just "algorithm A is better than algorithm B"?

Yes.  The decision about whether it belongs in a package or in base R is 
about who should maintain the code.  If I think it is fantastic code, 
but you will do a better job of maintaining it than I will, then there's 
no way I'd put it in base R.

> I'd think that the argument is relatively easy if the need for the change is
> driven by consumer demand.  But, even here I am not sure how to make an
> argument to the R-core to consider the big changes.  For example, there is a
> reasonable demand for constrained (smooth) optimization algorithms in R
> (based on R-help queries).  Currently, there are only 3 packages that can
> handle this.  However, in the base distribution only `constrOptim' function
> is provided, which cannot handle anything more than linear, inequality
> constraints.  I think that the base distribution needs to have a package for
> constrained optimization that can handle linear/nonlinear and
> equality/inequality constraints.

As Doug said, "I don't see anything in what you are proposing that could 
not be incorporated in a contributed package."

I think I answered your followup question above:  the rationale for 
including it in base R is because someone in the core team is in a 
better position to maintain the code than an outside package maintainer 
would be.

Duncan Murdoch

> John, thanks for raising an important issue.
>
> Thanks&  Best,
> Ravi.
>
> -------------------------------------------------------
> Ravi Varadhan, Ph.D.
> Assistant Professor,
> Division of Geriatric Medicine and Gerontology School of Medicine Johns
> Hopkins University
>
> Ph. (410) 502-2619
> email: rvaradhan at jhmi.edu
>
>
> -----Original Message-----
> From: r-devel-bounces at r-project.org [mailto:r-devel-bounces at r-project.org]
> On Behalf Of Duncan Murdoch
> Sent: Friday, December 03, 2010 11:13 AM
> To: nashjc at uottawa.ca
> Cc: r-devel at r-project.org
> Subject: Re: [Rd] Competing with one's own work
>
> On 03/12/2010 10:57 AM, Prof. John C Nash wrote:
> >  No, this is not about Rcpp, but a comment in that overly long discussion
> raised a question
> >  that has been in my mind for a while.
> >
> >  This is that one may have work that is used in R in the base functionality
> and there are
> >  improvements that should be incorporated.
> >
> >  For me, this concerns the BFGS, Nelder-Mead and CG options of optim(),
> which are based on
> >  the 1990 edition (Pascal codes) of my 1979 book "Compact numerical
> methods...", which were
> >  themselves derived from other people's work. By the time Brian Ripley took
> that work (with
> >  permission, even though not strictly required. Thanks!) there were already
> some
> >  improvements to these same algorithms (mainly bounds and masks) in the
> BASIC codes of the
> >  1987 book by Mary Walker-Smith and I. However, BASIC to R is not something
> I'd wish on
> >  anyone.
> >
> >  Now there are some R packages, including some I've been working on, that
> do offer
> >  improvements on the optim() offerings. I would not say mine are yet fully
> ready for
> >  incorporation into the base, but they are pretty close. Equally I think
> some of the tools
> >  in the base should be deprecated and users encouraged to try other
> routines. It is also
> >  getting more and more important that novice users be provided with
> sensible guidance and
> >  robust default settings and choices. In many areas, users are faced with
> more choice than
> >  is efficient for the majority of problems.
> >
> >  My question is: How should such changes be suggested / assisted? It seems
> to me that this
> >  is beyond a simple feature request. Some discussion on pros and cons would
> be appropriate,
> >  and those like myself who are familiar with particular tools can and
> should offer help.
> >
> >  Alternatively, is there a document available in the style "Writing R
> Extensions" that has
> >  a title like "How the R Base Packages are Updated"? A brief search was
> negative.
> >
> >  I'm happy to compete with my own prior work to provide improvements. It
> would be nice to
> >  see some of those improvements become the benchmark for further progress.
>
>
> There are answers at many different levels to your questions.  The
> simplest is that base packages are part of R, so they get updated when a
> member of R Core updates them, and the updates get released when a new
> version of R is released.
>
> So if you want a change, you need to convince a member of the core to
> make it.  Pointing out a bug is the easiest way to do this:  bugs
> usually get fixed quickly, if they are clearly demonstrated.
>
> If you want a bigger change, you need to make a convincing argument in
> favour of it.  If you pick a topic that is of particular interest to one
> core member, and you can convince him to make the change, then it will
> happen.  If pick some obscure topic that's not of interest to anyone,
> you'll need a very strong argument to make it interesting.  Part of any
> of these arguments is explaining why the change needs to be made to the
> base, why it can't just be published in a contributed package.  (That's
> why bug fixes are easy, and big additions to the base packages are not.)
>
> Duncan Murdoch
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>