[Rd] Packages manually setting NeedsCompilation

[Kurt Hornik]

>>>>> Hervé Pagès writes:

> FWIW NeedsCompilation is a misnomer. IIRC when I was discussing this 
> feature with R core members many years ago, it doesn't only flag 
> packages that require compilation (and thus contain arch-specific binary 
> files), but it is more generally intended to flag packages that the user 
> wouldn't be able to install **from source** on a pristine Windows or Mac 
> system because the installation process depends on tools that are not 
> necessarily present on such systems.

> For example I think I remember seeing at the time some packages that 
> were using 'make' for non-compilation related business. These packages 
> still needed to be flagged with 'NeedsCompilation: yes' in order to 
> recognize them as packages for which "binaries" needed to be made for 
> the purpose distributing them.

> Packages without 'NeedsCompilation' field (or with 'NeedsCompilation: 
> no') don't need binaries because the source tarballs is guaranteed to be 
> installable on **any** system, in particular on systems that don't have 
> any development tools.

Indeed, this is what NeedsCompilation is about.  When (on Windows only?)
we run install.packages(type = "both") looking for both binary and
source packages to possibly install, and we find a package for which the
version of the source is higher than that of the binary (or there is no
binary), we double-check with the user whether to install the source
when NeedsCompilation: yes is found (as this likely will not work on a
system with "only R" installed.

(See the comments in utils/R/packages2.R for what precisely is going
on.)

I.e., "NeedsCompilation: no" really means "should always be installable
from source".

Since R 3.2.0, R CMD build adds a 'NeedsCompilation' field to
DESCRIPTION if one is not already present, setting to 'yes' if there is
a 'src' subdir and to 'no' otherwise.

On CRAN, there are currently 14 packages which have a 'src' subdir but
no source files R knows to compile.  12 of these have 'install.libs.R'
(see below), one has a Makefile for a subdir, and one (mixl, reported by
Iñaki) uses Makevars to print if OpenMP is supported (I'll complain
about this one).

In WRE we say

  If a package wants to install other binaries (for example executable
  programs), it should provide an R script ‘src/install.libs.R’ which
  will be run as part of the installation in the ‘src’ build directory
  _instead of_ copying the shared objects/DLLs.

and most of the 12 above actually comply (but compile in a subdir).

So if a package has a 'src' subdir, it should really need
'NeedsCompilation: yes'.

On the other hand, packages with no 'src' subdir are not necessarily
always installable from source.  One could have configure code or top
level R code which does not work with "only R" installed.  These are the
cases when maintainers should manually set 'NeedsCompilation: yes' in
their DESCRIPTION file.  As always, it would be better if this was not
set manually but could be provided automagically, but I don't see how
this can be accomplished.

Best
-k


> Best,

> H.

> On 15/05/2025 03:38, Iñaki Ucar wrote:
>> On Thu, 15 May 2025 at 11:42, Jeroen Ooms <jeroenooms using gmail.com> wrote:
>>> On Wed, May 14, 2025 at 11:05 PM Simon Urbanek
>>> <simon.urbanek using r-project.org> wrote:
>>>> Can you give an example, please? I wonder if there is a real use-case or just bad package design.
>>> The case that bit me yesterday was a Bioconductor package Rigraphlib,
>>> see https://bioconductor.org/packages/release/bioc/src/contrib/Rigraphlib_1.0.0.tar.gz
>>> 
>>> This package builds a static library entirely from a configure script.
>>> Because there is no 'src' dir, we assumed the package to be all-arch,
>>> and get linking errors when trying to use it on another arch then it
>>> was built for. Arguably this is indeed bad package design, but that is
>>> hard to protect against.
>> The opposite case is also true: there is a src directory but no
>> compilation happens. This is a problem for deb/rpm shipment for two
>> reasons. In rpm packaging at least, archful packages are inspected to
>> extract debuginfo; since there is no compiled output there, this fails
>> and the whole build fails. If the build succeeds (e.g. debuginfo is
>> disabled), then we are building the same thing for all architectures,
>> when there should be just one, noarch, for all.
>> 
>> I can list at least 3 examples of this:
>> 
>> - BANOVA: requires compilation, there is a src directory, nothing
>> happens there, no source files, just a Makefile sitting there.
>> - mixl: same, and the Makefile is just used to print if OpenMP is supported (?).
>> - mathjaxr: same, and the Makefile is used to install a javascript library (!).
>> 
>>>> I wouldn't think that should happen as configure is supposed to only guide the compilation in src - if there is no src no binaries are expected as the package did not provide any native sources hence there should be no binary content. This looks like something that could be added to R CMD check?
>>> In that case maybe R should warn if it runs a package configure script
>>> but there is no "src"? And check if NeedsCompilation: yes is set?
>>> 
>>> One way or another, what matters is that when R runs configure or make
>>> during install, the value in the "Built" field in the binary package
>>> should contain the platform value. This is also important when distros
>>> ship R packages in deb or rpm format, to prevent packages that have
>>> compiled code from getting "Architecture: all" and be installed on the
>>> wrong platforms.
>> IMHO, the flag NeedsCompilation in the DESCRIPTION file should be the
>> one and only source of truth here, regardless of the presence or not
>> of a src directory. Then I see two paths here:
>> 
>> - Either the developer is responsible for setting this flag, and the R
>> CMD check should fail if it's yes but no compilation happens or the
>> other way around.
>> - Or R CMD build is responsible for setting this flag after checking
>> if compilation happens or not.
>> 
> -- 
> Hervé Pagès

> Bioconductor Core Team
> hpages.on.github using gmail.com

> ______________________________________________
> R-devel using r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel