[Rd] package development

Fri Dec 12 17:05:11 CET 2008

Jeff Ryan wrote:
> A quick comment on two subjects:
> 
> First to get the line # from the command line:
> 
> cat pkg/R/* > all.R
> wc all.R
> vim +LINE all.R  (or pick you favorite method here...)

The tmp file is also indicated in error message, so you can edit that, 
go to the line number, find context, grep the files in R/, and then go 
directly to the line that needs to be fixed in the source file. Couldn't 
be simpler?

> 
> From there it should be easy enough to get context.  At least enough
> to grep back into the R directory.
> 
> Also, the use of unit-tests is IMO the best answer  for managing a
> separate test suite that you can selectively run (all of part of)
> during the build phase.
> 
> Simply editing/adding an .Rbuildignore is all you need to turn it off
> or on.  This makes it easy not to distribute the tests as well, if you
> choose not too.

Yes, there are several options for not distributing tests.  I was 
thinking more about how to distribute them with a simple mechanism for 
anyone to run them, but in a way that they are not run by the usual R 
CMD check.

Paul
> 
> RUnit is a good tool to look into.
> 
> HTH
> Jeff
> 
> On Fri, Dec 12, 2008 at 8:12 AM, Paul Gilbert
> <pgilbert at bank-banque-canada.ca> wrote:
> 
>>
>>Duncan Murdoch wrote:
>>
>>>On 11/12/2008 6:04 PM, Terry Therneau wrote:
>>>
>>>> I'm making the move of the survival package from my own environment to,
>>>>and have stumbled into a vacuum.   The R Extensions manual has really
>>>>nice
>>>>instructions about how to lay out the directories, order the files, and
>>>>run tests for DISTRIBUTION of a product, but I can't find anything on how
>>>>to set up a reasonable DEVELOPMENT environment.
>>>>  In my local world, I had the .c and .s files in a common directory,
>>>>with
>>>>a Makefile that I had created, and the test suite in a subdirectory.
>>>> Debugging
>>>>and development was quite nice.
>>>>   make
>>>>   cd test
>>>>   R
>>>>   attach("..")
>>>>       try something and perhaps it fails
>>>>   q()
>>>>   cd ..
>>>>Fix and repeat. The Makefile took some time to create, but paid for
>>>>itself a
>>>>hundred times over.
>>>> So, I've now rearranged everything into standard R order.  Then I did
>>>>the
>>>>only thing I could find
>>>>   R CMD INSTALL ~/myRlib  survival  where "survival" is said directory.
>>>>  This turns out to be not useful at all.
>>>>The survival package is large, and I rather suspected that I would goof
>>>>something up, and I did, resulting in the following message
>>>>
>>>>   Error in parse(n = -1, file = file) : unexpected end of input at
>>>>       14450: }
>>>>       14451:
>>>>It is not exactly obvious which of the 132 files in my R/ directory is
>>>>the culprit here.
>>>
>>>That's something I would like to fix too.  There are (at least) two
>>>possible ways:  stop concatenating the files (which is not really needed
>>>nowadays, most packages install saved images), or add some markup to the
>>>concatenated file so that the parser can report on the original filename and
>>>line number (like the #LINE directives output by the C preprocessor).
>>
>>I would certainly appreciate a fix for this.  When this has happened to me,
>>I usually end up just sourcing the individual files into an R session until
>>I find the bad file, and get a report with the right line number. It always
>>seems like a lot of work for something that should be trivial.
>>
>>
>>>>  In general:
>>>>1. The library is large, and recompiling/reparsing everything is very far
>>>>from
>>>>instantaneous.  It is not the edit/load cycle I desire.
>>>
>>>If you install from the directory, the compiling should only be done once
>>>(unless you change a file, of course).  (The alternative is installing from
>>>the tarball, which is recommended for later stages of testing before
>>>distribution, because it's possible something could go wrong in building the
>>>tarball.  But it won't include any object files, so you'll recompile every
>>>time.)
>>>
>>>You can also use option "--docs=none" to skip building the help system;
>>>this will save a bit of time.
>>>
>>>
>>>
>>>>2. I take testing seriously: the test suite takes on the order of 15
>>>>minutes to
>>>>run on a fast machine.  I most certainly don't want to run it in the mid
>>>>cycle.
>>>
>>>I don't quite follow this.  If you want to run all your tests, you would
>>>use R CMD check.  If you only want to run some of them, then you can source
>>>things out of the tests directory while running interactively.
>>>
>>>
>>>>  Someone must have tackled this.  I'm hoping that there is some
>>>>documentation
>>>>that I have managed to overlook which discussess a good setup for this
>>>>middle
>>>>ground between concieving of a library and packaging it for delivery; the
>>>>"build, test, make sure it acually works" part of development.
>>
>>In my experience there are two differ sorts of problems that I think could
>>benefit from some improvement. The first is that there should be a standard
>>way to have extra tests, that do not get run in the normal CRAN testing
>>cycle, or by developers when using a "quick" R CMD check, but can be run
>>with a standard mechanism. I do this by putting the tests in a separate
>>package, and I have seen reports of different mechanisms, but I think they
>>are all somewhat ad hoc. Currently, if you put too much testing in your
>>package then all the testing gets omitted on some CRAN testing platforms.
>>Just a common directory like extraTests/ would be a good start.
>>
>>The second problem is that a developer usually needs to run tests/ when code
>>in R/ has been changed, but probably not run tests/ when the changes are
>>only in man/, demos/, or inst/doc/.  The checking that needs to be done in
>>those cases is reduced from the full R CMD check.  A common way to attack
>>this is with a good Makefile. I wrote an article in R News a few years ago
>>about my attempts to do this, and my Makefiles are available, but there is
>>some customization necessary, and there is lots of room for improvement. It
>>does (mostly) work with make -j, so days of testing on a single processor
>>machine can be accomplished in a few hours on multicore machines (for me,
>>mileage may vary). I have not addressed the idea of trying to specialize
>>files in tests/ to specific code files in R/. (I think others have tried to
>>do this with a "unit testing" approach.)
>>
>>Paul Gilbert
>>
>>
>>>I find the process I follow is to organize the files in the distribution
>>>structure from the beginning.   When adding new functions, I'll generally
>>>use source() a few times to get the syntax right, and perhaps run simple
>>>tests.  (But remember, if you use a NAMESPACE, the functions may not behave
>>>the same when they're sourced into the global environment.)  In the early
>>>stages, I'll do a lot of installs of the packages.
>>>
>>>If I was porting a big package and wanted to find syntax errors, to work
>>>around the not-very-helpful error message you saw I'd do something like
>>>
>>>for (f in list.files("pkg/R", full=TRUE)) source(f)
>>>
>>>which will report the error more informatively.
>>>
>>>Duncan Murdoch
>>>
>>>______________________________________________
>>>R-devel at r-project.org mailing list
>>>https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>>====================================================================================
>>
>>La version française suit le texte anglais.
>>
>>------------------------------------------------------------------------------------
>>
>>This email may contain privileged and/or confidential in...{{dropped:26}}
>>
>>______________________________________________
>>R-devel at r-project.org mailing list
>>https://stat.ethz.ch/mailman/listinfo/r-devel
>>
> 
> 
> 
> 
====================================================================================

La version française suit le texte anglais.

------------------------------------------------------------------------------------

This email may contain privileged and/or confidential in...{{dropped:26}}