[Rd] tests Rin and Rout
Paul Gilbert
pgilbert at bank-banque-canada.ca
Mon Mar 31 21:09:51 CEST 2008
Martin Maechler wrote:
>>>>>> "ChrG" == cgenolin <cgenolin at u-paris10.fr>
>>>>>> on Mon, 31 Mar 2008 10:58:33 +0200 writes:
>>>>>>
>
> ChrG> Martin Maechler <maechler at stat.math.ethz.ch> a écritÂ
> ChrG> :
> >>>>>>> "CG" == Christophe Genolini <cgenolin at u-paris10.fr>
> >>>>>>> on Mon, 31 Mar 2008 00:31:55 +0200 writes:
> >>
> >> >>
> >> >> Generally I find it's good to look at examples that
> >> work. >> For examples of packages using tests, look at
> >> source >> packages on CRAN. Run the tests on them (using
> >> R CMD >> check), and see what gets produced.
> >> >>
> CG> Do you have the name of a package that use it ? I try
> CG> the 10 first package, and 10 other at random, but none
> CG> of them use tests...
> >>
> >> hmm, I see 219 out 1378 CRAN packages having a 'tests'
> >> subdirectory, so it seems you have been a bit
> >> unlucky. ;-)
>
Hmmm. Well, since I have several of the 219, and I suspect a handful of
other developers also individually have several, it looks a bit like
most developers are not using them. Perhaps there needs to be more
encouragement.
More specifically on the question:
- You might look at the package numDeriv, which is fairly small and does
testing in the way I have more recently found to be most convenient. In
some of my older packages my mechanism was a bit awkward until the
package settled down.
-I now find it is generally simpler to have several small test files
rather than one big one.
-If there is a logical order for running the tests then this can be
arranged using a naming convention. I think tests are run in lexical
order by filename, which can be different on different systems but, it
can be made to work by putting numbers n the beginning of the filenames.
-If one of the functions that you run actually fails to exit properly,
then the test will fail.
-If numerical results rather than printed output is what you want to
test, then .R files are probably preferred. In the test file you should
check a value and stop() if it is not good, something like
fuzz <- 1e-14
if (fuzz < abs(sq(2) - 4)) stop("sq(2) test failed.")
-If you just check the printed value using the Rin / Rout mechanism, the
rounding in print() will hide errors that you should catch. If you try
to fix this by printing to higher precision, the tests will fail on
different architectures and with different compilers.
- For some calculations, fuzz may need to be somewhat larger than you
might expect, in order to accommodate different architectures and
compilers. (This can be an indication that the algorithm or the specific
problem is a bit unstable).
- It is nice to have test values (4 in the above example) that are known
theoretically, but I usually have to use a value that came from an
initial run of the program (so I am just testing if the program results
have changed). In this case it is useful to print the test value to a
high precision, say digits=18, just before you do the test, so you know
what the new/wrong result is when the test fails. This is also useful
for tracing down how big the fuzz needs to be to accommodate different
architectures.
- If you use random numbers then you need to set the random number
generator to a known state in each test file. The package setRNG may be
helpful for this. (The logical here is that you are testing the code for
changes and errors, not running a random experiment.)
-If you use random numbers then you may want to run the program
random.number.test() in setRNG as one of your first tests. This function
checks that the RNG and normal generator have not been changed. (They
have not changed in a long time, but I am a bit sensitive to this point,
having wasted a lot of time after a small change between two versions of
Splus.)
-If you use the Rin / Rout mechanism, be careful about what you print,
otherwise you may have a high maintenance problem because the printed
output can change a bit for reasons you may not anticipate. (It may be
possible to make this robust in ways I don't understand - so if someone
can add to this I would appreciate hearing. I'm especially curious how
language support works with this.)
-You can test error checking and warnings in your functions using try()
in the test files, but you might consider leaving this until you have
mastered some of the other pieces.
- Graphics output can be put in files (I think in Rplot.ps by default)
but I don't know of any way to automatically check the graphs to see if
they are good. As far as I know, you have to actually look at them. If
anyone knows of an automatic mechanism I would really like to hear about it.
-Each test file needs to attach any needed packages, including the one
being tested.
-Be a bit careful about what gets found in your default search path.
This can be especially important if you are checking the tests files
outside of the R CMD check environment, and after you have installed
your package and are testing a new revised version.
-If you are testing multiple packages that are inter-related, then you
may need to be careful about some additional things, like not picking up
one of the test packages from the site library rather than your testing
library.
HTH
Paul Gilbert
> ChrG> Do you imply that I say I try and I did not ?
>
> not at all! I am sorry if you (or anyone) got this impression!
>
> ChrG> Well, lets check that :
>
> ChrG> P_unluck_1 <- 1-(219/1379)
> ChrG> (P_unluck_10 <- P_unluck_1^10) [1] 0.1773933
>
> ChrG> So P>0.05, once can not say that I am not a liar.
> ChrG> Further investigation are needed :-)
>
> :-)
>
> >> Isn't all this is explained nicely in the "Writing R
> >> Extensions" Manual?
>
> ChrG> NO ! There is 8 lines on it, and the structure of the
> ChrG> Rin and Rout is not given. The writing R Ext. say "the
> ChrG> Test code can be in .R or in .Rin" but I do not know
> ChrG> what "test code" is suppose to be. I try simple things
> ChrG> like some function with argument, but it does not
> ChrG> works. I try more complicated things, I did not manage
> ChrG> to find the right things to do.
>
> Ok; so we should expand the part in the manual.
>
> First: '.Rin' are typically only used very rarely; so, normally
> the input file is 'foo.R' and hence (well ``hence'') must be
> valid R code, typically starting by
> library(<mypackage>).
>
> ChrG> It also say ".Rin file containing code which in turn
> ChrG> in .R", but there is also no information about the
> ChrG> what the .Rin should be. May be it is obvious for some
> ChrG> people, but not for me.
>
> >> Or are there sections we should expand ?
>
> ChrG> I do not know, if there is only few people using it,
> ChrG> may be it is not worth. I guess you have so many other
> ChrG> thinks to do... On the other hand, if some tool are
> ChrG> to difficult to understand, it is normal that people
> ChrG> do not use them... I don't know.
>
> ChrG> I will work on package example that Pr Ripley point to
> ChrG> me. Then when I will have understand the way it works,
> ChrG> would you like me to add little paragraph in "Writing
> ChrG> R extention" ? I mean, not an paragraph by expert but
> ChrG> a paragraph by a regular medium level user for some
> ChrG> other regular medium level user.
>
> That maybe very useful, thank you for the offer!
> Note that the manual source is in
> https://svn.r-project.org/R/trunk/doc/manual/R-exts.texi
>
> >> For answering your subsequent questions, you should
> >> probably both look at an example package *and* read the
> >> 'Writing R Extensions' manual a bit more "closely".
>
> ChrG> Well, I try to not disturb people for little obvious
> ChrG> thinks. I can tell you I did work hard before asking,
> ChrG> specialy before asking on r-devel...
>
> Okay, I apologize for having misjudged your efforts.
>
> Martin
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
====================================================================================
La version française suit le texte anglais.
------------------------------------------------------------------------------------
This email may contain privileged and/or confidential in...{{dropped:26}}
More information about the R-devel
mailing list