[Rd] Benchmark code, but avoid printing

[Simon Urbanek]

On Jan 2, 2015, at 12:02 PM, Gábor Csárdi <csardi.gabor at gmail.com> wrote:

> Dear all,
> 
> I am trying to benchmark code that occasionally prints on the screen
> and I want to
> suppress the printing. Is there an idiom for this?
> 
> If I do
> 
> sink(tempfile)
> microbenchmark(...)
> sink()
> 
> then I'll be also measuring the costs of writing to tempfile. I could
> also sink to /dev/null, which is probably fast, but that is not
> portable.
> 
> Is there a better solution? Is writing to a textConnection() better?
> 

Define better - you're just trading off one output code for another - it will be still measuring the cost of the output, obviously, and since the output is part of the code you're profiling it's correctly so. Each output method has different beavior - e.g. text connection can be faster, but it can also trigger additional garbage collection so it will affect results. Example:

> f=textConnection("x", "w")
> sink(f)
> m=microbenchmark({ for (i in 1:100) { print("foo"); sum(rnorm(1e3)) } })
> sink()
> m
Unit: milliseconds
                                                                           expr
 {     for (i in 1:100) {         print("foo")         sum(rnorm(1000))     } }
      min       lq     mean   median       uq      max neval
 12.76462 15.34483 17.85341 17.02435 19.56384 63.09329   100
> sink("/dev/null")
> m=microbenchmark({ for (i in 1:100) { print("foo"); sum(rnorm(1e3)) } })
> sink()
> m
Unit: milliseconds
                                                                           expr
 {     for (i in 1:100) {         print("foo")         sum(rnorm(1000))     } }
     min       lq     mean  median       uq      max neval
 13.0191 13.03601 13.41815 13.0534 13.16496 16.25288   100

As you can see /dev/null is more predictable, because it's straight output, but text connection can be faster in the beginning and becomes progressively slower.

As Henrik said, you're probably best off using simply /dev/null - the only oddball is Windows, and that's a trivial condition on .Platform$OS.type.

Cheers,
S