From simon.urbanek at r-project.org  Fri Jan  2 21:44:55 2015
From: simon.urbanek at r-project.org (Simon Urbanek)
Date: Fri, 2 Jan 2015 15:44:55 -0500
Subject: [Rd] Benchmark code, but avoid printing
In-Reply-To: <CABtg=KmgV-6QEffrGc4HzOSUM-jvE3ExHFB1KefAgv16S=+oGA@mail.gmail.com>
References: <CABtg=KmgV-6QEffrGc4HzOSUM-jvE3ExHFB1KefAgv16S=+oGA@mail.gmail.com>
Message-ID: <DC70B272-B6C8-4692-AE73-399C8ED90489@r-project.org>

On Jan 2, 2015, at 12:02 PM, G?bor Cs?rdi <csardi.gabor at gmail.com> wrote:

> Dear all,
> 
> I am trying to benchmark code that occasionally prints on the screen
> and I want to
> suppress the printing. Is there an idiom for this?
> 
> If I do
> 
> sink(tempfile)
> microbenchmark(...)
> sink()
> 
> then I'll be also measuring the costs of writing to tempfile. I could
> also sink to /dev/null, which is probably fast, but that is not
> portable.
> 
> Is there a better solution? Is writing to a textConnection() better?
> 

Define better - you're just trading off one output code for another - it will be still measuring the cost of the output, obviously, and since the output is part of the code you're profiling it's correctly so. Each output method has different beavior - e.g. text connection can be faster, but it can also trigger additional garbage collection so it will affect results. Example:

> f=textConnection("x", "w")
> sink(f)
> m=microbenchmark({ for (i in 1:100) { print("foo"); sum(rnorm(1e3)) } })
> sink()
> m
Unit: milliseconds
                                                                           expr
 {     for (i in 1:100) {         print("foo")         sum(rnorm(1000))     } }
      min       lq     mean   median       uq      max neval
 12.76462 15.34483 17.85341 17.02435 19.56384 63.09329   100
> sink("/dev/null")
> m=microbenchmark({ for (i in 1:100) { print("foo"); sum(rnorm(1e3)) } })
> sink()
> m
Unit: milliseconds
                                                                           expr
 {     for (i in 1:100) {         print("foo")         sum(rnorm(1000))     } }
     min       lq     mean  median       uq      max neval
 13.0191 13.03601 13.41815 13.0534 13.16496 16.25288   100

As you can see /dev/null is more predictable, because it's straight output, but text connection can be faster in the beginning and becomes progressively slower.

As Henrik said, you're probably best off using simply /dev/null - the only oddball is Windows, and that's a trivial condition on .Platform$OS.type.

Cheers,
S



[Rd] Benchmark code, but avoid printing
Simon Urbanek