From murdoch.duncan at gmail.com  Tue Jul  5 16:21:51 2011
From: murdoch.duncan at gmail.com (Duncan Murdoch)
Date: Tue, 05 Jul 2011 10:21:51 -0400
Subject: [Rd] Recent and upcoming changes to R-devel
In-Reply-To: <4E131CF5.5000206@openanalytics.eu>
References: <alpine.OSX.1.00.1107041124460.59432@tystie.local>	<3D59DF35E7CF3E42BA70C23BD5CBE6A85F4C199E22@exvic-mbx04.nexus.csiro.au>
	<4E12ED06.5040603@openanalytics.eu> <4E1310D9.6050901@gmail.com>
	<4E131CF5.5000206@openanalytics.eu>
Message-ID: <4E131DFF.7060808@gmail.com>

On 05/07/2011 10:17 AM, Tobias Verbeke wrote:
> Dear Duncan,
>
> On 07/05/2011 03:25 PM, Duncan Murdoch wrote:
> >  On 05/07/2011 6:52 AM, Tobias Verbeke wrote:
> >>  L.S.
> >>
> >>  On 07/05/2011 02:16 AM, Mark.Bravington at csiro.au wrote:
> >>  >  I may have misunderstood, but:
> >>  >
> >>  >  Please could we have an optional installation that does not*not*
> >>  byte-compile base and recommended?
> >>  >
> >>  >  Reason: it's not possible to debug byte-compiled code-- at least not
> >>  with the 'debug' package, which is quite widely used. I quite often
> >>  end up using 'mtrace' on functions in base/recommended packages to
> >>  figure out what they are doing. And sometimes I (and others)
> >>  experiment with changing functions in base/recommended to improve
> >>  functionality. That seems to be harder with BC versions, and might
> >>  even be impossible, as best I can tell from hints in the documentation
> >>  of 'compile').
> >>  >
> >>  >  Personally, if I had to choose only one, I'd rather live with the
> >>  speed penalty from not byte-compiling. But of course, if both are
> >>  available, I could install both.
> >>
> >>  I completely second this request. All speed improvements and the byte
> >>  compiler in particular are leaps forward and I am very grateful and
> >>  admiring towards the people that make this happen.
> >>
> >>  That being said, 'moving away' from the sources (with the lazy loading
> >>  files and byte-compilation) may be a step back for R package developers
> >>  that (during development and maybe on separate development installations
> >>  [as opposed to production installations of R]) require
> >>  the sources of all packages to be efficient in their work.
> >>
> >>  As many of you know there is an open source Eclipse/StatET visual
> >>  debugger ready and for that application as well (similar to Mark's
> >>  request) presence of non-compiled code is highly desirable.
> >>
> >>  For the particular purpose of debugging R packages, I would even plead
> >>  to go beyond the current options and support the addition of an
> >>  R package install option that allows to include the sources (e.g. in
> >>  a standard folder Rsrc/) in installed packages.
> >>
> >>  I am fully aware that one can always fetch the source tarballs from
> >>  CRAN for that purpose, but it would be much more easy if a simple
> >>  installation option could put the R sources of a package in a separate
> >>  folder [or archive inside an existing folder] such that R development
> >>  tools (such as the Eclipse/StatET IDE) can offer inspection of sources
> >>  or display them (e.g. during debugging) out of the box.
> >>
> >>  If one has the srcref, one can always load the absolutely correct source
> >>  code this way, even if one doesn't know the parent function with
> >>  the source attribute.
> >>
> >>  Any comments?
> >
> >  I think these requests have already been met. If you modify the body of
> >  a closure (as trace() does), then the byte compiled version is
> >  discarded, and you go back to the regular interpreted code. If you
> >  install packages with the R_KEEP_PKG_SOURCE=yes environment variable
> >  set, the you keep all source for all functions. (It's attached to the
> >  function itself, not as a file that may be out of date.) It's possible
> >  that byte compiling turns off R_KEEP_PKG_SOURCE, but that is something
> >  that is either easily fixed, or avoided by re-installing without byte
> >  compiling.
>
> Many thanks for your reaction. Is the R_KEEP_PKG_SOURCE=yes environment
> variable also supported during R installation ?

Yes, other than the error you saw below, which is a temporary problem.  
Not sure which function exceeded the length limit, but the length limit 
is going away before 2.14.0 is released.

Duncan Murdoch

> I hope I'm not overlooking anything, but when compiling
>
> ftp://ftp.stat.math.ethz.ch/Software/R/R-devel.tar.gz
>
> a few minutes ago I encountered the following issue:
>
> [...]
>
> building package 'tools'
> mkdir -p -- ../../../library/tools
> make[4]: Entering directory `/home/tobias/rAdmin/R-devel/src/library/tools'
> mkdir -p -- ../../../library/tools/R
> mkdir -p -- ../../../library/tools/po
> make[4]: Leaving directory `/home/tobias/rAdmin/R-devel/src/library/tools'
> make[4]: Entering directory `/home/tobias/rAdmin/R-devel/src/library/tools'
> make[5]: Entering directory
> `/home/tobias/rAdmin/R-devel/src/library/tools/src'
> making text.d from text.c
> making init.d from init.c
> making Rmd5.d from Rmd5.c
> making md5.d from md5.c
> gcc -std=gnu99 -I../../../../include  -I/usr/local/include
> -fvisibility=hidden -fpic  -g -O2 -c text.c -o text.o
> gcc -std=gnu99 -I../../../../include  -I/usr/local/include
> -fvisibility=hidden -fpic  -g -O2 -c init.c -o init.o
> gcc -std=gnu99 -I../../../../include  -I/usr/local/include
> -fvisibility=hidden -fpic  -g -O2 -c Rmd5.c -o Rmd5.o
> gcc -std=gnu99 -I../../../../include  -I/usr/local/include
> -fvisibility=hidden -fpic  -g -O2 -c md5.c -o md5.o
> gcc -std=gnu99 -shared -L/usr/local/lib64 -o tools.so text.o init.o
> Rmd5.o md5.o -L../../../../lib -lR
> make[6]: Entering directory
> `/home/tobias/rAdmin/R-devel/src/library/tools/src'
> make[6]: `Makedeps' is up to date.
> make[6]: Leaving directory
> `/home/tobias/rAdmin/R-devel/src/library/tools/src'
> make[6]: Entering directory
> `/home/tobias/rAdmin/R-devel/src/library/tools/src'
> mkdir -p -- ../../../../library/tools/libs
> make[6]: Leaving directory
> `/home/tobias/rAdmin/R-devel/src/library/tools/src'
> make[5]: Leaving directory
> `/home/tobias/rAdmin/R-devel/src/library/tools/src'
> make[4]: Leaving directory `/home/tobias/rAdmin/R-devel/src/library/tools'
> Error in parse(n = -1, file = file) :
>     function is too long to keep source (at line 2967)
> Error: unable to load R code in package ?tools?
> Execution halted
>
> [...]
>
> tobias at openanalytics:~/rAdmin$ echo $R_KEEP_PKG_SOURCE
> yes
>
> I do not have this issue when R_KEEP_PKG_SOURCE is set
> to 'false' during compilation.
>
> Best,
> Tobias
>
> >>  P.S. One could even consider a post-install option e.g. to add 'real'
> >>  R sources (and source references) to Windows packages (which are by
> >>  definition already 'installed' and for which such information is not
> >>  by default included in the CRAN binaries of these packages).
> >>
> >>  >>  >  Prof Brian Ripley wrote:
> >>  >>  >  There was an R-core meeting the week before last, and various
> >>  planned
> >>  >>  >  changes will appear in R-devel over the next few weeks.
> >>  >>  >
> >>  >>  >  These are changes planned for R 2.14.0 scheduled for Oct 31. As we
> >>  >>  >  are sick of people referring to R-devel as '2.14' or '2.14.0', that
> >>  >>  >  version number will not be used until we reach 2.14.0 alpha. You
> >>  >>  >  will be able to have a package depend on an svn version number when
> >>  >>  >  referring to R-devel rather than using R (>= 2.14.0).
> >>  >>  >
> >>  >>  >  All packages are installed with lazy-loading (there were 72 CRAN
> >>  >>  >  packages and 8 BioC packages which opted out). This means that the
> >>  >>  >  code is always parsed at install time which inter alia simplifies
> >>  the
> >>  >>  >  descriptions. R 2.13.1 RC warns on installation about packages which
> >>  >>  >  ask not to be lazy-loaded, and R-devel ignores such requests (with a
> >>  >>  >  warning).
> >>  >>  >
> >>  >>  >  In the near future all packages will have a name space. If the
> >>  >>  >  sources do not contain one, a default NAMESPACE file will be added.
> >>  >>  >  This again will simplify the descriptions and also a lot of internal
> >>  >>  >  code. Maintainers of packages without name spaces (currently 42% of
> >>  >>  >  CRAN) are encouraged to add one themselves.
> >>  >>  >
> >>  >>  >  R-devel is installed with the base and recommended packages
> >>  >>  >  byte-compiled (the equivalent of 'make bytecode' in R 2.13.x, but
> >>  >>  >  done less inefficiently). There is a new option R CMD INSTALL
> >>  >>  >  --byte-compile to byte-compile contributed packages, but that
> >>  remains
> >>  >>  >  optional.
> >>  >>  >  Byte-compilation is quite expensive (so you definitely want to do it
> >>  >>  >  at install time, which requires lazy-loading), and relatively few
> >>  >>  >  packages benefit appreciably from byte-compilation. A larger number
> >>  >>  >  of packages benefit from byte-compilation of R itself: for example
> >>  >>  >  AER runs its checks 10% faster. The byte-compiler technology is
> >>  >>  >  thanks to Luke Tierney.
> >>  >>  >
> >>  >>  >  There is support for figures in Rd files: currently with a
> >>  first-pass
> >>  >>  >  implementation (thanks to Duncan Murdoch).
> >>  >  ______________________________________________
> >>  >  R-devel at r-project.org mailing list
> >>  >  https://stat.ethz.ch/mailman/listinfo/r-devel
> >>  >
> >>
> >>  ______________________________________________
> >>  R-devel at r-project.org mailing list
> >>  https://stat.ethz.ch/mailman/listinfo/r-devel
> >
>



[Rd] Recent and upcoming changes to R-devel
Duncan Murdoch