From ripley at stats.ox.ac.uk  Thu Jul 12 14:17:48 2007
From: ripley at stats.ox.ac.uk (Prof Brian Ripley)
Date: Thu, 12 Jul 2007 13:17:48 +0100 (BST)
Subject: [Rd] cleanup and Makevars
In-Reply-To: <Pine.LNX.4.64.0707120954100.8241@gannet.stats.ox.ac.uk>
References: <A153F980-E93F-4623-B097-E7635151386B@stat.berkeley.edu>
	<Pine.LNX.4.64.0707100553350.16628@gannet.stats.ox.ac.uk>
	<637BDDED-F9CD-48D1-A0C1-D982AAA9DA29@stat.Berkeley.EDU>
	<Pine.LNX.4.64.0707120954100.8241@gannet.stats.ox.ac.uk>
Message-ID: <Pine.LNX.4.64.0707121313210.16834@gannet.stats.ox.ac.uk>

It seems this is the BioC package affxparser.
So my suggestion does work with

R CMD INSTALL affxparser

and using the preferred form of OBJECTS instead of OBJS also works.

BDR

On Thu, 12 Jul 2007, Prof Brian Ripley wrote:

> You seem extraordinarily reluctant to tell us what you are actually doing: no 
> command appears here.  I presume this is a call to R CMD INSTALL, but why do 
> I have to keep guessing?
>
> Since you have taken over the make action via the all: target in your 
> Makevars, you need also to take over responsibility for cleaning.  I should 
> think
>
> all: myclean $(SHLIB)
>
> myclean:
> 	rm -f $(OBJS)
>
> should do it.  And BTW, the use of $(OBJS) to set the objects is 
> undocumented, at least in 'Writing R Extensions'.
>
> On Wed, 11 Jul 2007, Kasper Daniel Hansen wrote:
>
>> 
>> On Jul 11, 2007, at 9:30 AM, Prof Brian Ripley wrote:
>> 
>>> On Mon, 9 Jul 2007, Kasper Daniel Hansen wrote:
>>> 
>>>> Hi
>>>> 
>>>> This is a question prompted by the mac version of R, but as I see it,
>>>> it should have broader interest.
>>>> 
>>>> These days the CRAN Mac binary per default compiles every package for
>>>> two architectures. First i386 and then ppc. In between the two
>>>> compilation runs, any object files in pkgname/src is removed. This
>>>> cleanup is necessary since otherwise Make would not recompile the
>>>> object files under the next architecture.
>>>> 
>>>> I have a package which includes a large SDK which I have put in
>>>>  src/fusion_sdk
>>>> (and subdirectories). I take care of that by using a Makevars file
>>>> with essentially (the whole Makevars file is reproduced below)
>>>> 
>>>> PKG_SOURCES = \
>>>>        fusion_sdk/calvin_files/data/src/CDFData.cpp
>>>> 
>>>> OBJS=$(PKG_SOURCES:.cpp=.o)
>>>> 
>>>> all: $(SHLIB)
>>>> 
>>>> However, this setup does not remove the object files in the
>>>> fusion_sdk subdirectory. From a posting on R-sig-mac by Simon Urbanek
>>>> I learned that I need to clean up these directories myself.
>>>> 
>>>> I have - per R extensions - attempted to do so using a pkgname/
>>>> cleanup script. While this works for cleaning up the sub directories
>>>> when I do R CMD build, it seems as if this script is not being run
>>>> between compilation runs (is this intentional btw.?).
>>> 
>>> What do you mean by 'compilation runs'?  The 'cleanup' script will be run 
>>> by R CMD INSTALL --cleanup (and without --cleanup it is intentionally not 
>>> run).
>>> 
>>>> So my question is now: how do I in a simple way clean up my
>>>> subdirectories? I would prefer it to be as simple as possible because
>>>> so far I have not really needed anything besides a Makevars file. In
>>>> fact the impression I have right now from looking at the SHLIB and
>>>> INSTALL scripts is that I might need a Makefile that removes the
>>>> object files _before_ starting the compilation, as I don't really see
>>>> any cleanup process (eg. making a target clean) - but this may very
>>>> well be due to my limited understanding of these scripts.
>>> 
>>> I see R CMD INSTALL --clean that runs a clean up afterwards, and 2.6.0 
>>> will also have R CMD INSTALL --preclean in case you forgot --clean on the 
>>> last run.
>>> 
>>>> Any help/hints on how to proceed? Especially if I want it to be
>>>> portable?
>>> 
>>> What does R CMD INSTALL --clean not do that you want?
>>> 
>>> If you are talking about R CMD SHLIB, that currently does not clean up but 
>>> this has been raised and it will most likely have a --clean option in 
>>> 2.6.0 that removes $(OBJECTS).
>> 
>> Perhaps I was not clear enough: The CRAN binary of R for Mac has two 
>> architectures inside $R_HOME/bin/exec, namely "ppc" and "i386". This means 
>> that whenever anyone installs a source package I see two compilation runs - 
>> the following is a shortened output:
>> 
>> * Installing to library '/Library/Frameworks/R.framework/Resources/library'
>> * Installing *source* package 'affxparser' ...
>> ** libs
>> ** arch - i386
>> g++-4.0 -arch i386 -isysroot /Developer/SDKs/MacOSX10.4u.sdk 
>> -no-cpp-precomp -I/Library/Frameworks/R.framework/Resources/include 
>> -I/Library/Frameworks/R.framework/Resources/include/i386 -imacros 
>> R_affx_constants.h -Ifusion_sdk/calvin_files/array/src 
>> -Ifusion_sdk/calvin_files/data/src -Ifusion_sdk/calvin_files/exception/src 
>> -Ifusion_sdk/calvin_files/fusion/src 
>> -Ifusion_sdk/calvin_files/fusion/src/GCOSAdapter 
>> -Ifusion_sdk/calvin_files/fusion/src/CalvinAdapter 
>> -Ifusion_sdk/calvin_files/parameter/src 
>> -Ifusion_sdk/calvin_files/parsers/src 
>> -Ifusion_sdk/calvin_files/portability/src 
>> -Ifusion_sdk/calvin_files/template/src -Ifusion_sdk/calvin_files/utils/src 
>> -Ifusion_sdk/calvin_files/writers/src -Ifusion_sdk/file 
>> -Ifusion_sdk/portability -D_USE_MEM_MAPPING_ -msse3    -fPIC  -g -O2 -Wall 
>> -O0 -c fusion_sdk/calvin_files/data/src/CDFData.cpp -o 
>> fusion_sdk/calvin_files/data/src/CDFData.o
>> 
>> (Now a lot of other files gets compiled)
>> 
>> g++-4.0 -arch i386 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -dynamiclib 
>> -Wl,-macosx_version_min -Wl,10.3 -undefined dynamic_lookup -single_module 
>> -multiply_defined suppress -L/usr/local/lib -o affxparser.so 
>> fusion_sdk/calvin_files/data/src/CDFData.o 
>> fusion_sdk/calvin_files/data/src/CDFProbeGroupInformation.o 
>> fusion_sdk/calvin_files/data/src/CDFProbeInformation.o 
>> fusion_sdk/calvin_files/data/src/CDFProbeSetInformation.o 
>> fusion_sdk/calvin_files/data/src/CDFQCProbeInformation.o 
>> fusion_sdk/calvin_files/data/src/CDFQCProbeSetInformation.o 
>> fusion_sdk/calvin_files/data/src/CELData.o 
>> fusion_sdk/calvin_files/data/src/DataGroup.o 
>> fusion_sdk/calvin_files/data/src/DataGroupHeader.o 
>> fusion_sdk/calvin_files/data/src/DataSet.o 
>> fusion_sdk/calvin_files/data/src/DataSetHeader.o 
>> fusion_sdk/calvin_files/data/src/FileHeader.o 
>> fusion_sdk/calvin_files/data/src/GenericData.o 
>> fusion_sdk/calvin_files/data/src/GenericDataHeader.o 
>> fusion_sdk/calvin_files/exception/src/ExceptionBase.o 
>> fusion_sdk/calvin_files/fusion/src/CalvinAdapter/CalvinCELDataAdapter.o 
>> fusion_sdk/calvin_files/fusion/src/FusionBPMAPData.o 
>> fusion_sdk/calvin_files/fusion/src/FusionCDFData.o 
>> fusion_sdk/calvin_files/fusion/src/FusionCDFQCProbeSetNames.o 
>> fusion_sdk/calvin_files/fusion/src/FusionCELData.o 
>> fusion_sdk/calvin_files/fusion/src/GCOSAdapter/GCOSCELDataAdapter.o 
>> fusion_sdk/calvin_files/parameter/src/ParameterNameValueType.o 
>> fusion_sdk/calvin_files/parsers/src/CDFFileReader.o 
>> fusion_sdk/calvin_files/parsers/src/CelFileReader.o 
>> fusion_sdk/calvin_files/parsers/src/DataGroupHeaderReader.o 
>> fusion_sdk/calvin_files/parsers/src/DataGroupReader.o 
>> fusion_sdk/calvin_files/parsers/src/DataSetHeaderReader.o 
>> fusion_sdk/calvin_files/parsers/src/DataSetReader.o 
>> fusion_sdk/calvin_files/parsers/src/FileHeaderReader.o 
>> fusion_sdk/calvin_files/parsers/src/FileInput.o 
>> fusion_sdk/calvin_files/parsers/src/GenericDataHeaderReader.o 
>> fusion_sdk/calvin_files/parsers/src/GenericFileReader.o 
>> fusion_sdk/calvin_files/utils/src/AffymetrixGuid.o 
>> fusion_sdk/calvin_files/utils/src/DateTime.o 
>> fusion_sdk/calvin_files/utils/src/FileUtils.o 
>> fusion_sdk/calvin_files/utils/src/StringUtils.o 
>> fusion_sdk/calvin_files/utils/src/checksum.o 
>> fusion_sdk/file/BPMAPFileData.o fusion_sdk/file/BPMAPFileWriter.o 
>> fusion_sdk/file/CDFFileData.o fusion_sdk/file/CELFileData.o 
>> fusion_sdk/file/FileIO.o fusion_sdk/file/FileWriter.o R_affx_cel_parser.o 
>> R_affx_cdf_parser.o R_affx_cdf_extras.o R_affx_bpmap_parser.o 
>> -F/Library/Frameworks/R.framework/.. -framework R
>> ** arch - ppc
>> g++-4.0 -arch ppc -isysroot /Developer/SDKs/MacOSX10.4u.sdk -no-cpp-precomp 
>> -I/Library/Frameworks/R.framework/Resources/include 
>> -I/Library/Frameworks/R.framework/Resources/include/ppc -imacros 
>> R_affx_constants.h -Ifusion_sdk/calvin_files/array/src 
>> -Ifusion_sdk/calvin_files/data/src -Ifusion_sdk/calvin_files/exception/src 
>> -Ifusion_sdk/calvin_files/fusion/src 
>> -Ifusion_sdk/calvin_files/fusion/src/GCOSAdapter 
>> -Ifusion_sdk/calvin_files/fusion/src/CalvinAdapter 
>> -Ifusion_sdk/calvin_files/parameter/src 
>> -Ifusion_sdk/calvin_files/parsers/src 
>> -Ifusion_sdk/calvin_files/portability/src 
>> -Ifusion_sdk/calvin_files/template/src -Ifusion_sdk/calvin_files/utils/src 
>> -Ifusion_sdk/calvin_files/writers/src -Ifusion_sdk/file 
>> -Ifusion_sdk/portability -D_USE_MEM_MAPPING_ -I/usr/local/include    -fPIC 
>> -g -O2 -Wall -O0 -c R_affx_cel_parser.cpp -o R_affx_cel_parser.o
>> 
>> As you see from this (and sorry for all the include statements), when R 
>> starts compiling for arch = "ppc", it starts with a with in /src whereas 
>> the first compilation run (or more precise the first architecture) starts 
>> from /src/fusion_sdk/calvin_files/data/src. This is of course because the 
>> object files in the subdirectories do not get removed between the two 
>> architectures.
>> 
>> This has - probably - something to do with R CMD SHLIB not cleaning up 
>> after itself. A method of removing $(OBJECTS) would be exactly what I need, 
>> and may I suggest that this is done per default instead of being set by an 
>> option. Anyone using the CRAN binary for the Mac will have the same problem 
>> as I.
>> 
>> I am happy to see that this may be fixed for R-2.6.0. Is there anything I 
>> can do in the meantime for getting my package to work under R-2.5.1? (I can 
>> always move my source files from the /src/fusion_sdk subdirectories 
>> directly into /src, but I would prefer to avoid that).
>> 
>>>> 
>>>> Kasper
>>>> 
>>>> The full Makevars file:
>>>> 
>>>> MYCXXFLAGS=-O0
>>> 
>>> You do realize that is highly non-portable?
>> 
>> Yes, but I do not know of any other way to downgrade the optimization 
>> level. The SDK I am using breaks (due to memory alignment issues if a 
>> specific optimization flag for GCC is include - the flag is certainly 
>> implied by -O2 and as far as I recall also -O1). I guess the solution is to 
>> switch to an autoconfigure script. The main reason for not doing this is 1) 
>> time and actually more 2) the fact that the SDK does not come with an ac 
>> script.
>> 
>> Thanks for the feedback, Kasper
>> 
>> 
>>>> %.o: %.cpp
>>>> 	$(CXX) $(ALL_CPPFLAGS) $(ALL_CXXFLAGS) $(MYCXXFLAGS) -c $< -o $@
>>>> 
>>>> PKG_CPPFLAGS=\
>>>> -imacros R_affx_constants.h\
>>>> -Ifusion_sdk/calvin_files/array/src\
>>>> -Ifusion_sdk/calvin_files/data/src\
>>>> -Ifusion_sdk/calvin_files/exception/src\
>>>> -Ifusion_sdk/calvin_files/fusion/src\
>>>> -Ifusion_sdk/calvin_files/fusion/src/GCOSAdapter\
>>>> -Ifusion_sdk/calvin_files/fusion/src/CalvinAdapter\
>>>> -Ifusion_sdk/calvin_files/parameter/src\
>>>> -Ifusion_sdk/calvin_files/parsers/src\
>>>> -Ifusion_sdk/calvin_files/portability/src\
>>>> -Ifusion_sdk/calvin_files/template/src\
>>>> -Ifusion_sdk/calvin_files/utils/src\
>>>> -Ifusion_sdk/calvin_files/writers/src\
>>>> -Ifusion_sdk/file\
>>>> -Ifusion_sdk/portability\
>>>> -D_USE_MEM_MAPPING_
>>>> 
>>>> PKG_SOURCES = \
>>>> 	fusion_sdk/calvin_files/data/src/CDFData.cpp\
>>>> 	fusion_sdk/calvin_files/data/src/CDFProbeGroupInformation.cpp\
>>>> 	fusion_sdk/calvin_files/data/src/CDFProbeInformation.cpp\
>>>> 	fusion_sdk/calvin_files/data/src/CDFProbeSetInformation.cpp\
>>>> 	fusion_sdk/calvin_files/data/src/CDFQCProbeInformation.cpp\
>>>> 	fusion_sdk/calvin_files/data/src/CDFQCProbeSetInformation.cpp\
>>>> 	fusion_sdk/calvin_files/data/src/CELData.cpp\
>>>> 	fusion_sdk/calvin_files/data/src/DataGroup.cpp\
>>>> 	fusion_sdk/calvin_files/data/src/DataGroupHeader.cpp\
>>>> 	fusion_sdk/calvin_files/data/src/DataSet.cpp\
>>>> 	fusion_sdk/calvin_files/data/src/DataSetHeader.cpp\
>>>> 	fusion_sdk/calvin_files/data/src/FileHeader.cpp\
>>>> 	fusion_sdk/calvin_files/data/src/GenericData.cpp\
>>>> 	fusion_sdk/calvin_files/data/src/GenericDataHeader.cpp\
>>>> 	fusion_sdk/calvin_files/exception/src/ExceptionBase.cpp\
>>>> 	fusion_sdk/calvin_files/fusion/src/CalvinAdapter/
>>>> CalvinCELDataAdapter.cpp\
>>>> 	fusion_sdk/calvin_files/fusion/src/FusionBPMAPData.cpp\
>>>> 	fusion_sdk/calvin_files/fusion/src/FusionCDFData.cpp\
>>>> 	fusion_sdk/calvin_files/fusion/src/FusionCDFQCProbeSetNames.cpp\
>>>> 	fusion_sdk/calvin_files/fusion/src/FusionCELData.cpp\
>>>> 	fusion_sdk/calvin_files/fusion/src/GCOSAdapter/GCOSCELDataAdapter.cpp\
>>>> 	fusion_sdk/calvin_files/parameter/src/ParameterNameValueType.cpp\
>>>> 	fusion_sdk/calvin_files/parsers/src/CDFFileReader.cpp\
>>>> 	fusion_sdk/calvin_files/parsers/src/CelFileReader.cpp\
>>>> 	fusion_sdk/calvin_files/parsers/src/DataGroupHeaderReader.cpp\
>>>> 	fusion_sdk/calvin_files/parsers/src/DataGroupReader.cpp\
>>>> 	fusion_sdk/calvin_files/parsers/src/DataSetHeaderReader.cpp\
>>>> 	fusion_sdk/calvin_files/parsers/src/DataSetReader.cpp\
>>>> 	fusion_sdk/calvin_files/parsers/src/FileHeaderReader.cpp\
>>>> 	fusion_sdk/calvin_files/parsers/src/FileInput.cpp\
>>>> 	fusion_sdk/calvin_files/parsers/src/GenericDataHeaderReader.cpp\
>>>> 	fusion_sdk/calvin_files/parsers/src/GenericFileReader.cpp\
>>>> 	fusion_sdk/calvin_files/utils/src/AffymetrixGuid.cpp\
>>>> 	fusion_sdk/calvin_files/utils/src/DateTime.cpp\
>>>> 	fusion_sdk/calvin_files/utils/src/FileUtils.cpp\
>>>> 	fusion_sdk/calvin_files/utils/src/StringUtils.cpp\
>>>> 	fusion_sdk/calvin_files/utils/src/checksum.cpp\
>>>> 	fusion_sdk/file/BPMAPFileData.cpp\
>>>> 	fusion_sdk/file/BPMAPFileWriter.cpp\
>>>> 	fusion_sdk/file/CDFFileData.cpp\
>>>> 	fusion_sdk/file/CELFileData.cpp\
>>>> 	fusion_sdk/file/FileIO.cpp\
>>>> 	fusion_sdk/file/FileWriter.cpp\
>>>> 	R_affx_cel_parser.cpp\
>>>> 	R_affx_cdf_parser.cpp\
>>>> 	R_affx_cdf_extras.cpp\
>>>> 	R_affx_bpmap_parser.cpp
>>>> 
>>>> OBJS=$(PKG_SOURCES:.cpp=.o)
>>>> 
>>>> all: $(SHLIB)
>>> 
>>> -- 
>>> Brian D. Ripley,                  ripley at stats.ox.ac.uk
>>> Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
>>> University of Oxford,             Tel:  +44 1865 272861 (self)
>>> 1 South Parks Road,                     +44 1865 272866 (PA)
>>> Oxford OX1 3TG, UK                Fax:  +44 1865 272595
>
>

-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595



[Rd] cleanup and Makevars
Prof Brian Ripley