From mtmorgan at fhcrc.org  Mon Dec 16 19:00:11 2013
From: mtmorgan at fhcrc.org (Martin Morgan)
Date: Mon, 16 Dec 2013 10:00:11 -0800
Subject: [Rd] External pointers and changing SEXPTYPE
In-Reply-To: <CAPxYnfp2btvLFUf9t6PQfToXqWVjBd5YXgnfTZresAZXk8ykjw@mail.gmail.com>
References: <CAPxYnfpR=0bVy2m2214f7765MBDmGupFs6=VB7L5K7cH35Z5Ng@mail.gmail.com>	<02A6F984-9A2F-47E4-9CC5-904592BE6803@gmail.com>	<CAPxYnfpj-Z0SNEFdNW33vr+GZLzD8nD_vCEUacfEUA0Mi_Q-9Q@mail.gmail.com>	<52AEE12F.1090506@stats.ox.ac.uk>
	<CAPxYnfp2btvLFUf9t6PQfToXqWVjBd5YXgnfTZresAZXk8ykjw@mail.gmail.com>
Message-ID: <52AF3FAB.2070609@fhcrc.org>

On 12/16/2013 09:18 AM, Krzysztof Mlynarczyk wrote:
> Yes, it turned out that using R_PreserveObject and R_ReleaseObject
> solved that problem.

Instead I think you want to use the third argument to R_MakeExternalPtr to 
protect the SEXP's you'd like to persist for the duration of the object, maybe 
having bundled your ans, ans_nms, R_z, R_a, R_b, R_c into a list. This is 
suggested by the argument name 'prot' and description in Writing R Extensions 
section 5.13.

Martin


> I sincerely apologize for posting only several chunks of code.
> Fortunately for me, the descritpion I gave was sufficient to track
> down what was missing.
> Thank you very much!
>
> Chris
>
> 2013/12/16 Prof Brian Ripley <ripley at stats.ox.ac.uk>:
>> On 16/12/2013 11:07, Krzysztof Mlynarczyk wrote:
>>>
>>> As far as I understood the documentation, external pointer should be
>>> automatically protected from gc when returned to environment. The
>>
>>
>> Yes, but you didn't give us the complete reproducible example the posting
>> guide asks for.  Peter's diagnosis is very likely right, but you have failed
>> to give us anything like enough to go on.
>>
>>
>>> solution you've just suggested would cause stack imbalance.
>>
>>
>> It would not work anyway: the pointer stack top is reset when .Call (or
>> similar) returns.
>>
>> There is R_PreserveObject for this purpose.
>>
>>
>>> Recently I've been thinking of encapsulating the pointer into a nice
>>> object using Rcpp. This sounds better that telling people to have fun
>>> with an external pointer itself.
>>>
>>> KM
>>>
>>> 2013/12/16 peter dalgaard <pdalgd at gmail.com>:
>>>>
>>>> Offhand, I'd say that if "all protects get unprotected before return"
>>>> mydata->ans is not protected against garbage collection, and thus very
>>>> likely collected and reused. If mydata is created by Calloc, the GC has no
>>>> way of knowing that it might have pointers to things that are intended to
>>>> persist.
>>>>
>>>> I haven't played with external pointers for a while, but I'd expect that
>>>> you'd need to retain a PROTECT on mydata->ans, and then UNPROTECT_PTR or so
>>>> in the finalizer.
>>>>
>>>> -pd
>>>>
>>>> On 16 Dec 2013, at 04:11 , Krzysztof Mlynarczyk <mitomaster at gmail.com>
>>>> wrote:
>>>>
>>>>> Dear Developers,
>>>>>
>>>>>
>>>>> I've been struggling through writing R extension in C. I've been using
>>>>> an external pointer to store my data (please see sample below). I
>>>>> encountered a very weird erroneous behaviour: when I tried to use my
>>>>> external pointer to a structure holding several types of data,
>>>>> including SEXPs, I discovered that SEXPs change their types between
>>>>> returning from initialization function and another one that uses the
>>>>> pointer.
>>>>>
>>>>> sample R code:
>>>>>
>>>>> # initializing
>>>>> a <- init_my_ptr(fname)
>>>>>
>>>>> # reading more data: error!
>>>>> df <- read_my_data(a)
>>>>>
>>>>> data structure in C:
>>>>> typedef struct {
>>>>>    SEXP ans, ans_nms, R_z, R_a, R_b, R_c;
>>>>>    FTYPE *datafile;
>>>>>    char *fname;
>>>>>    float *a, *b, *c;
>>>>>    int f_type;
>>>>>    float t, p, l;
>>>>>    int st, na, result, bFlags;
>>>>>    XXX z;
>>>>> } my_data_ptr;
>>>>>
>>>>> // In a C function initializing the external pointer:
>>>>> my_data_ptr *mydata = Calloc( 1, my_data_ptr ) ;
>>>>> SEXP Rdata;
>>>>> PROTECT(Rdata = R_MakeExternalPtr( mydata, R_fname, R_NilValue ));
>>>>> ...
>>>>> mydata->a = Calloc(mydata->na, float);
>>>>> // same for b and c
>>>>> // initializing names so that I could use e.g. df$a where df is
>>>>> returned by read_my_data()
>>>>> PROTECT(mydata->ans_nms = Rf_allocVector(STRSXP, efldNR ));
>>>>>    for( ix = 0; ix < efldNR; ix++ )
>>>>>      SET_STRING_ELT(mydata->ans_nms, ix, mkChar(vnames[ix]));
>>>>>
>>>>> // later I bind values of non-R variables from my data structure to a
>>>>> proper vector
>>>>> PROTECT(mydata->ans = Rf_allocVector(VECSXP, efldNR ));
>>>>>
>>>>>    Rf_setAttrib(mydata->ans, R_NamesSymbol, mytraj->ans_nms);
>>>>>    SET_VECTOR_ELT(mydata->ans, 0,      mydata->R_a );
>>>>>    SET_VECTOR_ELT(mydata->ans, 1,      mydata->R_b );
>>>>> ...
>>>>> // all protects get unprotected before return
>>>>> // finalizer is registered as well
>>>>> return Rdata;
>>>>>
>>>>> Later on in read_my_data() I read the pointer:
>>>>> my_data_ptr *mydata = (my_data_ptr*) R_ExternalPtrAddr(Rdata);
>>>>>
>>>>> // and REAL(mydata->R_a) yields error since TYPEOF(mydata->R_a) is not
>>>>> REALSXP as it should be but RAWSXP for some reason // (sometimes it's
>>>>> STRSXP or INTSXP while it should always be REALSXP)
>>>>> // The error message says:
>>>>> // REAL() can only be applied to a 'numeric', not a 'raw'
>>>>>
>>>>> // mydata->ans is the object returned to R where all the data is made
>>>>> available to R user:
>>>>> return mydata->ans;
>>>>>
>>>>> // end of example code
>>>>>
>>>>> Could you please point the possible reasons for the error along with
>>>>> the ways of fixing this issue? I've been trying in R-3.0.2, 3.0.1 and
>>>>> even 2.15 -- the problem happens in each of them.
>>>>>
>>>>>
>>>>> Regards,
>>>>> Christopher
>>>>>
>>>>> ______________________________________________
>>>>> R-devel at r-project.org mailing list
>>>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>>>
>>>>
>>>> --
>>>> Peter Dalgaard, Professor
>>>> Center for Statistics, Copenhagen Business School
>>>> Solbjerg Plads 3, 2000 Frederiksberg, Denmark
>>>> Phone: (+45)38153501
>>>> Email: pd.mes at cbs.dk  Priv: PDalgd at gmail.com
>>>>
>>>
>>> ______________________________________________
>>> R-devel at r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>>
>>
>>
>> --
>> Brian D. Ripley,                  ripley at stats.ox.ac.uk
>> Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
>> University of Oxford,             Tel:  +44 1865 272861 (self)
>> 1 South Parks Road,                     +44 1865 272866 (PA)
>> Oxford OX1 3TG, UK                Fax:  +44 1865 272595
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>


-- 
Computational Biology / Fred Hutchinson Cancer Research Center
1100 Fairview Ave. N.
PO Box 19024 Seattle, WA 98109

Location: Arnold Building M1 B861
Phone: (206) 667-2793



[Rd] External pointers and changing SEXPTYPE
Martin Morgan