From user3855417 at gmail.com  Wed Oct 22 16:53:31 2014
From: user3855417 at gmail.com (SO User)
Date: Wed, 22 Oct 2014 16:53:31 +0200
Subject: [Rd] Using a custom memory allocation function in R
Message-ID: <CACrQQoLKCDJnzBhut3Uhc3JErfe+=UQA-0GMbBGQR0pz+o90pQ@mail.gmail.com>

Heads up: I posted this question on Stackoverflow yesterday
(http://stackoverflow.com/questions/26484103/using-a-custom-memory-allocation-function-in-r).
So far I haven?t gotten a response and I thought this could be an even
better place to ask such a question.

I would like to be able to use my own memory allocation function for
certain data structures (real valued vectors and arrays) in R. The
reason for this is that I need my data to be 64bit aligned and I would
like to use the numa library for having control over which memory node
is used (I'm working on compute nodes with four 12-core AMD Opteron
6174 CPUs).

Now I have two functions for allocating and freeing memory:
numa_alloc_onnode and numa_free(courtesy of
http://stackoverflow.com/questions/8154162/numa-aware-cache-aligned-memory-allocation).
I'm using R version 3.1.1, so I have access to the function
allocVector3(src/main/memory.c), which seems to me as the intended way
of adding a custom memory allocator. I also found the struct
R_allocator in src/include/R_ext

However it is not clear to me how to put these pieces together. Let's
say, in R, I want the result res of an evaluation such as

res <- Y - mean(Y)

to be saved in a memory area allocated with my own function, how would
I do this? Can I integrate allocVector3 directly at the R level? I
assume I have to go through the R-C interface. As far as I know, I
cannot just return a pointer to the allocated area, but have to pass
the result as an argument. So in R I call something like

n <- length(Y)
res <- numeric(length=1)
.Call("R_allocate_using_myalloc", n, res)
res <- Y - mean(Y)

and in C

#include <R.h>
#include <Rinternals.h>
#include <numa.h>

SEXP R_allocate_using_myalloc(SEXP R_n, SEXP R_res){

  PROTECT(R_n = coerceVector(R_n, INTSXP));
  PROTECT(R_res = coerceVector(R_res, REALSXP));
  int *restrict n = INTEGER(R_n);

  R_allocator_t myAllocator;
  myAllocator.mem_alloc = numa_alloc_onnode;
  myAllocator.mem_free = numa_free;
  myAllocator.res = NULL;
  myAllocator.data = ???;

  R_res = allocVector3(REALSXP, n, myAllocator);

  UNPROTECT(2);
  return R_res;
}

Unfortunately I cannot get beyond a variable has incomplete type
'R_allocator_t' compilation error (I had to remove the .data line
since I have no clue as to what I should put there). Also the function
signature for numa_alloc_onnode is (size_t size, int node). How do I
pass the number of the memory node to the numa_alloc_onnode? Is that
somehow done through the .data field? Does any of the above code make
sense? Is there an easier way of achieving what I want to? It seems a
bit odd to have to allocate a small vector in R and the change its
location in memory in C just to be able to both control the memory
allocation and have the vector available in R...

I'm trying to avoid using Rcpp, as I'm modifying a fairly large
package and do not want to convert all C calls and thought that mixing
different C interfaces could perform sub-optimally.

Any help is greatly appreciated.



[Rd] Using a custom memory allocation function in R
SO User