[Rd] C - R integration: Memory Issues
Simon Urbanek
simon.urbanek at r-project.org
Fri Apr 20 15:51:30 CEST 2012
On Apr 20, 2012, at 5:06 AM, Nikolaos Bezirgiannidis wrote:
> On Thu, 19 Apr 2012 07:21:12 -0500, Dirk Eddelbuettel wrote:
>> On 18 April 2012 at 17:40, Nikolaos Bezirgiannidis wrote:
>> | Hi all,
>> |
>> | I am a PhD student and I am working on a C project that involves some
>> | statistical calculations. So, I tried to embed R into C, in order to
>> | call R functions from a C program. My program seems to get the correct
>> | results from R. However, it appears to have a lot of memory allocation
>> | issues, in contrast to the small amounts of memory that my code
>> | allocates. Some additional info that might be useful: I have build R
>> | from source with shared libraries enabled and the compiler I use is gcc
>> | version 4.61 in a Ubuntu 11.10 linux machine.
>>
>> [ Well I suspect "sudo apt-get install r-base" would have given you the same;
>> see the README at $CRAN/src/bin/linux/ubuntu ]
>>
>> | This is my function:
>> |
>> | static int prediction(double *berHistory, int berValues, double *ber)
>> | {
>> | SEXP e;
>> | SEXP bers;
>> | SEXP mean;
>> | int i;
>> | int errorOccurred;
>> | static int init = 0;
>> | char *argv[] = {"REmbeddedPostgres", "--gui=none", "--silent"};
>> | int argc = sizeof(argv)/sizeof(argv[0]);
>> |
>> | // Initialize Embedded R
>> | if (init == 0)
>> | {
>> | Rf_initEmbeddedR(argc, argv);
>> | }
>> | init = 1;
>> |
>> | // Allocate bers and copy values
>> | PROTECT(bers = allocVector(REALSXP, berValues));
>> |
>> | for (i = 0; i < berValues; i++)
>> | {
>> | REAL(bers)[i] = berHistory[i];
>> | }
>> |
>> | PROTECT(mean = allocVector(REALSXP, 1));
>> | PROTECT(e = lang2(install("mean"), bers));
>> | mean = R_tryEval(e, R_GlobalEnv, &errorOccurred);
>> | if (errorOccurred)
>> | {
>> | printf("error occurred in mean\n");
>> | }
>> |
>> | for (i = 0; i < berValues; i++)
>> | {
>> | REAL(bers)[i] = REAL(bers)[i] / REAL(mean)[0];
>> | }
>> |
>> | *ber = REAL(mean)[0];
>> |
>> | Rf_endEmbeddedR(0);
>> | UNPROTECT(3);
>> | return 0;
>> | }
>> |
>> | And these are the errors from Valgrind output:
>> |
>> | HEAP SUMMARY:
>> | ==2909== in use at exit: 18,832,260 bytes in 6,791 blocks
>> | ==2909== total heap usage: 21,758 allocs, 14,967 frees, 30,803,476
>> | bytes allocated
>> | ==2909==
>> | ==2909== 160 (40 direct, 120 indirect) bytes in 1 blocks are definitely
>> | lost in loss record 179 of 1,398
>> | ==2909== at 0x4028876: malloc (vg_replace_malloc.c:236)
>> | ==2909== by 0x41B364C: nss_parse_service_list (nsswitch.c:626)
>> | ==2909== by 0x41B3C59: __nss_database_lookup (nsswitch.c:167)
>> | ==2909== by 0x59272F8: ???
>> | ==2909== by 0x5928CCC: ???
>> | ==2909== by 0x416ABA6: getpwuid_r@@GLIBC_2.1.2 (getXXbyYY_r.c:256)
>> | ==2909== by 0x416A4ED: getpwuid (getXXbyYY.c:117)
>> | ==2909== by 0x439CCB9: do_fileinfo (platform.c:944)
>> | ==2909== by 0x43289ED: bcEval (eval.c:4430)
>> | ==2909== by 0x4332CA4: Rf_eval (eval.c:397)
>> | ==2909== by 0x43377E0: Rf_applyClosure (eval.c:855)
>> | ==2909== by 0x432F17E: bcEval (eval.c:4410)
>> | ==2909==
>> | ==2909== 160 (40 direct, 120 indirect) bytes in 1 blocks are definitely
>> | lost in loss record 180 of 1,398
>> | ==2909== at 0x4028876: malloc (vg_replace_malloc.c:236)
>> | ==2909== by 0x41B364C: nss_parse_service_list (nsswitch.c:626)
>> | ==2909== by 0x41B3C59: __nss_database_lookup (nsswitch.c:167)
>> | ==2909== by 0x5926148: ???
>> | ==2909== by 0x5926F3C: ???
>> | ==2909== by 0x41694A6: getgrgid_r@@GLIBC_2.1.2 (getXXbyYY_r.c:256)
>> | ==2909== by 0x4168CAD: getgrgid (getXXbyYY.c:117)
>> | ==2909== by 0x439CCEB: do_fileinfo (platform.c:947)
>> | ==2909== by 0x43289ED: bcEval (eval.c:4430)
>> | ==2909== by 0x4332CA4: Rf_eval (eval.c:397)
>> | ==2909== by 0x43377E0: Rf_applyClosure (eval.c:855)
>> | ==2909== by 0x432F17E: bcEval (eval.c:4410)
>> | ==2909==
>> | ==2909== LEAK SUMMARY:
>> | ==2909== definitely lost: 80 bytes in 2 blocks
>> | ==2909== indirectly lost: 240 bytes in 20 blocks
>> | ==2909== possibly lost: 0 bytes in 0 blocks
>> | ==2909== still reachable: 18,831,940 bytes in 6,769 blocks
>>
>> Well I think there is no real bad error here. You lost 80 and 240 bytes. That
>> is nothing to worry about, and somewhat normal. R is a dynamic system,
>> valgrind measures "with some error".
>>
>> You can compare the result to doing an allocation of a longer vector and not
>> freeing it.
>>
>> You could see if freeing your 'mean' variable at the end makes a difference,
>> or using a pointer to a single (stack) instance instead of an allocation
>> makes a difference. Likewise, you could returns bers as well. Or free
>> it. Right now I am not entirely what it is that your 'prediction' function is
>> trying to do.
>
> It is actually a more complex function with many more allocations and R_tryEval calls; the simplified one that I included leads to exactly the same lost bytes, which means that it is not caused by allocVector, but probably by the R_tryEval call. Is there any other, more proper way to evaluate R expressions from C?
>
> In my function I only need to return ber, so I could easily free all R objects (bers and mean here). But C's free() has no result and I am not familiar with any R function that frees objects allocated by allocVector. I have also seen in some R tutorials and examples that objects allocated by allocVector are not freed in any way.
>
R uses a garbage collector, so I think you're chasing a ghost here...
But if you look at the output of lost blocks this is simply storage allocated by your glibc for non-reentrant calls so it has nothing to do with R.
Cheers,
Simon
>>
>> | Reachable error summary is far too long to include in this mail. The
>> | interesting thing is that reachable errors are all caused by this small
>> | function.
>> |
>> | Any ideas? I would also appreciate any suggestions on how to improve
>> | the R-C integration in my code.
>>
>> Have you considered using R and C++ instead, and looked at Rcpp and
>> RInside?
>>
>> http://dirk.eddelbuettel.com/code/rcpp.html
>>
>> http://dirk.eddelbuettel.com/code/rinside.html
>>
>> RInside in particular is a lot simpler, at least to me. But some people
>> really want plain C in which case you know which route to take
>>
>> Dirk
>
> No, unfortunately I cannot switch to C++, because my program is a part of a bigger C project.
>
> Thanks for your reply and suggestions,
> Nikos
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
>
More information about the R-devel
mailing list