[Rd] R/C++/memory leaks
Ernest Turro
ernest.turro at ic.ac.uk
Tue Feb 27 19:01:33 CET 2007
On 27 Feb 2007, at 17:45, Luke Tierney wrote:
> On Tue, 27 Feb 2007, Ernest Turro wrote:
>
>> Hi Ross,
>>
>> On 26 Feb 2007, at 22:34, Ross Boylan wrote:
>>
>>> On Mon, 2007-02-26 at 16:08 +0000, Ernest Turro wrote:
>>>> Thanks for your comments Ross. A couple more comments/queries
>>>> below:
>>>>
>>>> On 26 Feb 2007, at 06:43, Ross Boylan wrote:
>>>>
>>>>> [details snipped]
>>>>>
>>>>> The use of the R api can be confined to a wrapper function. But
>>>>> I can
>>>>> think of no reason that a change to the alternate approach I
>>>>> outlined
>>>>> would solve the apparent leaking you describe.
>>>>>
>>>>
>>>> I'm not sure I see how a wrapper function using the R API would
>>>> suffice. Example:
>>> It doesn't sound as if it would suffice. I was responding to your
>>> original remark that
>>>
>>>> Since this is a standalone C++ program too, I'd rather use the R
>>>> API
>>>> as little as possible... But I will look at your solution if I find
>>>> it is really necessary.. Thanks
>>>
>>> I thought that was expressing a concern about using the alternate
>>> approach I outlined because it would use the R API. If you need to
>>> use
>>> that API for other reasons, you're still stuck with it :)
>>>>
>>>> During heavy computation in the C++ function I need to allow
>>>> interrupts from R. This means that R_CheckUserInterrupt needs to be
>>>> called during the computation. Therefore, use of the R API can't be
>>>> confined to just the wrapper function.
>>>>
>>>> In fact, I'm worried that some of the libraries I'm using are
>>>> failing
>>>> to release memory after interrupt and that that is the problem. I
>>>> can't see what I could do about that... E.g.
>>>>
>>>> #include <valarray>
>>>>
>>>> valarray<double> foo; // I don't know 100% that the foo object
>>>> hasn't
>>>> allocated some memory. if the program is interrupted it wouldn't be
>>>> released....
>>> That's certainly possible, but you seem to be overlooking the
>>> possibility that all the code is releasing memory appropriately,
>>> but the
>>> process's memory footprint isn't going down correspondingly. In my
>>> experience that's fairly typical behavior.
>>>
>>
>> OK, but does this still explain why the footprint keeps increasing
>> indefinitely when i do run, interrupt, run, interrupt, run,
>> interrupt......?
>>
>>
>>> In that case, depending on your point of view, you either don't
>>> have a
>>> problem or you have a hard problem. If you really want the memory
>>> released back to the system, it's a hard problem. If you don't
>>> care, as
>>> long as you have no leaks, all's well.
>>>
>>>>
>>>> I find it's very unfortunate that R_CheckUserInterrupt doesn't
>>>> return
>>>> a value. If it did (e.g. if it returned true if an interrupt has
>>>> occurred), I could just branch off somewhere, clean up properly and
>>>> return to R.
>>>>
>>>> Any ideas on how this could be achieved?
>>> I can't tell from the info page what function gets called in R if
>>> there
>>> is an interrupt, but it sounds as you could do the following hack:
>>> The R interrupt handler gets a function that calls a C function of
>>> your
>>> devising. The C function sets a flag meaning "interrupt requested".
>>> Then in your main code, you periodically call R_CheckUserInterrupt.
>>> When it returns you check the flag; if it's set, you cleanup and
>>> exit.
>>> Ross
>>>
>>
>> If this is feasible, it's by far the best solution.
>>
>> in error.c:
>>
>> void R_CheckUserInterrupt(void)
>> {
>> R_CheckStack();
>> /* This is the point where GUI systems need to do enough event
>> processing to determine whether there is a user interrupt
>> event
>> pending. Need to be careful not to do too much event
>> processing though: if event handlers written in R are allowed
>> to run at this point then we end up with concurrent R
>> evaluations and that can cause problems until we have proper
>> concurrency support. LT */
>> #if ( defined(HAVE_AQUA) || defined(Win32) )
>> R_ProcessEvents();
>> #else
>> if (R_interrupts_pending)
>> onintr();
>> #endif /* Win32 */
>> }
>>
>> Leaving aside the HAVE_AQUA and Win32 cases, I would like to write a
>> new function:
>
> Unfortunately we can't leave those aside. If standard unix where
> interrupts arrive as signals is all you care about then you can just
> save, replace and restore the R SIGINT handler around your code with
> one that sets a flag of your own. Things are not so simple on GUI
> systems where detecting a user interrupt action requires event
> processing, which might result in errors and non-local exits in
> response to those.
>
> There is an internal mechanism for registering C level on.exit
> routines but this is not in a form that can be made public as it would
> tie down implementation decisions too much. It is principle possible
> to build something around R_ToplevelExec, but that is not at this
> point part of the public API and so is subject to change. We might
> consider providing a variant of R_CheckInterrupts that either just
> checks or that executes cleanup code sometime after 2.5 is released.
>
Yes, I think it would be great if you added a variant of
R_CheckUserInterrupt() that actually returned something (e.g. a
boolean) to the R API.
So in your view, is the best solution to pass a memory cleanup C
function to on.exit() before the call? What about memory allocated in
some of the external libraries that I use? E.g. as I mentioned above:
>>>>
>>>> #include <valarray>
>>>> // ...
>>>> valarray<double> foo; // I don't know 100% that the foo object
>>>> hasn't
>>>> allocated some memory. if the program is interrupted it wouldn't be
>>>> released....
Thanks for your help,
Ernest
> Best,
>
> luke
>
>>
>> int R_CheckInterruptsPending(void)
>> {
>> R_CheckStack();
>> return R_interrupts_pending;
>> }
>>
>> and then in my C++ code:
>>
>> if(R_checkInterruptsPending) {
>> // clean up
>> // ...
>> R_CheckInterruptsPending();
>> }
>>
>> R_CheckStack() is declared in R_ext/Utils.h but the variable
>> R_interrupts_pending isn't, so how could I access it? In other words,
>> how can I extend error.c .....
>>
>>
>> Thanks,
>>
>> E
>>
>> ______________________________________________
>> R-devel at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>
> --
> Luke Tierney
> Chair, Statistics and Actuarial Science
> Ralph E. Wareham Professor of Mathematical Sciences
> University of Iowa Phone: 319-335-3386
> Department of Statistics and Fax: 319-335-3017
> Actuarial Science
> 241 Schaeffer Hall email: luke at stat.uiowa.edu
> Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu
More information about the R-devel
mailing list