[Rd] RProfmem output format
Henrik Bengtsson
henrik.bengtsson at gmail.com
Tue May 30 21:51:16 CEST 2017
UPDATE: Tomas Kalibera has fixed this bug (on missing newlines in
Rprofmem output) in R-devel r72747
(https://github.com/wch/r-source/commit/ba6665deace4a8dc239aebec977c17d0975fbc27).
Using the example of this thread, Rprofmem() of R-devel now outputs
with newlines:
$ R Under development (unstable) (2017-05-30 r72747) -- "Unsuffered
Consequences"
Copyright (C) 2017 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)
[...]
> Rprofmem(); x <- raw(2000); Rprofmem("")
> cat(readLines("Rprofmem.out", n=5, warn=FALSE), sep="\n")
232 :
472 :
472 :
1064 :
2040 :"raw"
/Henrik
On Sat, Jun 4, 2016 at 12:40 PM, Henrik Bengtsson
<henrik.bengtsson at gmail.com> wrote:
> I'm picking up this 5-year old thread.
>
> 1. About the four memory allocations without a stacktrace
>
> I think the four memory allocations without a stacktrace reported by Rprofmem():
>
>> Rprofmem(); x <- raw(2000); Rprofmem("")
>> cat(readLines("Rprofmem.out", n=5, warn=FALSE), sep="\n")
> 192 :360 :360 :1064 :2040 :"raw"
>
> are due to some initialization of R that is independent of Rprofmem(),
> because they can be avoided if one allocates some memory before (in a
> fresh R session):
>
>> z <- raw(1000); dummy <- gc()
>> Rprofmem(); x <- raw(2000); Rprofmem("")
>> cat(readLines("Rprofmem.out", n=5, warn=FALSE), sep="\n")
> 2040 :"raw"
>
>
> 2. About missing newlines when stacktrace is empty
>
> As a refresher, the problem is that memory allocations an empty
> stracktrace are reported without newlines, i.e.
>
> 192 :360 :360 :1064 :2040 :"raw"
>
> The question is why this is not reported as:
>
> 192 :
> 360 :
> 360 :
> 1064 :
> 2040 :"raw"
>
> This was/is because C function R_OutputStackTrace() - part of
> src/main/memory.c - looks like:
>
> static void R_OutputStackTrace(FILE *file)
> {
> int newline = 0;
> RCNTXT *cptr;
>
> for (cptr = R_GlobalContext; cptr; cptr = cptr->nextcontext) {
> if ((cptr->callflag & (CTXT_FUNCTION | CTXT_BUILTIN))
> && TYPEOF(cptr->call) == LANGSXP) {
> SEXP fun = CAR(cptr->call);
> if (!newline) newline = 1;
> fprintf(file, "\"%s\" ",
> TYPEOF(fun) == SYMSXP ? CHAR(PRINTNAME(fun)) :
> "<Anonymous>");
> }
> }
> if (newline) fprintf(file, "\n");
> }
>
>
> Thomas, your last comment was:
>
>> Yes. It's obviously better to always print a newline, and so clearly
>> deliberate not to, that I suspect there may have been a good reason.
>> If I can't work it out (after my grant deadline this week) I will just
>> assume it's wrong.
>
> When I search the code and the commit history
> (https://github.com/wch/r-source/commit/3d5eb2a09f2d75893efdc8bbf1c72d17603886a0),
> it appears that this was there from the very first commit. Also,
> searching the code for usages of R_OutputStackTrace(), I only find
> R_ReportAllocation() and R_ReportNewPage(), both part of of
> src/main/memory.c (see below).
>
> static void R_ReportAllocation(R_size_t size)
> {
> if (R_IsMemReporting) {
> if(size > R_MemReportingThreshold) {
> fprintf(R_MemReportingOutfile, "%lu :", (unsigned long) size);
> R_OutputStackTrace(R_MemReportingOutfile);
> }
> }
> return;
> }
>
> static void R_ReportNewPage(void)
> {
> if (R_IsMemReporting) {
> fprintf(R_MemReportingOutfile, "new page:");
> R_OutputStackTrace(R_MemReportingOutfile);
> }
> return;
> }
>
>
> Could it be that when you wrote it you had another usage for
> R_OutputStackTrace() in mind as well? If so, it makes sense that
> R_OutputStackTrace() shouldn't output a newline if the stack trace was
> empty. But if the above is the only usage, to me it looks pretty safe
> to always add a newline.
>
>> sessionInfo()
> R version 3.3.0 Patched (2016-05-26 r70682)
> Platform: x86_64-w64-mingw32/x64 (64-bit)
> Running under: Windows 7 x64 (build 7601) Service Pack 1
>
> locale:
> [1] LC_COLLATE=English_United States.1252
> [2] LC_CTYPE=English_United States.1252
> [3] LC_MONETARY=English_United States.1252
> [4] LC_NUMERIC=C
> [5] LC_TIME=English_United States.1252
>
> attached base packages:
> [1] stats graphics grDevices utils datasets methods base
>
> /Henrik
>
>
> On Sun, May 15, 2011 at 1:16 PM, Thomas Lumley <tlumley at uw.edu> wrote:
>> On Mon, May 16, 2011 at 1:02 AM, Hadley Wickham <hadley at rice.edu> wrote:
>>> So what causes allocations when the call stack is empty? Something
>>> internal? Does the garbage collector trigger allocations (i.e. could
>>> it be caused by moving data to contiguous memory)?
>>
>> The garbage collector doesn't move anything, it just swaps pointers in
>> a linked list.
>>
>> The lexer, parser, and evaluator all have to do some work before a
>> function context is set up for the top-level function, so I assume
>> that's where it is happening.
>>
>>> Any ideas what the correct thing to do with these memory allocations?
>>> Ignore them because they're not really related to the function they're
>>> attributed to? Sum them up?
>>>
>>>> I don't see why this is done, and I may well be the person who did it
>>>> (I don't have svn on this computer to check), but it is clearly
>>>> deliberate.
>>>
>>> It seems like it would be more consistent to always print a newline,
>>> and then it would obvious those allocations occurred when the call
>>> stack was empty. This would make parsing the file a little bit
>>> easier.
>>
>> Yes. It's obviously better to always print a newline, and so clearly
>> deliberate not to, that I suspect there may have been a good reason.
>> If I can't work it out (after my grant deadline this week) I will just
>> assume it's wrong.
>>
>>
>> -thomas
>>
>> --
>> Thomas Lumley
>> Professor of Biostatistics
>> University of Auckland
>>
>> ______________________________________________
>> R-devel at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
More information about the R-devel
mailing list