[Rd] segfault / crash when asking for large memory via strrep()

[luke-tierney at uiowa.edu]

That would be because the product nc * ni overflows in

     cbuf = buf = CallocCharBuf(nc * ni);

Since we disallow strings with more than 2^31-1 bytes we could test
and reject this. It might be more future-proof to change the
declaration of

     int j, ni, nc;

to

     R_xlen_t j, ni, nc;

and let the character allocation code reject, but that would create a
memory leak since the Free call isn't reached. This is a problem in
any case though, as

SET_STRING_ELT(s, is, markKnown(cbuf, STRING_ELT(x, ix)));

could throw errors for a number of reasons and then the Free() is not
reached. It would be better to use R_alloc or register a cleanup
function to call Free on a jump.

Best,

luke

On Wed, 1 Jun 2016, Martin Maechler wrote:

> We've had this more general topic on R-help,  and also in R-devel recently.
> There's one case here where I get the feeling R never gets into
> swapping but more directly aborts possibly from a bug we can
> more easily fix.
>
> Today I've been working (successfully! - not yet committed) at
> fixing  str() for very large strings.
>
> In this process, I've found that
>
>   pc <- function(.) paste(., collapse=".1.2.3.4.5.")
>   p  <- function(.) strrep(pc(.), 64L)
>   p(p(p(p(LETTERS))))
>
> produces a (memory related) segmentation fault (aka "crash")
> very reproducibly and relatively quickly
> both on my Linux (Fedora 22) desktop and on our Windows server.
>
> *** caught segfault ***
> address 0x7fc52dc89000, cause 'memory not mapped'
>
> Traceback:
> 1: strrep(pc(.), 64L)
> 2: p(p(p(p(LETTERS))))
> 3: system.time(L2 <- p(p(p(p(LETTERS)))))
>
> In the debugger, the symptoms point to the possibility of a
> bug just in the C parts of strrep() :
>
>
> Program received signal SIGSEGV, Segmentation fault.
> 0x00007ffff54d6223 in __strcpy_sse2_unaligned () from /usr/lib64/libc.so.6
> Missing separate debuginfos, use: dnf debuginfo-install bzip2-libs-1.0.6-14.fc22.x86_64 libgcc-5.3.1-6.fc22.x86_64 libgfortran-5.3.1-6.fc22.x86_64 libgomp-5.3.1-6.fc22.x86_64 libicu-54.1-4.fc22.x86_64 libquadmath-5.3.1-6.fc22.x86_64 libstdc++-5.3.1-6.fc22.x86_64 ncurses-libs-5.9-18.20150214.fc22.x86_64 pcre-8.38-4.fc22.x86_64 readline-6.3-5.fc22.x86_64 xz-libs-5.2.0-2.fc22.x86_64 zlib-1.2.8-7.fc22.x86_64
> (gdb) bt
> #0  0x00007ffff54d6223 in __strcpy_sse2_unaligned () from /usr/lib64/libc.so.6
> #1  0x0000000000457def in do_strrep (call=<optimized out>, op=<optimized out>, args=<optimized out>,
>    env=<optimized out>) at ../../../R/src/main/character.c:1658
> #2  0x00000000004d6844 in bcEval (body=body at entry=0xd66840, rho=rho at entry=0x45253b8,
>    useCache=useCache at entry=TRUE) at ../../../R/src/main/eval.c:5648
> #3  0x00000000004dd240 in Rf_eval (e=0xd66840, rho=0x45253b8) at ../../../R/src/main/eval.c:616
> #4  0x00000000004dedaf in Rf_applyClosure (call=call at entry=0x45250a8, op=op at entry=0xd668e8,
>    arglist=0x45251f8, rho=rho at entry=0x4525000, suppliedvars=0xa57188)
>    at ../../../R/src/main/eval.c:1134
> #5  0x00000000004dd3b1 in Rf_eval (e=0x45250a8, rho=0x4525000) at ../../../R/src/main/eval.c:732
> #6  0x00000000004dedaf in Rf_applyClosure (call=call at entry=0x4525718, op=op at entry=0x4524d28,
>    arglist=0x4524f90, rho=rho at entry=0xa8ea30, suppliedvars=0xa57188)
>    at ../../../R/src/main/eval.c:1134
> #7  0x00000000004dd3b1 in Rf_eval (e=0x4525718, rho=0xa8ea30) at ../../../R/src/main/eval.c:732
> #8  0x00000000004e0cde in do_set (call=0x4525670, op=0xa61358, args=<optimized out>, rho=0xa8ea30)
>    at ../../../R/src/main/eval.c:2196
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

-- 
Luke Tierney
Ralph E. Wareham Professor of Mathematical Sciences
University of Iowa                  Phone:             319-335-3386
Department of Statistics and        Fax:               319-335-3017
    Actuarial Science
241 Schaeffer Hall                  email:   luke-tierney at uiowa.edu
Iowa City, IA 52242                 WWW:  http://www.stat.uiowa.edu