[Rd] segfault when applying strange coercion (PR#2923)

[Peter Dalgaard BSA]

Peter Dalgaard BSA <p.dalgaard at biostat.ku.dk> writes:

> ligges at statistik.uni-dortmund.de writes:
> 
> > I made a mistake -- and R crashed. A (more or less) reproducible example 
> > seems to be:
> > 
> >    x <- matrix(nrow=20000, ncol=20)
> >    x$any <- numeric(0)
> > 
> > It is possible to crash R by executing these lines several times (1 - 
> > 20) on Windows as well as on Solaris (might be related to PR#2880 - 
> > let's call it "non-existing list elements")
> 
> Linux too: (r-devel)
> 
> ...
> > x <- matrix(nrow=20000, ncol=20) ;    x$any <- numeric(0)
> Warning message:
> Coercing LHS to a list
> > x <- matrix(nrow=20000, ncol=20) ;    x$any <- numeric(0)
> 
> Program received signal SIGSEGV, Segmentation fault.
> Rf_copyMostAttrib (inp=0xa2325a8, ans=0x402f9008)
>     at ../../../R/src/main/attrib.c:155
> 155             if ((TAG(s) != R_NamesSymbol) &&
> 
> Tried running it with gctorture() but haven't gotten the prompt back
> yet.... 

For once, gctorture didn't seem to be of any use except to make things
take longer. 

A little further debugging reveals that the crash is happening in this
loop

154         for (s = ATTRIB(inp); s != R_NilValue; s = CDR(s)) {
155             if ((TAG(s) != R_NamesSymbol) &&
156                 (TAG(s) != R_DimSymbol) &&
157                 (TAG(s) != R_DimNamesSymbol)) {
158                 installAttrib(ans, TAG(s), CAR(s));
159             }
 
where inp is

(gdb) p *inp
$32 = {sxpinfo = {type = 4, obj = 0, named = 3, gp = 4789, mark = 0,
    debug = 1, trace = 0, fin = 0, gcgen = 0, gccls = 2}, attrib =0x4212b5c4,
  gengc_next_node = 0x83a8d50, gengc_prev_node = 0x83a8d50, u =
{primsxp = {...

[type = 4 means that it is an ENVSXP -- I don't think they generally
want to have attributes?]

(gdb) p *(inp->attrib)
$33 = {sxpinfo = {type = 28, obj = 1, named = 2, gp = 4789, mark = 0,
    debug = 1, trace = 0, fin = 0, gcgen = 0, gccls = 2}, attrib = 0x4212b5bc,

[type = 28 is undefined, so something is clearly wrong at this point]

Comes from R_subassign3_dflt ..hmm.. We have 

1680            if (!(isNewList(x) || isExpression(x))) {
1681                warning("Coercing LHS to a list");
1682                x = coerceVector(x, VECSXP);
1683            }

which requires reprotection of x does it not?

-- 
   O__  ---- Peter Dalgaard             Blegdamsvej 3  
  c/ /'_ --- Dept. of Biostatistics     2200 Cph. N   
 (*) \(*) -- University of Copenhagen   Denmark      Ph: (+45) 35327918
~~~~~~~~~~ - (p.dalgaard at biostat.ku.dk)             FAX: (+45) 35327907