[Rd] [External] Possible ALTREP bug

Toby Hocking tdhock5 @end|ng |rom gm@||@com
Wed Jun 16 16:22:08 CEST 2021


By the way, where is the documentation for INTEGER_ELT, REAL_ELT, etc? I
looked in Writing R Extensions and R Internals but I did not see any
mention.
REAL_ELT is briefly mentioned on
https://svn.r-project.org/R/branches/ALTREP/ALTREP.html
Would it be possible to please add some mention of them to Writing R
Extensions?
- how many of these _ELT functions are there? INTEGER, REAL, ... ?
- in what version of R were they introduced?
- I guess input types are always SEXP and int?
- What are the output types for each?

On Fri, May 28, 2021 at 5:16 PM <luke-tierney using uiowa.edu> wrote:

> Since the INTEGER_ELT, REAL_ELT, etc, functions are fairly new it may
> be possible to check that places where they are used allow for them to
> allocate. I have fixed the one that got caught by Gabor's example, and
> a rchk run might be able to pick up others if rchk knows these could
> allocate. (I may also be forgetting other places where the _ELt
> methods are used.)  Fixing all call sites for REAL, INTEGER, etc, was
> never realistic so there GC has to be suspended during the method
> call, and that is done in the dispatch mechanism.
>
> The bigger problem is jumps from inside things that existing code
> assumes will not do that. Catching those jumps is possible but
> expensive; doing anything sensible if one is caught is really not
> possible.
>
> Best,
>
> luke
>
> On Fri, 28 May 2021, Gabriel Becker wrote:
>
> > Hi Jim et al,
> > Just to hopefully add a bit to what Luke already answered, from what I am
> > recalling looking back at that bioconductor thread Elt methods are used
> in
> > places where there are hard implicit assumptions that no garbage
> collection
> > will occur (ie they are called on things that aren't PROTECTed), and
> beyond
> > that, in places where there are hard assumptions that no error (longjmp)
> > will occur. I could be wrong, but I don't know that suspending garbage
> > collection would protect from the second one. Ie it is possible that an
> > error *ever* being raised from R code that implements an elt method could
> > cause all hell to break loose.
> >
> > Luke or Tomas Kalibera would know more.
> >
> > I was disappointed that implementing ALTREPs in R code was not in the
> cards
> > (it was in my original proposal back in 2016 to the DSC) but I trust Luke
> > that there are important reasons we can't safely allow that.
> >
> > Best,
> > ~G
> >
> > On Fri, May 28, 2021 at 8:31 AM Jim Hester <james.f.hester using gmail.com>
> wrote:
> >       From reading the discussion on the Bioconductor issue tracker it
> >       seems like
> >       the reason the GC is not suspended for the non-string ALTREP Elt
> >       methods is
> >       primarily due to performance concerns.
> >
> >       If this is the case perhaps an additional flag could be added to
> >       the
> >       `R_set_altrep_*()` functions so ALTREP authors could indicate if
> >       GC should
> >       be halted when that particular method is called for that
> >       particular ALTREP
> >       class.
> >
> >       This would avoid the performance hit (other than a boolean
> >       check) for the
> >       standard case when no allocations are expected, but allow
> >       authors to
> >       indicate that R should pause GC if needed for methods in their
> >       class.
> >
> >       On Fri, May 28, 2021 at 9:42 AM <luke-tierney using uiowa.edu> wrote:
> >
> >       > integer and real Elt methods are not expected to allocate. You
> >       would
> >       > have to suspend GC to be able to do that. This currently can't
> >       be done
> >       > from package code.
> >       >
> >       > Best,
> >       >
> >       > luke
> >       >
> >       > On Fri, 28 May 2021, Gábor Csárdi wrote:
> >       >
> >       > > I have found some weird SEXP corruption behavior with
> >       ALTREP, which
> >       > > could be a bug. (Or I could be doing something wrong.)
> >       > >
> >       > > I have an integer ALTREP vector that calls back to R from
> >       the Elt
> >       > > method. When this vector is indexed in a lapply(), its first
> >       element
> >       > > gets corrupted. Sometimes it's just a type change to
> >       logical, but
> >       > > sometimes the corruption causes a crash.
> >       > >
> >       > > I saw this on macOS from R 3.5.3 to 4.2.0. I created a small
> >       package
> >       > > that demonstrates this:
> >       https://github.com/gaborcsardi/redfish
> >       > >
> >       > > The R callback in this package calls
> >       `loadNamespace("Matrix")`, but
> >       > > the same crash happens for other packages as well, and
> >       sometimes it
> >       > > also happens if I don't load any packages at all. (But that
> >       example
> >       > > was much more complicated, so I went with the package
> >       loading.)
> >       > >
> >       > > It is somewhat random, and sometimes turning off the JIT
> >       avoids the
> >       > > crash, but not always.
> >       > >
> >       > > Hopefully I am just doing something wrong in the ALTREP code
> >       (see
> >       > >
> >       https://github.com/gaborcsardi/redfish/blob/main/src/test.c),
> >       and it
> >       > > is not actually a bug.
> >       > >
> >       > > Thanks,
> >       > > Gabor
> >       > >
> >       > > ______________________________________________
> >       > > R-devel using r-project.org mailing list
> >       > > https://stat.ethz.ch/mailman/listinfo/r-devel
> >       > >
> >       >
> >       > --
> >       > Luke Tierney
> >       > Ralph E. Wareham Professor of Mathematical Sciences
> >       > University of Iowa                  Phone:
> >        319-335-3386
> >       > Department of Statistics and        Fax:
> >        319-335-3017
> >       >     Actuarial Science
> >       > 241 Schaeffer Hall                  email:
> >        luke-tierney using uiowa.edu
> >       > Iowa City, IA 52242                 WWW:
> >       http://www.stat.uiowa.edu
> >       > ______________________________________________
> >       > R-devel using r-project.org mailing list
> >       > https://stat.ethz.ch/mailman/listinfo/r-devel
> >       >
> >
> >               [[alternative HTML version deleted]]
> >
> >       ______________________________________________
> >       R-devel using r-project.org mailing list
> >       https://stat.ethz.ch/mailman/listinfo/r-devel
> >
> >
> >
>
> --
> Luke Tierney
> Ralph E. Wareham Professor of Mathematical Sciences
> University of Iowa                  Phone:             319-335-3386
> Department of Statistics and        Fax:               319-335-3017
>     Actuarial Science
> 241 Schaeffer Hall                  email:   luke-tierney using uiowa.edu
> Iowa City, IA 52242                 WWW:  http://www.stat.uiowa.edu
> ______________________________________________
> R-devel using r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

	[[alternative HTML version deleted]]



More information about the R-devel mailing list