[Rd] Bracketed paste issues on Linux
Cesko Voeten
c@c@voeten @end|ng |rom hum@|e|denun|v@n|
Mon Jun 14 12:06:05 CEST 2021
Making it 1024 times larger gives:
installing 'sysdata.rda'
Error: segfault from C stack overflow
Making it only 4 times larger provides a usable R. In my test case of copying&pasting mgcv::gam, I observe the same visual corruption at the prompt as before, but when pressing return it has actually been received correctly. My real-world problem involved a file 33KiB in size, which - as expected, since 16KiB < 33KiB - still has the same problem as before.
I know nothing about readline, but I presume that there is no way for this buffer size to be dynamically resized at run time. In that case, maybe R should simply force-disable readline's bracketed paste? By the way, according to readline's changelog, this does indeed seem to be a feature that changed (viz. was enabled in more places) from readline-8.0 to readline-8.1.
Finally, please disregard my earlier comment about vim and nano working just fine. They do, but they don't actually use readline (according to ldd), so don't provide a valid comparison.
Thanks for your efforts!
Cesko
On 14-06-2021 at 08:33, Tomas Kalibera wrote:
> Thanks, Cesko, for more debugging. As you are already compiling the code, could you please try increasing CONSOLE_BUFFER_SIZE in ./include/Defn.h from 4096 to some very large value (e.g. 1024 times), rebuild R and check if the problems (not all bytes received correctly, visual corruption) go away for texts of the size you looked at before?
>
>
> Thanks,
> Tomas
>
>
>
> On 6/13/21 10:59 AM, Voeten, C.C. wrote:
>>
>> Thanks for looking into this! I've just compiled today's R-devel snapshot, and it shows the same issue. extSoftVersion() from that build:
>>
>> zlib
>> "1.2.11"
>> bzlib
>> "1.0.8, 13-Jul-2019"
>> xz
>> "5.2.5"
>> PCRE
>> "10.37 2021-05-26"
>> ICU
>> "69.1"
>> TRE
>> "TRE 0.8.0 R_fixes (BSD)"
>> iconv
>> "glibc 2.33"
>> readline
>> "8.1"
>> BLAS
>> "/home/cesko/r-devel/usr/lib64/R/lib/libRblas.so"
>>
>> Thanks for your observation that it works on your system - that implicates my readline-8.1 as being the culprit. Unfortunately, I don't dare attempt to downgrade it on my system to test, and regardless we still don't know why other readline-using programs can paste in the same text with no issues.
>>
>>
>> I've made some further progress on debugging: I noticed that text <4096 bytes in size arrives fine (although sometimes with visual corruption), but text >4096 bytes doesn't. Pasting in the result of perl -e 'print ("if(T)cat(\"a\")\n"x292)' works as expected, changing the 292 to 293 causes R to print a bunch of a's followed by the source code of the cat function.
>>
>>
>> To still answer your question: with mgcv::gam, pasting in the first 94 lines (as printed by R with options(width=80)) produces a visual corruption of the prompt (it reads "G$family <- familyar.summaryintercept = drop.intercept)) control$scalePenalty,") but if I press return and type the closing "}" the code has actually arrived just fine. The text up to and including that line is 4023 bytes in size; when trying to add in more, it fails again.
>>
>>
>> Cesko
>> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>> *Van:* Tomas Kalibera <tomas.kalibera using gmail.com>
>> *Verzonden:* zondag 13 juni 2021 10:00:27
>> *Aan:* Voeten, C.C.; r-devel using r-project.org
>> *Onderwerp:* Re: [Rd] Bracketed paste issues on Linux
>> Thanks for the report. Could you please also post output from
>> extSoftVersion() ?
>>
>> What happens if you paste just a smaller part of the code before the
>> long line? Is the output still corrupted? If so, is it corrupted the
>> same way, at the same places?
>>
>> (It seems to be working on my Ubuntu 20.04, readline 8.0, R-devel)
>>
>> Thanks
>> Tomas
>>
>> On 6/12/21 3:44 PM, Cesko Voeten wrote:
>> > I am on an up-to-date Arch Linux system, using the GNOME desktop environment. By default, this turns on bracketed paste in terminal emulators; for those not familiar with this concept: it makes it so that if you paste in multiple lines of code, they are received in a single chunk. This works just fine with R, up to a certain amount of text: for chunks past a certain length, some amount of text in the middle of the chunk goes missing. For example, if I print the source of mgcv::gam into my R session and then attempt to copy and paste it back in, what I end up with is:
>> >
>> > <snip 53 perfectly good lines>
>> > pmf$formula <- gp$pf
>> > pmf <- eval(pmf, parent.frame())
>> > } objectvironment(attr(object$pred.formula, "full")) <- .GlobalEnv<- environment(object$terms) <- environment(object$pterms) <- .GlobalEnv
>> >
>> > So:
>> > - the first 55 lines in this example arrive perfectly fine
>> > - then a bunch go completely missing
>> > - then various parts of the last few lines are jumbled together into one line
>> >
>> > For reference on the third point, the actual last 10 lines of my version of mgcv::gam are:
>> > if (is.null(object$deviance))
>> > object$deviance <- sum(residuals(object, "deviance")^2)
>> > names(object$gcv.ubre) <- method
>> > environment(object$formula) <- environment(object$pred.formula) <- environment(object$terms) <- environment(object$pterms) <- .GlobalEnv
>> > if (!is.null(object$model))
>> > environment(attr(object$model, "terms")) <- .GlobalEnv
>> > if (!is.null(attr(object$pred.formula, "full")))
>> > environment(attr(object$pred.formula, "full")) <- .GlobalEnv
>> > object
>> > }
>> >
>> > parts of which can be recognized in the last line of what was pasted.
>> > Naturally, the pasted function is not parsed properly: if I press return I get the expected "+" signaling that the REPL is expecting more input. So it is not merely a visual issue.
>> >
>> > I can reproduce this both in GNOME Terminal and in xterm, so it is not a bug specific to my terminal emulator. In addition, pasting the exact same code into either vim or nano running within the same terminal works fine. So I believe that this may be a bug in R itself. It's easy to work around by disabling bracketed paste in the terminal, but it would be great if this could actually be made to work, especially given that bracketed paste is the default on my desktop environment.
>> >
>> > If given an account, I would be happy to file this as a bug; let me know if that is desired. In the meantime, have others run into this and perhaps identified the root cause and/or a different workaround?
>> >
>> > Thanks,
>> > Cesko
>> >
>> > sessionInfo():
>> >
>> > R version 4.1.0 (2021-05-18)
>> > Platform: x86_64-pc-linux-gnu (64-bit)
>> > Running under: Arch Linux
>> >
>> > Matrix products: default
>> > BLAS/LAPACK: /opt/intel/mkl/lib/intel64/libmkl_gf_lp64.so
>> >
>> > locale:
>> > [1] LC_CTYPE=nl_NL.UTF-8 LC_NUMERIC=C
>> > [3] LC_TIME=nl_NL.UTF-8 LC_COLLATE=nl_NL.UTF-8
>> > [5] LC_MONETARY=nl_NL.UTF-8 LC_MESSAGES=nl_NL.UTF-8
>> > [7] LC_PAPER=nl_NL.UTF-8 LC_NAME=C
>> > [9] LC_ADDRESS=C LC_TELEPHONE=C
>> > [11] LC_MEASUREMENT=nl_NL.UTF-8 LC_IDENTIFICATION=C
>> >
>> > attached base packages:
>> > [1] stats graphics grDevices utils datasets methods base
>> >
>> > loaded via a namespace (and not attached):
>> > [1] compiler_4.1.0 Matrix_1.3-4 mgcv_1.8-36 splines_4.1.0
>> > [5] nlme_3.1-152 grid_4.1.0 lattice_0.20-44
>> >
>> > ______________________________________________
>> > R-devel using r-project.org mailing list
>> > https://stat.ethz.ch/mailman/listinfo/r-devel <https://stat.ethz.ch/mailman/listinfo/r-devel>
More information about the R-devel
mailing list