[Rd] R Bug: write.table for matrix of more than 2, 147, 483, 648 elements
spencer.graves at prodsyse.com
Thu Apr 19 01:03:40 CEST 2018
On 2018-04-18 17:38, Steven McKinney wrote:
> Hi Colton,
> You could divide your write task into chunks that do not violate the 2^31-1 limit.
> write.table has an append argument (default FALSE).
> Figure out a row chunk size nri < nr such that nri * nc is under 2^31-1 and use
> write.table() to write that out.
> Then use
> write.table( append = TRUE, )
> for the next chunk of rows, looping over chunks until done. Two chunks will get your 2.8 billion entries done.
Magnificent: Is that something that could be implemented inside
> Steven McKinney, Ph.D.
> Molecular Oncology and Breast Cancer Program
> British Columbia Cancer Research Centre
>> -----Original Message-----
>> From: R-devel [mailto:r-devel-bounces at r-project.org] On Behalf Of Tousey,
>> Sent: April-18-18 2:08 PM
>> To: R-core at r-project.org; simon.urbanek at R-project.org; R-devel at r-
>> Subject: [Rd] R Bug: write.table for matrix of more than 2, 147, 483, 648
>> I want to report a bug in R that is limiting my capabilities to export a
>> matrix with write.csv or write.table with over 2,147,483,648 elements (C's
>> int limit). I found this bug already reported about before: https://bugs.r-
>> project.org/bugzilla/show_bug.cgi?id=17182. However, there appears to be no
>> solution or fixes in upcoming R version releases.
>> The error message is coming from the writetable part of the utils package
>> in the io.c source code(https://svn.r-
>> /* quick integrity check */
>> if(XLENGTH(x) != (R_len_t)nr * nc)
>> error(_("corrupt matrix -- dims not not match
>> The issue is that nr*nc is an integer and the size of my matrix, 2.8
>> billion elements, exceeds C's limit, so the check forces the code to fail.
>> My version:
>>  "x86_64-w64-mingw32"
>>  "x86_64"
>>  "mingw32"
>>  "x86_64, mingw32"
>>  ""
>>  "3"
>>  "4.3"
>>  "2017"
>>  "11"
>>  "30"
>> $`svn rev`
>>  "73796"
>>  "R"
>>  "R version 3.4.3 (2017-11-30)"
>>  "Kite-Eating Tree"
>> Thank you,
>> Colton Tousey
>> Research Associate II
>> P: 816.585.0300 E: colton.tousey at kc.frb.org
>> FEDERAL RESERVE BANK OF KANSAS CITY
>> 1 Memorial Drive * Kansas City, Missouri 64198 *
>> [[alternative HTML version deleted]]
>> R-devel at r-project.org mailing list
> R-devel at r-project.org mailing list
More information about the R-devel