[Rd] Support for long arrays

[George Ostrouchov]

With that much RAM, you probably have many cores. Consider splitting 
your long array across several R sessions with distributed parallel 
processing so that no one segment is over the limit. You will likely 
need the parallel processing anyway to make your computation time 
tolerable. See the pbdDEMO package and the rest of pbdR packages (see 
r-pbd.org).

Cheers,
George

On 1/6/14 6:00 AM, r-devel-request at r-project.org wrote:
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Sun, 5 Jan 2014 20:18:06 +0000
> From: Boris Aronshtam <baronshtam at lightminersystems.com>
> To: "r-devel at r-project.org" <r-devel at r-project.org>
> Cc: Boris Aronshtam <baronshtam at lightminersystems.com>
> Subject: [Rd] Support for long arrays
> Message-ID:
> 	<9D80368C6A00834189C1982FB64EFA3007EE9002 at ORD2MBX07C.mex05.mlsrvr.com>
> Content-Type: text/plain
>
> My machine has hundreds of GB of RAM.  Is there a way to create an array having more than 2 Billion (2^31) rows?
>
> I am trying to create a large array using:
>
> d = c(1e+10,2)
> a=array(0,d)
>
> This results in the error:
>
> Error in array(0, d) : negative length vectors are not allowed
> In addition: Warning message:
> In array(0, d) : NAs introduced by coercion
>
> Note that I can create a vector of the same large size:
>
> v=c(1:2e+10)
>
>
>
> 	[[alternative HTML version deleted]]
>
>
>
> ------------------------------
>
> Message: 2
> Date: Sun, 05 Jan 2014 15:41:18 -0500
> From: Duncan Murdoch <murdoch.duncan at gmail.com>
> To: Boris Aronshtam <baronshtam at lightminersystems.com>,
> 	"r-devel at r-project.org" <r-devel at r-project.org>
> Subject: Re: [Rd] Support for long arrays
> Message-ID: <52C9C36E.6030903 at gmail.com>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> On 14-01-05 3:18 PM, Boris Aronshtam wrote:
>> My machine has hundreds of GB of RAM.  Is there a way to create an array having more than 2 Billion (2^31) rows?
> Not currently.  Individual dimensions are limited to signed 32 bit
> values.  The overall count of elements is limited to 2^52 or so.
>
> Duncan Murdoch
>
>> I am trying to create a large array using:
>>
>> d = c(1e+10,2)
>> a=array(0,d)
>>
>> This results in the error:
>>
>> Error in array(0, d) : negative length vectors are not allowed
>> In addition: Warning message:
>> In array(0, d) : NAs introduced by coercion
>>
>> Note that I can create a vector of the same large size:
>>
>> v=c(1:2e+10)
>>
>>
>>
>> 	[[alternative HTML version deleted]]
>>
>> ______________________________________________
>> R-devel at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>
>
> ------------------------------
>