[R] R: "in-place" appending to a matrix.

[culpritNr1]

Hello Gabor,

First of all, thanks for your reply.

Indeed, your strategy solves the problem of speed=). My question was,
however, more of an R design question.

Your suggestion of pre-allocating a big and continuous portion of memory is
probably the first manual work around to try. And it will work.

Now, modern high level languages like the continually improving R, Python
and also Matlab and Perl come with built-in goodies that run at C/C++ speed.
Those built-in functions are high level versions of functionalities that we
would code "manually" in low level languages.

Being relatively new to R, my expectations are that in a high level language
like this, I do not need to manually program a simple operation such as
in-place appending. Manipulation of rectangular data should be extremely
efficient. (I use it to make computation of DNA properties in organisms with
huge genomes like human and mouse.)

So, do you know whether or not such functions exists or is planned to be
incorporated in the near future? An optional module perhaps?

Again, thank you,

culpritNr1





Gabor Grothendieck wrote:
> 
> Create an empty matrix first and then fill it in.   That
> will avoid the overhead in repeatedly expanding it.  If
> you don't know how many rows then make it 1000 rows
> and remove the unused ones once finished.
> 
> On Wed, Oct 15, 2008 at 4:05 PM, culpritNr1 <ig2ar-saf1 at yahoo.co.uk>
> wrote:
>>
>> Hello fellow R sufferers,
>>
>> Is there a way to perform an appending operation in place?
>>
>> Currently, the way my pseudo-code goes is like this
>>
>> for (i in 1:1000) {
>>    if (some condition) {
>>        newRow <- myFunction(myArguments)
>>        X <- rbind(X, newRow)     #  <- this is the bottleneck!!
>>    }
>> }
>>
>> As you can see, it works but as the matrix X gets the size of a few
>> million
>> rows, the code runs very slow.
>>
>> I am looking for something like the natively "in place" appending python
>> function called "append()" or the perl function "push". "In-place"
>> operations would allow me to do (in pseudocode)
>>
>> for (i in 1:1000) {
>>    if (some condition) {
>>        newRow <- myFunction(myArguments)
>>        append(X, newRow)
>>    }
>> }
>>
>> You see? I do not have to call and re-assign the giant X matrix every
>> loop
>> cycle.
>>
>> Any help?
>>
>> Thank you,
>>
>> Your culpritNr1
>>
>>
>>
>> --
>> View this message in context:
>> http://www.nabble.com/R%3A-%22in-place%22-appending-to-a-matrix.-tp20001258p20001258.html
>> Sent from the R help mailing list archive at Nabble.com.
>>
>> ______________________________________________
>> R-help at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
> 
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
> 
> 

-- 
View this message in context: http://www.nabble.com/R%3A-%22in-place%22-appending-to-a-matrix.-tp20001258p20014954.html
Sent from the R help mailing list archive at Nabble.com.