[R] How to make this for() loop memory efficient?

[Ray Brownrigg]

Steve:

I don't understand why you couldn't get the original code working.  You just have to 
notice that one comment overflows its line.

However I couldn't get your code to match the output of the original - almost, but not 
quite!

Ray

On Wed, 11 Jan 2012, Steve Lianoglou wrote:
> I'm having a really difficult time understanding what you're trying to
> get -- copy and pasting your code is failing to run, and your question
> isn't clear, ie:
> 
> "For each phone call that BEGINS with the module which is denoted by 81
> (i.e. of the form 81X,XXX), what is the expected number of modules in these
> calls?"
> 
> How does one calculate the expected number of "modules" in this
> module? What does that even mean?
> 
> Anyway, here's some using your `data` data.frame that calculates the
> number of unique calls and other statistics on the "call id" within
> each module prefix. I'm using both data.table and plyr ... there are
> no for loops.
> 
> You will want to do `whatever it is you really want to do` inside the
> "blocks" below.
> 
> ## R code
> data <- transform(data, module.prefix=substring(modules, 1, 2))
> 
> ## take a look at `data` now
> 
> ## calulate "stuff" inside each module.prefix using data.table
> xx <- data.table(data, key="module.prefix")
> 
> ans <- xx[, {
>   ## the columns of the particular subset of your data.table
>   ## are "injected" into the scope for this expression block
>   ## which is where the `calls` variable below comes from
>   tabled <- table(as.character(calls))
>   list(unique.calls=length(tabled), min=min(tabled),
> median=as.numeric(median(tabled)), max=max(tabled))
>   ## you will want to return your own list of "stuff"
> }, by='module.prefix']
> 
> 
> ## with plyr
> library(plyr)
> ans <- ddply(data, "module.prefix", function(x) {
>   ## `x` is a data.frame that all share the same module.prefix
>   ## do whatever you want with it here
>   tabled <- table(as.character(x$calls))
>   c(unique.calls=length(tabled), min=min(tabled),
> median=median(tabled), max=max(tabled))
> })
> 
> You'll have to read up on the particulars of data.table and plyr. Both
> are really powerful packages ... you should get familiar with at least
> one.
> 
> plyr is a bit more flexible in some ways.
> 
> data.table is a bit more strict (cf. the need for
> `as.numeric(median(tabled))`), but also tends to be (much) faster when
> working over large datasets
> 
> HTH,
> -steve