[Rd] Speed up code, profiling, optimization, lapply vs. loops

[Thorn Thaler]

High everybody,

currently I'm writinig a package that, for a given family of variance 
functions depending on a parameter theta, say, computes the extended 
quasi likelihood (eql) function for different values of theta.

The computation involves a couple of calls of the 'glm' routine. What 
I'm doing now is to call 'lapply' for a list of theta values and a 
function, that constructs a family object for the particular choice of 
theta, computes the glm and uses the results to get the eql. Not 
surprisingly the function is not very fast. Depending on the size of the 
parameter space under consideration it takes a couple of minutes until 
the function finishes. Testing ~1000 Parameters takes about 5 minutes on 
my machine.

I know that loops in R are slow more often than not. Thus, I thought 
using 'lapply' is a better way. But anyways, it is just another way of a 
loop. Besides, it involves some overhead for the function call and hence 
i'm not sure wheter using 'lapply' is really the better choice.

What I like to know is to figure out, where the bottleneck lies. 
Vectorization would help, but since I don't think that there is 
vectorized 'glm' function, which is able to handle a vector of family 
objects. I'm not aware if there is any choice aside from using a loop.

So my questions:
- how can I figure out where the bottleneck lies?
- is 'lapply' always superior to a loop in terms of execution time?
- are there any 'evil' commands that should be avoided in a loop, for 
they slow down the computation?
- are there any good books, tutorials about how to profile R code 
efficiently?

TIA 4 ur help,

Thorn