[R] problem with nested loop for regression
rcoder
mpdotbook at gmail.com
Tue Aug 5 20:41:52 CEST 2008
Hi Jim,
Thanks very much for your reply and suggestions. Although the statement
Preg[,k]<-coef(lm(tt~sel_col)) is applying lm() between the same vectors,
the contents of one of the vectors (sel_col) should be changing with each
cycle through the outer loop. My mistake is that I didn't specify
information on the rows the data should go into. I have modified the
statement to resemble something like this:
pEUUreg[j:(j+1),k]<-coef(lm(tt~sel_col)), where j+1 ensures a jump of 2
rows, as slope and intercept are arranged vertically in the o/p matrix. I
tried it on a section of one column (to save time) and it seemed to o/p some
(varying) data to Preg.
I now want to just extract the slope data into a new matrix, so I have added
a statement for this. I was just wondering if there is a neater way to tell
the lm() function to o/p slope data only? Perhaps one way would be to use
[j,k] vs [j:(j+1),k) in the regression statement - forcing successive slope
data to overwrite cell locations containing intercept data.
Also, the full code takes an awfully long time to run. Would anyone be able
to suggest a way for mee to speed it up - perhaps in the regression
algorithm?
Thanks,
rcoder
jholtman wrote:
>
> Actually now that I read it closer, I see what your problem is. what
> did you think the statement:
>
> Preg[,k]<-coef(lm(tt~sel_col))
>
> was going to do? Preg is a 200x100 matrix and you are only storing
> two values (the coefficients) so they will be repeated 100 times in
> the column. So there is nothing wrong with your nested "for" loops;
> it is in the algorithm that you are using. You might want to use the
> browser and you would see something like this:
>
> Browse[1]> str(sel_col)
> num [1:5] -0.115 -2.666 -0.811 1.440 -0.879
> Browse[1]> str(tt)
> Time-Series [1:5] from 2 to 10: 2 4 6 8 10
> Browse[1]> n
> debug: if (!all(is.na(sel_col))) {
> Preg[, k] <- coef(lm(tt ~ sel_col))
> }
> Browse[1]> n
> debug: Preg[, k] <- coef(lm(tt ~ sel_col))
> Browse[1]> k
> [1] 1
> Browse[1]> n
> debug: k
> Browse[1]> n
> debug: sel_col <- PmatWt[, k]
> Browse[1]> k
> [1] 2
> Browse[1]> str(sel_col)
> num [1:5] -0.115 -2.666 -0.811 1.440 -0.879
> Browse[1]> str(Preg)
> num [1:200, 1:100] 6.356 0.587 6.356 0.587 6.356 ...
> Browse[1]> str(PmatWt)
> num [1:5, 1:100] -0.115 -2.666 -0.811 1.440 -0.879 ...
> Browse[1]> coef(lm(tt~sel_col))
> (Intercept) sel_col
> 6.355760 0.586699
> Browse[1]> str(Preg)
> num [1:200, 1:100] 6.356 0.587 6.356 0.587 6.356 ...
> Browse[1]>
>
> This would help you to understand what is happening.
>
> On Mon, Aug 4, 2008 at 8:35 PM, jim holtman <jholtman at gmail.com> wrote:
>> Exactly what problem are you having? There is nothing wrong with
>> nested for loops, so what is leading you to believe you have a
>> problem? I ran your program and it seems to terminate. Most of the
>> time seems to have been spent in the following statement:
>>
>> if(!all(is.na(sel_col)))
>> {Preg[,k]<-coef(lm(tt~sel_col))}
>>
>> Given that you appear to be executing this statement about 9000 times,
>> it is not surprising. So can you provide more details?
>>
>> 0 51.2 root
>> 1. 44.5 coef
>> 2. . 44.1 lm
>> 3. . . 15.7 eval
>> 4. . . . 15.6 eval
>> 5. . . . | 15.6 model.frame
>> 6. . . . | . 15.3 model.frame.default
>> 7. . . . | . . 6.0 sapply
>> 8. . . . | . . . 3.8 lapply
>> 9. . . . | . . . . 3.0 FUN
>> 10. . . . | . . . . | 1.1 %in%
>> 11. . . . | . . . . | . 1.0 match
>> 12. . . . | . . . . | . . 0.8 is.factor
>> 13. . . . | . . . . | . . . 0.7 inherits
>> 10. . . . | . . . . | 1.0 .deparseOpts
>> 11. . . . | . . . . | . 0.3 pmatch
>> 9. . . . | . . . . 0.5 as.list
>> 8. . . . | . . . 1.7 unique
>> 9. . . . | . . . . 0.7 unique.default
>> 9. . . . | . . . . 0.6 unlist
>> 10. . . . | . . . . | 0.4 lapply
>> 8. . . . | . . . 0.3 unlist
>> 7. . . . | . . 4.0 na.omit
>> 8. . . . | . . . 3.8 na.omit.data.frame
>> 9. . . . | . . . . 3.0 [
>> 10. . . . | . . . . | 2.9 [.data.frame
>> 11. . . . | . . . . | . 0.6 duplicated
>> 12. . . . | . . . . | . . 0.3 duplicated.default
>> 11. . . . | . . . . | . 0.3 [
>> 12. . . . | . . . . | . . 0.2 [.ts
>> 11. . . . | . . . . | . 0.3 [[
>>
>> On Mon, Aug 4, 2008 at 12:58 PM, rcoder <mpdotbook at gmail.com> wrote:
>>>
>>> Hi,
>>>
>>> I guess my question is really more about the nested for loop construct
>>> and
>>> whether it is doing what I intend it to do in my code in the previous
>>> post.
>>> I would be grateful if anyone who has used nested loops could let me
>>> know if
>>> I am doing something wrong.
>>>
>>> Thanks,
>>>
>>> rcoder
>>>
>>>
>>>
>>> rcoder wrote:
>>>>
>>>> Hi everyone,
>>>>
>>>> I'm experiencing difficulty getting the results I want when I use a
>>>> nested
>>>> for loop. I have a data set to which I perform some calculations, and
>>>> then
>>>> try to apply a regression over a rolling window. The code runs, but the
>>>> regression results I am getting (intercept and slope) are simply the
>>>> same,
>>>> repeated again and again in the results matrix. The regression does not
>>>> seem to be obeying the instructions of the nested loop, which intend it
>>>> to
>>>> calculate regression coefficients over a data one row at a time. I have
>>>> been struggling with the code for many days now, testing various parts,
>>>> and I cannot seem to get the nested loop to work as I want it to. I
>>>> would
>>>> be very grateful for any suggestions. Below is a brief version of my
>>>> code:
>>>>
>>>> #Code start
>>>> library(zoo)
>>>> seed.set(1)
>>>> Pmat<-matrix(rnorm(1000), nrow=100, ncol=100)
>>>> maxcol<-ncol(Pmat)
>>>> maxrow<-nrow(Pmat)
>>>> startrow<-10
>>>> period<-10
>>>> wind<-2 #roll window
>>>> subdiv<-period/wind
>>>> rollstart<-11 #start roll at period+1
>>>> #converting Pmat into ts for rollapply() to work...
>>>> PmatTS<-ts(Pmat)
>>>> Preg<-matrix(NA,ncol=maxcol,nrow=2*maxrow)
>>>> PmatWt<-matrix(NA, nrow=subdiv,ncol=maxcol)
>>>> mult_col<-matrix(1:5, nrow=5, ncol=1)
>>>> #rolling calculations...
>>>> for (i in (rollstart):maxrow)
>>>> {
>>>> #extract the relevant timeframe...
>>>> dslice<-PmatTS[(i-period):i,]
>>>> dslicets<-ts(dslice)
>>>> #operating on sliced data...
>>>> Pmin<-rollapply(dslicets, wind, by=wind, min, na.rm=F)
>>>> Pmax<-rollapply(dslicets, wind, by=wind, max, na.rm=F)
>>>> Pmult<-Pmin*Pmax #calculating product
>>>> tt<-time(Pmult)
>>>> for (j in 1:5) #1st nested loop
>>>> {
>>>> PmatWt[j,]<-Pmult[j,]*mult_col[j,]
>>>> }
>>>> #rolling regression analysis...
>>>> for (k in 1:maxcol) #2nd nested loop
>>>> {
>>>> sel_col<-PmatWt[,k]
>>>> if(!all(is.na(sel_col))) {Preg[,k]<-coef(lm(tt~sel_col))}
>>>> }
>>>> }
>>>> #Code End
>>>>
>>>> Thanks,
>>>>
>>>> rcoder
>>>>
>>>
>>> --
>>> View this message in context:
>>> http://www.nabble.com/problem-with-nested-loop-for-regression-tp18792841p18815273.html
>>> Sent from the R help mailing list archive at Nabble.com.
>>>
>>> ______________________________________________
>>> R-help at r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>> PLEASE do read the posting guide
>>> http://www.R-project.org/posting-guide.html
>>> and provide commented, minimal, self-contained, reproducible code.
>>>
>>
>>
>>
>> --
>> Jim Holtman
>> Cincinnati, OH
>> +1 513 646 9390
>>
>> What is the problem that you are trying to solve?
>>
>
>
>
> --
> Jim Holtman
> Cincinnati, OH
> +1 513 646 9390
>
> What is the problem that you are trying to solve?
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
>
--
View this message in context: http://www.nabble.com/problem-with-nested-loop-for-regression-tp18792841p18837399.html
Sent from the R help mailing list archive at Nabble.com.
More information about the R-help
mailing list