[R] loop is going to take 26 hours - needs to be quicker!

Jenny Barnes jmb at mssl.ucl.ac.uk
Thu Dec 14 16:48:21 CET 2006


Dear Patrick,

Thank you for the link - I'd advise anyone who's started using R to have a look 
at these as well - any help is always appreciated. I've downloaded the S Poetry 
and will hit the books tomorrow and get reading it!

Jenny


>
>S Poetry may be of use to you -- especially the chapter
>on arrays which discusses 3 dimensional arrays in particular.
>
>Patrick Burns
>patrick at burns-stat.com
>+44 (0)20 8525 0696
>http://www.burns-stat.com
>(home of S Poetry and "A Guide for the Unwilling S User")
>
>Jenny Barnes wrote:
>
>>Dear R-help,
>>
>>I forgot to mention that I need the array in that format because I am going to 
>>do the same thing for another dataset of precipitation (ncep.data2) so they 
are 
>>both arrays of dimensions [144,72,46] so that I can correlate them globally 
and 
>>plot a visual image of the global correlations between the 2 datasets.... One 
of 
>>the datasets has a land mask applied to it already so it should be clear to 
see 
>>the land and pick ot the locations (i.e.over Europe) where there is strongest 
>>and weakest correlation.....that is the ultimate goal.
>>
>>Following Rainer's response I should also point out that the columns in 
>>gpcc.data2 (with dimensions dim(gpcc.data2) = [476928,5]) are:
>>
>>[,1]="Year", [,2]="month" (which is just january so always 1), 
[,3]="latitude", 
>>[,4]="longitude" and [,5]="data". All I want in the gpcc.array is the data not 
>>the longitudes and latitude values...hope that helps clear it up a bit!
>>
>>I look forward to hearing any more ideas, thanks again for your time in 
reading 
>>this,
>>
>>Jenny Barnes
>>
>>  
>>
>>>Jenny Barnes wrote:
>>>    
>>>
>>>>Dear R-help,
>>>>
>>>>I have a loop, which is set to take about 26 hours to run at the rate it's 
>>>>      
>>>>
>>going 
>>  
>>
>>>>- this is ridiculous and I really need your help to find a more efficient 
way 
>>>>      
>>>>
>>of 
>>  
>>
>>>>loading up my array gpcc.array:
>>>>
>>>>#My data is stored in a table format with all the data in one long column 
>>>>#running though every longitute, for every latitude, for every year. The 
>>>>#original data is sotred as gpcc.data2 where dim(gpcc.data2) = [476928,5] 
>>>>      
>>>>
>>where 
>>  
>>
>>>>#the 5th column is the data:
>>>>
>>>>#make the array in the format I need [longitude,latitude,years]
>>>>
>>>>gpcc.array <- array(NA, c(144,72,46)) 
>>>>
>>>>n=0
>>>>for(k in 1:46){
>>>>for(j in 1:72){
>>>>for(i in 1:144){
>>>>n <- n+1
>>>>gpcc.array[i,j,k] <- gpcc.data2[n,5]
>>>>print(j)
>>>>}
>>>>}
>>>>}
>>>>      
>>>>
>>>I don't know if it is faster - but adding three columns to qpcc.data, 
>>>one for longitude, one for lattitude and one for year (using rep() as 
>>>they are in sequence) and the using reshape() might be faster?
>>>
>>>
>>>    
>>>
>>>>So it runs through all the longs for every lat for every year - which is the 
>>>>order the data is running down the column in gpcc.data2 so n increses by 1 
>>>>      
>>>>
>>each 
>>  
>>
>>>>time and each data point is pulled off....
>>>>
>>>>It needs to be a lot quicker, I'd appreciate any ideas!
>>>>
>>>>Many thanks for taking time to read this,
>>>>
>>>>Jenny Barnes
>>>>
>>>>~~~~~~~~~~~~~~~~~~
>>>>Jennifer Barnes
>>>>PhD student - long range drought prediction
>>>>Climate Extremes
>>>>Department of Space and Climate Physics
>>>>University College London
>>>>Holmbury St Mary, Dorking
>>>>Surrey
>>>>RH5 6NT
>>>>01483 204149
>>>>07916 139187
>>>>Web: http://climate.mssl.ucl.ac.uk
>>>>
>>>>______________________________________________
>>>>R-help at stat.math.ethz.ch mailing list
>>>>https://stat.ethz.ch/mailman/listinfo/r-help
>>>>PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>>>>and provide commented, minimal, self-contained, reproducible code.
>>>>      
>>>>
>>>-- 
>>>Rainer M. Krug, Dipl. Phys. (Germany), MSc Conservation
>>>Biology (UCT)
>>>
>>>Department of Conservation Ecology and Entomology
>>>University of Stellenbosch
>>>Matieland 7602
>>>South Africa
>>>
>>>Tel:		+27 - (0)72 808 2975 (w)
>>>Fax:		+27 - (0)86 516 2782
>>>Fax:		+27 - (0)21 808 3304 (w)
>>>Cell:		+27 - (0)83 9479 042
>>>
>>>email:	RKrug at sun.ac.za
>>>      	Rainer at krugs.de
>>>    
>>>
>>
>>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>Jennifer Barnes
>>PhD student - long range drought prediction
>>Climate Extremes
>>Department of Space and Climate Physics
>>University College London
>>Holmbury St Mary, Dorking
>>Surrey
>>RH5 6NT
>>01483 204149
>>07916 139187
>>Web: http://climate.mssl.ucl.ac.uk
>>
>>______________________________________________
>>R-help at stat.math.ethz.ch mailing list
>>https://stat.ethz.ch/mailman/listinfo/r-help
>>PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>>and provide commented, minimal, self-contained, reproducible code.
>>
>>
>>  
>>

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Jennifer Barnes
PhD student - long range drought prediction
Climate Extremes
Department of Space and Climate Physics
University College London
Holmbury St Mary, Dorking
Surrey
RH5 6NT
01483 204149
07916 139187
Web: http://climate.mssl.ucl.ac.uk



More information about the R-help mailing list