[R] help usin scan on large matrix (caveats to what has been discussed before)

Martin Tomko martin.tomko at geo.uzh.ch
Thu Aug 12 16:52:07 CEST 2010


Hi Peter,
apologies, too fast copying and pasting.
So, here is the explanation:
f<-"C:/test/mytab.txt";
R<-readLines(con=f);

where mytab.txt is a table formatted as noted in previous post (space 
delimited, with header, rownames, containing integers).

Now, my understandign of scan was that I have to specify the FULL number 
of values in it (examples specify things like 200*2000 for a matrix 
etc). That's why I thought that I need to do cols*rows as well. Avoiding 
the first line with headers is simple, avoiding the first column is not 
- hence my questions.
Sorry, the corrected, matching parentheses are here - why did the 
previous execute is a wonder...
c<-scan(file=f,what=rep(c(list(NULL),rep(list(0L),cols-1)),rows-1), skip=1)
here, my reasoning was:

* c(list(NULL),rep(list(0L),cols-1)) specifies a template for any line 
(first elelement to be ignored => NULL, it is a string in the table 
specified, and then a repetition of integers - I am still not sure how 
you derived 0L, and what it means and where to find a doc for that.);
* the previous needs to be repeated rows-1 times, hence 
what=rep(c(list(NULL),rep(list(0L),cols-1)),rows-1)

I do nto understand the following:

  You need an unlist(c). And more than likely NOT byrow=TRUE. However, I think do.call(cbind,c) should do the trick more easily.

what will unlist(c) do; why should it not be bywrow=TRUE, and how would 
you go about integrating do.call(cbind,c) with matrix. Apologies to 
naive questions, I am a newbie, in principle.

Cheers
Martin




On 8/12/2010 4:29 PM, peter dalgaard wrote:
> On Aug 12, 2010, at 1:34 PM, Martin Tomko wrote:
>
>    
>> Hi Peter,
>> thank you for your reply. I still cannot get it to work.
>> I have modified your code as follows:
>> rows<-length(R)
>> cols<- max(unlist(lapply(R,function(x) length(unlist(gregexpr(" ",x,fixed=TRUE,useBytes=TRUE))))))
>>      
> Notice that the above is completely useless to the reader unless you tell us what R is (except for a statistical programming language ;-))
>
>    
>> c<-scan(file=f,what=rep(c(list(NULL),rep(list(0L),cols-1),rows-1)), skip=1)
>>      
> What's the outer rep() and rows-1 doing in there???! Notice that the parentheses don't match up as I think you think they do, so there's really only one argument to rep(), making it a no-op. The rows-1 is going inside the c, which might be causing the apparent extra column. And the number of rows should not affect 'what=' anyway. Now if you had done what I wrote...
>
>    
>> m<-matrix(c, nrow = rows-1, ncol=cols+1,byrow=TRUE);
>>      
> If you make a matrix from a list, odd things will happen. You need an unlist(c). And more than likely NOT byrow=TRUE. However, I think do.call(cbind,c) should do the trick more easily.
>
>    
>> the list c seems ok, with all the values I would expect. Still, length(c) gives me a value = cols+1, which I find odd (I would expect =cols).
>> I thine repeated it rows-1 times (to account for the header row). The values seem ok.
>> Anyway, I tried to construct the matrix, but when I print it, the values are odd:
>>      
>>> m[1:10,1:10]
>>>        
>>       [,1] [,2]       [,3]       [,4]       [,5]       [,6]       [,7]
>> [1,] NULL Integer,15 Integer,15 Integer,15 Integer,15 Integer,15 Integer,15
>> [2,] NULL Integer,15 Integer,15 Integer,15 Integer,15 Integer,15 Integer,15
>> [3,] NULL Integer,15 Integer,15 Integer,15 Integer,15 Integer,15 Integer,15
>> [4,] NULL Integer,15 Integer,15 Integer,15 Integer,15 Integer,15 Integer,15
>> [5,] NULL Integer,15 Integer,15 Integer,15 Integer,15 Integer,15 Integer,15
>> [6,] NULL Integer,15 Integer,15 Integer,15 Integer,15 Integer,15 Integer,15
>> [7,] NULL Integer,15 Integer,15 Integer,15 Integer,15 Integer,15 Integer,15
>> [8,] NULL Integer,15 Integer,15 Integer,15 Integer,15 Integer,15 Integer,15
>> [9,] NULL Integer,15 Integer,15 Integer,15 Integer,15 Integer,15 Integer,15
>> [10,] NULL Integer,15 Integer,15 Integer,15 Integer,15 Integer,15 Integer,15
>> ....
>>
>> Any idea where the values are gone?
>> Thanks
>> Martin
>>
>> Hence, I filled it into the matrix of dimensions
>>
>> On 8/12/2010 12:24 PM, peter dalgaard wrote:
>>      
>>> On Aug 12, 2010, at 11:30 AM, Martin Tomko wrote:
>>>
>>>
>>>        
>>>> c<-scan(file=f,what=list(c("",(rep(integer(0),cols)))), skip=1)
>>>> m<-matrix(c, nrow = rows, ncol=cols,byrow=TRUE);
>>>>
>>>> for some reason I end up with a character matrix, which I don't want. Is this the proper way to skip the first column (this is not documented anywhere - how does one skip the first column in scan???). is my way of specifying "integer(0)" correct?
>>>>
>>>>          
>>> No. Well, integer(0) is just superfluous where 0L would do, since scan only looks at the types not the contents, but more importantly, what= wants a list of as many elements as there are columns and you gave it
>>>
>>>
>>>        
>>>> list(c("",(rep(integer(0),5))))
>>>>
>>>>          
>>> [[1]]
>>> [1] ""
>>>
>>> I think what you actually meant was
>>>
>>> c(list(NULL),rep(list(0L),5))
>>>
>>>
>>>
>>>
>>>        
>>>> And finally - would any sparse matrix package be more appropriate, and can I use a sparse matrix for the image() function producing typical heat,aps? I have seen that some sparse matrix packages produce different looking outputs, which would not be appropriate.
>>>>
>>>> Thanks
>>>> Martin
>>>>
>>>> ______________________________________________
>>>> R-help at r-project.org mailing list
>>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>>>> and provide commented, minimal, self-contained, reproducible code.
>>>>
>>>>          
>>>
>>>        
>>
>> -- 
>> Martin Tomko
>> Postdoctoral Research Assistant
>>
>> Geographic Information Systems Division
>> Department of Geography
>> University of Zurich - Irchel
>> Winterthurerstr. 190
>> CH-8057 Zurich, Switzerland
>>
>> email: 	martin.tomko at geo.uzh.ch
>> site:	http://www.geo.uzh.ch/~mtomko
>> mob: 	+41-788 629 558
>> tel: 	+41-44-6355256
>> fax: 	+41-44-6356848
>>
>>      
>    


-- 
Martin Tomko
Postdoctoral Research Assistant

Geographic Information Systems Division
Department of Geography
University of Zurich - Irchel
Winterthurerstr. 190
CH-8057 Zurich, Switzerland

email: 	martin.tomko at geo.uzh.ch
site:	http://www.geo.uzh.ch/~mtomko
mob: 	+41-788 629 558
tel: 	+41-44-6355256
fax: 	+41-44-6356848



More information about the R-help mailing list