[R] ff package: reading selected columns from csv

[Jan van der Laan]

Looking at the source code for read.table.ffdf what seems to happen is  
that when reading the first block of data by read.table (standard 1000  
lines) the specified colClasses are used. In subsequent calls the  
types of the columns of the ffdf object are used as colClasses. In  
your case the ffdf object had only one column. This probably causes  
the error.

What you could try is to use the packages ffbase and LaF (untested):

library(ffbase)
library(LaF)

x.class <- c('character', 'numeric','character','character',
     'character', 'character', 'character')
laf <- laf_open_csv(file=csvfile, header=FALSE,
      skip=100, column_types=x.class)
yourdata <- laf_to_ffdf(laf, columns=2)

I specify column type 'character' as a type is needed. However, by  
using the column=2 argument only the second column is read.

It looks like you have a decent amount of memory, so you could also try

yourdata <- laf[,2]

to read the data in as a standard R vector.

HTH,

Jan




threshold <r.kozarski at gmail.com> schreef:

> Dear Jan, thank you for your answer.
> I am basically following the code Ive been using with read.table, where
> x.class <- c('NULL', 'numeric','NULL','NULL','NULL', 'NULL', 'NULL')
> has been working fine.
>
> Reading all columns works with me but take much longer than allowed time
> constrains.. (460 such sets+ time for processing). The number of columns
> remains 7 over the whole data set.
>
> Best, Robert
>
>
>
>
> --
> View this message in context:  
> http://r.789695.n4.nabble.com/ff-package-reading-selected-columns-from-csv-tp4637794p4637896.html
> Sent from the R help mailing list archive at Nabble.com.
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.