[R] Reading big files in chunks-ff package

Mav mastorvarela at gmail.com
Sun Mar 25 19:52:56 CEST 2012

Thank you Jan

My problem is the following:
For instance, I have 2 files with different number of rows (15 million and 8
million of rows each).
I would like to read the first one in chunks of 5 million each. However
between the first and second chunk, I would like to analyze those first 5
million of rows, write the analysis in a new csv and then proceed to read
and analyze the second chunk and so on until the third chunk. With the
second file, I would like to do the same...read the first chunk, analyze it
and continue to read the second and analyze it.

Basically my problem is that I manage to read the files....but with so many
rows...I cannot do any analyses (even filtering the rows) because of the RAM

Sorry if is still not clear.

Thank you

View this message in context: http://r.789695.n4.nabble.com/Reading-big-files-in-chunks-ff-package-tp4502070p4503642.html
Sent from the R help mailing list archive at Nabble.com.

More information about the R-help mailing list