[R] How to save very large matrix?
Rui Barradas
ruipbarradas at sapo.pt
Tue Oct 29 21:42:47 CET 2013
Hello,
You can use the argument to write.csv or write.table append = TRUE to
write the matrix in chunks. Something like the following.
bigwrite <- function(x, file, rows = 1000L, ...){
passes <- NROW(x) %/% rows
remaining <- NROW(x) %% rows
k <- 1L
write.table(x[k:rows, ], file, row.names = FALSE, ...)
k <- k + rows
for(i in seq_len(passes)[-1]){
write.table(x[k:(rows*i), ], file, append = TRUE, row.names = FALSE,
col.names = FALSE, ...)
k <- k + rows
}
if(remaining > 0)
write.table(x[k:NROW(x), ], file, append = TRUE, row.names = FALSE,
col.names = FALSE, ...)
}
f <- "temp"
m <- matrix(0, 50012, 10)
bigwrite(m, f, sep = ",") # Use 'sep' to get a csv file
Hope this helps,
Rui Barradas
Em 29-10-2013 19:27, Petar Milin escreveu:
> Hello!
> I have a very large matrix of results: 50000x100000. I saved it as RDS, but I would also need to save it as txt or csv. Is there a way to do it? Now, with write.table I am receiving an error:
> Error in .External2(C_writetable, x, file, nrow(x), p, rnames, sep, eol, :
> long vectors not supported yet: io.c:1116
>
> Please, help! Many thanks!
>
> PM
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
More information about the R-help
mailing list