[R] R Linux Memory Limit error for calculating dissimilarity matrix with Gower distance
@erenbkt@ @ending from gm@il@com
Wed Oct 24 16:15:55 CEST 2018
I am trying to cluster data file with 50K rows and 10 columns with R. Since
data contains mixed data types (nominal, continuous etc.) I am using gower
distance to create a dissimilarity matrix. But while I was creating the
matrix, I got Error: cannot allocate vector of size 4.1 Gb error. I am
using 64 bit 16 GB RAM.
1-My question is how to use all my memory for R (memory.limit does not work
2-Can I chunk dissimilarty matrix while I am creting it with ;
gower_dist <- daisy(i_cluster_ready,metric = "gower")
3-Can daisy function directly write to ROM to avoid error of RAM usace?
I have tried ff and bigmemory packages. Bigmemory does not work with
mixed data. and I could not figure ff function out.
Thanks for your help
[[alternative HTML version deleted]]
More information about the R-help