[R] How to create a high-dimensional matrix

R. Michael Weylandt michael.weylandt at gmail.com
Wed Oct 17 14:45:20 CEST 2012

On Tue, Oct 16, 2012 at 10:56 PM, lrl <liurl1221 at gmail.com> wrote:
> Should it be .Machine$integer.max- 429497 ^2?

6 of one; half dozen of the other. Your matrix is still absolutely
massive: the development version of R does allow for lager matrices so
I think you can store a 429497 by 429497 in memory then, but it won't
be hugely performant.

> The matrix is not a sparse matrix, but a symmetric matrix.
> It's a process of feature selection. I am choosing the most important
> variables from 439497 variables. Now I am considering divide the whole
> dataset into several part and process a small part first.

The use of a training set is generally good practice, but I'm terribly
skeptical of any automated feature selection from a pool of ~400k
selectors. To wit,

fortune("fancy random")


More information about the R-help mailing list