[R] How to create a high-dimensional matrix

[R. Michael Weylandt]

On Tue, Oct 16, 2012 at 10:56 PM, lrl <liurl1221 at gmail.com> wrote:
> Should it be .Machine$integer.max- 429497 ^2?
>

6 of one; half dozen of the other. Your matrix is still absolutely
massive: the development version of R does allow for lager matrices so
I think you can store a 429497 by 429497 in memory then, but it won't
be hugely performant.

> The matrix is not a sparse matrix, but a symmetric matrix.
>
> It's a process of feature selection. I am choosing the most important
> variables from 439497 variables. Now I am considering divide the whole
> dataset into several part and process a small part first.
>
>

The use of a training set is generally good practice, but I'm terribly
skeptical of any automated feature selection from a pool of ~400k
selectors. To wit,

library(fortunes)
fortune("fancy random")

Cheers,
Michael