[R] Large Dataset

[Edwin Sendjaja]

Hi Simons,

Is SAS more powerfull than R? 

Well, I think I cannot afford to buy SAS.

actually, my computer isn't  really slow. I think 4GB RAM is big enough for 
personal PC.  I am just wondering, why R running so slow with these specs to 
handling 3 GB data set. What if the data set were 1 TB?mmm..


Edwin

> Hi,
>
> I am not very knowledgeable about this kind of stuff but my guess is that
> if you have a fairly slow computer and massive data sets there isnt alot
> you can do except get a better computer, buy more RAM or use something like
> SAS instead?
>
> Hopefully someone else will chip in Edwin, best of luck.
>
> Simon.
>
>
> ----- Original Message -----
> From: "Edwin Sendjaja" <edwin7 at web.de>
> To: "Simon Pickett" <simon.pickett at bto.org>
> Cc: <r-help at r-project.org>
> Sent: Tuesday, January 06, 2009 2:53 PM
> Subject: Re: [R] Large Dataset
>
> > Hi Simon,
> >
> > My RAM is only 3.2 GB (actually it should be 4 GB, but my Motherboard
> > doesnt
> > support it.
> >
> > R use almost of all my RAM and half of my swap. I think memory.limit will
> > not
> > solve my problem.  It seems that I need  RAM.
> >
> > Unfortunately, I can't buy more RAM.
> >
> > Why R is slow reading big data set?
> >
> >
> > Edwin
> >
> >> Only a couple of weeks ago I had to deal with this.
> >>
> >> adjust the memory limit as follows, although you might not want 4000,
> >> that
> >> is quite high....
> >>
> >> memory.limit(size = 4000)
> >>
> >> Simon.
> >>
> >> ----- Original Message -----
> >> From: "Edwin Sendjaja" <edwin7 at web.de>
> >> To: "Simon Pickett" <simon.pickett at bto.org>
> >> Cc: <r-help at r-project.org>
> >> Sent: Tuesday, January 06, 2009 12:24 PM
> >> Subject: Re: [R] Large Dataset
> >>
> >> > Hi Simon,
> >> >
> >> > Thank for your reply.
> >> > I have read ?Memory but I dont understand how to use. I am not sure if
> >> > that
> >> > can solve my problem. Can you tell me more detail?
> >> >
> >> > Thanks,
> >> >
> >> > Edwin
> >> >
> >> >> type
> >> >>
> >> >> ?memory
> >> >>
> >> >> into R and that will explain what to do...
> >> >>
> >> >> S
> >> >> ----- Original Message -----
> >> >> From: "Edwin Sendjaja" <edwin7 at web.de>
> >> >> To: <r-help at r-project.org>
> >> >> Sent: Tuesday, January 06, 2009 11:41 AM
> >> >> Subject: [R] Large Dataset
> >> >>
> >> >> > Hi alI,
> >> >> >
> >> >> > I  have a 3.1 GB Dataset ( with  11 coloumns and lots data in int
> >> >> > and
> >> >> > string).
> >> >> > If I use read.table; it takes very long. It seems that my RAM is
> >> >> > not big
> >> >> > enough (overload) I have 3.2 RAM and  7GB SWAP, 64 Bit Ubuntu.
> >> >> >
> >> >> > Is there a best sultion to read a large data R? I have seen, that
> >> >> > people
> >> >> > suggest to use bigmemory package, ff. But it seems very
> >> >> > complicated. I dont
> >> >> > know how to start with that packages.
> >> >> >
> >> >> > i have tried to use bigmemory. But I got some kind of errors.  Then
> >> >> > I
> >> >> > gave up.
> >> >> >
> >> >> >
> >> >> > can someone give me an simple example how ot use ff or bigmemory?or
> >> >> > maybe
> >> >> > re
> >> >> > better sollution?
> >> >> >
> >> >> >
> >> >> >
> >> >> > Thank you in advance,
> >> >> >
> >> >> >
> >> >> > Edwin
> >> >> >
> >> >> > ______________________________________________
> >> >> > R-help at r-project.org mailing list
> >> >> > https://stat.ethz.ch/mailman/listinfo/r-help
> >> >> > PLEASE do read the posting guide
> >> >> > http://www.R-project.org/posting-guide.html
> >> >> > and provide commented, minimal, self-contained, reproducible code.