[R] Working with very large datasets and generating an executable file

Bert Gunter bgunter@4567 @end|ng |rom gm@||@com
Thu May 7 01:30:03 CEST 2020


To supplement Jeff's comments:

Big Data:
https://CRAN.R-project.org/view=HighPerformanceComputing

To deploy models:
https://cran.r-project.org/web/views/ModelDeployment.html

Opinion: Executables are a security risk. I wouldn't touch one unless
from a trusted source. I think I understand what you want to do, but I
would second Jeff's comment about using R packages. Don't bother to
disagree with me -- just dismiss if you do-- as this is wandering O/T
anyway.


Bert Gunter

"The trouble with having an open mind is that people keep coming along
and sticking things into it."
-- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )

On Wed, May 6, 2020 at 4:23 PM Jeff Newmiller <jdnewmil using dcn.davis.ca.us> wrote:
>
> Large data... yes, though how this can be done may vary. I have used machines with 128G of RAM before with no special big data packages.
>
> Making an executable... theoretically, yes, though there are some significant technical (and possibly legal) challenges that will most likely make you question whether it was worth it if you try, particularly if your intent is to obscure your code from the recipient. I (as a random user and programmer on the Internet) would strongly discourage such efforts... it will almost certainly be more practical to deliver code in script/package form.
>
> On May 6, 2020 2:20:47 PM PDT, Paul Bernal <paulbernal07 using gmail.com> wrote:
> >Dear R friends,
> >
> >Hope you are doing well. I have two questions, the first one is, can I
> >work
> >with very large datasets in R? That is, say I need to test several
> >machine
> >learning algorithms, like (random forest, multiple linear regression,
> >etc.)
> >on datasets having between 50 to 100 columns and 20 million
> >observations,
> >is there any way that R can handle data that large?
> >
> >The second question is, is there a way I can develop an R model and
> >turn it
> >into an executable program that can work on any OS?
> >
> >Any help and/or guidance will be greatly appreciated,
> >
> >Best regards,
> >
> >Paul
> >
> >       [[alternative HTML version deleted]]
> >
> >______________________________________________
> >R-help using r-project.org mailing list -- To UNSUBSCRIBE and more, see
> >https://stat.ethz.ch/mailman/listinfo/r-help
> >PLEASE do read the posting guide
> >http://www.R-project.org/posting-guide.html
> >and provide commented, minimal, self-contained, reproducible code.
>
> --
> Sent from my phone. Please excuse my brevity.
>
> ______________________________________________
> R-help using r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.



More information about the R-help mailing list