[R] Reading very large text files into R
jdnewm|| @end|ng |rom dcn@d@v|@@c@@u@
Thu Sep 29 17:31:36 CEST 2022
"Confusion" is the size of the file. Try specifying the colClasses argument to nail down the number and type of the columns.
On September 29, 2022 8:16:34 AM PDT, Bert Gunter <bgunter.4567 using gmail.com> wrote:
>I had no trouble reading your text snippet with
>"... your text... ")
>There were 15 columns. The last column was all empty except for the row
>containing the "B".
>So there seems to be some confusion here.
>On Thu, Sep 29, 2022 at 6:54 AM Nick Wray <nickmwray using gmail.com> wrote:
>> Hello I may be offending the R purists with this question but it is
>> linked to R, as will become clear. I have very large data sets from the UK
>> Met Office in notepad form. Unfortunately, I can’t read them directly
>> into R because, for some reason, although most lines in the text doc
>> consist of 15 elements, every so often there is a sixteenth one and R
>> doesn’t like this and gives me an error message because it has assumed that
>> every line has 15 elements and doesn’t like finding one with more. I have
>> tried playing around with the text document, inserting an extra element
>> into the top line etc, but to no avail.
>> Also unfortunately you need access permission from the Met Office to get
>> the files in question so this link probably won’t work:
>> So what I have done is simply to copy and paste the text docs into excel
>> csv and then read them in, which is time-consuming but works. However the
>> later datasets are over the excel limit of 1048576 lines. I can paste in
>> the first 1048576 lines but then trying to isolate the remainder of the
>> text doc to paste it into a second csv doc is proving v difficult – the
>> only way I have found is to scroll down by hand and that’s taking ages. I
>> cannot find another way of editing the notepad text doc to get rid of the
>> part which I have already copied and pasted.
>> Can anyone help with a)ideally being able to simply read the text tables
>> into R or b)suggest a way of editing out the bits of the text file I have
>> already pasted in without laborious scrolling?
>> Thanks Nick Wray
>> [[alternative HTML version deleted]]
>> R-help using r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> PLEASE do read the posting guide
>> and provide commented, minimal, self-contained, reproducible code.
> [[alternative HTML version deleted]]
>R-help using r-project.org mailing list -- To UNSUBSCRIBE and more, see
>PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>and provide commented, minimal, self-contained, reproducible code.
Sent from my phone. Please excuse my brevity.
More information about the R-help