[R] read txt file - date - no space
Diego Avesani
d|ego@@ve@@n| @end|ng |rom gm@||@com
Wed Aug 1 14:29:53 CEST 2018
Dear Pikal, Deal all,
again really thank.
it seems not working.
Some specifications: My non data are -999, but I could change it.
My final results is:
1 1 -55.86242 -55.84764660 -277.4775
2 2 -55.47554 -94.58921682 -277.4845
3 3 -55.47095 -99.20239198 -277.4709
4 4 -55.46470 -55.45952932 -392.9071
5 5 -55.43335 -55.40171682 -388.4110
6 6 -55.40108 -55.37399691 -332.9068
7 7 -55.39201 -55.35156250 -332.8902
8 8 -110.87184 0.16136188 -281.8230
9 9 -110.95077 -55.63856096 -332.9564
10 10 -157.64430 -0.06602705 -315.3840
11 11 -105.06157 0.11507675 -315.4152
12 12 -70.08677 -52.54501096 -316.7247
So it is not correct.
For example for the first day in my csv I would have expected 0.167.
I am going to post what you have suggested:
for str(MyData)
'data.frame': 160008 obs. of 4 variables:
$ date: POSIXct, format: "1998-01-10 00:00:00" "1998-01-10 01:00:00"
"1998-01-10 02:00:00" "1998-01-10 03:00:00" ...
$ str1: num 0.6 0.2 0.6 0 0 0 0 0.2 0.6 0.2 ...
$ str2: num 0 0.2 0.2 0 0 0 0 0 0.2 0.4 ...
$ str3: num 0 0.2 0.4 0.6 0 0 0 0 0 0.4 ...
unfortunately, I am not able to post
dput(str[1:20,])
it gives me
Error in str[1:20, ] : object of type 'closure' is not subsettable
Thanks again,
I hope that what I posted could be enough in order to help me.
Diego
On 1 August 2018 at 13:58, PIKAL Petr <petr.pikal using precheza.cz> wrote:
> Hi
>
> I did not get through all answers you already got and you probably
> obtained similar advice as mine.
>
> # read data (if you have csv file just use read.csv)
> > test<-read.table("clipboard", header=T, sep=",")
>
> # control your object(s)
> > str(test)
> 'data.frame': 8 obs. of 4 variables:
> $ date: Factor w/ 8 levels "10/1/1998 0:00",..: 1 2 3 4 5 6 7 8
> $ str1: num 0.6 0.2 0.6 0 0 0 0 0.2
> $ str2: num 0 0.2 0.2 0 0 0 0 0
> $ str3: num 0 0.2 0.4 0.6 0 0 0 0
>
> #if it is OK change first column to real date by POSIXct
> test$date<-as.POSIXct(test$date, format="%d/%m/%Y %H:%M")
>
> #attach lubridate
> > library(lubridate)
>
> # aggregate your object(s) and use lubridate function
>
> > aggregate(test[,-1], list(day(test$date)), mean)
> Group.1 str1 str2 str3
> 1 10 0.2 0.05 0.15
>
> # or format function
>
> > aggregate(test[,-1], list(format(test$date, "%Y-%m-%d")), mean)
> Group.1 str1 str2 str3
> 1 1998-01-10 0.2 0.05 0.15
>
> If it does not work with your data you should post at least result of
>
> str(yourdata)
>
> or preferably
>
> dput(yourdata[1:20,])
>
> Cheers
> Petr
>
>
> > -----Original Message-----
> > From: R-help <r-help-bounces using r-project.org> On Behalf Of Diego Avesani
> > Sent: Wednesday, August 1, 2018 8:55 AM
> > To: Jeff Newmiller <jdnewmil using dcn.davis.ca.us>
> > Cc: r-help mailing list <r-help using r-project.org>
> > Subject: Re: [R] read txt file - date - no space
> >
> > Dear all,
> > I am sorry, I did a lot of confusion. I am sorry, I have to relax and
> stat all again
> > in order to understand.
> > If I could I would like to start again, without mixing strategy and
> waiting for
> > your advice.
> >
> > I am really appreciate you help, really really.
> > Here my new file, a *.csv file (buy the way, it is possible to attach it
> in the
> > mailing list?)
> >
> > date,str1,str2,str3
> > 10/1/1998 0:00,0.6,0,0
> > 10/1/1998 1:00,0.2,0.2,0.2
> > 10/1/1998 2:00,0.6,0.2,0.4
> > 10/1/1998 3:00,0,0,0.6
> > 10/1/1998 4:00,0,0,0
> > 10/1/1998 5:00,0,0,0
> > 10/1/1998 6:00,0,0,0
> > 10/1/1998 7:00,0.2,0,0
> >
> >
> > I read it as:
> > MyData <- read.csv(file="obs_prec.csv",header=TRUE, sep=",")
> >
> > at this point I would like to have the daily mean.
> > What would you suggest?
> >
> > Really Really thanks,
> > You are my lifesaver
> >
> > Thanks
> >
> >
> >
> > Diego
> >
> >
> > On 1 August 2018 at 01:01, Jeff Newmiller <jdnewmil using dcn.davis.ca.us>
> wrote:
> >
> > > ... and the most common source of NA values in time data is wrong
> > > timezones. You really need to make sure the timezone that is assumed
> > > when the character data are converted to POSIXt agrees with the data.
> > > In most cases the easiest way to insure this is to use
> > >
> > > Sys.setenv(TZ="US/Pacific")
> > >
> > > or whatever timezone from
> > >
> > > OlsonNames()
> > >
> > > corresponds with your data. Execute this setenv function before the
> > > strptime or as.POSIXct() function call.
> > >
> > > You can use
> > >
> > > MyData[ is.na(MyData$datetime), ]
> > >
> > > to see which records are failing to convert time.
> > >
> > > [1] https://github.com/jdnewmil/eci298sp2016/blob/master/QuickHowto1
> > >
> > > On July 31, 2018 3:04:05 PM PDT, Jim Lemon <drjimlemon using gmail.com>
> > wrote:
> > > >Hi Diego,
> > > >I think the error is due to NA values in your data file. If I extend
> > > >your example and run it, I get no errors:
> > > >
> > > >MyData<-read.table(text="103001930 103001580 103001530
> > > >1998-10-01 00:00:00 0.6 0 0
> > > >1998-10-01 01:00:00 0.2 0.2 0.2
> > > >1998-10-01 02:00:00 0.6 0.2 0.4
> > > >1998-10-01 03:00:00 0 0 0.6
> > > >1998-10-01 04:00:00 0 0 0
> > > >1998-10-01 05:00:00 0 0 0
> > > >1998-10-01 06:00:00 0 0 0
> > > >1998-10-01 07:00:00 0.2 0 0
> > > >1998-10-01 08:00:00 0.6 0 0
> > > >1998-10-01 09:00:00 0.2 0.2 0.2
> > > >1998-10-01 10:00:00 0.6 0.2 0.4
> > > >1998-10-01 11:00:00 0 0 0.6
> > > >1998-10-01 12:00:00 0 0 0
> > > >1998-10-01 13:00:00 0 0 0
> > > >1998-10-01 14:00:00 0 0 0
> > > >1998-10-01 15:00:00 0.2 0 0
> > > >1998-10-01 16:00:00 0.6 0 0
> > > >1998-10-01 17:00:00 0.2 0.2 0.2
> > > >1998-10-01 18:00:00 0.6 0.2 0.4
> > > >1998-10-01 19:00:00 0 0 0.6
> > > >1998-10-01 20:00:00 0 0 0
> > > >1998-10-01 21:00:00 0 0 0
> > > >1998-10-01 22:00:00 0 0 0
> > > >1998-10-01 23:00:00 0.2 0 0
> > > >1998-10-02 00:00:00 0.6 0 0
> > > >1998-10-02 01:00:00 0.2 0.2 0.2
> > > >1998-10-02 02:00:00 0.6 0.2 0.4
> > > >1998-10-02 03:00:00 0 0 0.6
> > > >1998-10-02 04:00:00 0 0 0
> > > >1998-10-02 05:00:00 0 0 0
> > > >1998-10-02 06:00:00 0 0 0
> > > >1998-10-02 07:00:00 0.2 0 0
> > > >1998-10-02 08:00:00 0.6 0 0
> > > >1998-10-02 09:00:00 0.2 0.2 0.2
> > > >1998-10-02 10:00:00 0.6 0.2 0.4
> > > >1998-10-02 11:00:00 0 0 0.6
> > > >1998-10-02 12:00:00 0 0 0
> > > >1998-10-02 13:00:00 0 0 0
> > > >1998-10-02 14:00:00 0 0 0
> > > >1998-10-02 15:00:00 0.2 0 0
> > > >1998-10-02 16:00:00 0.6 0 0
> > > >1998-10-02 17:00:00 0.2 0.2 0.2
> > > >1998-10-02 18:00:00 0.6 0.2 0.4
> > > >1998-10-02 19:00:00 0 0 0.6
> > > >1998-10-02 20:00:00 0 0 0
> > > >1998-10-02 21:00:00 0 0 0
> > > >1998-10-02 22:00:00 0 0 0
> > > >1998-10-02 23:00:00 0.2 0 0",
> > > >skip=1,stringsAsFactors=FALSE)
> > > >names(MyData)<-c("date","time","st1","st2","st3")
> > > >MyData$datetime<-strptime(paste(MyData$date,MyData$time),
> > > > format="%Y-%m-%d %H:%M:%S")
> > > >MyData$datetime
> > > >st1_daily<-by(MyData$st1,MyData$date,mean)
> > > >st2_daily<-by(MyData$st2,MyData$date,mean)
> > > >st3_daily<-by(MyData$st3,MyData$date,mean)
> > > >st1_daily
> > > >st2_daily
> > > >st3_daily
> > > >
> > > >Try adding na.rm=TRUE to the "by" calls:
> > > >
> > > >st1_daily<-by(MyData$st1,MyData$date,mean,na.rm=TRUE)
> > > >st2_daily<-by(MyData$st2,MyData$date,mean,na.rm=TRUE)
> > > >st3_daily<-by(MyData$st3,MyData$date,mean,na.rm=TRUE)
> > > >
> > > >Jim
> > > >
> > > >On Tue, Jul 31, 2018 at 11:11 PM, Diego Avesani
> > > ><diego.avesani using gmail.com> wrote:
> > > >> Dear all,
> > > >>
> > > >> I have still problem with date.
> > > >> Could you please tel me how to use POSIXct.
> > > >> Indeed I have found this command:
> > > >> timeAverage, but I am not able to convert MyDate to properly date.
> > > >>
> > > >> Thank a lot
> > > >> I hope to no bother you, at least too much
> > > >>
> > > >>
> > > >> Diego
> > > >>
> > > >>
> > > >> On 31 July 2018 at 11:12, Diego Avesani <diego.avesani using gmail.com>
> > > >wrote:
> > > >>>
> > > >>> Dear Jim, Dear all,
> > > >>>
> > > >>> thanks a lot.
> > > >>>
> > > >>> Unfortunately, I get the following error:
> > > >>>
> > > >>>
> > > >>> st1_daily<-by(MyData$st1,MyData$date,mean)
> > > >>> Error in tapply(seq_len(0L), list(`MyData$date` = c(913L, 914L,
> > > >925L, :
> > > >>> arguments must have same length
> > > >>>
> > > >>>
> > > >>> This is particularly strange. indeed, if I apply
> > > >>>
> > > >>>
> > > >>> mean(MyData$str1,na.rm=TRUE)
> > > >>>
> > > >>>
> > > >>> it works
> > > >>>
> > > >>>
> > > >>> Sorry, I have to learn a lot.
> > > >>> You are really boosting me
> > > >>>
> > > >>> Diego
> > > >>>
> > > >>>
> > > >>> On 31 July 2018 at 11:02, Jim Lemon <drjimlemon using gmail.com> wrote:
> > > >>>>
> > > >>>> Hi Diego,
> > > >>>> One way you can get daily means is:
> > > >>>>
> > > >>>> st1_daily<-by(MyData$st1,MyData$date,mean)
> > > >>>> st2_daily<-by(MyData$st2,MyData$date,mean)
> > > >>>> st3_daily<-by(MyData$st3,MyData$date,mean)
> > > >>>>
> > > >>>> Jim
> > > >>>>
> > > >>>> On Tue, Jul 31, 2018 at 6:51 PM, Diego Avesani
> > > ><diego.avesani using gmail.com>
> > > >>>> wrote:
> > > >>>> > Dear all,
> > > >>>> > I have found the error, my fault. Sorry.
> > > >>>> > There was an extra come in the headers line.
> > > >>>> > Thanks again.
> > > >>>> >
> > > >>>> > If I can I would like to ask you another questions about the
> > > >imported
> > > >>>> > data.
> > > >>>> > I would like to compute the daily average of the different date.
> > > >>>> > Basically I
> > > >>>> > have hourly data, I would like to ave the daily mean of them.
> > > >>>> >
> > > >>>> > Is there some special commands?
> > > >>>> >
> > > >>>> > Thanks a lot.
> > > >>>> >
> > > >>>> >
> > > >>>> > Diego
> > > >>>> >
> > > >>>> >
> > > >>>> > On 31 July 2018 at 10:40, Diego Avesani
> > > >>>> > <diego.avesani using gmail.com>
> > > >>>> > wrote:
> > > >>>> >>
> > > >>>> >> Dear all,
> > > >>>> >> I move to csv file because originally the date where in csv
> > > >file.
> > > >>>> >> In addition, due to the fact that, as you told me, read.csv is
> > > >>>> >> a special case of read.table, I prefer start to learn from the
> > > >>>> >> simplest
> > > >one.
> > > >>>> >> After that, I will try also the *.txt format.
> > > >>>> >>
> > > >>>> >> with read.csv, something strange happened:
> > > >>>> >>
> > > >>>> >> This us now the file:
> > > >>>> >>
> > > >>>> >> date,st1,st2,st3,
> > > >>>> >> 10/1/1998 0:00,0.6,0,0
> > > >>>> >> 10/1/1998 1:00,0.2,0.2,0.2
> > > >>>> >> 10/1/1998 2:00,0.6,0.2,0.4
> > > >>>> >> 10/1/1998 3:00,0,0,0.6
> > > >>>> >> 10/1/1998 4:00,0,0,0
> > > >>>> >> 10/1/1998 5:00,0,0,0
> > > >>>> >> 10/1/1998 6:00,0,0,0
> > > >>>> >> 10/1/1998 7:00,0.2,0,0
> > > >>>> >> 10/1/1998 8:00,0.6,0.2,0
> > > >>>> >> 10/1/1998 9:00,0.2,0.4,0.4
> > > >>>> >> 10/1/1998 10:00,0,0.4,0.2
> > > >>>> >>
> > > >>>> >> When I apply:
> > > >>>> >> MyData <- read.csv(file="obs_prec.csv",header=TRUE, sep=",")
> > > >>>> >>
> > > >>>> >> this is the results:
> > > >>>> >>
> > > >>>> >> 10/1/1998 0:00 0.6 0.00 0.0 NA
> > > >>>> >> 2 10/1/1998 1:00 0.2 0.20 0.2 NA
> > > >>>> >> 3 10/1/1998 2:00 0.6 0.20 0.4 NA
> > > >>>> >> 4 10/1/1998 3:00 0.0 0.00 0.6 NA
> > > >>>> >> 5 10/1/1998 4:00 0.0 0.00 0.0 NA
> > > >>>> >> 6 10/1/1998 5:00 0.0 0.00 0.0 NA
> > > >>>> >> 7 10/1/1998 6:00 0.0 0.00 0.0 NA
> > > >>>> >> 8 10/1/1998 7:00 0.2 0.00 0.0 NA
> > > >>>> >>
> > > >>>> >> I do not understand why.
> > > >>>> >> Something wrong with date?
> > > >>>> >>
> > > >>>> >> really really thanks,
> > > >>>> >> I appreciate a lot all your helps.
> > > >>>> >>
> > > >>>> >> Diedro
> > > >>>> >>
> > > >>>> >>
> > > >>>> >> Diego
> > > >>>> >>
> > > >>>> >>
> > > >>>> >> On 31 July 2018 at 01:25, MacQueen, Don <macqueen1 using llnl.gov>
> > > >wrote:
> > > >>>> >>>
> > > >>>> >>> Or, without removing the first line
> > > >>>> >>> dadf <- read.table("xxx.txt", stringsAsFactors=FALSE,
> > > >>>> >>> skip=1)
> > > >>>> >>>
> > > >>>> >>> Another alternative,
> > > >>>> >>> dadf$datetime <- as.POSIXct(paste(dadf$V1,dadf$V2))
> > > >>>> >>> since the dates appear to be in the default format.
> > > >>>> >>> (I generally prefer to work with datetimes in POSIXct class
> > > >rather
> > > >>>> >>> than
> > > >>>> >>> POSIXlt class)
> > > >>>> >>>
> > > >>>> >>> -Don
> > > >>>> >>>
> > > >>>> >>> --
> > > >>>> >>> Don MacQueen
> > > >>>> >>> Lawrence Livermore National Laboratory
> > > >>>> >>> 7000 East Ave., L-627
> > > >>>> >>> Livermore, CA 94550
> > > >>>> >>> 925-423-1062
> > > >>>> >>> Lab cell 925-724-7509
> > > >>>> >>>
> > > >>>> >>>
> > > >>>> >>>
> > > >>>> >>> On 7/30/18, 4:03 PM, "R-help on behalf of Jim Lemon"
> > > >>>> >>> <r-help-bounces using r-project.org on behalf of
> > > >drjimlemon using gmail.com>
> > > >>>> >>> wrote:
> > > >>>> >>>
> > > >>>> >>> Hi Diego,
> > > >>>> >>> You may have to do some conversion as you have three
> > > >>>> >>> fields
> > > >in
> > > >>>> >>> the
> > > >>>> >>> first line using the default space separator and five
> > > >fields in
> > > >>>> >>> subsequent lines. If the first line doesn't contain any
> > > >important
> > > >>>> >>> data
> > > >>>> >>> you can just delete it or replace it with a meaningful
> > > >header
> > > >>>> >>> line
> > > >>>> >>> with five fields and save the file under another name.
> > > >>>> >>>
> > > >>>> >>> It looks as thought you have date-time as two fields. If
> > > >so, you
> > > >>>> >>> can
> > > >>>> >>> just read the first field if you only want the date:
> > > >>>> >>>
> > > >>>> >>> # assume you have removed the first line
> > > >>>> >>> dadf<-read.table("xxx.txt",stringsAsFactors=FALSE
> > > >>>> >>> dadf$date<-as.Date(dadf$V1,format="%Y-%m-%d")
> > > >>>> >>>
> > > >>>> >>> If you want the date/time:
> > > >>>> >>>
> > > >>>> >>>
> > > >dadf$datetime<-strptime(paste(dadf$V1,dadf$V2),format="%Y-%m-%d
> > > >>>> >>> %H:%M:%S")
> > > >>>> >>>
> > > >>>> >>> Jim
> > > >>>> >>>
> > > >>>> >>> On Tue, Jul 31, 2018 at 12:29 AM, Diego Avesani
> > > >>>> >>> <diego.avesani using gmail.com> wrote:
> > > >>>> >>> > Dear all,
> > > >>>> >>> >
> > > >>>> >>> > I am dealing with the reading of a *.txt file.
> > > >>>> >>> > The txt file the following shape:
> > > >>>> >>> >
> > > >>>> >>> > 103001930 103001580 103001530
> > > >>>> >>> > 1998-10-01 00:00:00 0.6 0 0
> > > >>>> >>> > 1998-10-01 01:00:00 0.2 0.2 0.2
> > > >>>> >>> > 1998-10-01 02:00:00 0.6 0.2 0.4
> > > >>>> >>> > 1998-10-01 03:00:00 0 0 0.6
> > > >>>> >>> > 1998-10-01 04:00:00 0 0 0
> > > >>>> >>> > 1998-10-01 05:00:00 0 0 0
> > > >>>> >>> > 1998-10-01 06:00:00 0 0 0
> > > >>>> >>> > 1998-10-01 07:00:00 0.2 0 0
> > > >>>> >>> >
> > > >>>> >>> > If it is possible I have a coupe of questions, which
> > > >>>> >>> will
> > > >sound
> > > >>>> >>> stupid but
> > > >>>> >>> > they are important to me in order to understand ho R
> > > >>>> >>> deal
> > > >with
> > > >>>> >>> file
> > > >>>> >>> or date.
> > > >>>> >>> >
> > > >>>> >>> > 1) Do I have to convert it to a *csv file?
> > > >>>> >>> > 2) Can a deal with space and not ","
> > > >>>> >>> > 3) How can I read date?
> > > >>>> >>> >
> > > >>>> >>> > thanks a lot to all of you,
> > > >>>> >>> > Thanks
> > > >>>> >>> >
> > > >>>> >>> >
> > > >>>> >>> > Diego
> > > >>>> >>> >
> > > >>>> >>> > [[alternative HTML version deleted]]
> > > >>>> >>> >
> > > >>>> >>> > ______________________________________________
> > > >>>> >>> > R-help using r-project.org mailing list -- To UNSUBSCRIBE and
> > > >more,
> > > >>>> >>> see
> > > >>>> >>> > https://stat.ethz.ch/mailman/listinfo/r-help
> > > >>>> >>> > PLEASE do read the posting guide
> > > >>>> >>> http://www.R-project.org/posting-guide.html
> > > >>>> >>> > and provide commented, minimal, self-contained,
> > > >reproducible
> > > >>>> >>> code.
> > > >>>> >>>
> > > >>>> >>> ______________________________________________
> > > >>>> >>> R-help using r-project.org mailing list -- To UNSUBSCRIBE and
> > > >more, see
> > > >>>> >>> https://stat.ethz.ch/mailman/listinfo/r-help
> > > >>>> >>> PLEASE do read the posting guide
> > > >>>> >>> http://www.R-project.org/posting-guide.html
> > > >>>> >>> and provide commented, minimal, self-contained,
> > > >reproducible
> > > >>>> >>> code.
> > > >>>> >>>
> > > >>>> >>>
> > > >>>> >>
> > > >>>> >
> > > >>>
> > > >>>
> > > >>
> > > >
> > > >______________________________________________
> > > >R-help using r-project.org mailing list -- To UNSUBSCRIBE and more, see
> > > >https://stat.ethz.ch/mailman/listinfo/r-help
> > > >PLEASE do read the posting guide
> > > >http://www.R-project.org/posting-guide.html
> > > >and provide commented, minimal, self-contained, reproducible code.
> > >
> > > --
> > > Sent from my phone. Please excuse my brevity.
> > >
> >
> > [[alternative HTML version deleted]]
> >
> > ______________________________________________
> > R-help using r-project.org mailing list -- To UNSUBSCRIBE and more, see
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide http://www.R-project.org/
> posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
> Osobní údaje: Informace o zpracování a ochraně osobních údajů obchodních
> partnerů PRECHEZA a.s. jsou zveřejněny na: https://www.precheza.cz/
> zasady-ochrany-osobnich-udaju/ | Information about processing and
> protection of business partner’s personal data are available on website:
> https://www.precheza.cz/en/personal-data-protection-principles/
> Důvěrnost: Tento e-mail a jakékoliv k němu připojené dokumenty jsou
> důvěrné a podléhají tomuto právně závaznému prohláąení o vyloučení
> odpovědnosti: https://www.precheza.cz/01-dovetek/ | This email and any
> documents attached to it may be confidential and are subject to the legally
> binding disclaimer: https://www.precheza.cz/en/01-disclaimer/
>
>
[[alternative HTML version deleted]]
More information about the R-help
mailing list