[R] read txt file - date - no space
Jeff Newmiller
jdnewm|| @end|ng |rom dcn@d@v|@@c@@u@
Wed Aug 1 01:01:48 CEST 2018
... and the most common source of NA values in time data is wrong timezones. You really need to make sure the timezone that is assumed when the character data are converted to POSIXt agrees with the data. In most cases the easiest way to insure this is to use
Sys.setenv(TZ="US/Pacific")
or whatever timezone from
OlsonNames()
corresponds with your data. Execute this setenv function before the strptime or as.POSIXct() function call.
You can use
MyData[ is.na(MyData$datetime), ]
to see which records are failing to convert time.
[1] https://github.com/jdnewmil/eci298sp2016/blob/master/QuickHowto1
On July 31, 2018 3:04:05 PM PDT, Jim Lemon <drjimlemon using gmail.com> wrote:
>Hi Diego,
>I think the error is due to NA values in your data file. If I extend
>your example and run it, I get no errors:
>
>MyData<-read.table(text="103001930 103001580 103001530
>1998-10-01 00:00:00 0.6 0 0
>1998-10-01 01:00:00 0.2 0.2 0.2
>1998-10-01 02:00:00 0.6 0.2 0.4
>1998-10-01 03:00:00 0 0 0.6
>1998-10-01 04:00:00 0 0 0
>1998-10-01 05:00:00 0 0 0
>1998-10-01 06:00:00 0 0 0
>1998-10-01 07:00:00 0.2 0 0
>1998-10-01 08:00:00 0.6 0 0
>1998-10-01 09:00:00 0.2 0.2 0.2
>1998-10-01 10:00:00 0.6 0.2 0.4
>1998-10-01 11:00:00 0 0 0.6
>1998-10-01 12:00:00 0 0 0
>1998-10-01 13:00:00 0 0 0
>1998-10-01 14:00:00 0 0 0
>1998-10-01 15:00:00 0.2 0 0
>1998-10-01 16:00:00 0.6 0 0
>1998-10-01 17:00:00 0.2 0.2 0.2
>1998-10-01 18:00:00 0.6 0.2 0.4
>1998-10-01 19:00:00 0 0 0.6
>1998-10-01 20:00:00 0 0 0
>1998-10-01 21:00:00 0 0 0
>1998-10-01 22:00:00 0 0 0
>1998-10-01 23:00:00 0.2 0 0
>1998-10-02 00:00:00 0.6 0 0
>1998-10-02 01:00:00 0.2 0.2 0.2
>1998-10-02 02:00:00 0.6 0.2 0.4
>1998-10-02 03:00:00 0 0 0.6
>1998-10-02 04:00:00 0 0 0
>1998-10-02 05:00:00 0 0 0
>1998-10-02 06:00:00 0 0 0
>1998-10-02 07:00:00 0.2 0 0
>1998-10-02 08:00:00 0.6 0 0
>1998-10-02 09:00:00 0.2 0.2 0.2
>1998-10-02 10:00:00 0.6 0.2 0.4
>1998-10-02 11:00:00 0 0 0.6
>1998-10-02 12:00:00 0 0 0
>1998-10-02 13:00:00 0 0 0
>1998-10-02 14:00:00 0 0 0
>1998-10-02 15:00:00 0.2 0 0
>1998-10-02 16:00:00 0.6 0 0
>1998-10-02 17:00:00 0.2 0.2 0.2
>1998-10-02 18:00:00 0.6 0.2 0.4
>1998-10-02 19:00:00 0 0 0.6
>1998-10-02 20:00:00 0 0 0
>1998-10-02 21:00:00 0 0 0
>1998-10-02 22:00:00 0 0 0
>1998-10-02 23:00:00 0.2 0 0",
>skip=1,stringsAsFactors=FALSE)
>names(MyData)<-c("date","time","st1","st2","st3")
>MyData$datetime<-strptime(paste(MyData$date,MyData$time),
> format="%Y-%m-%d %H:%M:%S")
>MyData$datetime
>st1_daily<-by(MyData$st1,MyData$date,mean)
>st2_daily<-by(MyData$st2,MyData$date,mean)
>st3_daily<-by(MyData$st3,MyData$date,mean)
>st1_daily
>st2_daily
>st3_daily
>
>Try adding na.rm=TRUE to the "by" calls:
>
>st1_daily<-by(MyData$st1,MyData$date,mean,na.rm=TRUE)
>st2_daily<-by(MyData$st2,MyData$date,mean,na.rm=TRUE)
>st3_daily<-by(MyData$st3,MyData$date,mean,na.rm=TRUE)
>
>Jim
>
>On Tue, Jul 31, 2018 at 11:11 PM, Diego Avesani
><diego.avesani using gmail.com> wrote:
>> Dear all,
>>
>> I have still problem with date.
>> Could you please tel me how to use POSIXct.
>> Indeed I have found this command:
>> timeAverage, but I am not able to convert MyDate to properly date.
>>
>> Thank a lot
>> I hope to no bother you, at least too much
>>
>>
>> Diego
>>
>>
>> On 31 July 2018 at 11:12, Diego Avesani <diego.avesani using gmail.com>
>wrote:
>>>
>>> Dear Jim, Dear all,
>>>
>>> thanks a lot.
>>>
>>> Unfortunately, I get the following error:
>>>
>>>
>>> st1_daily<-by(MyData$st1,MyData$date,mean)
>>> Error in tapply(seq_len(0L), list(`MyData$date` = c(913L, 914L,
>925L, :
>>> arguments must have same length
>>>
>>>
>>> This is particularly strange. indeed, if I apply
>>>
>>>
>>> mean(MyData$str1,na.rm=TRUE)
>>>
>>>
>>> it works
>>>
>>>
>>> Sorry, I have to learn a lot.
>>> You are really boosting me
>>>
>>> Diego
>>>
>>>
>>> On 31 July 2018 at 11:02, Jim Lemon <drjimlemon using gmail.com> wrote:
>>>>
>>>> Hi Diego,
>>>> One way you can get daily means is:
>>>>
>>>> st1_daily<-by(MyData$st1,MyData$date,mean)
>>>> st2_daily<-by(MyData$st2,MyData$date,mean)
>>>> st3_daily<-by(MyData$st3,MyData$date,mean)
>>>>
>>>> Jim
>>>>
>>>> On Tue, Jul 31, 2018 at 6:51 PM, Diego Avesani
><diego.avesani using gmail.com>
>>>> wrote:
>>>> > Dear all,
>>>> > I have found the error, my fault. Sorry.
>>>> > There was an extra come in the headers line.
>>>> > Thanks again.
>>>> >
>>>> > If I can I would like to ask you another questions about the
>imported
>>>> > data.
>>>> > I would like to compute the daily average of the different date.
>>>> > Basically I
>>>> > have hourly data, I would like to ave the daily mean of them.
>>>> >
>>>> > Is there some special commands?
>>>> >
>>>> > Thanks a lot.
>>>> >
>>>> >
>>>> > Diego
>>>> >
>>>> >
>>>> > On 31 July 2018 at 10:40, Diego Avesani <diego.avesani using gmail.com>
>>>> > wrote:
>>>> >>
>>>> >> Dear all,
>>>> >> I move to csv file because originally the date where in csv
>file.
>>>> >> In addition, due to the fact that, as you told me, read.csv is a
>>>> >> special
>>>> >> case of read.table, I prefer start to learn from the simplest
>one.
>>>> >> After that, I will try also the *.txt format.
>>>> >>
>>>> >> with read.csv, something strange happened:
>>>> >>
>>>> >> This us now the file:
>>>> >>
>>>> >> date,st1,st2,st3,
>>>> >> 10/1/1998 0:00,0.6,0,0
>>>> >> 10/1/1998 1:00,0.2,0.2,0.2
>>>> >> 10/1/1998 2:00,0.6,0.2,0.4
>>>> >> 10/1/1998 3:00,0,0,0.6
>>>> >> 10/1/1998 4:00,0,0,0
>>>> >> 10/1/1998 5:00,0,0,0
>>>> >> 10/1/1998 6:00,0,0,0
>>>> >> 10/1/1998 7:00,0.2,0,0
>>>> >> 10/1/1998 8:00,0.6,0.2,0
>>>> >> 10/1/1998 9:00,0.2,0.4,0.4
>>>> >> 10/1/1998 10:00,0,0.4,0.2
>>>> >>
>>>> >> When I apply:
>>>> >> MyData <- read.csv(file="obs_prec.csv",header=TRUE, sep=",")
>>>> >>
>>>> >> this is the results:
>>>> >>
>>>> >> 10/1/1998 0:00 0.6 0.00 0.0 NA
>>>> >> 2 10/1/1998 1:00 0.2 0.20 0.2 NA
>>>> >> 3 10/1/1998 2:00 0.6 0.20 0.4 NA
>>>> >> 4 10/1/1998 3:00 0.0 0.00 0.6 NA
>>>> >> 5 10/1/1998 4:00 0.0 0.00 0.0 NA
>>>> >> 6 10/1/1998 5:00 0.0 0.00 0.0 NA
>>>> >> 7 10/1/1998 6:00 0.0 0.00 0.0 NA
>>>> >> 8 10/1/1998 7:00 0.2 0.00 0.0 NA
>>>> >>
>>>> >> I do not understand why.
>>>> >> Something wrong with date?
>>>> >>
>>>> >> really really thanks,
>>>> >> I appreciate a lot all your helps.
>>>> >>
>>>> >> Diedro
>>>> >>
>>>> >>
>>>> >> Diego
>>>> >>
>>>> >>
>>>> >> On 31 July 2018 at 01:25, MacQueen, Don <macqueen1 using llnl.gov>
>wrote:
>>>> >>>
>>>> >>> Or, without removing the first line
>>>> >>> dadf <- read.table("xxx.txt", stringsAsFactors=FALSE, skip=1)
>>>> >>>
>>>> >>> Another alternative,
>>>> >>> dadf$datetime <- as.POSIXct(paste(dadf$V1,dadf$V2))
>>>> >>> since the dates appear to be in the default format.
>>>> >>> (I generally prefer to work with datetimes in POSIXct class
>rather
>>>> >>> than
>>>> >>> POSIXlt class)
>>>> >>>
>>>> >>> -Don
>>>> >>>
>>>> >>> --
>>>> >>> Don MacQueen
>>>> >>> Lawrence Livermore National Laboratory
>>>> >>> 7000 East Ave., L-627
>>>> >>> Livermore, CA 94550
>>>> >>> 925-423-1062
>>>> >>> Lab cell 925-724-7509
>>>> >>>
>>>> >>>
>>>> >>>
>>>> >>> On 7/30/18, 4:03 PM, "R-help on behalf of Jim Lemon"
>>>> >>> <r-help-bounces using r-project.org on behalf of
>drjimlemon using gmail.com>
>>>> >>> wrote:
>>>> >>>
>>>> >>> Hi Diego,
>>>> >>> You may have to do some conversion as you have three fields
>in
>>>> >>> the
>>>> >>> first line using the default space separator and five
>fields in
>>>> >>> subsequent lines. If the first line doesn't contain any
>important
>>>> >>> data
>>>> >>> you can just delete it or replace it with a meaningful
>header
>>>> >>> line
>>>> >>> with five fields and save the file under another name.
>>>> >>>
>>>> >>> It looks as thought you have date-time as two fields. If
>so, you
>>>> >>> can
>>>> >>> just read the first field if you only want the date:
>>>> >>>
>>>> >>> # assume you have removed the first line
>>>> >>> dadf<-read.table("xxx.txt",stringsAsFactors=FALSE
>>>> >>> dadf$date<-as.Date(dadf$V1,format="%Y-%m-%d")
>>>> >>>
>>>> >>> If you want the date/time:
>>>> >>>
>>>> >>>
>dadf$datetime<-strptime(paste(dadf$V1,dadf$V2),format="%Y-%m-%d
>>>> >>> %H:%M:%S")
>>>> >>>
>>>> >>> Jim
>>>> >>>
>>>> >>> On Tue, Jul 31, 2018 at 12:29 AM, Diego Avesani
>>>> >>> <diego.avesani using gmail.com> wrote:
>>>> >>> > Dear all,
>>>> >>> >
>>>> >>> > I am dealing with the reading of a *.txt file.
>>>> >>> > The txt file the following shape:
>>>> >>> >
>>>> >>> > 103001930 103001580 103001530
>>>> >>> > 1998-10-01 00:00:00 0.6 0 0
>>>> >>> > 1998-10-01 01:00:00 0.2 0.2 0.2
>>>> >>> > 1998-10-01 02:00:00 0.6 0.2 0.4
>>>> >>> > 1998-10-01 03:00:00 0 0 0.6
>>>> >>> > 1998-10-01 04:00:00 0 0 0
>>>> >>> > 1998-10-01 05:00:00 0 0 0
>>>> >>> > 1998-10-01 06:00:00 0 0 0
>>>> >>> > 1998-10-01 07:00:00 0.2 0 0
>>>> >>> >
>>>> >>> > If it is possible I have a coupe of questions, which will
>sound
>>>> >>> stupid but
>>>> >>> > they are important to me in order to understand ho R deal
>with
>>>> >>> file
>>>> >>> or date.
>>>> >>> >
>>>> >>> > 1) Do I have to convert it to a *csv file?
>>>> >>> > 2) Can a deal with space and not ","
>>>> >>> > 3) How can I read date?
>>>> >>> >
>>>> >>> > thanks a lot to all of you,
>>>> >>> > Thanks
>>>> >>> >
>>>> >>> >
>>>> >>> > Diego
>>>> >>> >
>>>> >>> > [[alternative HTML version deleted]]
>>>> >>> >
>>>> >>> > ______________________________________________
>>>> >>> > R-help using r-project.org mailing list -- To UNSUBSCRIBE and
>more,
>>>> >>> see
>>>> >>> > https://stat.ethz.ch/mailman/listinfo/r-help
>>>> >>> > PLEASE do read the posting guide
>>>> >>> http://www.R-project.org/posting-guide.html
>>>> >>> > and provide commented, minimal, self-contained,
>reproducible
>>>> >>> code.
>>>> >>>
>>>> >>> ______________________________________________
>>>> >>> R-help using r-project.org mailing list -- To UNSUBSCRIBE and
>more, see
>>>> >>> https://stat.ethz.ch/mailman/listinfo/r-help
>>>> >>> PLEASE do read the posting guide
>>>> >>> http://www.R-project.org/posting-guide.html
>>>> >>> and provide commented, minimal, self-contained,
>reproducible
>>>> >>> code.
>>>> >>>
>>>> >>>
>>>> >>
>>>> >
>>>
>>>
>>
>
>______________________________________________
>R-help using r-project.org mailing list -- To UNSUBSCRIBE and more, see
>https://stat.ethz.ch/mailman/listinfo/r-help
>PLEASE do read the posting guide
>http://www.R-project.org/posting-guide.html
>and provide commented, minimal, self-contained, reproducible code.
--
Sent from my phone. Please excuse my brevity.
More information about the R-help
mailing list