[R] Help with readBin
William Dunlap
wdunlap at tibco.com
Tue Jun 19 04:02:19 CEST 2012
I suspect that "junk" (the four bytes after the 2nd record marker), read as a 4-byte integer
instead of four 1-byte integers, is the number of bytes of data following it. Its value
in your example is 18920 = 8 * 43 * 55, where 43 and 55 are two integers in the header,
probably the dimensions of the array containing the double precision data.
Bill Dunlap
Spotfire, TIBCO Software
wdunlap tibco.com
> -----Original Message-----
> From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org] On Behalf
> Of William Dunlap
> Sent: Monday, June 18, 2012 6:43 PM
> To: kapo coulibaly; r-help at r-project.org
> Subject: Re: [R] Help with readBin
>
> You didn't give much of a description of what sort of numbers you expected
> in the header so this is pretty much a guess. However, by reading the tail of
> the file with offsets 0 through 7 bytes we get numbers in the 30-40 range for
> an offset of 4 bytes. I called that field "junk" below and placed it so that the
> two recordMarker fields were the same (52 decimal). Do the numbers in the
> header look right?
>
> f <- function (filename)
> {
> con <- file(filename, "rb")
> on.exit(close(con))
> rbl <- function(...) readBin(..., endian = "little")
> recordMarkerA <- rbl(con, what = "integer", size = 4, n = 1)
> twoIntegers <- rbl(con, what = "integer", size = 4, n = 2)
> twoDoubles <- rbl(con, what = "numeric", size = 8, n = 2)
> oneString <- rawToChar(rbl(con, what = "raw", size = 1, n = 16))
> threeIntegers <- rbl(con, what = "integer", size = 4, n = 3)
> recordMarkerB <- rbl(con, what = "integer", size = 4, n = 1)
> junk <- rbl(con, what = "raw", size = 1, n = 4)
> # the 100 below should be (file.info(filename)$size - headerSize)/8
> doubles <- rbl(con, what = "numeric", size = 8, n = 100)
> list(recordMarkerA = recordMarkerA, twoIntegers = twoIntegers,
> twoDoubles = twoDoubles, oneString = oneString, threeIntegers = threeIntegers,
> recordMarkerB = recordMarkerB, junk = junk, doubles = doubles)
> }
>
> > f(tf)
> $recordMarkerA
> [1] 52
>
> $twoIntegers
> [1] 1 1
>
> $twoDoubles
> [1] 1 1
>
> $oneString
> [1] " HEAD"
>
> $threeIntegers
> [1] 43 55 1
>
> $recordMarkerB
> [1] 52
>
> $junk
> [1] e8 49 00 00
>
> $doubles
> [1] 33.674 34.272 34.736 35.098 35.378 35.628
> [7] 35.838 36.046 36.324 36.604 36.856 37.112
> [13] 37.398 37.694 38.008 38.364 38.742 39.134
> [19] 39.494 39.844 40.128 40.372 40.562 40.712
> [25] 40.818 40.880 40.900 40.882 40.830
>
> Bill Dunlap
> Spotfire, TIBCO Software
> wdunlap tibco.com
>
> From: kapo coulibaly [mailto:kmcoulib at gmail.com]
> Sent: Monday, June 18, 2012 5:55 PM
> To: William Dunlap; r-help at r-project.org
> Subject: Re: [R] Help with readBin
>
> No. But here it is:
> c(52L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L,
> 0L, 0L, 0L, -16L, 63L, 0L, 0L, 0L, 0L, 0L, 0L, -16L, 63L, 32L,
> 32L, 32L, 32L, 32L, 32L, 32L, 32L, 32L, 32L, 32L, 32L, 72L, 69L,
> 65L, 68L, 43L, 0L, 0L, 0L, 55L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 52L,
> 0L, 0L, 0L, -24L, 73L, 0L, 0L, -125L, -64L, -54L, -95L, 69L,
> -42L, 64L, 64L, -119L, 65L, 96L, -27L, -48L, 34L, 65L, 64L, -111L,
> -19L, 124L, 63L, 53L, 94L, 65L, 64L, 6L, -127L, -107L, 67L, -117L,
> -116L, 65L, 64L, -86L, -15L, -46L, 77L, 98L, -80L, 65L, 64L,
> -86L, -15L, -46L, 77L, 98L, -48L, 65L, 64L, 37L, 6L, -127L, -107L,
> 67L, -21L, 65L, 64L, -39L, -50L, -9L, 83L, -29L, 5L, 66L, 64L,
> -74L, -13L, -3L, -44L, 120L, 41L, 66L, 64L, 90L, 100L, 59L, -33L,
> 79L, 77L, 66L, 64L, 33L, -80L, 114L, 104L, -111L, 109L, 66L,
> 64L, 117L, -109L, 24L, 4L, 86L, -114L, 66L, 64L, 109L, -25L,
> -5L, -87L, -15L, -78L, 66L, 64L, 70L, -74L, -13L, -3L, -44L,
> -40L, 66L, 64L, 27L, 47L, -35L, 36L, 6L, 1L, 67L, 64L, 59L, -33L,
> 79L, -115L, -105L, 46L, 67L, 64L, -27L, -48L, 34L, -37L, -7L,
> 94L, 67L, 64L, -2L, -44L, 120L, -23L, 38L, -111L, 67L, 64L, -84L,
> 28L, 90L, 100L, 59L, -65L, 67L, 64L, 121L, -23L, 38L, 49L, 8L,
> -20L, 67L, 64L, -86L, -15L, -46L, 77L, 98L, 16L, 68L, 64L, 86L,
> 14L, 45L, -78L, -99L, 47L, 68L, 64L, 14L, 45L, -78L, -99L, -17L,
> 71L, 68L, 64L, 66L, 96L, -27L, -48L, 34L, 91L, 68L, 64L, 98L,
> 16L, 88L, 57L, -76L, 104L, 68L, 64L, 113L, 61L, 10L, -41L, -93L,
> 112L, 68L, 64L, 51L, 51L, 51L, 51L, 51L, 115L, 68L, 64L, 55L,
> -119L, 65L, 96L, -27L, 112L, 68L, 64L, 10L, -41L, -93L, 112L,
> 61L, 106L, 68L, 64L, -53L, -95L, 69L, -74L)
>
> On Mon, Jun 18, 2012 at 8:39 PM, William Dunlap
> <wdunlap at tibco.com<mailto:wdunlap at tibco.com>> wrote:
> Did you ever send the output of dput to R-help?
> On Thu, May 3, 2012 at 5:00 PM, William Dunlap
> <wdunlap at tibco.com<mailto:wdunlap at tibco.com>> wrote:
> You can do the following to allow others to recreate your problem.
>
> yourFileBytes <- readBin("yourFile", what="integer", size=1, n=300) # is 300 bytes
> enough to see the problem?
> dput(yourFileBytes)
>
> Put the output of dput(yourFileBytes) in your mail.
>
> Bill Dunlap
> Spotfire, TIBCO Software
> wdunlap tibco.com<http://tibco.com>
>
> From: kapo coulibaly [mailto:kmcoulib at gmail.com<mailto:kmcoulib at gmail.com>]
> Sent: Monday, June 18, 2012 5:35 PM
> To: William Dunlap
> Cc: r-help at r-project.org<mailto:r-help at r-project.org>
>
> Subject: Re: [R] Help with readBin
>
> I still haven't found a working solution. Is it allowed to attach a file so that somebodyelse
> can reproduce the problem?
> On Thu, May 3, 2012 at 5:00 PM, William Dunlap
> <wdunlap at tibco.com<mailto:wdunlap at tibco.com>> wrote:
> You can do the following to allow others to recreate your problem.
>
> yourFileBytes <- readBin("yourFile", what="integer", size=1, n=300) # is 300 bytes
> enough to see the problem?
> dput(yourFileBytes)
>
> Put the output of dput(yourFileBytes) in your mail. Someone can (and you should)
> recreate the problem with
> bytes <- ... copy 'n paste the printout of dput(bytes) here ...
> tf <- tempfile()
> stopifnot(is.integer(bytes) && all(abs(bytes)<=128)) # to make sure bytes was copied
> correctly
> writeBin(bytes, con=tf, size=1)
>
> Then show just the commands needed to read a couple of rows of your file, along with
> the expected output, as precisely and you can. E.g.,
> con <- file(tf, "rb")
> readBin(con, what="integer", size=4, n=2) # expect 3 then something less than 10
> readBin(con, what="numeric", size=8, n=3) # expect 2 numbers in range (0, 32] then 2.57
> ...
>
> Bill Dunlap
> Spotfire, TIBCO Software
> wdunlap tibco.com<http://tibco.com>
>
>
> > -----Original Message-----
> > From: r-help-bounces at r-project.org<mailto:r-help-bounces at r-project.org> [mailto:r-
> help-bounces at r-project.org<mailto:r-help-bounces at r-project.org>] On Behalf
> > Of kapo coulibaly
> > Sent: Thursday, May 03, 2012 10:57 AM
> > To: r-help at r-project.org<mailto:r-help at r-project.org>
> > Subject: Re: [R] Help with readBin
> >
> > I believe here is the structure of the file I'm trying to read:
> > record marker (4 bytes), 2 integers (4 bytes each), 2 doubles (8 bytes
> > each), one string (16 bytes or 16 characters), 3 integers (4 bytes each), 1
> > record marker (4 bytes) and a big array of doubles (8 bytes each).
> > Everything in the file is read correctly except for the doubles.
> > If any indication, I've read similar file before with readBin the only
> > difference is this one was created with a code compiled with gfortran in
> > linux 64 bit. I was able to read the same output binary file when the
> > fortran source code was compiled in windows xp 32 bit. The values I'm
> > expecting should be between 0 and about 32.
> >
> >
> >
> >
> > The code I used is:
> >
> >
> >
> > # Loading Required libraries
> > library(tcltk)
> >
> > # Tk inputbox function
> > inputBox<-function() {
> > tt<-tktoplevel()
> > Zmin<-tclVar("0")
> > Zmax<-tclVar("0")
> > dZ<-tclVar("0")
> > entry.Zmin<-tkentry(tt,width="20",textvariable=Zmin)
> > entry.Zmax<-tkentry(tt,width="20",textvariable=Zmax)
> > entry.dZ<-tkentry(tt,width="20",textvariable=dZ)
> > lbl.Zmin<-tklabel(tt,text="Number of layers")
> > lbl.Zmax<-tklabel(tt,text="Number of Stress Periods")
> > lbl.dZ<-tklabel(tt,text="dZ")
> > tkgrid(lbl.Zmin,entry.Zmin)
> > tkgrid(entry.Zmin)
> > tkgrid(lbl.Zmax,entry.Zmax)
> > tkgrid(entry.Zmax)
> > #tkgrid(lbl.dZ,entry.dZ)
> > #tkgrid(entry.dZ)
> >
> > OnOK <- function()
> > {
> > # NameVal <- c(tclvalue(Zmin),tclvalue(Zmax),tclvalue(dZ))
> > tkdestroy(tt)
> > }
> > OK.but <-tkbutton(tt,text=" OK ",command=OnOK)
> > # tkbind(entry.Name, "<Return>",OnOK)
> > tkgrid(OK.but,columnspan=3)
> > tkfocus(tt)
> > tkwait.window(tt)
> > res<-as.numeric(c(tclvalue(Zmin),tclvalue(Zmax)))#,tclvalue(dZ)))
> > return(res)
> > }
> >
> >
> ########################################################################
> > ########
> > # Main program
> >
> ########################################################################
> > ########
> >
> > # Model Parameters input (number of layers and stress periods)
> > param<-inputBox()
> >
> > # Select and open Modflow Binary file for reading
> > fich<-tclvalue(tkgetOpenFile(title="Modflow Binary File",filetypes="{{hds
> > binary Files} {.hds}} {{All files} *}"))
> > zz <- file(fich, "rb")
> >
> > # Cycling thru time steps and layers
> > for (k in 1:param[2]) {
> > for (i in 1:param[1]) {
> > readBin(zz,what="numeric",n=1,size=4) # record marker typical of
> > fortran access="sequential" in gfortran
> > readBin(zz,what="integer",n=2,size=4)->N1
> > readBin(zz,what="double",n=2,size=8)->N2
> > readChar(zz,16)->txt1
> > print(txt1)
> > readBin(zz,what="integer",n=3,size=4)->N3
> > tnber<-N3[1]*N3[2]
> > readBin(zz,what="integer",n=1,size=4) # record marker typical of
> > fortran access="sequential" in gfortran
> > readBin(zz,what=real(),n=tnber,size=4)->N4
> > readBin(zz,what="integer",n=2,size=4) # record marker typical of
> > fortran access="sequential" in gfortran
> > print(N4[1:10])
> >
> >
> > }
> >
> > }
> >
> > close(zz)
> >
> > On Thu, May 3, 2012 at 1:26 PM, Duncan Murdoch
> > <murdoch.duncan at gmail.com<mailto:murdoch.duncan at gmail.com>>wrote:
> >
> > > On 03/05/2012 12:41 PM, kapo coulibaly wrote:
> > >
> > >> I'm trying to read a binary file created by a fortran code using readBin
> > >> and readChar. Everything reads fine (integers and strings) except for
> > >> double precision numbers, they are read as huge or very small number
> > >> (1E-250,...). I tried various endianness, swap, But nothing has worked so
> > >> far.
> > >> I also tried on R 64 bit for linux and windows (R 2.14) and R 2.11 on
> > >> windows XP 32 bit.
> > >> Any help would be appreciated.
> > >>
> > >
> > > As I wrote to someone else with a similar problem a couple of weeks ago:
> > >
> > > You need to see what's in the file. The hexView package can dump it in
> > > various formats; see example(viewFormat) for a couple.
> > >
> > > Duncan Murdoch
> > >
> >
> > [[alternative HTML version deleted]]
> >
> > ______________________________________________
> > R-help at r-project.org<mailto:R-help at r-project.org> mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
>
>
>
> [[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
More information about the R-help
mailing list