[Rd] Issues with libcurl + HTTP status codes (eg. 403, 404)

Tue Aug 25 22:30:22 CEST 2015

Hi Martin,

Indeed it does (and I should have confirmed myself with R-patched and
R-devel before posting...)

Thanks, and sorry for the noise.
Kevin

On Tue, Aug 25, 2015, 13:11 Martin Morgan <mtmorgan at fredhutch.org> wrote:

> On 08/25/2015 12:54 PM, Kevin Ushey wrote:
> > Hi all,
> >
> > The following fails for me (on OS X, although I imagine it's the same
> > on other platforms using libcurl):
> >
> >      options(download.file.method = "libcurl")
> >      options(repos = c(CRAN = "https://cran.rstudio.com/", CRANextra =
> > "http://www.stats.ox.ac.uk/pub/RWin"))
> >      install.packages("lattice") ## could be any package
> >
> > gives me:
> >
> >      > options(download.file.method = "libcurl")
> >      > options(repos = c(CRAN = "https://cran.rstudio.com/", CRANextra
> > = "http://www.stats.ox.ac.uk/pub/RWin"))
> >      > install.packages("lattice") ## coudl be any package
> >      Installing package into ‘/Users/kevinushey/Library/R/3.2/library’
> >      (as ‘lib’ is unspecified)
> >      Error: Line starting '<!DOCTYPE HTML PUBLI ...' is malformed!
> >
> > This seems to come from a call to `available.packages()` to a URL that
> > doesn't exist on the server (likely when querying PACKAGES on the
> > CRANextra repo)
> >
> > Eg.
> >
> >      > URL <- "http://www.stats.ox.ac.uk/pub/RWin"
> >      > available.packages(URL, method = "internal")
> >      Warning: unable to access index for repository
> > http://www.stats.ox.ac.uk/pub/RWin
> >           Package Version Priority Depends Imports LinkingTo Suggests
> > Enhances License License_is_FOSS
> >          License_restricts_use OS_type Archs MD5sum NeedsCompilation
> > File Repository
> >      > available.packages(URL, method = "libcurl")
> >      Error: Line starting '<!DOCTYPE HTML PUBLI ...' is malformed!
> >
> > It looks like libcurl downloads and retrieves the 403 page itself,
> > rather than reporting that it was actually forbidden, e.g.:
> >
> >      > download.file("
> http://www.stats.ox.ac.uk/pub/RWin/bin/macosx/mavericks/contrib/3.2/PACKAGES.gz
> ",
> > tempfile(), method = "libcurl")
> >      trying URL '
> http://www.stats.ox.ac.uk/pub/RWin/bin/macosx/mavericks/contrib/3.2/PACKAGES.gz
> '
> >      Content type 'text/html; charset=iso-8859-1' length 339 bytes
> >      ==================================================
> >      downloaded 339 bytes
> >
> > Using `method = "internal"` gives an error related to the inability to
> > access that URL due to the HTTP status 403.
> >
> > The overarching issue here is that package installation shouldn't fail
> > even if libcurl fails to access one of the repositories set.
> >
>
> With
>
>  > R.version.string
> [1] "R version 3.2.2 Patched (2015-08-25 r69179)"
>
> the behavior is to warn with an indication of the repository for which the
> problem occurs
>
>  > URL <- "http://www.stats.ox.ac.uk/pub/RWin"
>  > available.packages(URL, method="libcurl")
> Warning: unable to access index for repository
> http://www.stats.ox.ac.uk/pub/RWin:
>    Line starting '<!DOCTYPE HTML PUBLI ...' is malformed!
>       Package Version Priority Depends Imports LinkingTo Suggests Enhances
>       License License_is_FOSS License_restricts_use OS_type Archs MD5sum
>       NeedsCompilation File Repository
>  > available.packages(URL, method="internal")
> Warning: unable to access index for repository
> http://www.stats.ox.ac.uk/pub/RWin:
>    cannot open URL 'http://www.stats.ox.ac.uk/pub/RWin/PACKAGES'
>       Package Version Priority Depends Imports LinkingTo Suggests Enhances
>       License License_is_FOSS License_restricts_use OS_type Archs MD5sum
>       NeedsCompilation File Repository
>
> Does that work for you / address the problem?
>
> Martin
>
> >> sessionInfo()
> > R version 3.2.2 (2015-08-14)
> > Platform: x86_64-apple-darwin13.4.0 (64-bit)
> > Running under: OS X 10.10.4 (Yosemite)
> >
> > locale:
> > [1] en_CA.UTF-8/en_CA.UTF-8/en_CA.UTF-8/C/en_CA.UTF-8/en_CA.UTF-8
> >
> > attached base packages:
> > [1] stats     graphics  grDevices utils     datasets  methods   base
> >
> > other attached packages:
> > [1] testthat_0.8.1.0.99  knitr_1.11           devtools_1.5.0.9001
> > [4] BiocInstaller_1.15.5
> >
> > loaded via a namespace (and not attached):
> >   [1] httr_1.0.0     R6_2.0.0.9000  tools_3.2.2    parallel_3.2.2
> whisker_0.3-2
> >   [6] RCurl_1.95-4.1 memoise_0.2.1  stringr_0.6.2  digest_0.6.4
>  evaluate_0.7.2
> >
> > Thanks,
> > Kevin
> >
> > ______________________________________________
> > R-devel at r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel
> >
>
>
> --
> Computational Biology / Fred Hutchinson Cancer Research Center
> 1100 Fairview Ave. N.
> PO Box 19024 Seattle, WA 98109
>
> Location: Arnold Building M1 B861
> Phone: (206) 667-2793
>

	[[alternative HTML version deleted]]