[Rd] internet.R test hangs if http proxy needed (PR#3108)

[Prof Brian Ripley]

On Tue, 27 May 2003, Barry Rowlingson wrote:

> Prof Brian Ripley wrote:
> 
> > We need help, as to reproduce this needs some particular local setup and 
> > there have been no reports from anyone during the beta-test period for 
> > 1.7.0, nor from previous versions.  (That makes me suspect it is rare 
> > local setup.)  Can you please debug this further.
> > 
> 
> It appears to be partly a bug in my impatience:
> 
> > 
> system.time(httpget("http://www.stats.ox.ac.uk/pub/datasets/csb/ch11b.dat"))
> Error in make.socket(host, port = port) : Socket not established
> Timing stopped at: 0 0 189 0 0
> 
>   The function times out after 189 seconds, whereas my impatience
> timeout is 3 minutes. I will upgrade my impatience to the latest version
> so that it is compatible with everyone elses.
> 
>   At first I thought it was taking 3 * options()$timeout to produce the
> timeout in make.socket. A quick investigation shows that to be purely 
> coincidental with the initial value of options()$timeout:
> 
> > options(timeout=2)
> > 
> system.time(httpget("http://www.stats.ox.ac.uk/pub/datasets/csb/ch11b.dat"))
> making socket
> Error in make.socket(host, port = port) : Socket not established
> Timing stopped at: 0 0 189.04 0 0
> 
>   The 189 second timeout appears to be part of our particular local
> setup (port 80 is blocked off-site).
> 
> options()$timeout seems to have no effect on connections that fail due
> to  non-use of our proxies. Is this a bug? The doc claims:
> 
> timeout: integer.  The timeout for Internet operations, in seconds.
>            Default 60 seconds.
> 
> which is a bit vague. If I do something like:
>   read.table("http://www.stats.ox.ac.uk/pub/datasets/csb/ch11b.dat")
> without setting my proxy I get 189 seconds, regardless of
> options()$timeout yet it is undoubtedly an internet operation.

It is several internet operations.  However, ?download.file says

     The remaining details apply to method `"internal"' only.

     The timeout for many parts of the transfer can be set by the
     option `timeout' which defaults to 60 seconds.

and note, not all parts.

> 
>   Back to the original problem: the httpget function in tests/internet.R 
> can be modified to obey an http_proxy setting with a fairly simple 
> adjustment near the start:
> 
>    hp <- Sys.getenv("http_proxy")
>    if(hp != ""){
> ## we need to extract host and port from the proxy:
> ## split on / or : thusly:
> ## [http]:[]/[]/[hostname]:[port]
> ##   1     2  3      4        5
>      hpbits <- strsplit(hp,"[/:]")[[1]]
>      host <- hpbits[4]
>      port <- hpbits[5]
> ## ask the proxy for the full URL with all its http:// glory:
>      rurl <- url
>    }else{
> ## no proxy, connect direct and ask for the URL relative to root:
>      host <- urlel[3]
>      rurl <- paste(c("", urlel[-(1:3)]), collapse = "/")
>    }

Just how general is that?  It will not cope with authenticating proxies, 
for example.  I would rather skip the test.

>   I still have some qualms about tests that rely on things external to 
> the test site, but without including a socket server program (which 
> would need its own set of tests, I suppose) I cant see any way of 
> testing socket functionality without dribbling on the internet at large.

Well, R can itself be a socket server, but we cannot presume that
there are resources to run two copies of R, for example.

-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595