[R] RCurl: authentication when posting forms

Duncan Temple Lang duncan at wald.ucdavis.edu
Fri Aug 29 02:40:44 CEST 2008


Valerie Obenchain wrote:
> Duncan,
> 
> Thank you for the examples.  I had tried all of these different options 
> for authentication but had no luck. I was getting a "100 continue" and 
> then a "401 unauthorized" response.  This morning the owners of the 
> server I was trying to access discovered a bug with their api when using 
> basic authorization. Evidently the code on their side was explicitly 
> checking that the request method was GET, hence why all of my POST 
> attempts were failing. They are in the process of fixing that code and 
> my guess is that when I am able to try again the RCurl post functions 
> will work just fine.

Thanks for the update.  Good to know that there isn't a problem
with the libcurl/RCurl code.

> 
> I did have one other RCurl question I'd like to ask you about the 
> parseHTTPHeader function.
> 
> The parser appears to parse on spaces, so when the error message is more 
> than one word (eg, "not found"), the message returned is "not". I have 
> modified the parseHTTPHeader function so that it works for me. I may not 
> have done this in the most efficient way but at least you can see what I 
> was trying to do.


Yes, this is a problem. Thanks for bringing it to my attention.
I did get your mail directly to me from about 2 weeks ago 
and I replied. Perhaps my reply got eaten by your filters.
In it, I said that it was a bug, and that it would be
very useful if you could send me an RCurl command
that illustrated the error so that I could add it to the RCurl
tests.
 

The only modification that can make your patch slightly
"better" is that the words in the statusMessage are essentially
all the words in the status variable, less the first two
(the status number and the protocol name). So

 header[["statusMessage"]] <- paste(els[-c(1,2)], collapse = " ")

is more convenient than

  hstring <- NULL
  for(i in 3:length(els)) hstring <- paste(hstring," ",els[i],sep="")
  hstring <- substr(hstring,2,nchar(hstring))
  header[["statusMessage"]] <- hstring


If you do have a URL that I can use to test the error handling code, I'd appreciate
if you could send it to me.  


 D.

> 
> Below I have pasted my modification for the parseHTTPHeader function 
> which I am calling parseHeader. Please let me know if you think this is 
> a bug or if I am just using the function incorrectly.
> 
> Thank you for the help.
> Valerie
> 
> #---------------------------------------------------
> #Sample code:
> reader <- basicTextGatherer()
> header <- basicTextGatherer()
> handle <- getCurlHandle()
> myopts <- curlOptions(  netrc=1, httpheader=c(Authorization=mypwd, 
> Accept="test/xml",
>                                       Accept="multipart/*", 
> 'Content-Type'="text/xml; charset=utf-8"),
>                                       postfields=body, 
> writefunction=reader$update, headerfunction=header$update,
>                                       ssl.verifyhost=FALSE, 
> ssl.verifypeer=FALSE, followlocation=TRUE)} else
> 
> curlPerform(url=myUrl, .opts=myopts, curl=handle)
> h <- parseHeader(header$value())
> status <- h$status
> message <- h$statusMessage
> #----------------------------------------------------
> 
> #Modified parse function:
> parseHeader <- function (lines)
> {
>   if (length(lines) < 1)
>       return(NULL)
>   if (length(lines) == 1)
>       lines = strsplit(lines, "\r\n")[[1]]
>   status = lines[1]
>   lines = lines[-c(1, length(lines))]
>   lines = gsub("\r\n", "", lines)
>   if (FALSE) {
>       header = lines[-1]
>       header <- read.dcf(textConnection(header))
>   }
>   else {
>       els <- sapply(lines, function(x) strsplit(x, ":[ ]*"))
>       header <- lapply(els, function(x) x[2])
>       names(header) <- sapply(els, function(x) x[1])
>   }
>   els <- strsplit(status, " ")[[1]]
>   header[["status"]] <- as.integer(els[2])
> # new code below
>   hstring <- NULL
>   for(i in 3:length(els)) hstring <- paste(hstring," ",els[i],sep="")
>   hstring <- substr(hstring,2,nchar(hstring))
>   header[["statusMessage"]] <- hstring
>   header
> }
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Duncan Temple Lang wrote:
> >
> >
> >Hi Valerie
> >
> >Valerie Obenchain wrote:
> >>Hi,
> >>
> >>Has anyone successfully used RCurl for posting data to a 
> >>password-protected site? 
> >
> >Yes. I just set up a sample form to test with and the following
> >all work
> >
> >
> ># Perl script (and HTML form for testing in the browser) taken from
> >#   http://www.elated.com/articles/form-validation-with-perl-and-cgi/
> >
> >
> ># Provide the login & password directly
> >postForm("http://www.omegahat.org/RCurl/testPassword/form_validation.cgi", 
> >
> >           your_name = "Duncan",
> >           your_age = "35-55",
> >           your_sex = "m",
> >           submit = "submit",
> >           .opts = list(userpwd = "bob:welcome"))
> >
> ># Get the login & password in ~/.netrc
> >postForm("http://www.omegahat.org/RCurl/testPassword/form_validation.cgi", 
> >
> >           your_name = "Duncan",
> >           your_age = "35-55",
> >           your_sex = "m",
> >           submit = "submit",
> >          .opts = list(netrc = TRUE))
> >
> ># Get the login & password from a different netrc file
> >
> >postForm("http://www.omegahat.org/RCurl/testPassword/form_validation.cgi", 
> >
> >           your_name = "Duncan",
> >           your_age = "35-55",
> >           your_sex = "m",
> >           submit = "submit",
> >           .opts = list(netrc = TRUE,
> >                        netrc.file = 
> >"/Users/duncan/Projects/org/omegahat/R/RCurl/inst/examples/omg.netrc"))
> >
> >
> >So let me know what problems you are having and more details
> >about the OS, version of libcurl, and a sample URL to which
> >to post, etc.
> >
> >  D.
> >
> >>I
> >>have tired using option netrc=1 with both postForm and curlPerform 
> >>(with postfields option) but can't authenticate.
> >>I would happily provide more details if some one has had some 
> >>experience with this.
> >>
> >>Thanks very much.
> >>Valerie
> >>
> >>______________________________________________
> >>R-help at r-project.org mailing list
> >>https://stat.ethz.ch/mailman/listinfo/r-help
> >>PLEASE do read the posting guide 
> >>http://www.R-project.org/posting-guide.html
> >>and provide commented, minimal, self-contained, reproducible code.

-- 
"There are men who can think no deeper than a fact" - Voltaire


Duncan Temple Lang                duncan at wald.ucdavis.edu
Department of Statistics          work:  (530) 752-4782
4210 Mathematical Sciences Bldg.  fax:   (530) 752-7099
One Shields Ave.
University of California at Davis
Davis, CA 95616, USA



-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <https://stat.ethz.ch/pipermail/r-help/attachments/20080828/d6b2d1fc/attachment.bin>


More information about the R-help mailing list