[R] RCurl and postForm()

[Duncan Temple Lang]

Hi Ryan

 postForm() is using a different style (or specifically Content-Type) of submitting the form than the curl -d command.
Switching the style = 'POST' uses the same type, but at a quick guess, the parameter name 'a' is causing confusion
and the result is the empty JSON array - "[]".

A quick workaround is to use curlPerform() directly rather than postForm()

 r = dynCurlReader()
 curlPerform(postfields = 'Archbishop Huxley', url = 'http://www.datasciencetoolkit.org/text2people', verbose = TRUE,
              post = 1L, writefunction = r$update)
 r$value()

This yields

[1]
"[{\"gender\":\"u\",\"first_name\":\"\",\"title\":\"archbishop\",\"surnames\":\"Huxley\",\"start_index\":0,\"end_index\":17,\"matched_string\":\"Archbishop
Huxley\"}]"

and you can use fromJSON() to transform it into data in R.

  D.

On 4/29/11 12:14 PM, Elmore, Ryan wrote:
> Hi everybody,
> 
> I think that I am missing something fundamental in how strings are passed from a postForm() call in R to the curl or libcurl functions underneath.  For example, I can do the following using curl from the command line:
> 
> $ curl -d "Archbishop Huxley" "http://www.datasciencetoolkit.org/text2people"
> [{"gender":"u","first_name":"","title":"archbishop","surnames":"Huxley","start_index":0,"end_index":17,"matched_string":"Archbishop Huxley"}]
> 
> Trying the same thing, or what I *think* is the same thing (obvious not) in R (Mac OS 10.6.7, R 2.13.0) produces:
> 
>> library(RCurl)
> Loading required package: bitops
>> api <- "http://www.datasciencetoolkit.org/text2people"
>> postForm(api, a="Archbishop Huxley")
> [1] "[{\"gender\":\"u\",\"first_name\":\"\",\"title\":\"archbishop\",\"surnames\":\"Huxley\",\"start_index\":44,\"end_index\":61,\"matched_string\":\"Archbishop Huxley\"},{\"gender\":\"u\",\"first_name\":\"\",\"title\":\"archbishop\",\"surnames\":\"Huxley\",\"start_index\":88,\"end_index\":105,\"matched_string\":\"Archbishop Huxley\"}]"
> attr(,"Content-Type")
>                 charset
> "text/html"     "utf-8"
> 
> I can match the result given on the DSTK API's website by using system(), but doesn't seem like the R-like way of doing something.
> 
>> system("curl -d 'Archbishop Huxley' 'http://www.datasciencetoolkit.org/text2people'")
> 158   141  141   141    0[{"gender":"u","first_name":"","title":"archbishop","surnames":"Huxley","start_index":0,"end_index":17,"matched_string":"Archbishop Huxley"}]    17    599     72 --:--:-- --:--:-- --:--:--   670
> 
> If you want to see some additional information related to this question, I posted on StackOverflow a few days ago:
> http://stackoverflow.com/questions/5797688/post-request-using-rcurl
> 
> I am working on this R wrapper for the data science toolkit as a way of illustrating how to make an R package for the Denver RUG and ran into this problem.  Any help to this problem will be greatly appreciated by the Denver RUG!
> 
> Cheers,
> Ryan
> 
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.