[R] Chinese characters encoding problem with XML

Wind windspeedo at qq.com
Wed Dec 31 02:29:02 CET 2008

XML is a good tool reading data from web within R.  But I wonder how could get the encoding correctly.

url <- 'http://www.szitic.com/docc/jz-lmzq.html'
xml <- htmlTreeParse(url, useInternal=TRUE)
q <- "//tbody/tr/td"
dat <- unlist(xpathApply(xml, q, xmlValue))
df <- as.data.frame(t(matrix(dat, 4)))

The first column of df is dates in Chinese.   dt is one of the Chinese dates.
When I copied the content of dt into the email, it become the following:
> dt
[1] "2008&#229;Gí¸‘Ù&#13312;&#12544;&#12800;¤Zöd\x8825&#230;¸Úí¸ã¸&#13568;&#8704;&#3328;&#2560;&#15872;&#8192;

Indeed in R,  it looks like:
[1] "2008\345\271\xb412\346\234\x8825\346\227\xa5"

and the color of the numbers differs a little.

> getOption("encoding")
[1] "native.enc"
> Sys.getlocale()
[1] "LC_COLLATE=Chinese (Simplified)_People's Republic of China.936;LC_CTYPE=Chinese (Simplified)_People's Republic of China.936;LC_MONETARY=Chinese (Simplified)_People's Republic of China.936;LC_NUMERIC=C;LC_TIME=Chinese (Simplified)_People's Republic of China.936"

Package:              XML
Version:              1.98-1
Date:                 2008/10/17

R version 2.8.0 (2008-10-20)
Windows Vista Basic, Simplified Chinese edition.

There is no problem using Chinese characters in R codes.

I wonder how could get the Chinese characters with XML.   Or is there any methods which could help me convert the encoding of characters from UTF-8 to unicode in R?



More information about the R-help mailing list