[R] Why is removeSparseTerms() not doing anything?

[John Kane]

Reproducibility
http://adv-r.had.co.nz/Reproducibility.html  
 http://stackoverflow.com/questions/5963269/how-to-make-a-great-r-reproducible-example


John Kane
Kingston ON Canada


> -----Original Message-----
> From: mikehall at y7mail.com
> Sent: Wed, 22 Apr 2015 18:52:45 +0000 (UTC)
> To: r-help at r-project.org
> Subject: [R] Why is removeSparseTerms() not doing anything?
> 
> Here's the code and results.  The corpus is the text version of a single
> book.   (r vs. 3.2)
>> docs <- tm_map(docs, stemDocument)
>> dtm <- DocumentTermMatrix(docs)
>> freq <- colSums(as.matrix(dtm))
>> ord <- order(freq)
>> freq[tail(ord)]
> one experi   will   can lucid dream
> 287   312   363   452   1018   2413
>> freq[head(ord)]
> abbey abdomin   abdu abraham absent   abus
>   1       1       1       1       1       1
>> dim(dtm)
> [1]   1 5265
>> dtms <- removeSparseTerms(dtm, 0.1)
>> dim(dtms)
> [1]   1 5265
>> dtms <- removeSparseTerms(dtm, 0.001)
>> dim(dtms)
> [1]   1 5265
>> dtms <- removeSparseTerms(dtm, 0.9)
>> dim(dtms)
> [1]   1 5265
>> 
> 
> 	[[alternative HTML version deleted]]
> 
> ______________________________________________
> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

____________________________________________________________
Can't remember your password? Do you need a strong and secure password?
Use Password manager! It stores your passwords & protects your account.