[R] calculation of entropy in R???

[David Reinke]

I've usually done my own calculations of entropy. Suppose p is a vector of
values (counts of outcomes, probabilities, etc.). Here's a simple function
to compute Shannon entropy:

shannon.entropy <- function(p)
{
	if (min(p) < 0 || sum(p) <= 0)
		return(NA)
	p.norm <- p[p>0]/sum(p)
	-sum(log2(p.norm)*p.norm)
}

The p > 0 qualifier is put in to avoid taking log of zero.

It gets a bit more complicated if you're trying to find things like
conditional entropy and mutual information, e.g., H(X|Y), I(X;Y), but the
above is a basic building block for anything more complex. My approach for
getting H(X|Y), etc. is to get a contingency table, then calculate
conditional entropy and mutual information from the table. You might also
consult a text such as Cover & Thomas, Elements of Information Theory.

Regards,
David

David Reinke

Senior Transportation Engineer/Economist
Dowling Associates, Inc.
180 Grand Avenue, Suite 250
Oakland, California 94612-3774
510.839.1742 x104 (voice)
510.839.0871 (fax)
www.dowlinginc.com

-----Original Message-----
From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org] On
Behalf Of muhammad faisal
Sent: Tuesday, July 08, 2008 1:14 PM
To: R Help Cran
Subject: [R] calculation of entropy in R???


 i want to calculate shannon entropy which is 
        H1,H2,H3....upto H7?

if there is any function or any package in 
        which i can find this entropy directly. do you have any information

        please share this and i will be very thankful to you.
Regards,
++++++++++++++++++++++++++++++++++++++++++++++
MUHAMMAD FAISAL  Department of Statistics and Decion Support system, 
University of Vienna,Austria      Res. # +4314796696104 
Cell # +436503605156
Office # +431427738662
+++++++++++++++++++++++++++++++++++++++++++++++



      
	[[alternative HTML version deleted]]

______________________________________________
R-help at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.