[R] Cohen's Kappa for beginners

[Peter Ehlers]

cohen.kappa(cbind(x,y)) works for me.

  -Peter Ehlers

On 2010-05-25 7:42, Scot W. McNary wrote:
> Hi,
>
> It doesn't seem happy with non-numeric data in the data frame version.
> Maybe a recode would work?
>
>  > x1 <- c(1, 2, 3, 1)
>  > y1 <- c(1, 3, 3, 1)
>  > cohen.kappa(data.frame(x = x1, y = y1))
> Call: cohen.kappa1(x = x, w = w, n.obs = n.obs, alpha = alpha)
>
> Cohen Kappa and Weighted Kappa correlation coefficients and confidence
> boundaries
> lower estimate upper
> unweighted kappa 0.098 0.60 1.1
> weighted kappa 0.601 0.86 1.1
>
> The unweighted kappa is .60.
>
> I also tried to create the p x p table version, thinking that it would
> be unhappy with less than a p x p table. The example dataset produces a
> 3 x 2 table, so this pads it out to a 3 x 3:
>
>  > x <- factor(x)
>  > y <- factor(y, levels = c("blue", "red", "yellow"))
>  > table(x,y)
> y
> x blue red yellow
> blue 1 0 0
> red 0 2 0
> yellow 1 0 0
>  > cohen.kappa(table(x,y))
> Call: cohen.kappa1(x = x, w = w, n.obs = n.obs, alpha = alpha)
>
> Cohen Kappa and Weighted Kappa correlation coefficients and confidence
> boundaries
> lower estimate upper
> unweighted kappa 0.098 0.6 1.10
> weighted kappa -0.693 0.0 0.69
>
> Number of subjects = 4>
>
> The unweighted kappa is the same as above and as found by Jim with other
> packages.
>
> Scot
>
> On 5/25/2010 7:31 AM, Jim Lemon wrote:
>> On 05/25/2010 06:01 PM, Jason Priem wrote:
>>> Hi,
>>> I've got two vectors with ratings from two coders, like this:
>>>
>>> x<-c("red", "yellow", "blue", "red") #coder number 1
>>> y<-c("red", "blue", "blue", "red") #coder number 2
>>>
>>> I want to find Cohen's Kappa using the wkappa function in the psych
>>> package. The only example in the docs is using a matrix, which I'm
>>> afraid I don't understand--I don't know how to get there from what I've
>>> got. Could anyone point me in the right direction? Thanks!
>>
>> Hi Jason,
>> You're not alone. I tried to work out how to run this and it took a
>> while. Both kappa2 in the "irr" package:
>>
>> kappa2(cbind(x,y))
>>
>> and classAgreement in the "e1071" package:
>>
>> classAgreement(table(x,y))
>>
>> produce a kappa of 0.6. I was unable to work out how to use the wkappa
>> function in the "psych" package:
>>
>> wkappa(table(x,y))
>>
>> The above led to a kappa of -0.8. Although wkappa is presented as a
>> test of the reliability of ratings of nominal level data, the example
>> uses numeric data.
>>
>> Jim
>>