[R] Cohen's kappa, unequal score ranges

[Julia Myatt]

Hi Jim,

Thanks for your help, I tried that function which did result in an output. The only issue now is my summary looks like this:

      Length Class  Mode   
table 16     -none- numeric
kappa  1     -none- numeric

Not what I was expecting, this was the case when my data was in the following format:

  ob1 ob2
1   3   4
2   4   3
3   2   1
4   2   1
5   5   1
6   2   4

I then tried entering my data as a matrix like the one below (all this data is made up until I get it to work!):

 p1 p2 p3 p4 p5
p1  3  1  0  4  2
p2  3  5  0  7  1
p3  2  3  4  6  3
p4  4  1  0  4  0
p5  8  2  0  2  5

and this resulted in the same type of output:

 Length Class  Mode   
table 36     -none- numeric
kappa  1     -none- numeric

If that is the kappa output (1) something isn't right as I purposely made the data without good interrater reliability! I was wondering if you have anymore tips as to what I am doing wrong? I'm afraid I'm very new to R so don't even know the basics!!

Thanks for all your help,

Julia.


________________________________________
From: Jim Lemon [jim at bitwrit.com.au]
Sent: 23 December 2009 02:15
To: Julia Myatt
Cc: r-help at r-project.org
Subject: Re: [R] Cohen's kappa, unequal score ranges

On 12/23/2009 01:58 AM, Julia Myatt wrote:
> Hi,
>
> I am having problems getting cohen's kappa to work. I have been using the function:
>
>
>> <-ckappa(x,y)
>>
> from the psy package.
>
> I am trying to test for inter-observer reliability, I have 2 observers and 26 categories, however, the two observers might not necessarily have the same range of categories (I have unequal score ranges). However, I thought R could cope with this.
>
> Each time I try the function above I get the following output:
>
>
>> Error in ckappa(ob1, ob2) : unused argument(s) (ob2)
>>
> I got this even when I altered the data to have equal score ranges, therefore I am not sure what this means or what I should do!!
>
> Any help in this area would be much appreciated, or anything about the best way to deal with inter-observer reliability (my data is all categorical),
>
>
Hi Julia,
The ckappa function only takes one argument, a rater by object matrix or
dataframe of scores. Try:

ckappa(cbind(ob1,ob2))

Jim