[R] reliability, scale scores in the psych package

[Ista Zahn]

Dear Professor Revelle and R-helpers,
This is a two-part question: the first part is general, and the second
is specific to the psych package.

First question: In the past I've constructed composite variables from
questionnaire data by using rowMeans(), and then correlating items
with the scale using cor() as an informal check for "bad" items. Over
the weekend I decided to take a look at some of the packages in the
psychometric task view, to see if there was a way to simplify this
process. I looked at several packages, including psychometric, CTT,
and psych. I'm interested in hearing from others who need to do this
kind of thing frequently. What approach do you use? Do you use one of
the packages mentioned above? Are there other packages I might want to
take a look at?

Second question: I spent some time with the psych package trying to
figure out how to use the score.items() function, and it's become
clear to me that I don't understand what it's doing. I assumed that
setting a key equal to -1 would result in the item being reverse
scored, but I get weird results, as shown below. When I try to reverse
score (by setting a value of -1 in the key), I get scale scores that
don't add up (e.g., the mean score is reported as being larger than
the maximum item score). How is the score.items() function intended to
be used? Do I need to reverse score items before using score.items()?

Thanks,
Ista

########## score.items() example begins here ##########
> library(psych)
> Data.score <- as.data.frame(matrix(c(40,29,40,32,1,1,3,1,5,3,3,44,24,47,31,4,4,1,1,4,2,1,13,5,14,5,5,4,3,3,4,4,3,7,2,2,0,5,4,2,2,4,4,4,7,6,5,4,1,1,3,4,3,2,1,18,15,21,8,6,6,1,1,6,6,6,9,10,15,7,5,4,2,1,5,5,5,10,7,12,6,2,2,4,4,3,3,3,8,7,13,8,1,1,4,2,2,2,1,10,5,13,7,4,3,3,3,3,3,3), nrow=10, byrow=TRUE))
> names(Data.score) <- c("s1","s2","s3","s4","imi1","imi2","imi3","imi4","imi5","imi6","imi7")
> Data.score
   s1 s2 s3 s4 imi1 imi2 imi3 imi4 imi5 imi6 imi7
1  40 29 40 32    1    1    3    1    5    3    3
2  44 24 47 31    4    4    1    1    4    2    1
3  13  5 14  5    5    4    3    3    4    4    3
4   7  2  2  0    5    4    2    2    4    4    4
5   7  6  5  4    1    1    3    4    3    2    1
6  18 15 21  8    6    6    1    1    6    6    6
7   9 10 15  7    5    4    2    1    5    5    5
8  10  7 12  6    2    2    4    4    3    3    3
9   8  7 13  8    1    1    4    2    2    2    1
10 10  5 13  7    4    3    3    3    3    3    3
>
> #This works fine
> key.list <- list(silence=1:4, interest=5:11)
> keys <- make.keys(length(names(Data.score)), key.list, item.labels=names(Data.score))
> scored <- score.items(keys, Data.score, missing=FALSE, totals=FALSE)
> scored$scores
      silence interest
 [1,]   35.25 2.428571
 [2,]   36.50 2.428571
 [3,]    9.25 3.714286
 [4,]    2.75 3.571429
 [5,]    5.50 2.142857
 [6,]   15.50 4.571429
 [7,]   10.25 3.857143
 [8,]    8.75 3.000000
 [9,]    9.00 1.857143
[10,]    8.75 3.142857
>
> #This does not do what I expected. Mean interest scores are higher than score of the highest item.
> key.list2 <- list(silence=1:4, interest=c(5,6,-7,-8,9,10,11))
> keys2 <- make.keys(length(names(Data.score)), key.list2, item.labels=names(Data.score))
> scored2 <- score.items(keys2, Data.score, missing=FALSE, totals=FALSE)
> scored2$scores
      silence interest
 [1,]   35.25 14.71429
 [2,]   36.50 15.28571
 [3,]    9.25 15.42857
 [4,]    2.75 15.85714
 [5,]    5.50 13.57143
 [6,]   15.50 17.42857
 [7,]   10.25 16.42857
 [8,]    8.75 14.14286
 [9,]    9.00 13.57143
[10,]    8.75 14.85714