KAPPA procedure

Calculates a kappa coefficient of agreement for nominally scaled data (A.J. Rook).

Option

PRINT = string

Whether to print kappa and its associated information (test); default test

Parameters

DATA = tables

Data sets, each consisting of an object × category table whose entries are the number of judges assigning the ith object to the jth category

STATISTIC = scalars

Save the value of kappa for each data table

VARIANCE = scalars

Save the corresponding variances

Description

The kappa coefficient provides a way of assessing the agreement between judges who have rated a set of N objects or subjects using a nominal scale: that is, each judge has allocated each object to one of M different categories. The data for KAPPA, specified by the DATA parameter, consists of an N × M table whose entries indicate the number of judges that have assigned the ith object to the jth category. This must not contain any missing values and all the row totals must be equal.

Kappa takes the value one when there is complete agreement and zero when there is none (except that expected by chance). The printing of the test statistic and its associated information is controlled by the PRINT option. With the default, test, the procedure prints the actual and expected proportion of times that the judges agree, the resulting value of kappa and its variance. When N is large, the sampling distribution of kappa is approximately Normal. The procedure thus also prints the value of kappa divided by the variance, and its probability assuming a Normal distribution. A warning is printed if N is less than 20.

The STATISTIC and VARIANCE parameters allow kappa and its variance to be saved, in scalars.

Option: PRINT. Parameters: DATA, STATISTIC, VARIANCE.

Method

The method used is that of Siegel & Castellan (1988, pages 284-291).

Reference

Siegel, S. & Castellan, N. J. (1988). Nonparametric Statistics for the Behavioural Sciences (second edition). McGraw-Hill, Singapore.