Inter-rater Reliability of Ward Rating Scales

¹ M.R.C. Research Fellow, University Department of Psychiatry, 15 Hyde Terrace, Leeds, LS2 9LT

1. Rating scales are useful devices for assessing patients, but their reliability is partly dependent on the situation in which they are used. When used in a ward situation, as opposed to an interview situation, several factors act together to reduce inter-rater reliability.

2. When choosing a method of calculating reliability, it may be necessary to assume that: (a) scores are distributed normally; (b) agreement is meaningful; (c) chance agreement is negligible; (d) total scores are meaningful; (e) mean scores of both raters are similar.

3. If these assumptions are not valid, the statistic weighted kappa may be applied. With a particular method of weighting it may be applied to any rating scale. The statistic is simple to calculate, and its variances may be calculated by a multiple-step operation.

4. Some comments are made on how to improve the reliability of ward rating scales, with particular reference to the construction of items of the scales.

This article has been cited by other articles:

				K. Rasheed, Y. S Rabinowitz, D. Remba, and M. J Remba Interobserver and intraobserver reliability of a classification scheme for corneal topographic patterns Br. J. Ophthalmol., December 1, 1998; 82(12): 1401 - 1406. [Abstract] [Full Text]

				D. V. Cicchetti, C. Lee, A. F. Fontana, and B. N. Dowds A Computer Program for Assessing Specific Category Rater Agreement for Qualitative Data Educational and Psychological Measurement, October 1, 1978; 38(3): 805 - 813. [Abstract]

				D. V. Cicchetti, S. L. Aivano, and J. Vitale Computer Programs for Assessing Rater Agreement and Rater Bias for Qualitative Data Educational and Psychological Measurement, April 1, 1977; 37(1): 195 - 201. [Abstract]