Large sample variance of kappa in the case of different sets of raters.
JL Fleiss, JC Nee, JR Landis - Psychological bulletin, 1979 - psycnet.apa.org
JL Fleiss, JC Nee, JR Landis
Psychological bulletin, 1979•psycnet.apa.orgPublished formulas for the large sample variance of the kappa statistic that are appropriate
for the case of different sets of raters for different Ss, when each set of raters is selected at
random from a larger pool of available raters, are determined to be incorrect. New formulas
are derived and checked by Monte Carlo simulation. Kappa is shown to be identical, except
for terms that go to zero as the number of Ss increases, to the intraclass correlation
coefficient resulting from applying a one-way ANOVA to the data.(10 ref)(PsycINFO …
for the case of different sets of raters for different Ss, when each set of raters is selected at
random from a larger pool of available raters, are determined to be incorrect. New formulas
are derived and checked by Monte Carlo simulation. Kappa is shown to be identical, except
for terms that go to zero as the number of Ss increases, to the intraclass correlation
coefficient resulting from applying a one-way ANOVA to the data.(10 ref)(PsycINFO …
Abstract
Published formulas for the large sample variance of the kappa statistic that are appropriate for the case of different sets of raters for different Ss, when each set of raters is selected at random from a larger pool of available raters, are determined to be incorrect. New formulas are derived and checked by Monte Carlo simulation. Kappa is shown to be identical, except for terms that go to zero as the number of Ss increases, to the intraclass correlation coefficient resulting from applying a one-way ANOVA to the data.(10 ref)(PsycINFO Database Record (c) 2016 APA, all rights reserved)
American Psychological Association