A Modified Weighted Kappa for the Comparison of Different Measurement Scales

This paper investigates possible causes of the discordance between two ordinal\r\nscales evaluating the same good or service. In psychometric literature, Cohen’s\r\nKappa is one of the most important index to evaluate the strength of agreement, or\r\ndisagreement, between two nominal variables, in particular in its weighted version.\r\nIn this paper a Modified Weighted Kappa is proposed. A proper procedure to determine\r\nthe lower and upper triangle in a non-square table is also implemented, so to\r\ngeneralize the index in order to compare two scales with different number of categories.\r\nWith the aim to verify the tendency of a scale to have a mainly positive or\r\nnegative rating compared to a different one, a parametric test is set up. A study with\r\nreal data is also conducted.