AGREE - software description

AGREEment on nominal data is a computer program developed for research situations where one or more judges classify objects into nominal scale categories. It allows the calculation of measures of agreement (Cohen's kappa or the D2 measure) between nominal scale judgements and provides several options. AGREE is a program that has been developed by Roel Popping from the University of Groningen.

In many fields of scientific and practical research, objects are being divided into classes according to their properties by one or more judges. For example, applicants for a position may be qualified or not, patients may be divided into healthy and unhealthy, proposals may be divided into workable and unworkable, etc.

Agreement is very often considered as a special kind of association. There are differences however. It is important to determine the similarity of the content of behavior (in a broad sense) between coders in general with the degree of identity of this behavior. The behavior of one coder does not have to be predicted from that of the other. In the case of association one investigates the strength of the linear relationship between variables. Here the goal is to predict the values of one variable from those of the other. With regard to agree-ment, most important is the similarity of the content of behavior between rates, with the goal of determining the degree of identity of this behavior. The basic idea of an agreement index is looking at the fraction of obser-vations on which rates agree.

Very often the coders have to assign the objects to a variable having a nominal scale. In the situation where a measure of nominal scale agreement is to be computed, the choice of the agreement index is determined by whether or not the coders knew the response categories a priori. When the set of categories is known, this set is identical for all coders, and all objects would be assigned to one of the categories. Another situation is when the response categories have to be developed by the coders during the assigning process. In this situation each coder may finish with a different set and number of categories. This situation arises most in pilot studies, where the investigator wishes to find a set of response categories that will be used in the main investigation. In the first situation the agreement index kappa performs best, while in the second situation the index D2 is to be preferred. Extensions of these indices are available for nearly all research situations that are realistic.

The computer program AGREE 7 can be used in computing the two indices kappa and D2 in many research situations.

Ordering

For price information and ordering, please visit the Prices and Ordering Page