Paired comparison model

The basic paired comparison model was presented by R.A. Bradley and M.E. Terry in [a1] (cf. also Bradley–Terry model). The paired comparison experiment has $ t $ objects $ T _ {1} \dots T _ {t} $ with $ n _ {ij } $ comparisons of $ T _ {i} $ and $ T _ {j} $, $ n _ {ij } \geq 0 $, $ n _ {ii } = 0 $, $ n _ {ji } = n _ {ij } $, $ i,j = 1 \dots t $. The model postulates the existence of treatment parameters $ \pi _ {i} $ for $ T _ {i} $( $ \pi _ {i} \geq 0 $), $ \sum _ {i} \pi _ {i} = 1 $, such that the probability $ \pi _ {i.ij } $ of selecting $ T _ {i} $, when compared with $ T _ {j} $, is equal to $ { {\pi _ {i} } / {( \pi _ {i} + \pi _ {j} ) } } $. A model for incorporating ties has been introduced by P.V. Rao and L.L. Kupper in [a2] and by R.R. Davidson in [a3].

Rao and Kupper introduced a threshold parameter, $ \eta _ {0} \geq 0 $, which is interpreted as the threshold of sensory perception for the judge. They model the probabilities of preference $ \pi _ {i.ij } $ and of no preference $ \pi _ {0.ij } $ as

$$ \pi _ {i.ij } = { \frac{\pi _ {i} }{( \pi _ {i} + \theta \pi _ {j} ) } } $$

and

$$ \pi _ {0.ij } = { \frac{\pi _ {i} \pi _ {j} ( \theta ^ {2} - 1 ) }{( \pi _ {i} + \theta \pi _ {j} ) ( \pi _ {j} + \theta \pi _ {i} ) } } , $$

where $ \theta = e ^ {\eta _ {0} } $. For $ \eta _ {0} = 0 $, the Rao–Kupper model coincides with the Bradley–Terry model. In [a4], R.J. Beaver and D.V. Gokhale have generalized the model to incorporate within-pair order effects. They assume the existence of parameters $ \delta _ {ij } $, $ i,j = 1 \dots t $, $ \delta _ {ij } = \delta _ {ji } $, associated with the pair $ ( i,j ) $, such that the preference probabilities for the ordered pair $ ( i,j ) $ are

$$ \pi _ {i,ij } = { \frac{\pi _ {i} + \delta _ {ij } }{\pi _ {i} + \pi _ {j} } } , $$

$$ \pi _ {j.ij } = { \frac{\pi _ {i} - \delta _ {ij } }{\pi _ {i} + \pi _ {j} } } . $$

Thurstone's model (see [a5]) assumes that a person receives in response to $ T _ {i} $ a sensation $ X _ {i} $ which is normally distributed with location $ \mu _ {i} $ on a subjective continuum (cf. also Normal distribution). A special case gives that the probability that $ T _ {i} $ is preferred to $ T _ {j} $ is equal to

$$ {\mathsf P} ( T _ {i} > T _ {j} ) = {\mathsf P} ( X _ {i} > X _ {j} ) = { \frac{1}{\sqrt {2 \pi } } } \int\limits _ {- ( \mu _ {i} - \mu _ {j} ) } ^ \infty {e ^ {- { {y ^ {2} } / 2 } } } {dy } . $$

If the normal density function is replaced by the logistic density function, the model is equal to the Bradley–Terry model with $ \mu _ {i} = { \mathop{\rm ln} } \pi _ {i} $. H. Stern has considered, [a6], models for paired comparison experiments based on comparison of gamma random variables. Different values of the shape parameter yield different models, including the Bradley–Terry model and the Thurstone model. Likelihood methods can be used to estimate the parameters of the models. The likelihood equations must be solved with iterative methods.

It is also possible to fit response surfaces in paired comparison experiments (see, e.g., [a7], [a8]). Mostly it is assumed that the parameters $ \pi _ {i} $, $ i = 1 \dots t $, are functions of continuous variables $ x _ {1} \dots x _ {s} $ such that the formulated model is linear in the unknown parameters $ \beta _ {j} $. If such a model is formulated, then it is possible to discuss the question of optimal design in paired comparison experiments. Many criteria for optimal design depend on the variance-covariance matrix of the estimators for the unknown parameters $ \beta _ {j} $. However, the asymptotic variance-covariance matrix itself depends on the unknown parameters (see, e.g., [a7], [a8]). A. Springall has defined, [a7], so-called analogue designs. These are designs in which the elements of the paired comparison variance-covariance matrix are proportional to the elements of the classical response surface variance-covariance matrix with the same design points.

In order to find designs, E.E.M. van Berkum has assumed, [a8], that the parameters $ \beta _ {j} $ are all equal. In that case the variance-covariance matrix is proportional to the variance-covariance matrix for the estimators in an ordinary linear model and general optimal design theory can be applied ( $ D $- optimality, $ G $- optimality, equivalence theorem). He also gives optimal designs for various factorial models.

There is much literature on paired comparison experiments and related topics such as generalized linear models, log-linear models, weighted least squares and non-parametric methods. A bibliography up to 1976 is given in [a9]. The state of the art as of 1976 is given in [a10], and as of 1992 in [a11].

References

[a1]	R.A. Bradley, M.E. Terry, "The rank analysis of incomplete block designs. I. The method of paired comparisons" Biometrika , 39 (1952) pp. 324–345
[a2]	P.V. Rao, L.L. Kupper, "Ties in paired-comparison experiments: A generalization of the Bradley–Terry model" J. Amer. Statist. Assoc. , 62 (1967) pp. 194–204
[a3]	R.R. Davidson, "On extending the Bradley–Terry model to accommodate ties in paired comparison experiments" J. Amer. Statist. Assoc. , 65 (1970) pp. 317–328
[a4]	R.J. Beaver, D.V. Gokhale, "A model to incorporate within-pair order effects in paired comparisons" Commun. in Statist. , 4 (1975) pp. 923–929
[a5]	L.L. Thurstone, "Psychophysical analysis" Amer. J. Psychol. , 38 (1927) pp. 368–389
[a6]	H. Stern, "A continuum of paired comparison models" Biometrika , 77 (1990) pp. 265–273
[a7]	A. Springall, "Response surface fitting using a generalization of the Bradley–Terry paired comparison model" Appl. Statist. , 22 (1973) pp. 59–68
[a8]	E.E.M. Van Berkum, "Optimal paired comparison designs for factorial experiments" , CWI Tracts , 31 , CWI , Amsterdam (1987)
[a9]	R.R. Davidson, P.H. Farquhar, "A bibliography on the method of paired comparisons" Biometrika , 32 (1976) pp. 241–252
[a10]	R.A. Bradley, "Science, statistics and paired comparisons" Biometrics , 32 (1976) pp. 213–232
[a11]	H.A. David, "Ranking and selection from paired-comparison data. With discussion" , The Frontiers of Modern Statistical Inference Procedures II (Sydney, 1987) , Math. Management Sci. , 28 , Amer. Sci. Press (1992) pp. 3–24