AbstractsMathematics

Measurement system analysis for binary tests

by Akkerhuis TS




Institution: Universiteit van Amsterdam
Department:
Year: 2016
Posted: 02/05/2017
Record ID: 2065575
Full text PDF: http://hdl.handle.net/11245/1.540065


Abstract

Binary tests classify items into two categories such as reject/accept, positive/negative or guilty/innocent. A binary test’s proneness to measurement error is usually expressed in terms of the misclassification probabilities FAP (false acceptance probability) and FRP (false rejection probability). The thesis studies the estimation of these probabilities under the common complication of an unavailable gold standard or reference standard, meaning that the 'true value' of an item cannot be obtained. Various methods based on latent variable modeling have been proposed to deal with this situation. However, there is sufficient reason to doubt the reliability of these methods. In this thesis, the reliability of these methods is put to the test. A mathematical exploration provides formal proof that, without a gold standard, FAP and FRP can only be estimated with unrealistically strict assumptions. A simulation study shows that small deviations from these strict assumptions are likely to lead to large biases in estimates of FAP and FRP. Moreover, this thesis shows that binary measurement error can be decomposed in a systematic and a random component, and that only the latter can be estimated without a gold standard. This is well known for numerical measurements, but a new insight for the non-numerical case. The random component of binary measurement error can be quantified by means of the probabilities of inconsistent classification IAP and IRP. A robust method of estimating these quantities is provided and evaluated in a simulation study. The final chapter generalizes the findings to nominal scales with more than two classes.