The calculation of the probability of correct selection (PCS) shows how likely it is
that the populations chosen as “best” truly are the top populations, according
to a well-defined standard. PCS is useful for the researcher with limited
resources or the statistician attempting to test the quality of two different
statistics. This paper explores the theory behind two selection goals for PCS,
-best and
-best,
and how they improve previous definitions of PCS for massive datasets. This paper
also calculates PCS for two applications that have already been analyzed by multiple
testing procedures in the literature. The two applications are in neuroimaging and
econometrics. It is shown through these applications that PCS not only supports the
multiple testing conclusions but also provides further information about the statistics
used.
Keywords
probability of correct selection (PCS), $d$-best, $G$-best,
ranking and selection, neuroimaging, econometrics