Prof. Alastair Young (Imperial College) Sampling Distributions of Likelihood-based p-values

We consider the problem of inference for a scalar interest parameter in the presence of a nuisance
parameter, using a likelihood-based statistic which is asymptotically normally distributed under the
null hypothesis. Two approaches to calculation of an approximate p-value are: analytic methods
based on normal approximation to an adjusted form of statistic; simulation (`bootstrap')
approximation to the null sampling distribution of the statistic. Higher-order expansions are used to
compare the sampling distributions, under a general contiguous alternative hypothesis, of p-values
calculated by these different approaches. We establish that comparisons in terms of power under an
alternative hypothesis are intrinsically linked to the extent to which testing procedures are
conservative or anti-conservative under the null. Empirical examples are discussed which
demonstrate that higher-order asymptotic effects may be clearly seen in small sample contexts.
This is joint work with Stephen Lee (University of Hong Kong).