Randomization tests in many ways are the most basic statistical test. A randomization procedure tests for the likelihood of a given type of pattern to appear in a data set, versus the null hypothesis, which states that the observed pattern has appeared purely by chance in a random set of observations. A randomization test seeks to determine whether the null hypothesis is reasonable in a given data set. For such a test, a test statistic St is determined that quantifies an observed pattern (e.g. a correlation coefficient). The observed value of St is then compared to the distribution of St that is obtained, when the data set is reorganized at random. If the null hypothesis is true, then all possible values of St are equally likely to occur. The significance of St can be calculated as the proportion of values equally or more extreme than the observed value of St.
Randomization has two distinct advantages, and two distinct disadvantages:
Randomization delivers valid significance levels without the random sampling from a larger data set required for the application of 'conventional' statistics, and as a direct consequence randomization procedures are largely exempt from any restrictions that apply to conventional parametric statistics in terms of distributions.
However, and for the same reason, results from randomization tests cannot directly be extrapolated to a larger, sampled data set; results initially only apply within a complete data set. In addition, small data sets do not directly lend themselves to the calculation of the many permutations needed to accurately obtain reasonable significance levels. This latter shortcoming can be addressed with special modifications of randomization procedures, Bootstrapping and Monte-Carlo Simulation.
Randomization procedures however are ideally suited for the analysis of large, finite data sets, and in particular for the analysis of telemetered time series data.
We are developing new methods to model pinniped survival rates that include randomization techniques, see our project Modelling of Pinniped Population Trends.
Check out this excellent publication on the topic of randomization and related methods:
Bryan F.J.Manly (1997) Randomization, Bootstrap and Monte Carlo Methods in Biology (2nd ed.). Texts in Statistical Science. Chapman & Hall, London, UK, 399 pp.
And this paper as an example of an application of randomization approaches for the analysis of telemetry data:
Horning M, Trillmich F (1999) Lunar cycles in diel prey migrations exert stronger effect on diving of juveniles than adult Galápagos fur seals. Proc. Royal Soc. Lond. B. 266 (1424): 1127-1132.
