APERMTEST procedure

Does random permutation tests for analysis-of-variance tables (R.W. Payne).

Options

PRINT = strings

Controls printed output (aovtable, critical); default aovt

NTIMES = scalar

Number of permutations to make; default 999

EXCLUDE = factors

Factors in the block model of the design whose levels are not to be randomized

SEED = scalar

Seed for the random number generator used to make the permutations; default 0 continues from the previous generation or (if none) initializes the seed automatically

Parameter

SAVE = ANOVA save structure

Save structure from the analysis of variance; default uses the save structure from the most recent ANOVA

Description

Random permutation tests provide an alternative to using the F probabilities printed for variance ratios in an analysis-of-variance table in situations where the assumptions of the analysis are not satisfied. These assumptions can be assessed by studying the residual plots produced by APLOT. In particular, the use of the F distribution to calculate the probabilities is based on the assumption that the residuals from each stratum have Normal distributions with equal variances, and so the histogram of residuals produced by APLOT should look reasonably close to the Normal, bell-shaped curve. Experience shows the analysis is robust to small departures from Normality. APERMTEST can be useful if the histogram looks very non-Normal (and you are unable to redefine the analysis as a generalized linear model; see FIT).

The simplest form of use is simply to specify the command

APERMTEST

straight after the ANOVA. APERMTEST recovers the necessary information about the analysis automatically, and performs 999 random permutations (made using a default seed). The probability for each variance ratio is then determined from its distribution over the randomly permuted datasets.

The NTIMES option of APERMTEST allows you to request another number of permutations, and the SEED option allows you to specify another seed. APERMTEST checks whether NTIMES is greater than the number of possible permutations available for the data set. If so, APERMTEST does an exact test instead, which uses each possible permutation once.

The information about the analysis is obtained from the save structure of the most recent ANOVA (which is stored automatically within GenStat). You can save the information from any analysis of variance explicitly using the SAVE parameter of ANOVA. You can then perform permutation tests for that analysis by using the save structure as the setting of the SAVE parameter of APERMTEST. The EXCLUDE option allows you to restrict the randomization so that one or more of the factors in the block model is not randomized. The most common instance where this is required is when one of the treatment factors involves time-order, which cannot be randomized.

Output is controlled by the PRINT option, with settings:

aovtable

for an analysis-of-variance table with the usual F probabilities replaced by those from the permutation test; and

critical

for a table giving critical values for each variance ratio.

Options: PRINT, NTIMES, EXCLUDE, SEED.

Parameter: SAVE.

Method

If there is no blocking and the treatment combinations have more than one replicate, APERMTEST uses SETALLOCATIONS to determine the number of unique permutations so that it can see whether an exact test is possible. If so, the permutations are formed using SETALLOCATIONS too. Otherwise the number of possible permutations is calculated using the FACTORIAL function, and the permutations for the exact test are formed using the PERMUTE procedure. For a permutation test, RANDOMIZE is used to perform the permutations, taking account of the block structure of the design. The AOVTABLE option of AKEEP is used to save the variance ratios, and the QUANTILES function is used to calculate the critical values.

Action with RESTRICT

APERMTEST takes account of any restrictions on the y-variate in the analysis of variance (i.e. the variate specified as the Y parameter in the earlier ANOVA command).