FDRBONFERRONI procedure

Estimates false discovery rates by a Bonferroni-type procedure (A. Glaser).

Options

PRINT = string

Controls printed output (pi0); default pi0

METHOD = string

Controls the method used for calculating π₀ (smoother, bootstrap); default smoo

LOGP = string

Whether to take logs of π₀ when METHOD=smoother (yes, no); default no

DF = scalar

Degrees of freedom for smoothing spline; default 3

PLOT = string

Controls plots (phistogram, qhistogram, pi0vslambda, qvsp, tests, expfalsepositive, inference, loginference); default phis, qhis, pi0v, qvsp, test, expf, infe, logi

WINDOW = scalar

Window for the graphs; default 1

KEYWINDOW = scalar

Window for the key (zero for none); default 2

Parameters

PROBABILITIES = variates

Significance values, must lie between 0 and 1

LAMBDA = scalars

or variates Values of tuning parameter λ, equivalent to significance levels at which to test the PROBABILITIES; default !(0, 0.05....0.9)

FDR = variates

Saves the False Discovery Rates (i.e. q-values) at the sorted p-values in PROBABILITIES

FRR = variates

Saves the False Rejection Rates at the sorted p-values in PROBABILITIES

PI0 = scalars

Saves the value of π₀, i.e. the maximum value of the FDR

LOWER = scalars

Lower bound of q-values to use with PLOT settings qvsp, tests and expfalsepositive; default 0

UPPER = scalar

Upper bound of q-values to to use with PLOT settings qvsp, tests and expfalsepositive; default 1, which indicates maximum q-value

Description

When testing m multiple hypotheses there are various outcomes that can occur, summarized in the table below

Decision on null hypothesis:	Accept	Reject	Total
Situation: null true	U	V	m₀
alternative true	T	S	m₁
Total	W	R	m

where R is the total number of rejected hypotheses and W = R - m. The proportion of tests that are truly null, π₀, is m₀ divided by m. The false discovery rate (FDR), also known as the q-value of a test, is a commonly used error measure in multiple-hypotheses, defined as

FDR = E(V/R | R > 0) × Pr(R > 0)

i.e. the expected proportion of false positives findings among all the rejected hypotheses multiplied by the probability of making at least one rejection; the FDR is zero when R = 0. Similarly the false rejection rate (FRR) is defined as

FDR = E(T/W | W > 0) × Pr(W > 0),

i.e. the expected proportion of false negatives findings among all the accepted hypotheses times the probability of accepting at least one test. We also define the power to be equal to E(S/m₁ | m₁ > 0) × Pr(m₁ > 0).

The p-values from the multiple hypotheses are supplied, in a variate, using the PROBABILITIES parameter. The analysis uses a Bonferroni-type multiple-testing procedure to calculate the corresponding q-values. The p-values are assumed independent, or may be weakly dependent if there are many of them. The parameter π₀ is calculated using the method of Storey (2002). This involves a tuning parameter λ, which can be set using the LAMBDA parameter; the default is a variate containing the numbers 0, 0.05, ... 0.9. Λ can be thought of as the value beyond which the individual p-values are considered null. As λ gets larger the bias of π₀ gets smaller, but its variance increases. If you set LAMBDA to a scalar, π₀ is estimated by dividing the number of null tests (i.e. the number of p-values greater than λ) by the expected number of null tests m × (1 - Λ). If you set LAMBDA to a variate with several values, two methods are available, selected by the following settings of the METHOD option:

smoother

fits a smoothing spline of λ onto initial estimates of π₀ calculated as for a single λ value, and takes the estimate of π₀ as the value corresponding to the largest value of λ;

bootstrap

estimates π₀ by bootstrap sampling from the variate of p-values.

The default is smoother, as the bootstrap method may be time-consuming when there are many p-values. The number of degrees of freedom to use in the smoothing is specified by the DF option (default 3). Also, you can set option LOGP=yes to do the smoothing on log-transformed π₀ values.

The PRINT option controls printed output, using the settings:

pi0

prints π₀, the estimate of the expected proportion of null p-values corresponding to the largest q-value.

By default PRINT=pi0.

Various graphs can be selected by the following settings of the PLOT option:

phistogram

histogram of p-values;

qhistogram

histogram of q-values (i.e. the FDR values);

pi0vslambda

λ against π₀ (when only one value of λ is specified the default values of LAMBDA are used);

qvsp

q-values against p-values;

tests

plot of the sorted tests against q-values;

expfalsepositive

plot of the number of expected false positives against the sorted tests;

inference

plot of the FDR, FRR and power statistics against the sorted p-values; and

loginference

plots the FDR, FRR and power statistics on log scales, against the sorted p-values restricted to p<0.5, with a background grid to enable estimates to be read for specific probability values. Due to the small numbers used in this plot the p-values and FDR, FRR & power statistics are displayed as percentages.

By default all the plots are produced. The WINDOW option specifies the window for the graphs, and the KEYWINDOW option species the window for keys.

Options: PRINT, METHOD, LOGP, ROBUST, DF, PLOT, WINDOW, KEYWINDOW.

Parameters: PROBABILITIES, LAMBDA, FDR, FRR, PI0, LOWER, UPPER.

Method

FDRBONFERRONI uses the method of Storey (2002), with the definitions of FRR and power given in Genovese & Wasserman (2002).

Action with RESTRICT

The PROBABILITIES parameter can be restricted. All output estimates will then be based only on those unrestricted units.

References

Storey, J.D. (2002). A direct approach to false discovery rates. Journal of the Royal Statistical Society Series B, 64, 479-498.

Genovese, C. & Wasserman, L. (2002). Operating characteristics and extensions of the false discovery rate procedure. Journal of the Royal Statistical Society Series B, 64, 499-518.