False Discovery Rate using Bonferroni
See Also  Example
This menu can be used to estimate false discovery rates (defined in the table below) using a Bonferroni-type procedure. This is a non-parametric approach, where for each value of lambda, the observed proportion of the sample that are not differentially expressed (π0) is calculated. The procedure uses two methods to get an overall measure of π0. The first uses bootstrapping to choose the value of π0 which minimises the mean squared error, and the second uses a spline smoother to smooth the values of π0 around the maximum value of lambda. Unadjusted q-values are then calculated from the estimate of π0 as π0*p*(Proportion of tests < p) (where p is the test probability) for each test value, and then the Bonferroni q values are defined as the minimum of the q values above each test value, stepping this procedure down through the sorted p values.

The following table defines some random variables related to m hypothesis tests:

Signficance Test # declared non-significant# declared significantTotal
# true null hypotheses U V m0
# non-true null hypotheses T S m1 = m − m0
Total W = m − R R m

m0 is the number of true null hypotheses
m − m0 is the number of false null hypotheses
U is the number of true negatives
V is the number of false positives
T is the number of false negatives
S is the number of true positives
H1...Hm are the null hypotheses being tested

In m hypothesis tests of which m0 are true null hypotheses, R is an observable random variable, and S, T, U, and V are unobservable random variables.

The proportion of tests that are truly null, π0, is m0 divided by m. The false discovery rate (FDR), also known as the q-value of a test, is a commonly used error measure in multiple-hypotheses, defined as FDR = E(V/R | R > 0) × Pr(R > 0), i.e. the expected proportion of false positives findings among all the rejected hypotheses multiplied by the probability of making at least one rejection; the FDR is zero when R = 0. Similarly the false rejection rate (FRR) is defined as FDR = E(T/W | W > 0) × Pr(W > 0), i.e. the expected proportion of false negatives findings among all the accepted hypotheses times the probability of accepting at least one test. We also define the power to be equal to E(S/m1 | m1 > 0) × Pr(m1 > 0).

A range of graphs can be plotted after the Bonferroni procedure has been used.

Method

Controls the method used for calculating π0:
Spline smoother Fits a smoothing spline of λ onto initial estimates of π0 calculated as for a single λ value, and takes the estimate of π0 as the value corresponding to the largest value of λ
Bootstrap Estimates π0 by bootstrap sampling from the variate of p-values.

Available Data

This lists data structures appropriate for the edit box which currently has focus. You can double-click a name to enter it in the edit box.

Probabilities to Model

This variate contains the probabilities to which the mixture distribution is fitted. All the values in the variate should be between 0 and 1 (although missing values (*) are allowed).

Lambda

The tuning parameter λ to be used (saclar or variate). The default is a variate containing the numbers 0, 0.05, ... 0.9. λ can be thought of as the value beyond which the individual p-values are considered null. As λ gets larger the bias of π0 gets smaller, but its variance increases. If you set λ to a scalar, π0 is estimated by dividing the number of null tests (i.e. the number of p-values greater than λ) by the expected number of null tests m × (1 - λ). If λ is set to a variate of values, then π0 is estimated at each value of the variate and the resulting values combined according to the method (spline smoother or bootstrap) selected above.

Spline DF

The degrees of freedom to use for the smoothing spline when this method is chosen. This should be an integer between 1 and 99.

Use Logged Pi 0 values when smoothing

Do the smoothing on log-transformed π0 values when this method is chosen.

Action Buttons

RunRun the analysis.
CancelClose the menu without further changes.
OptionsOpens a dialog where additional options and settings can be specified for the analysis.
DefaultsSet the menu settings back to the default settings. Clicking the right mouse on this button produces a pop-up menu where you can choose to set the menu using the currently stored defaults or the GenStat default settings.
StoreOpens a dialog to specify names of structures to store the results from the analysis. The names to save the structures should be supplied before running the analysis.

Example

The following menu shows the estimates of FDR using the Bonferroni procedure

The options used were:

and the Store button was used to save results back to a spreadsheet:

The resulting graphs are show below:





See Also