BREGRESSION procedure

Constructs a regression tree (R.W. Payne).


Options

PRINT = strings
Controls printed output (summary, details, indented, bracketed, labelleddiagram, numbereddiagram, graph, monitoring); default * i.e. none

Y = variate
Response variate for the regression

TREE = tree
Saves the tree that has been constructed

MSLIMIT = scalar
Limit on the mean square of the observations at a node at which to stop making splits; default 0

OWNBSELECT = string
Indicates whether or not your own version of the BSELECT procedure is to be used, as explained in the Method section (yes, no); default no


Parameters

X = variates or factors
Independent variables available for constructing the tree

ORDERED = strings
Whether factor levels are ordered (yes, no); default no


Description

A regression tree is a mechanism for predicting a response variable from a set of independent variables (see Chapter 8 of Breiman et al.). The tree is constructed using data on a set of observations. Their values for the response variable are specified (in a variate) using the Y option, and their values for the independent variables are specified (in a list of variates or factors) using the X parameter. Factors may have either ordered or unordered levels, according to whether the corresponding value ORDERED parameter is set to yes or no. For example, a factor called Dose with levels 1, 1.5, 2 and 2.5 would usually be treated as having ordered levels, whereas levels labelled 'Morphine', 'Amidone', 'Phenadoxone' and 'Pethidine' of a factor called Drug would be regarded as unordered.

   The construction process splits the observations into subsets. With an x-variate or a factor with ordered levels, the subsets are formed by taking the observations with values less then or greater than some split point p. For a factor with unordered levels, all possible ways of dividing its levels into two subsets are tried. The aim is to form subsets that have similar values for the response variate. The predicted value of the response variable for each node of the tree is the mean of its value for the subset of observations at that node. The accuracy of the node is the squared distance of the values of the response variate from their mean for the observations at the node, divided by the total number of observations. The potential splits at the node are assessed by their effect on the accuracy, that is the difference between the accuracy of the node and the sum of the accuracies of the two potential successor nodes. The node will become a terminal node if none of the splits provides any improvement in accuracy, or if the mean square of the observations at the node is less than a limit specified by the MSLIMIT option (default 0).

   The resulting tree can be saved using the TREE option. Details of the tree can be printed as selected by the PRINT option, with settings:

    summary
prints a summary of the properties of the tree;

    details
gives detailed information about the nodes of the tree;

    bracketed
display as used to represent an identification key in "bracketed" form (printed node by node).

    indented
display as used to represent an identification key in "indented" form (printed branch by branch);

    labelleddiagram
diagrammatic display including the node labels;

    numbereddiagram
diagrammatic display with the nodes labelled by their numbers;

    graph
plots the tree using high-resolution graphics.

    monitoring
prints information monitoring the construction process.

BREGRESSION stores the information required for printing as part of the tree. For variates and ordered factors, the labels are generally formed as "identifier<p" and "identifier>p", where p is the value chosen to partition the data for the variate concerned. Alternatively, if you have defined an "extra" text for the variate (using the EXTRA parameter of the VARIATE command), this will be used instead. The labels are then "extra-text < p" and "extra-text > p". The style is similar for unordered factors, but here the labels involve the operators .IN. and .NI. instead of < and >.

   Generally the construction will result in over-fitting, that is it will form a tree that keeps making splits beyond the point that can be justified statistically. The solution is to prune the tree to remove the uninformative sub-branches, and this can be performed using the BPRUNE procedure. It is best, if possible, to base the pruning on an independent set of data. The pruning uses the accuracy figures, which are stored with the tree. The BRVALUES procedure can be used to calculate new accuracy (and prediction) values, from another data set.

   Finally, once the tree has been pruned, the value predicted for a new set of independent values can be obtained by supplying their values to the BRPREDICT procedure. This runs the values through the tree to see which terminal node they reach. The prediction is then provided by the value predicted at that node.

 

Options: PRINT, Y, TREE, MSLIMIT, OWNBSELECT.

Parameters: X, ORDERED.


Method

BREGRESSION calls procedure BCONSTRUCT to form the tree. This uses a special-purpose procedure BSELECT, which is customized specifically to select splits for use in regression trees and stored with BREGRESSION. You can use your own method of selection by providing your own BSELECT and setting option OWNBSELECT=yes. In the standard version of BSELECT, the BASSESS directive is used to assess the potential splits.


Action with RESTRICT

Any restrictions on the Y or X variates are removed.


Reference

Breiman, L., Friedman, J.H., Olshen, R.A. & Stone, C.J. (1984). Classification and Regression Trees. Wadsworth, Monterey.