Test a Model

After a model is built, you test it to measure how well the model predicts the target. A model is tested by applying it to data for which the target value is known (actual values) and then comparing the predicted values with the actual values. Note that the data used for model test must be prepared in the same way that the data used for model build was.

Oracle Data Miner supports testing two types of models:

Classification Models

Testing a classification model calculates in any or all of the following test metrics:

The three metrics are briefly described in this topic. For more information about testing classification models, see ODM Concepts in Where to Find More Information.

Lift

Lift measures how well the model improves predictions over using a random value.

Confusion Matrix

The confusion matrix indicates the types of errors that the model is likely to make. The columns are predictions and the rows are actual values. For example, if you are predicting a target with values 0 and 1, the number in the upper right cell of the confusion matrix indicates the false-positive predictions, that is, predictions of 1 when the actual value is 0.

Receiver Operating Characteristics (ROC)

Calculating Receiver Operating Characteristics results in a ROC curve. ROC curves are similar to lift charts in that they provide a means of comparison between individual models and determine thresholds which yield a high proportion of positive hits. Also, the area under the ROC curve measures the discriminating ability of a binary classification model.

Regression Models

Testing a regression model results calculates the following measures of model accuracy:

These statistics are the metrics most commonly used to test regression models.

You can also create a residual plot to evaluate a regression model. See Residual Plot for more information.