After a model is built, you test it to measure how well the model predicts the target. A model is tested by applying it to data for which the target value is known (actual values) and then comparing the predicted values with the actual values. Note that the data used for model test must be prepared in the same way that the data used for model build was.
Oracle Data Miner supports testing two types of models:
Testing a classification model calculates in any or all of the following test metrics:
The three metrics are briefly described in this topic. For more information about testing classification models, see ODM Concepts in Where to Find More Information.
Lift measures how well the model improves predictions over using a random value.
The confusion matrix indicates the types of errors that the model is likely to make. The columns are predictions and the rows are actual values. For example, if you are predicting a target with values 0 and 1, the number in the upper right cell of the confusion matrix indicates the false-positive predictions, that is, predictions of 1 when the actual value is 0.
Calculating Receiver Operating Characteristics results in a ROC curve. ROC curves are similar to lift charts in that they provide a means of comparison between individual models and determine thresholds which yield a high proportion of positive hits. Also, the area under the ROC curve measures the discriminating ability of a binary classification model.
Testing a regression model results calculates the following measures of model accuracy:
These statistics are the metrics most commonly used to test regression models.
You can also create a residual plot to evaluate a regression model. See Residual Plot for more information.
Copyright © 2005, Oracle. All rights reserved.