Performance Binominal Classification (AI Studio Core)

Parameters

• main_criterion

  The main criterion is used when performance vectors are compared, e.g., parameter Optimization or Attribute selection. If not selected, the main criterion is the first criterion in the output performance vector.

  If performance vectors are not compared, the main criterion is ignored.

  Range:
• manually_set_positive_class

  Check this box to use the positive class parameter to manually specify the positive class. Otherwise the positive class is derived from the label's internal mapping. It is recommended to manually set the desired positive class.

  Range:
• positive_class

  Use this parameter to manually set the positive class.

  In rare cases the suggested values in the drop down menu may not match the actual label's values. In this case the correct positive class can be specified by typing its name manually instead of selecting it from the drop down menu.

  Range:
• accuracy

  accuracy = (Correct predictions)/(Number of Examples) = (TP + TN) / (TP + FP + FN + TN)

  Range:
• classification_error

  classification error = (Incorrect predictions)/(Number of Examples) = (FP + FN) / (TP + FP + FN + TN)

  Range:
• kappa

  Cohen's kappa = (po - pe)/(1 - pe)

  where:

  po = observed accuracy = (TP + TN) / (TP + FP + FN + TN)

  pe = expected accuracy = [(TP + FP)(TP + FN) + (FN + TN)(FP + TN)] / [(TP + FP + FN + TN)^2]

  Range:
• AUC (optimistic)

  When the ROC graph is plotted, before calculating the area under the curve (AUC), the predictions are sorted by score, from highest to lowest, and the graph is plotted Example by Example. If two or more Examples have the same score, the ordering is not well-defined; in this case, the optimistic version of AUC plots the positive Examples before plotting the negative Examples.

  Range:
• AUC

  When the ROC graph is plotted, before calculating the area under the curve (AUC), the predictions are sorted by score, from highest to lowest, and the graph is plotted Example by Example. If two or more Examples have the same score, the ordering is not well-defined. The normal version of AUC calculates the area by taking the average of AUC (optimistic) and AUC (pessimistic).

  Range:
• AUC (pessimistic)

  When the ROC graph is plotted, before calculating the area under the curve (AUC), the predictions are sorted by score, from highest to lowest, and the graph is plotted Example by Example. If two or more Examples have the same score, the ordering is not well-defined; in this case, the pessimistic version of AUC plots the negative Examples before plotting the positive Examples.

  Range:
• precision

  precision = (True positive predictions)/(All positive predictions) = TP / (TP + FP)

  Range:
• recall

  recall = (True positive predictions)/(Number of positive Examples) = TP / (TP + FN)

  Range:
• lift

  lift is the ratio of two quantities, representing the improvement over random sampling:

  1. The probability of choosing a positive Example from the group of all positive predictions: TP / (TP + FP)

  2. The probability of choosing a positive Example from the group of all Examples: (TP + FN) / (TP + FP + FN + TN)

  lift = [TP / (TP + FP)] / [(TP + FN) / (TP + FP + FN + TN)]

  Range:
• fallout

  fallout = (False positive predictions)/(Number of negative Examples) = FP / (FP + TN)

  Range:
• f_measure

  F1 = 2 (precision * recall) / (precision + recall) = 2TP / (2TP + FP + FN)

  Range:
• false_positive

  The number of false positive predictions: FP

  Range:
• false_negative

  The number of false negative predictions: FN

  Range:
• true_positive

  The number of true positive predictions: TP

  Range:
• true_negative

  The number of true negative predictions: TN

  Range:
• sensitivity

  sensitivity = recall = (True positive predictions)/(Number of positive Examples) = TP / (TP + FN)

  Range:
• specificity

  specificity = (True negative predictions)/(Number of negative Examples) = TN / (TN + FP)

  Range:
• youden

  Sometimes called informedness or DeltaP'.

  J = sensitivity + specificity - 1

  Range:
• positive_predictive_value

  PPV = precision = (True positive predictions)/(All positive predictions) = TP / (TP + FP)

  Range:
• negative_predictive_value

  NPV = (True negative predictions)/(All negative predictions) = TN / (TN + FN)

  Range:
• psep

  Sometimes called markedness or DeltaP.

  psep = PPV + NPV - 1

  Range:
• skip_undefined_labels

  When this parameter is true, Examples not belonging to a defined class are ignored.

  Range:
• comparator_class

  The fully qualified classname of the PerformanceComparator implementation is specified here.

  Range:
• use_example_weights

  This parameter has no effect if no Attribute has the weight role.

  Range: