Find Threshold (Meta)
(AI Studio Core)
Synopsis
Determines confidence thresholds based on misclassification costs, also possible to define costs for the option non-classified.Description
This operator uses a set of class weights and also allows a weight for the fact that an example is not classified at all (marked as unknown). Based on the predictions of the model of the inner learner this operator optimized a set of thresholds regarding the defined weights.
This operator might be very useful in cases where it is better to not classify an example then to classify it in a wrong way. This way, it is often possible to get very high accuracies for the remaining examples (which are actually classified) for the cost of having some examples which must still be manually classified.
Input
training set (IOObject)
Output
model (Model)- example set (Data Table)
This is an example set output port
Parameters
- class weightsThe weights for all classes, empty: using 1 for all classes. The costs for not classifying at all are defined with class name '?'.
- allow unkown predictionsThis indicates if unkown predictions are allowed. If checked, the costs for unkown predictions must be specified.
- predict unknown costsUse this cost value for predicting an example as unknown.
- training ratioUse this amount of input data for model learning and the rest for threshold optimization.
- number of iterationsDefines the number of optimization iterations.
- use local random seedIndicates if a local random seed should be used.
- local random seedSpecifies the local random seed