Performance (User-Based)
(AI Studio Core)
Synopsis
This operator delivers as output a list of performance values according to a list of user defined performance criteria.Description
This performance evaluator operator should be used for regression tasks, i.e. in cases where the label attribute has a numerical value type. The operator expects a test ExampleSet as input, whose elements have both true and predicted labels, and delivers as output a list of performance values according to a list of performance criteria that it calculates. If an input performance vector was already given, this is used for keeping the performance values.
Additional user-defined implementations of PerformanceCriterion can be specified by using the parameter list additional_performance_criteria . Each key/value pair in this list must specify a fully qualified classname (as the key), and a string parameter (as value) that is passed to the constructor. Please make sure that the class files are in the classpath (this is the case if the implementations are supplied by a plugin) and that they implement a one-argument constructor taking a string parameter. It must also be ensured that these classes extend MeasuredPerformance since the PerformanceEvaluator operator will only support these criteria. Please note that only the first three user defined criteria can be used as logging value with names "user1", ... , "user3".
The resulting performance vectors are usually compared with a standard performance comparator which only compares the fitness values of the main criterion. Other implementations than this simple comparator can be specified using the parameter comparator_class . This may for instance be useful if you want to compare performance vectors according to the weighted sum of the individual criteria. In order to implement your own comparator, simply subclass PerformanceComparator . Please note that for true multi-objective optimization usually another selection scheme is used instead of simply replacing the performance comparator.
Input
labelled data (IOObject)
performance (Performance Vector)
Output
- performance (Performance Vector)
- example set (Data Table)
This is an example set output port
Parameters
- skip undefined labelsIf set to true, examples with undefined labels are skipped.
- comparator classFully qualified classname of the PerformanceComparator implementation.
- use example weightsIndicated if example weights should be used for performance calculations if possible.
- main criterionThe criterion used for comparing performance vectors.
- additional performance criteriaList of classes that implement com.rapidminer.operator.performance.PerformanceCriterion.