Optimize Parameters (Grid) (RapidMiner Studio Core)

Synopsis

This operator finds the optimal values of the selected parameters of the operators in its subprocess.

Description

The Optimize Parameters (Grid) operator has a subprocess in it. It executes the subprocess for all combinations of selected values of the parameters and then delivers the optimal parameter values through the parameter port. The performance vector for optimal values of parameters is delivered through the performance port. Any additional results of the subprocess are delivered through the result ports.

The entire configuration of this operator is done through the edit parameter settings parameter. Complete description of this parameter is described in the parameters section.

Please note that selecting a large number of parameters and/or large number of steps (or possible values of parameters) results in a huge number of combinations. For example, if you select 3 parameters and 25 steps for each parameter then the total number of combinations would be above 390625 (i.e. 25 x 25 x 25). The subprocess is executed for all possible combinations. Running a subprocess for such a huge number of iterations will take a lot of time. So always carefully limit the parameters and their steps.

This operator returns an optimal parameter set which can also be written to a file with the Write Parameters operator. This parameter set can be read in another process using the Read Parameters operator.

Other parameter optimization schemes are also available. The Optimize Parameters (Evolutionary) operator might be useful if the best ranges and dependencies are not known at all. Another operator which works similar to this parameter optimization operator is the Loop Parameters operator. In contrast to the optimization operators, this operator simply iterates through all parameter combinations. This might be especially useful for plotting purposes.

Differentiation

Optimize Parameters (Evolutionary)

The Optimize Parameters (Evolutionary) operator finds the optimal values for a set of parameters using an evolutionary approach which is often more appropriate than a grid search (as in the Optimize Parameters (Grid) operator) or a greedy search (as in the Optimize Parameters (Quadratic) operator) and leads to better results. The Optimize Parameters (Evolutionary) operator might be useful if the best ranges and dependencies are not known at all.

Input

  • input (Data Table)

    This operator can have multiple inputs. When one input is connected, another input port becomes available which is ready to accept another input (if any). The order of inputs remains the same. The Object supplied at the first input port of this operator is available at the first input port of the nested chain (inside the subprocess). Do not forget to connect all inputs in correct order. Make sure that you have connected the right number of ports at the subprocess level.

Output

  • performance (Performance Vector)

    This port delivers the Performance Vector for the optimal values of the selected parameters. A Performance Vector is a list of performance criteria values.

  • parameters (Parameter Set)

    This port delivers the optimal values of the selected parameters. This optimal parameter set can also be written to a file with the Write Parameters operator. The written parameter set can be read in another process using the Read Parameters operator.

  • result (IOObject)

    Any additional results of the subprocess are delivered through the result ports. This operator can have multiple outputs. When one result port is connected, another result port becomes available which is ready to deliver another output (if any). The order of outputs remains the same. The Object delivered at the first result port of the subprocess is delivered at the first result port of the operator. Don't forget to connect all outputs in correct order. Make sure that you have connected the right number of ports.

Parameters

  • edit_parameter_settingsThe parameters are selected through the edit parameter settings menu. You can select the parameters and their possible values through this menu. This menu has an Operators window which lists all the operators in the subprocess of this operator. When you click on any operator in the Operators window, all parameters of that operator are listed in the Parameters window. You can select any parameter through the arrow keys of the menu. The selected parameters are listed in the Selected Parameters window. Only those parameters should be selected for which you want to find optimal values. This operator finds optimal values of the parameters in the specified range. The range of every selected parameter should be specified. When you click on any selected parameter (parameter in Selected Parameters window) the Grid/Range and Value List option is enabled. These options allow you to specify the range of values of the selected parameters. The Min and Max fields are for specifying the lower and upper bounds of the range respectively. As all values within this range cannot be checked, the steps field allows you to specify the number of values to be checked from the specified range. Finally the scale option allows you to select the pattern of these values. You can also specify the values in form of a list. Range:
  • error_handlingThis parameter allows you to select the method for handling errors occurring during the execution of the inner process. It has the following options:
    • fail_on_error: In case an error occurs, the execution of the process will fail with an error message.
    • ignore_error: In case an error occurs, the error will be ignored and the execution of the process will continue with the next iteration.
    Range: selection

Tutorial Processes

Finding optimal values of parameters of the SVM operator

The 'Weighting' data set is loaded using the Retrieve operator. The Optimize Parameters (Grid) operator is applied on it. Have a look at the Edit Parameter Settings parameter of the Optimize Parameters (Grid) operator. You can see in the Selected Parameters window that the C and gamma parameters of the SVM operator are selected. Click on the SVM.C parameter in the Selected Parameters window, you will see that the range of the C parameter is set from 0.001 to 100000. 11 values are selected (in 10 steps) logarithmically. Now, click on the SVM.gamma parameter in the Selected Parameters window, you will see that the range of the gamma parameter is set from 0.001 to 1.5. 11 values are selected (in 10 steps) logarithmically. There are 11 possible values of 2 parameters, thus there are 121 ( i.e. 11 x 11) combinations. The subprocess will be executed for all combinations of these values, thus it will iterate 121 times. In every iteration, the value of the C and/or gamma parameters of the SVM(LibSVM) operator is changed. The value of the C parameter is 0.001 in the first iteration. The value is increased logarithmically until it reaches 100000 in the last iteration. Similarly, the value of the gamma parameter is 0.001 in the first iteration. The value is increased logarithmically until it reaches 1.5 in the last iteration

Have a look at the subprocess of the Optimize Parameters (Grid) operator. First the data is split into two equal partitions using the Split Data operator. The SVM (LibSVM) operator is applied on one partition. The resultant classification model is applied using two Apply Model operators on both the partitions. The statistical performance of the SVM model on both testing and training partitions is measured using the Performance (Classification) operators. At the end the Log operator is used to store the required results.

The log parameter of the Log operator stores five things. The iterations of the Optimize Parameters (Grid) operator are counted by apply-count of the SVM operator. This is stored in a column named 'Count'. The value of the classification error parameter of the Performance (Classification) operator that was applied on the Training partition is stored in a column named 'Training Error'. The value of the classification error parameter of the Performance (Classification) operator that was applied on the Testing partition is stored in a column named 'Testing Error'. The value of the C parameter of the SVM (LibSVM) operator is stored in a column named 'SVM C'. The value of the gamma parameter of the SVM (LibSVM) operator is stored in a column named 'SVM gamma'. Also note that the stored information will be written into a file as specified in the filename parameter.

At the end of the process, the Write Parameters operator is used for writing the optimal parameter set in a file. This file can be read using the Read Parameters operator to use these parameter values in another process.

Run the process and turn to the Results Workspace. You can see that the optimal parameter set has the following values: SVM.C = 100000.0 and SVM.gamma = 0.0010. Now have a look at the values saved by the Log operator to verify these values. Switch to Table View to see the stored values in tabular form. You can see that the minimum Testing Error is 0.008 (in 11th iteration). The values of the C and gamma parameters for this iteration are the same as given in the optimal parameter set.