1. Documentation
2. 2024.1 (Latest) 2024.0 10.3 10.2 10.1 10.0 9.10 9.9 9.8 9.7 9.6 9.5 9.4 9.3 9.2 9.1 9.0 8.2 8.1 8.0 7.6
3. Server
4. Administration
5. Process Execution Queues

Process Execution Queue Management

One of the most powerful features of Rapidminer Server is that it can execute RapidMiner processes either immediately or as scheduled processes. Processes can be scheduled to be executed once at a specific point in time or periodically at specific intervals.

Rapidminer Server utilizes a queueing infrastructure where processes are submitted when they are ready for execution. The execution engine picks up processes once they are made available in the queue. When multiple processes are available at the same time the execution engine will execute them concurrently.

Process execution queues

Currently the queueing infrastructure supports two types of queues:

• Simple Queue
• Blocking Queue

RapidMiner Server is shipped with a pre-configured queue of type Simple Queue named DEFAULT, but it is possible for the user to configure multiple queues of each type.

To view the configured queues go to Process > Process-Scheduler. Click on Queues located on the right side of the page:

This will redirect you to the Queues page where each configured queue is listed along with it's status information:

Go to Simple Queue and Blocking Queue sections to find out more about the status information displayed in the Queues view.

Simple Queue

Processes submitted to a Simple Queue will be picked up for execution as soon as their execution time arrives. This setup requires minimal configuration and is the default setup of RapidMiner Server.

Status information

The following table explains the status information associated with a Simple Queue

Field	Description
Type	the type of the queue
Running processes	the number of processes that are currently being executed
Backlog	the number of processes in the queue that await to be executed. In the case of a Simple Queue this is always 0 as the execution engine starts execution immediately for each process in the queue
Permitted groups	a list of all the groups that are allowed to submit processes to the queue for execution

Blocking Queue

A Blocking Queue can help manage process execution workload more effectively as it allows the user to set a maximum limit to the number of processes that can be executed concurrently regardless of the number of submitted processes; if the number of submitted processes reaches this limit then new processes will be inserted in the queue in a waiting state until an already executing process completes.

Besides the number of concurrently executing processes, the number of waiting processes as well as the maximum time a process can be in a waiting state are configurable.

Status information

This is the status information for a configured Blocking Queue:

The following table explains the status information:

Field	Description
Type	The type of the queue. Can be either Simple Queue or Blocking Queue.
Running processes	the number of processes that are currently being executed
Backlog	the number of processes in the queue that await to be executed
Maximum load	Sets the maximum number of concurrently executed processes.
Limit queue length	Specifies whether there is a maximum number of processes allowed in the queue. When set to true, the system allows up to Max queue length processes in the queue. If queue-length reaches the maximum, new processes for this queue are not added (they are dropped).
Maximum queue length	Sets the number of processes allowed in the queue. Only applicable if Limit queue length is set to true. The minimum limit is one and there is no maximum. When the threshold is reached, processes are dropped from the queue.
Limit waiting time	Specifies whether there is a maximum time the process is in a waiting state before execution. When true and the Maximum waiting time threshold is exceeded the process is dropped from the queue.
Max waiting time	The time threshold in seconds a process is allowed to wait before being executed.
Permitted groups	a list of all the groups that are allowed to submit processes to the queue for execution

Adding a Queue

To configure a new queue navigate to Process > Process-Scheduler. Click on Queues located on the right side of the page. This will bring you to the Queues interface. Click the Add Queue button:

The Manage Queue dialog will appear. There you can select the type of the queue to add. The Manage Queue dialog is different depending on the type selected so please refer to Adding a Simple Queue or Adding a Blocking Queue sections on how to proceed.

Adding a Simple Queue

To add a Simple Queue enter the preferred name for the queue and select Simple Queue as type in the Manage Dialog:

Click the Add button and close the dialog. The new queue should be immediately visible in the Queues page

Adding a Blocking Queue

To add a Blocking Queue enter the preferred name and select Blocking Queue as the type in the Manage Dialog. This will convert the dialog as follows.

The following table contains an explanation of the configurable queue parameters and their possible values.

Parameter	Description	Values
Type	The type of the queue	Can be either Simple Queue or Blocking Queue
Maximum load	Sets the maximum number of processes that can be executed concurrently	[1,...,1000000]
Limit queue length	Specifies whether there is a maximum number of processes in the queue.	Can be true (checked) or false (unchecked).
Maximum queue length	Sets the number of processes allowed in the queue. Only applicable if Limit queue length is set to true. When the threshold is reached, processes are dropped from the queue	[1,...,1000000]
Limit waiting time	Specifies whether there is a maximum time the process is in a waiting state before execution. When true and the Maximum waiting time threshold is exceeded the process is dropped from the queue.	Can be true (checked) or false (unchecked).
Max waiting time	The time threshold in seconds a process is allowed to wait before being executed. Only applicable if Limit waiting time is set to true	[1,...,1000000]

Enter the parameters and click the Add button. The new queue should be immediately visible in the Queues page.

Remove Queues

To remove an existing queue, navigate to Process > Process-Scheduler. Click on Queues on the right side of the page. This will bring you to the Queues interface.

Click on the delete button next to the queue name you want to delete.

Edit Existing Queues

To edit an existing queue, navigate to Process > Process-Scheduler. Click on Queues on the right side of the page. This will bring you to the Queues interface.

Click the small notepad button next to the queue name you want to edit.

This will bring up a dialog with the parameters that can be edited:

This table contains the parameters that can be edited along with the allowed values.

Managing permissions to queues

A queue can be configured to allow process execution to specific groups. To change these permissions navigate to Process > Process-Scheduler. Click on Queues on the right side of the page. This will bring you to the Queues interface.

Click the Manage groups link next to the Permitted groups property of the queue.

This will bring up the Manage permissions dialog:

To permit a group to execute processes from the queue, select the group name from the left hand side group list and click Copy. This action will move the selected group(s) to the group list on the right hand side.

To restrict a group from executing processes from the queue, select the group from the group list on the right hand side and click Remove. This action will move the selected group(s) to the group list on the left hand side.

When done click Update.

Upgrading to RapidMiner Server 7.2.0

With RapidMiner Server version 7.2.0 the process execution queues have been updated with the following changes

• An interface that allows viewing, modifying, adding, and removing queues has been added Process execution queues configuration has been moved to the database
• Additionally queues can be assigned to groups so that processes execution can be restricted

When a RapidMiner Server upgrade is performed from RapidMiner Server 7.1.0 or earlier where the queues.xml file was used for queue configuration, a migration step will take place that will have the following effects:

• The DEFAULT simple queue will be added
• Old process queue definitions in queues.xml will be converted and moved from the repository to the database
• Existing triggers will be moved to the DEFAULT queue