GPU-enabled Job Agent

Before you create your own GPU-enabled Job Agent, be aware that Altair RapidMiner provides pre-configured Docker images for Deep Learning. Nevertheless, if you want more technical detail, continue reading!

The main points are the following:

Your Job Agent must be installed on a computer with CUDA-compatible GPU.
You must install the Nvidia library CUDA and should install the library cuDNN for enhanced performance.
You must install the following extensions:
- Deep Learning
- ND4J Back End

Pay careful attention to the compatibility matrix:

CUDA compatibility

Deep Learning Extension	ND4J Back-End	Supported CUDA	Supported cuDNN
1.1.2	1.0	10.1	7.6
1.1.1	1.0	10.1	7.6
1.1.0	1.0	10.1	7.6
1.0.1	1.0	10.1	7.6
1.0	1.0	10.1	7.6
0.9.4	0.1.1	10.0	7.4
0.9.3	0.1.0	10.0	-
0.9.1	-	9.0	-
0.9.0	-	9.0	-
0.8.1	-	9.1	-
0.8.0	-	9.1	-

Create a GPU-enabled Job Agent

A Job Agent can take advantage of a GPU for processing images or training and scoring neural networks. Currently, one GPU can be used per Job Agent.

Take the following steps:

Install the Job Agent on a computer that has a CUDA-compatible GPU.
Follow the installation instructions for CUDA 10.1 (and cuDNN version 7.6).
Download the ND4J Back End and the Deep Learning extensions from the marketplace, and move them to the extensions folder belonging to the Job Agent {homeDir}/resources/extensions/.

Create the settings file {homeDir}/config/rapidminer/rapidminer.properties as follows:

 rapidminer.backend.nd4j=GPU-CUDA
 rapidminer.backend.nd4j.max_bytes=32G
 rapidminer.backend.nd4j.max_physical_bytes=48G
 rapidminer.deeplearning.training_ui.ports=60080

Settings

The settings are defined by a properties file, {homeDir}/config/rapidminer/rapidminer.properties, located in the home directory of the Job Agent.

Parameter Key	Possible Parameter Value	Explanation
`rapidminer.backend.nd4j`	`CPU-OpenBLAS`, `CPU-MKL`, `GPU-CUDA`	Choose the computation back-end to use for calculation.
`rapidminer.backend.nd4j.max_bytes`	`1024M`, `16G`	The JVM off-heap memory limit for the computation backend (~native memory limit)
`rapidminer.backend.nd4j.max_physical_bytes`	`1024M`, `16G`	The maximum bytes for the entire process - usually set to max-bytes plus Xmx plus a bit extra, in case other libraries require some off-heap memory as well.
`rapidminer.deeplearning.training_ui.ports`	`1`-`65535`	Choose a port a Job Agents training UI should be listening to. The port `0` is also allowed, but will assign a random port.

Categories

Versions

GPU-enabled Job Agent

Create a GPU-enabled Job Agent

Settings