Connect External Job Agents to a Deployment

If you need to scale your deployment beyond the physical limits of your host machine, it's possible to run your Job Agent containers on another physical machine.

To do this, you need to create a second set of definition files (.env. and docker-compose.yml) but with modified content, and launch that stack on the other host machine.

In your duplicated docker-compose.yml file, remove (or comment) every service except the Job Agent service you want to launch on that machine.

Next, edit JOBAGENT_QUEUE_ACTIVEMQ_URI, AIHUB_CONNECTION_PROTOCOL, AIHUB_CONNECTION_HOST, and AIHUB_CONNECTION_PORT to point to the external (publicly accessible) address of your RapidMiner Server. This is running on the original machine you wish to connect your new Job Agent to.

If needed, adjust the memory settings according to the memory tuning guide.

Finally, run docker-compose up -d to launch the Job Agents. You can check in your RapidMiner Server UI if the new Job Agents connected successfully.