Categories

Versions

  1. Documentation
  3. Deployment
  4. Templates
  5. Deploy RapidMiner for testing and development (single host)

You are viewing the RapidMiner Deployment documentation for version 9.6 - Check here for latest version

Deploy RapidMiner for testing and development (single host)

The template defined below is meant for testing and development.

It deploys RapidMiner on a single host with the following components:

Compared to the production template, the only difference is the number of Job Agents. Additional production templates are available from the templates page.

Note that the easiest way to get started is to deploy to AWS or Azure using our preconfigured cloud images.

System requirements

Minimum recommended hardware configuration

Each virtual or physical machine should at least have:

  • Dual core
  • 16GB RAM
  • >10GB free disk space

The amount of memory needed depends heavily on the amount of data that will be processed in the Server. An environment as small as 8GB could work for a "Hello world" kind of test. For anything else, we recommend 16GB or beyond depending on user data.

Instructions

To deploy the template, take the following steps:

  1. If you have not yet done so, install Docker.

  2. From the page below, copy the two files that define the docker-compose deployment to a folder on the server host:

  3. Connect to the server host (ssh), and change directory (cd) to the folder containing those two files.

  4. For JupyterHub, create the following external Docker network:

     docker network create jupyterhub-user-net

  5. Deploy the template:

     docker-compose up -d

    Via this command, the Docker images will be automatically downloaded from Docker Hub (if they were not downloaded previously), and the services started.

Services

Once the deployment is running, the reverse proxy listens on the standard HTTP port (80) by default, as well as the HTTPS port (443), if an HTTPS certificate is configured. The deployment makes the following services / URLs available:

URL Description Default login credentials
http://<deployment-url> Login screen for RapidMiner Server Username: admin
Password: changeit
http://<deployment-url>/python-admin Python Environment Manager Username: admin
Password: changeit
http://<deployment-url>/jupyter JupyterHub Integrated to the RapidMiner Server login (SSO)
http://<deployment-url>/grafana Dashboards Username: admin
Password: grafanaadminpass

The environment file (.env)

#
# Global parameters
#

REGISTRY=rapidminer/
AUTH_SECRET=TTY5MjUxbzRBN2ZIWThpNGVKNGo4V2xqOHk0dTNV
BROKER_ACTIVEMQ_USERNAME=amq-user
BROKER_ACTIVEMQ_PASSWORD=M69251o4A7fHY8i4eJ4j8Wlj8y4u3U
RAPIDMINER_SERVER_HOST=rm-server-svc
RAPIDMINER_SERVER_URL=http://rm-server-svc:8080
TZ=UTC

#
# Rapidminer server
#

SERVER_VERSION=9.6.0
SERVER_DBHOST=rm-postgresql-svc
SERVER_DBSCHEMA=rapidminer-server-db
SERVER_DBUSER=rmserver-db-user
SERVER_DBPASS=w61J784XSb24K4LRV97MbE16i8xa9O
SERVER_MAX_MEMORY=2048M

#
# Job Agent
#

JOBAGENT_QUEUE_ACTIVEMQ_URI=failover:(tcp://rm-server-svc:5672)
JOBAGENT_CONTAINER_COUNT=2
JOB_QUEUE=DEFAULT
JOBAGENT_CONTAINER_MEMORYLIMIT=2048
RAPIDMINER_JOBAGENT_OPTS="-Djobagent.python.registryBaseUrl=http://pem-webui-svc:82/"

#
# Proxy
#

PROXY_VERSION=9.6.0
JUPYTER_BACKEND=http://rm-jupyterhub-svc:8000
JUPYTER_URL_SUFFIX=/jupyter/
GRAFANA_BACKEND=http://rm-grafana-svc:3000
GRAFANA_URL_SUFFIX=/grafana
PEM_BACKEND=http://pem-webui-svc:82/
PEM_URL_SUFFIX=/python-admin
HTTPS_CRT_PATH=/rapidminer/uploaded/certs/validated_cert.crt
HTTPS_KEY_PATH=/rapidminer/uploaded/certs/validated_cert.key
HTTPS_DH_PATH=/rapidminer/uploaded/certs/dhparam.pem

#
# Jupyterhub
#

RM_JUPYTER_VERSION=9.6.0
JHUB_DOCKER_NOTEBOOK_IMAGE=rapidminer/rapidminer-jupyter_notebook:9.6.0
JHUB_POSTGRES_HOST=rm-jupyterhub-db-svc
JHUB_POSTGRES_DB=jupyterhub
JHUB_POSTGRES_USER=jupyterhubdbuser
JHUB_POSTGRES_PASSWORD=FoExRExzQsL7UpgEYa5sO7mhiGhi3ne

JHUB_HOSTNAME=rm-jupyterhub-svc
JHUB_ADMIN_USER=admin
JUPYTERHUB_CRYPT_KEY=e0084caa38f521cc985d675168bcf7b7137cf5b0d4a3e8b99d2e3542705584eb
JHUB_DEBUG=False
JHUB_PROXY_DEBUG=False
JHUB_DB_DEBUG=False
JHUB_SPAWNER_DEBUG=False
DOCKER_NOTEBOOK_CPU_LIMIT=100
DOCKER_NOTEBOOK_MEM_LIMIT=2g

#
# Python environment manager
#
PEM_VERSION=9.6.0

#
# Grafana
#
RM_GRAFANA_VERSION=9.6.0

The docker-compose definition (docker-compose.yml)

version: '3'
services:
  rm-proxy-svc:
    image: ${REGISTRY}rapidminer-proxy:${PROXY_VERSION}
    hostname: rm-proxy-svc
    restart: always
    environment:
      - RMSERVER_BACKEND=${RAPIDMINER_SERVER_URL}
      - JUPYTER_BACKEND=${JUPYTER_BACKEND}
      - JUPYTER_URL_SUFFIX=${JUPYTER_URL_SUFFIX}
      - GRAFANA_BACKEND=${GRAFANA_BACKEND}
      - GRAFANA_URL_SUFFIX=${GRAFANA_URL_SUFFIX}
      - PEM_BACKEND=${PEM_BACKEND}
      - PEM_URL_SUFFIX=${PEM_URL_SUFFIX}
      - HTTPS_CRT_PATH=${HTTPS_CRT_PATH}
      - HTTPS_KEY_PATH=${HTTPS_KEY_PATH}
      - HTTPS_DH_PATH=${HTTPS_DH_PATH}
      - DEBUG_CONF_INIT=false
    ports:
      - 80:80
      - 443:443
    networks:
      rm-platform-int-net:
        aliases:
          - rm-proxy-svc
    volumes:
      - ./ssl:/etc/nginx/ssl:ro
      - pem-uploaded-vol:/rapidminer/pem/uploaded/
  rm-postgresql-svc:
    image: postgres:9.6
    hostname: rm-postgresql-svc
    restart: always
    environment:
      - POSTGRES_DB=${SERVER_DBSCHEMA}
      - POSTGRES_USER=${SERVER_DBUSER}
      - POSTGRES_PASSWORD=${SERVER_DBPASS}
    volumes:
      - rm-postgresql-vol:/var/lib/postgresql/data
    networks:
      rm-platform-int-net:
        aliases:
          - rm-postgresql-svc
  rm-server-svc:
    image: ${REGISTRY}rapidminer-server:${SERVER_VERSION}
    hostname: rm-server-svc
    restart: always
    environment:
      - DBHOST=${SERVER_DBHOST}
      - DBSCHEMA=${SERVER_DBSCHEMA}
      - DBUSER=${SERVER_DBUSER}
      - DBPASS=${SERVER_DBPASS}
      - SERVER_MAX_MEMORY=${SERVER_MAX_MEMORY}
      - BROKER_ACTIVEMQ_USERNAME=${BROKER_ACTIVEMQ_USERNAME}
      - BROKER_ACTIVEMQ_PASSWORD=${BROKER_ACTIVEMQ_PASSWORD}
      - JOBSERVICE_AUTH_SECRET=${AUTH_SECRET}
      - JUPYTER_URL_SUFFIX=${JUPYTER_URL_SUFFIX}
      - GRAFANA_URL_SUFFIX=${GRAFANA_URL_SUFFIX}
      - TZ=${TZ}
    volumes:
      - rm-server-bootstrap-vol:/bootstrap.d
      - rm-server-home-vol:/persistent-rapidminer-home
    depends_on:
      - rm-postgresql-svc
    networks:
      jupyterhub-user-net:
        aliases:
          - rm-server-svc
      rm-platform-int-net:
        aliases:
          - rm-server-svc
  rm-server-job-agent-svc:
    image: ${REGISTRY}rapidminer-execution-jobagent:${SERVER_VERSION}
    hostname: rm-server-job-agent-svc
    restart: always
    environment:
      - RMSERVER_HOST=${RAPIDMINER_SERVER_HOST}
      - JOBAGENT_QUEUE_ACTIVEMQ_URI=${JOBAGENT_QUEUE_ACTIVEMQ_URI}
      - JOBAGENT_QUEUE_ACTIVEMQ_USERNAME=${BROKER_ACTIVEMQ_USERNAME}
      - JOBAGENT_QUEUE_ACTIVEMQ_PASSWORD=${BROKER_ACTIVEMQ_PASSWORD}
      - JOBAGENT_AUTH_SECRET=${AUTH_SECRET}
      - JOBAGENT_CONTAINER_COUNT=${JOBAGENT_CONTAINER_COUNT}
      - JOB_QUEUE=${JOB_QUEUE}
      - JOBAGENT_CONTAINER_MEMORYLIMIT=${JOBAGENT_CONTAINER_MEMORYLIMIT}
      - RAPIDMINER_JOBAGENT_OPTS=${RAPIDMINER_JOBAGENT_OPTS}
      - TZ=${TZ}
    volumes:
      - rm-server-bootstrap-ja-vol:/bootstrap.d
    depends_on:
      - rm-server-svc
    networks:
      rm-platform-int-net:
        aliases:
          - rm-server-job-agent-svc
  pem-webui-svc:
    image: ${REGISTRY}python-environment-manager-webui:${PEM_VERSION}
    hostname: pem-webui-svc
    restart: always
    volumes:
      - pem-uploaded-vol:/var/www/html/uploaded/
    networks:
      rm-platform-int-net:
        aliases:
          - pem-webui-svc
  pem-cron-svc:
    image: ${REGISTRY}python-environment-manager-cron:${PEM_VERSION}
    hostname: pem-cron-svc
    restart: always
    volumes:
      - pem-cron-log-vol:/var/log/
      - pem-uploaded-vol:/rapidminer/uploaded/
      - /var/run/docker.sock:/var/run/docker.sock
    networks:
      rm-platform-int-net:
        aliases:
          - pem-cron-svc
  rm-jupyterhub-db-svc:
    image: postgres:9.6
    hostname: rm-jupyterhub-db-svc
    restart: always
    environment:
      - POSTGRES_DB=${JHUB_POSTGRES_DB}
      - POSTGRES_USER=${JHUB_POSTGRES_USER}
      - POSTGRES_PASSWORD=${JHUB_POSTGRES_PASSWORD}
    volumes:
      - jupyterhub-postgresql-vol:/var/lib/postgresql/data
    networks:
      rm-platform-int-net:
        aliases:
          - rm-jupyterhub-db-svc
  rm-jupyterhub-svc:
    image: ${REGISTRY}rapidminer-jupyterhub-jupyterhub:${RM_JUPYTER_VERSION}
    hostname: rm-jupyterhub-svc
    restart: always
    environment:
      - JHUB_HOSTNAME=${JHUB_HOSTNAME}
      - SERVER_BASE_URL=${RAPIDMINER_SERVER_URL}
      - ADMIN_USER=${JHUB_ADMIN_USER}
      - POSTGRES_HOST=${JHUB_POSTGRES_HOST}
      - POSTGRES_DB=${JHUB_POSTGRES_DB}
      - POSTGRES_USER=${JHUB_POSTGRES_USER}
      - POSTGRES_PASSWORD=${JHUB_POSTGRES_PASSWORD}
      - DOCKER_NOTEBOOK_IMAGE=${JHUB_DOCKER_NOTEBOOK_IMAGE}
      - JUPYTERHUB_CRYPT_KEY=${JUPYTERHUB_CRYPT_KEY}
      - JOBSERVICE_AUTH_SECRET=${AUTH_SECRET}
      - DOCKER_NOTEBOOK_CPU_LIMIT=${DOCKER_NOTEBOOK_CPU_LIMIT}
      - DOCKER_NOTEBOOK_MEM_LIMIT=${DOCKER_NOTEBOOK_MEM_LIMIT}
      - JHUB_DEBUG=${JHUB_DEBUG}
      - JHUB_PROXY_DEBUG=${JHUB_PROXY_DEBUG}
      - JHUB_DB_DEBUG=${JHUB_DB_DEBUG}
      - JHUB_SPAWNER_DEBUG=${JHUB_SPAWNER_DEBUG}
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:rw
    depends_on:
      - rm-jupyterhub-db-svc
      - rm-server-svc
    networks:
      rm-platform-int-net:
        aliases:
          - rm-jupyterhub-svc
      jupyterhub-user-net:
        aliases:
          - rm-jupyterhub-svc
  rm-grafana-svc:
    image: ${REGISTRY}rapidminer-grafana:${RM_GRAFANA_VERSION}
    hostname: rm-grafana-svc
    restart: always
    environment:
      - GF_SERVER_ROOT_URL=%(protocol)s://%(domain)s:%(http_port)s/grafana/
      - GF_SECURITY_ADMIN_PASSWORD=grafanaadminpass
      - TZ=${TZ}
    volumes:
      - grafana-home:/var/lib/grafana
      - ./grafana-provisioning/:/etc/grafana/provisioning/
    depends_on:
      - rm-grafana-proxy-svc
    networks:
      rm-platform-int-net:
        aliases:
          - rm-grafana-svc
  rm-grafana-proxy-svc:
    image: ${REGISTRY}rapidminer-grafana-proxy:${RM_GRAFANA_VERSION}
    hostname: rm-grafana-proxy-svc
    restart: always
    environment:
      - RAPIDMINER_URL=${RAPIDMINER_SERVER_URL}
    depends_on:
      - rm-server-svc
    networks:
      rm-platform-int-net:
        aliases:
          - rm-grafana-proxy-svc

volumes:
  rm-postgresql-vol:
  rm-server-bootstrap-vol:
  rm-server-home-vol:
  rm-server-bootstrap-ja-vol:
  pem-uploaded-vol:
  pem-cron-log-vol:
  jupyterhub-postgresql-vol:
  grafana-home:

networks:
  rm-platform-int-net:
  jupyterhub-user-net:
    external:
      name: jupyterhub-user-net
Privacy Policy © 2024 Altair Engineering Inc. All Rights Reserved. We are currently listed on Nasdaq as ALTR.