Kubernetes deployment with Helm

It has been nearly a year since our Kubernetes documentation was updated, and in the meantime large changes have occurred. It’s past time for an update.

Following some internal experiments at RapidMiner, we have attempted to reduce the complexity of Kubernetes configuration by introducing Helm Charts, but the results are still not what you would call “plug and play”.

Nevertheless, in the interest of progress, knowing that some of our users are already experienced with Kubernetes, we have decided to release some skeletal documentation – “skeletal” in the sense that it is not complete, but perhaps adequate for experienced users who know how to fill in the gaps.

In the weeks ahead, we plan to incrementally improve this documentation, based on feedback. If you made some effort to install and run the software, please let us know what we can do to improve the documentation.

For a simpler, single-host deployment of RapidMiner AI Hub, see Docker-compose deployment or the cloud images.

For RapidMiner AI Hub 9.10, our Docker Images have been improved to meet the latest security guidelines, including those of OpenShift and Rootless Docker. We tested our example configuration with the following Kubernetes services:

To deploy RapidMiner AI Hub with Kubernetes / Helm:

Table of contents

Before you begin

To deploy the Helm chart, you need basic Kubernetes infrastructure. This documentation will not explain Kubernetes infrastructure setup. The links below are intended as hints for getting started.

  1. Create Kubernetes infrastructure.

  2. As part of your Kubernetes setup, create NFS storage with a root folder:

    whose name we recommend you set to <NAMESPACE-PLACEHOLDER>, the same as your namespace, so that you can support multiple deployments on the same cluster, with the same NFS storage -- see productNS and nfsPath in values.yaml. To enable non-root container users to read and write files in this folder that is dedicated to your RapidMiner stack, set the following permissions:

     chown -R 2011.root <NAMESPACE-PLACEHOLDER>
     chmod g+w <NAMESPACE-PLACEHOLDER>
    
  3. Create a namespace, also with name <NAMESPACE-PLACEHOLDER>.

  4. Have your server certificate ready. Alternatively, use the built-in Let's Encrypt.

Introduction to Helm

Helm is a package manager for Kubernetes. A Helm Chart bundles the Kubernetes YAML files as templates, which you then configure via the file values.yaml. The details of this configuration depend on the details of your Kubernetes deployment. You and I may share the same templates, but our configurations (values.yaml) will differ. A typical Chart is a folder resembling the following:

mychart/
  Chart.yaml
  values.yaml
  charts/
  templates/
Chart.yaml
The Chart.yaml file contains a description of the chart. You can access it from within a template.
values.yaml
The file that defines your configuration, it contains the default values for a chart. These values may be overridden during helm install or helm upgrade.
charts/
The charts/ directory may contain other charts, called subcharts.
templates/
This folder contains the Kubernetes YAML files, as templates. When Helm evaluates a chart, it will send all of the files in the templates/ directory through the template rendering engine. It then collects the results of those templates and sends them on to Kubernetes. The placeholders in the YAML files are defined by values.yaml.

Read more:

Introductory videos:

Instructions

To simplify the configuration of the Kubernetes YAML files, we use Helm, the package manager for Kubernetes.

  1. Make sure that your Kubernetes infrastructure is in place, including Helm.

  2. Download the Helm archive, and extract values.yaml, renaming it to custom-values.yaml:

     helm show values ./rapidminer-aihub-9.10.0.tgz > custom-values.yaml
    
  3. Edit custom-values.yaml and define your configuration by setting the appropriate values.

  4. Then apply the following command to the Kubernetes cluster:

helm upgrade -n <NAMESPACE-PLACEHOLDER> --install rapidminer-aihub --values custom-values.yaml ./rapidminer-aihub-9.10.0.tgz

Note that the value <NAMESPACE-PLACEHOLDER> is the same as the one you gave in custom-values.yaml for the key productNS.

EBS volumes are sensitive to multi-attach errors during rolling updates. It is best to scale down all the deployments before the update.

The HELM configuration file (values.yaml)

common:
  domain: "<FQDN-PLACEHOLDER>"
# If you need to obtain LetsEncrypt certificate first, please temporarly change the protocol to http://
  public_url: "https://<FQDN-PLACEHOLDER>"
  sso_public_url: "https://<FQDN-PLACEHOLDER>"
# The namespace of the deployment
  productNS: "<NAMESPACE-PLACEHOLDER>"
# The docker image tag
  mainVersion: "9.10.0-gen2"
# The docker image tag for Coding Environment Storage
  cesVersion: "9.10.0-gen2"
# Docker registry prefix
  dockerURL: "rapidminer/"
# The timezone of the deployment
  timeZone: "<TIMEZONE-PLACEHOLDER>"
# platform related Values, please choose one from:
# "OpenShift" : OpenShift related security and other infrastructure settings
# "EKS" : Amazon Elastic Kubernetes related security and other infrastructure settings
  platfrom: "EKS"
# If nodeSelector value is not set, then the nodeselector feature will not inserted to deployments
# any label of the worker nodes can be used
#  nodeSelector:
#    <NODE-LABEL-1-NAME-PLACEHOLDER>: "<NODE-LABEL-1-VALUE-PLACEHOLDER>"
#    <NODE-LABEL-2-NAME-PLACEHOLDER>: "<NODE-LABEL-2-VALUE-PLACEHOLDER>"
  nodeSelector: {}
# If the name of the image pull secret is empty, this part of deployment will not inserted
  imagePullSecret: []
# The name of the kubernetes secret, which contains the RapidMiner License
# If empty, the default admin user can provide it on the webui
  licenseSecret: ""
# the key of the license in the Kubernetes secret, default value is "SERVER_LICENSE"
  licenSecretKey: "SERVER_LICENSE"
# This will be the initial user, which will have admin permission in the deployment.
  initialUser: "admin"
# Initial password for the initial user
  initialPass: "<ADMIN-PASS-PLACEHOLDER>"
# The built in OIDC server realm, this realm will be used by the components in the SSO communication (KeyCloak)
  defaultSSORealm: "master"
# Default SSL requirement to access KeyCloak SSO
  ssoSSL: "external"
# Default storageclass, for example ebs-us-west-2a, see templates/002-storageclasses.yaml
  defaultstorageClass: "<STORAGECLASS-PLACEHOLDER>"
# IP / FQDN of the NFS Server
  nfsServer: "<NFS-SERVER-PLACEHOLDER>"
# Path on the NFS Server, this folder should exists before deploying
# If you have multiple deployment on the same cluster, it is recommended to include the namespace in that folder name
  nfsPath: "/<NAMESPACE-PLACEHOLDER>"
  nfsMountOpts:
    - vers=4
    - minorversion=1
    - sec=sys
# If you have multiple deployment on the same cluster, it is recommended to include the namespace in that name
  nfsStorageName: nfs-<NAMESPACE-PLACEHOLDER>
  createNamespace: "false"

AWS:
# Only the value true is supported yet
  allowEBS: "true"
  # initEBSStorageClasses should be true if the storageclasses were created by helm
  initEBSStorageClasses: "true"
# The region of the ebs volumes, for example us-west-2, this value should align to defaultStorageClass property
  EBSRegion: "<EBS-REGION-PLACEHOLDER>"
OKD: {}

proxy:
  serviceName: "proxy-svc-pub"
  imageName: "rapidminer-proxy"
  version: "9.10.0-gen2"
  configName: "proxy-config"
  storageClass: "<STORAGECLASS-PLACEHOLDER>"
  debug: "false"

letsEncrypt:
  imageName: "rm-letsencrypt-client"
  version: "9.10.0-gen2"
  configName: "letsencrypt-client-config"
  allowLetsEncrypt: "true"
  certsHome: "/certificates/"
  webMasterEmail: "<WEBMASTER-EMAIL-PLACEHOLDER>"

landingPage:
  serviceName: "landing-page-svc-priv"
  imageName: "rapidminer-deployment-landing-page"
  version: "9.10.0-gen2"
  configName: "landing-page-config"
  storageClass: "<STORAGECLASS-PLACEHOLDER>"
  ssoClientId: "urn:rapidminer:landing-page"
  ssoClientSecret: "<LANDING-PAGE-OIDC-CLIENT-SECRET-PLACEHOLDER>"
  debug: "false"

serverDB:
  serviceName: "rapidminer-server-postgres-svc-priv"
  imageName: "postgres-10"
  version: "9.10.0-gen2"
  configName: "rapidminer-server-postgres-config"
  storageClass: "<STORAGECLASS-PLACEHOLDER>"
  dbName: "<SERVER-DB-NAME-PLACEHOLDER>"
  dbUser: "<SERVER-DB-USER-PLACEHOLDER>"
  dbPass: "<SERVER-DB-PASS-PLACEHOLDER>"


server:
  serviceName: "rapidminer-server-svc-priv"
  imageName: "rapidminer-server"
  version: "9.10.0-gen2"
  configName: "rapidminer-server-config"
  storageClass: "<STORAGECLASS-PLACEHOLDER>"
  activeMQServiceName: "rapidminer-server-amq-svc-priv"
  activeMQUser: "<SERVER-AMQ-USER-NAME-PLACEHOLDER>"
  activeMQPass: "<SERVER-AMQ-PASS-PLACEHOLDER>"
  ssoClientId: "urn:rapidminer:server"
  ssoClientSecret: "<SERVER-OIDC-CLIENT-SECRET-PLACEHOLDER>"
  memLimit: "2048M"
  platformAdminSyncDebug: "False"
  legacyRESTBasicAuth: "false"

jobagent:
  imageName: "rapidminer-execution-jobagent"
  version: "9.10.0-gen2"
  configName: "job-agents-config"
  storageClass: "<STORAGECLASS-PLACEHOLDER>"
  authSecret: "<SERVER-AMQ-SECRET-PLACEHOLDER>"
  jobQueue: "DEFAULT"
  containerCount: "1"
  containerMemLimit: "2048"
  containerJavaOpts: ""
  javaOpts: "-Djobagent.container.jvmCustomProperties=Dlogging.level.com.rapidminer=INFO"

keycloak:
  serviceName: "keycloak-svc-priv"
  imageName: "rapidminer-keycloak"
  version: "9.10.0-gen2"
  configName: "keycloak-config"
  storageClass: "<STORAGECLASS-PLACEHOLDER>"
  proxyAddrForward: "true"

keycloakDB:
  serviceName: "keycloak-postgres-svc-priv"
  imageName: "postgres-10"
  version: "9.10.0-gen2"
  configName: "keycloak-postgres-config"
  storageClass: "<STORAGECLASS-PLACEHOLDER>"
  vendor: "POSTGRES"
  dbName: "<KEYCLOAK-DB-NAME-PLACEHOLDER>"
  dbUser: "<KEYCLOAK-DB-USER-PLACEHOLDER>"
  dbPass: "<KEYCLOAK-DB-PASS-PLACEHOLDER>"
  dbSchema: "public"
  useSSL: "false"

rmInit:
  imageName: "rapidminer-deployment-init"
  version: "9.10.0-gen2"
  configName: "rm-deployment-init-config"
  storageClass: "<STORAGECLASS-PLACEHOLDER>"
  debug: "false"

platformAdmin:
  serviceName: "platform-admin-webui-svc-priv"
  imageName: "rapidminer-platform-admin-webui"
  version: "9.10.0-gen2"
  configName: "platform-admin-webui-config"
  storageClass: "<STORAGECLASS-PLACEHOLDER>"
  proxyURLSuffix: "/platform-admin"
  proxyRTSWebUISuffix: "/rts-admin"
  ssoClientId: "urn:rapidminer:platform-admin"
  ssoClientSecret: "<PLATFORM-ADMIN-OIDC-CLIENT-SECRET-PLACEHOLDER>"
  disablePython: "false"
  disableRTS: "false"
  debug: "false"

ces:
  imageName: "rapidminer-coding-environment-storage"
  version: "9.10.0-gen2"
  configName: "rapidminer-coding-environment-storage-config"
  pythonPackageLink: "git+https://github.com/rapidminer/python-rapidminer.git@9.9.0.0"
  ubuntuUid: "9999"
  ubuntuGid: "9999"
  rapidMinerUser: "rapidminer"

rts:
  serviceName: "rts-agent-svc-priv"
  imageName: "rapidminer-execution-scoring"
  version: "9.10.0-gen2"
  configName: "rts-agent-config"
  storageClass: "<STORAGECLASS-PLACEHOLDER>"
  proxyURLSuffix: "/rts"
  initSharedCondaSettings: "true"
  waitForLicenses: "1"
  basicAuth: "true"
  rtsServerLicense: "false"

jupyterDB:
  serviceName: "rm-jupyterhub-db-svc"
  imageName: "rapidminer-jupyterhub-postgres"
  version: "9.10.0-gen2"
  configName: "jupyterhub-postgres-config"
  storageClass: "<STORAGECLASS-PLACEHOLDER>"
  vendor: "POSTGRES"
  dbName: "<JUPYTERHUB-DB-NAME-PLACEHOLDER>"
  dbUser: "<JUPYTERHUB-DB-USER-PLACEHOLDER>"
  dbPass: "<JUPYTERHUB-DB-PASS-PLACEHOLDER>"

jupyterHub:
  proxyServiceName: "jupyterhub-proxy-svc-priv"
  proxyAPIServiceName: "jupyterhub-proxy-api-svc-priv"
  serviceName: "jupyterhub-hub-svc-priv"
  imageName: "rapidminer-jupyterhub-jupyterhub"
  version: "9.10.0-gen2"
  configName: "jupyterhub-config"
  storageClass: "<STORAGECLASS-PLACEHOLDER>"
  proxyURLSuffix: "/jupyter"
  cryptKey: "51fd01d4bdf18df039744272fe400dc06cb2b00a41add94b3f3fd90ede301df2"
  debug: "False"
  tokenDebug: "False"
  proxyDebug: "False"
  dbDebug: "False"
  spawnerDebug: "False"
  stackName: "default"
  ssoClientId: "urn:rapidminer:jupyterhub"
  ssoClientSecret: "<JUPYTERHUB-OIDC-CLIENT-SECRET-PLACEHOLDER>"
  ssoUserNameKey: "preferred_username"
  ssoResourceAccKey: "resource_access"
  spawner: "kubespawner"
  apiProtocol: "http"
  k8sCMD: "/entrypoint.sh"
  k8sArgs: "[]"
  proxyPort: "8000"
  apiPort: "8001"
  appPort: "8081"
  envVolumeName: "rm-coding-shared-vol"

jupyterNoteBook:
  imageName: "rapidminer-jupyter_notebook"
  version: "9.10.0-gen2"
  memLimit: "2G"
  cpuLimit: "100"
  ssoUidKey: "X_NB_UID"
  ssoGidKey: "X_NB_GID"
  ssoCustomBindMountsKey: "X_NB_CUSTOM_BIND_MOUNTS"
  customBindMounts: ""
  storageAccessMode: "ReadWriteOnce"
  storageSize: "5Gi"
  storageClass: "<STORAGECLASS-PLACEHOLDER>"
  # Should be the name of the CES pvc (-nfs-shared-pvc)
  pvcName: "<NAMESPACE-PLACEHOLDER>-nfs-shared-pvc"
  pvcSubPath: "coding-shared"
  imagePullAtStartup: "False"
#  nodeSelector:
#    key: "<NODE-LABEL-1-NAME-PLACEHOLDER>"
#    value: "<NODE-LABEL-1-VALUE-PLACEHOLDER>"
  nodeSelector: {}

grafanaProxy:
  serviceName: "grafana-proxy-svc-priv"
  imageName: "rapidminer-grafana-proxy"
  version: "9.10.0-gen2"
  configName: "grafana-proxy-config"

grafanaAnonProxy:
  serviceName: "grafana-anonymous-proxy-svc-priv"
  imageName: "rapidminer-grafana-proxy"
  version: "9.10.0-gen2"
  configName: "grafana-anonymous-proxy-config"

grafana:
  serviceName: "grafana-svc-priv"
  imageName: "rapidminer-grafana"
  version: "9.10.0-gen2"
  configName: "grafana-config"
  storageClass: "<STORAGECLASS-PLACEHOLDER>"
  proxyURLSuffix: "/grafana"
  ssoClientId: "urn:rapidminer:grafana"
  ssoClientSecret: "<GRAFANA-OIDC-CLIENT-SECRET-PLACEHOLDER>"
  disableSanitizeHTML: "true"

tokenTool:
  serviceName: "token-tool-svc-priv"
  imageName: "rapidminer-deployment-landing-page"
  version: "9.10.0-gen2"
  configName: "token-tool-config"
  storageClass: "<STORAGECLASS-PLACEHOLDER>"
  proxyURLSuffix: "/get-token"
  ssoClientId: "urn:rapidminer:token-tool"
  ssoClientSecret: "<TOKEN-TOOL-OIDC-CLIENT-SECRET-PLACEHOLDER>"
  ssoCustomScope: "openid info offline_access"
  customContent: "get-token"
  debug: "false"