You are viewing the RapidMiner Hub documentation for version 10.3 - Check here for latest version
Upgrade from 9.10.11 to 10.0.0
Preliminaries
To obtain a more satisfactory result, read these useful pre-migration hints. In particular,
- if you have any data in a repository in AI Hub 9, you should first move it into a project, and
- any web service functionality should be deployed to a scoring agent.
- Migration is not supported for Windows systems, only Linux.
- The license is migrated from the
.env
file and NOT from license files. If you set your license by uploading it to the tool or copying a file, please also paste it to the.env
file (SERVER_LICENSE
). - Download migration-bundle.tar.gz -- it contains all the files needed to run the migration.
An important point to observe is you can run the old stack even after migration, because the data is copied, not overwritten.
Instructions
By default, the migration process uses the compose standalone docker-compose
(notice the hyphen!). Newer docker installations contain compose
as a submodule and would normally replace docker-compose
with calls to docker compose
(without a hyphen!). However, because the migration script migration-orchestrator.sh
also uses docker-compose
and you would otherwise have to rewrite the script, it is recommended that you install the standalone docker-compose
, as follows:
sudo curl -SL https://github.com/docker/compose/releases/download/v2.12.2/docker-compose-linux-x86_64 -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
- Create a backup of all of your data and stop your 9.10.11 AI HUB compose stack
- Extract the contents of
migration-bundle.tar.gz
into the folder of your current stack, so the migration files are in the same folder as yourdocker-compose.yml
and.env
files - In the original
.env
file, please add a valid server license string to theSERVER_LICENSE
variable. Make sure to pull the images related to the migration:
docker-compose -f docker-compose-migration.yaml --env-file .env-migration pull
Run
migration-orchestrator.sh
:bash migration-orchestrator.sh
It executes the following main steps:
- Creates a backup from
docker-compose.yml
and.env
(docker-compose.yml.bak
and.env-bak
) - Compares your
docker-compose.yml
and.env
files to the stock 9.10.11-gen2 versions - Migrates the values from the original
.env
and does the variable renamings - Migrates the Keycloak database
- Migrates and updates Keycloak roles and clients (manual review still needed)
- Runs AI HUB backend with migration enabled
- Runs Job Agent with migration enabled (separate/additional Job Agent deployments should be handled manually)
- Runs Scoring Agent with migration enabled
- Migrates Grafana content (eg. dashboards, plugins)
- Handles volume restructuring related to the new stack
- Stops all services used during the migration process
- Creates a backup from
Pull the images for the new stack
docker-compose pull
Start the new stack with:
docker-compose up -d deployment-init
wait for this container to exit, then run:
docker-compose up -d
The new stack should be up and running and you should be able to log in with your original credentials.
Post migration instructions
It is always recommended to manually review the settings of the new stack as your old installation may contain customizations that the migration automation might skip. Some of these aspects can be found below (without being exhaustive).
Old Keycloak clients
The old stack contained several Keycloak clients named urn:rapidminer:<component-name>
(eg. urn:rapidminer:grafana
). The new stack uses new client names without the urn:rapidminer:
prefix (eg. grafana
), however these clients are newly created next to the old ones, avoiding to disrupt customized client settings.
By default, the new AI HUB 10.0.0 does not use the old clients, therefore they can be manually deleted if not needed.
New Keycloak roles
The AI HUB 10.0.0 introduces some new realm roles in Keycloak (eg. aihub:admin
) for access control. It is recommended to review the assignement of these roles to users and clients extensively after the migration.
The migration automation sets the following role assignements to users based on their existing effective roles:
- administrator
- Users with existing effective
administrator
role are givenaihub:admin
,aihub:queues:create
,aihub:projects:create
andaihub:schedule
roles
- Users with existing effective
- analyst
- Users with existing effective
analyst
role are givenaihub:projects:create
andaihub:schedule
roles
- Users with existing effective
Using unsigned Grafana plugins
Starting from AI HUB 10.0.0 the stack uses the official Grafana image. The migration automation handles copying your existing dashboards and plugins from the old stack.
In case there are any plugins that are unsigned by Grafana in your setup, you shall set the GF_PLUGINS_ALLOW_LOADING_UNSIGNED_PLUGINS
environment variable with the comma separated list of the unsigned plugins.
If you update/create this variable, make sure to include rapidminer-aihub-datasource
Running the old stack after a successful migration
The migration process does not alter any data in the old AI HUB stack, so it remains fully functional.
After a successful migration process, the old stack can be started with the following commands:
You should stop the new stack before trying to start the old stack after a successful migration.
docker-compose -f docker-compose-customer.yml --env-file .env-customer up -d rm-init-svc
You can follow the usual initialization process with
docker-compose -f docker-compose-customer.yml --env-file .env-customer logs -f rm-init-svc
After the following text is printed repeatedly to the screen, typically after 1-2 minutes:
[License initialization] Waiting for RapidMiner Server startup in order to install the license...
Start the whole old stack with
docker-compose -f docker-compose-customer.yml --env-file .env-customer up -d
Reset
In case of something goes wrong, here are some useful commands to reset the state.
Stop all migration related services
docker-compose -f docker-compose-migration.yaml --env-file .env-migration down
Restore files from backup
cp .env-bak .env && mv docker-compose.yml.bak docker-compose.yml
Delete migration related files
rm .env-customer docker-compose-customer.yml \
docker-compose-9.10.11-gen2.yml \
docker-compose-aix.yml \
docker-compose-migration.yaml \
.env-9.10.11-gen2 \
.env-aix \
.env-bak \
.env-migration \
migration-orchestrator.sh \
docker-compose-ja-migration.yaml \
.env-ja-migration
If the migration stack is started from an inconsistent state (some errors occured earlier and the folder was not cleaned properly), a .env-aix
folder may be present. It should be deleted.
Delete new, AI HUB 10 related volumes
docker volume rm $(basename $PWD)_aihub-db-vol \
$(basename $PWD)_aihub-home-vol \
$(basename $PWD)_job-agent-vol \
$(basename $PWD)_keycloak-db-vol \
$(basename $PWD)_scoring-agent-vol \
$(basename $PWD)_grafana-home-vol \
$(basename $PWD)_keycloak-kcadm-vol