You are viewing the RapidMiner Hub documentation for version 2024.0 - Check here for latest version
Custom logging
If the default logging behaviour is not sufficient, you might want to alter the logging configuration and provide custom log appenders. This guide will give you an overview on how to customize the log appenders for different components.
Job Agent
The RapidMiner Job Agent uses logback as its logging framework. The following XML file describes the default configuration:
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<include resource="org/springframework/boot/logging/logback/defaults.xml" />
<include resource="org/springframework/boot/logging/logback/console-appender.xml" />
<property name="LOG_FILE" value="${LOG_FILE:-${LOG_PATH:-${LOG_TEMP:-${java.io.tmpdir:-/tmp}}}/agent.log}"/>
<appender name="AGENTLOGFILE"
class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${LOG_PATH}/agent.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${LOG_PATH}/agent.log.%d{yyyy-MM-dd}</fileNamePattern>
</rollingPolicy>
<encoder>
<pattern>${FILE_LOG_PATTERN}</pattern>
</encoder>
</appender>
<root level="INFO">
<appender-ref ref="CONSOLE" />
<appender-ref ref="AGENTLOGFILE" />
</root>
</configuration>
By default two appenders will be used: The CONSOLE and the AGENTLOGFILE. The CONSOLE appender is responsible for
logging to shell, whereas the AGENTLOGFILE appender will create a rolling log file on disk.
Custom appender
To register a custom log appender, you need to create a configuration XML, save it as a file (logback.xml), copy or bind it into the
docker deployment and register the location in an environment variable
LOGGING_CONFIG = ${jobagent.baseDir}/home/config/logback.xml
The documentation about different log appenders can be found in the logback documentation.
Example: SMTP appender
Some appenders require third-party libraries. Before adding an appender check the dependency list
and add the required libraries to the libs/ folder of the Job Agent docker deployment. In this case the SMTP appender requires the
Java Mail API 1.6.0 and the JavaBeans Activation Framework 1.1 library.
The SMTP appender will collect logging events and sends the content via email. By default, the appender is triggered by
the logging precedence ERROR and above. The detailed documentation about this appender can be found in the logback documentation.
The following configuration adds an EMAIL appender to the default logging configuration:
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<include resource="org/springframework/boot/logging/logback/defaults.xml" />
<include resource="org/springframework/boot/logging/logback/console-appender.xml" />
<property name="LOG_FILE" value="${LOG_FILE:-${LOG_PATH:-${LOG_TEMP:-${java.io.tmpdir:-/tmp}}}/agent.log}"/>
<appender name="EMAIL" class="ch.qos.logback.classic.net.SMTPAppender">
<smtpHost>$SMTP_HOST</smtpHost>
<smtpPort>$SMTP_PORT</smtpPort>
<SSL>true</SSL>
<username>$USERNAME</username>
<password>$PASSWORD</password>
<to>$MAIL_TO</to>
<from>$MAILFROM</from>
<subject>Error: %logger{20} - %m</subject>
<layout class="ch.qos.logback.classic.PatternLayout">
<pattern>%date %-5level %logger{35} - %message%n</pattern>
</layout>
</appender>
<appender name="AGENTLOGFILE"
class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${LOG_PATH}/agent.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${LOG_PATH}/agent.log.%d{yyyy-MM-dd}</fileNamePattern>
</rollingPolicy>
<encoder>
<pattern>${FILE_LOG_PATTERN}</pattern>
</encoder>
</appender>
<root level="INFO">
<appender-ref ref="EMAIL" />
<appender-ref ref="CONSOLE" />
<appender-ref ref="AGENTLOGFILE" />
</root>
</configuration>
Please make sure to replace the following properties before actually using this appender:
$SMTP_HOST: the hostname of your SMTP server, e.g. mail.your-organization.com$SMTP_PORT: the port of your SMTP server, e.g. 465$USERNAME: the username of the account which is responsible for sending the mails$PASSWORD: the corresponding password$MAIL_TO: the mail address which should receive the log events$MAIL_FROM: the mail address of the sender
Job Container
The RapidMiner Job Container uses logback as its logging framework. The following XML file describes the default configuration:
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<include resource="org/springframework/boot/logging/logback/defaults.xml" />
<include resource="org/springframework/boot/logging/logback/console-appender.xml" />
<appender name="CONTAINERLOGFILE"
class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${LOG_FILE}</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${LOG_FILE}.%d{yyyy-MM-dd}</fileNamePattern>
</rollingPolicy>
<encoder>
<pattern>${FILE_LOG_PATTERN}</pattern>
</encoder>
</appender>
<logger name="com.rapidminer.Process" level="OFF"/>
<logger name="com.rapidminer.execution.jobcontainer.execution.ExecutionProcessListener" level="OFF"/>
<root level="INFO">
<appender-ref ref="CONSOLE" />
<appender-ref ref="CONTAINERLOGFILE" />
</root>
</configuration>
By default two appenders will be used: The CONSOLE and the CONTAINERLOGFILE. The CONSOLE appender is responsible for
logging to shell, whereas the CONTAINERLOGFILE appender will create a rolling log file on disk.
Custom appender
To register a custom log appender, you need to create the XML, save it as a file (logback-jobcontainer.xml),
copy or bind it into the docker deployment, preferably in the home/config folder of the Job Agent. Afterwards you need to instruct the Job Agent to forward
this configuration to all spawned Job Containers. Add the following environment variables
JOBAGENT_CONTAINER_JVM_CUSTOM_OPTIONS = -Dlogging.config=${jobagent.baseDir}/home/config/logback-jobcontainer.xml
The documentation about different log appenders can be found in the logback documentation.