1. Documentation
2. 2024.0 (Latest) 10.3 10.2 10.1 10.0 9.10 9.9 9.8 9.7 9.6 9.5 9.4 9.3 9.2 9.1 9.0 8.2 8.1 8.0 7.6
3. Studio
4. How to
5. Cloud connectivity
6. Using Google Cloud Storage

Using the Google Cloud Storage Connector

The Google Cloud Storage Connector allows you to access your Google Cloud Storage directly from RapidMiner Studio. Both read and write operations are supported. You can also read from a set of files in a Google Cloud Storage directory, using the Loop Google Storage operator. This document will walk you through how to:

• Connect to your Google Cloud Storage account
• Read from Google Cloud Storage

Connect to your Google Cloud Storage account

Before you can use the Google Cloud Storage connector, you have to configure a new Google Cloud Storage connection. For this purpose, you will need the connection details of your account. This includes a project id and either an access token, or a private key for a service account.

1. Open the Manage Connections dialog in RapidMiner Studio by going to Connections > Manage Connections.

2. Click on Add Connection in the lower left:

   

3. Enter a name for the new connection and select Google Cloud Storage Connection as the Connection Type:

   

4. Fill in the connection details of your Google Storage account. You have two alternative options for that, see next two steps for details.

5. You may use an access token that you get after you allow RapidMiner to access your cloud account on a consent screen. This is the default option: leave use service account unchecked and follow the steps below.

   1. To the right of the access token field, click the button to request an access token.

   2. Click on Request access token to open the Google website in your browser. If you are not already logged into your Google Cloud account, you will have to do so now. You can manually copy the URL by clicking on Show URL instead.

      

   3. Click Allow to give RapidMiner access to your Google Cloud account and to generate a token. This will bring you to a page where you can see the access token. Copy the code you get there.

      

   4. Return to RapidMiner Studio, enter the access token, and click Complete:

      

   5. Specify the project id for the connection as well.

6. Alternatively, you may setup a Service account for your project. In this case, check use service account flag and follow the steps below.

   1. After setting up the Service account, create and download a JSON key for it. Paste the entire JSON file content (e.g. using a text editor and the clipboard) into the private key file content field.

      

   2. Specify the project id for the connection as well.

7. While not required, we recommend testing your new Google Cloud Storage connection by clicking the Test button.

8. Click Save all changes to save your connection and close the Manage Connections window. You can now start using the Google Cloud Storage operators.

Read from Google Cloud Storage

The Read Google Storage operator reads data from your Google Cloud Storage account. The operator can be used to load arbitrary file formats, since it only downloads and does not process the files. To process the files you will need to use additional operators such as Read CSV, Read Excel, or Read XML.

Let us start with reading a simple csv file from Google Cloud Storage.

1. Open a new process in RapidMiner Studio and choose Blank Project from the list. Drag the Read Google Storage operator into the Process view, and connect its output port to the result port of the process:

   

2. Select your Google Cloud Storage connection from the connection drop down menu in the Parameters view.

   

3. Click on the file chooser button to view the files in your Google Cloud Storage account. Select the file that you want to load and click Open. Note that you need storage.buckets.list permissions on the project to be able to list the buckets and use the file chooser. If you do not have that permission, please type the path from which you want to read from directly into the parameter field.

   

   As mentioned above, the Read Google Storage operator does not process the contents of the specified file. In our example, we have chosen a csv file (a comma separated values file). This file type can be processed via the Read CSV operator.

4. Add a Read CSV operator between the Read Google Storage operator and the result port. You may set the parameters of the Read CSV operator - such as column separator -, depending on the format of your csv file:

   

5. Run the process! In the Results perspective, you should see a table containing the rows and columns of your choosen csv file:

   

You could now use further operators to work with this document, e.g., to determine the commonness of certain events. To write results back to Google Cloud Storage, you can use the Write Google Storage operator. It uses the same connection type as the Read Google Storage operator and has a similar interface. You can also read from a set of files in a Google Cloud Storage directory, using the Loop Google Storage operator. For this you need to specify the connection and the folder, which you want to process, as well the steps of the processing loop with nested operators. For more details read the help of the Loop Google Storage operator.