Introduction

Reltio provides a thorough step-by-step documentation on how to configure your Reltio GBQ Connector. You will be guided on how to create a project, a dataset, how to add roles and principals to a service account. However, there are a few things that you can miss while following the documentation guide, so here are some insights on how to properly create a connection between Reltio and GBQ.

Step-by-step guide

1. Create a GCP project in your GBQ account

In Google Cloud Platform a project is a place where your Google Cloud resources are allocated. It is the entity that holds the metadata of your application such as settings and permissions. When those resources are within a single project they can communicate and work together easily.

As our first step towards establishing a GBQ connection, we need to have a GCP project where we can provide permissions and create datasets.

Documentation on how to do that can be found here: Create a GCP project in your GBQ account

Information like Project name, Project ID and Project number can be found in the menu, in the top-right corner of your screen. These will be needed later.

2. Create a BigQuery dataset in the GCP project

In Google Cloud Platform a dataset is a logical container that organizes and controls the access to your resources in GCP such as tables, views, procedures and functions.

We need to create a dataset as our next step, because this will enable the event delivery from your Reltio tenant to the GBQ dataset which is the final destination of the connector.

Documentation on how to do that can be found here: Create a BigQuery dataset in the GCP project

After the dataset is created, you need to authorize it so that it can be accessed by Reltio. In order to do that, you need to go to Google Big Query Studio and select the dataset. Click the Sharing button on the menu and then Authorize dataset. In the window that pops up, select the name of the dataset and after that click Add authorization.

If you do not authorize the dataset for your Reltio tenant, the connection might result in some errors.

3. Create a custom IAM role at a project level and at a dataset level

In GBQ, IAM lets you grant granular access to specific resources in BigQuery. In GBQ a principal can be a user, a group of users or a service account. When a concrete principal calls the GBQ API, it must be granted the right IAM permissions to use the resource. This is done via granting an IAM role to the principal. Roles can be assigned at a project and at a dataset level.

Documentation on how to do that can be found here: Create a custom IAM role at a project/dataset level

Reltio advises to add the following permissions to the roles:

• For the IAM role at the project level:
  • bigquery.jobs.create
  • bigquery.jobs.get
  • serviceusage.services.use
• For the IAM role at the dataset level:
  • bigquery.tables.create
  • bigquery.tables.delete
  • bigquery.tables.get
  • bigquery.tables.getData
  • bigquery.tables.list
  • bigquery.tables.update
  • bigquery.tables.updateData

4. Grant the roles permissions to your GCP service account

Prerequisites

Before you start, make sure that you know the appropriate Google Cloud Platform (GCP) service account:

• Reltio owned GCP service account for Non-HIPAA Tenants: data-pipeline-hub-prod@customer-facing.iam.gserviceaccount.com
• Reltio owned GCP service account for HIPAA Tenants: data-pipeline-hub-hipaa@customer-facing-hipaa.iam.gserviceaccount.com

Guide

Documentation on how to do that can be found here: Grant the roles permissions to your GCP service account

The following roles should be granted to the Service account:

• Reltio GBQ connector Project Level role
• Reltio GBQ connector Dataset Level role
• Service Usage Consumer

It is very important to check in the Permissions of your Dataset whether you have under BigQuery Data Owner → {project_name}@appspot.gserviceaccount.com principal.

5. Configure a new GBQ pipeline in Reltio’s Console

You can configure the a new GBQ pipeline either through Reltio’s API or in Reltio’s Console application.

This documentation shows how to make the configuration in Reltio’s Console: Configure a new GBQ pipeline in Reltio’s Console

There are few things to be noted here:

• Pipeline connection section
  • Be very careful when choosing pipeline name as its length must be between 3 and 20 characters and cannot include dashes (“-”) or underscore (“_”), otherwise you will receive an error related to the tenant.
• Authentication section
  • The Service Account Key File can be found when you go to GBQ → IAM & Admin → Service accounts and choose the service account that you have – check step 4 of this document (Depending on the GBQ environment, you can have a normal account instead of a service account). In the Permissions tab, go to Keys and click Add Key. Choose JSON as a format and the key will be downloaded to your device.
  • Every time you add new roles/permissions to the service account, you need to generate a new key, because it cannot be updated automatically.
• Project and Dataset details section
  • This information can be found when you go to Google BigQuery Studio, select the dataset and click on the Copy button in the top-right corner of the screen. You will see the Project (GCP project name), Dataset (GCP dataset name) and Location. The location parameter is optional, but it is recommended.
• Data delivery options section
  • On the right side of this section, choose JSON for storage format and put a tick on Split Table option (this will help you to find data and navigate in GBQ easier).

After you are done, you can click on Save and create tables/views button. If everything is configured correctly, then all the options in the menu of the pipeline will be enabled. Also, check whether the Status toggle is on as well and if not, click on it to enable it.