August 3, 2023 / August 23, 2024 by Danail Iliev

Reltio Lambda LCA: “beforeMerge”

What Are We Building?
Prerequisites
Step 1: Setup AWS-Side of Things
Step 2: Collaboration with Reltio Support
Step 3: Configuring the LCA in L3
Step 4: Experimenting & Exploring the Input Structure
Step 5: The Actual Logic
The Conditions
Accepting the Merge
Rejecting the Merge
First Reltio API Call
Putting the Pieces Together
Summary
Further Improvements
Resources

Reltio Life Cycle Actions (widely known as LCA) allow you to execute custom operations on events in occurring in the life cycle of entities (or relationships). Some of the supported life cycle hooks include: beforeMerge , afterMerge, beforeSave, afterDelete (for exhaustive list visit the first link in the Resources section of this article).

LCAs open up entirely new range of possibilities for your Reltio implementation. In this article we are going to take a closer look into the beforeMerge LCA hook and how you could leverage it for minimizing Data Steward’s mistakes on merge. Such mistakes could possibly lead to a lot of work needed for fixing the issue afterwards – especially if your mistakes are automatically propagated to downstream consumer systems.

What Are We Building?

This article is going to walk you through creating a simplified Reltio LCA via AWS Lambda. The LCA will be executed before any merge on a Reltio tenant, and will serve as a “judge” – only letting the merge pass through if certain conditions are met. While any Lambda-supported language could be used to create the LCA, we believe JavaScript is a great choice because of its big popularity and the simplicity of the language itself which would allow for virtually anyone to read the code and understand its meaning.

The code examples in this article do not follow the best programming principles or teach you how to work with Node.js & AWS Lambda in general – the emphasis is on the LCA principles instead.

Prerequisites

Basic understanding of the Reltio platform and the concepts of merge & crosswalks
Some experience with AWS Lambda
Node.js
Reltio tenant & AWS account (if you follow along)

Step 1: Setup AWS-Side of Things

Lets start by preparing the development environment for our LCA. Since Reltio should invoke the Lambda from their side we have to create a user with the appropriate permissions for it to do so. Working with AWS is not the purpose of this article so we will only briefly mention it, but in the end the deliverable should be a pair of access tokens for programmatic access. Make sure you:

Have created the user and have access key and secret key
Have created a JavaScript Lambda function
The user has permissions to trigger the Lambda

Step 2: Collaboration with Reltio Support

Next, we have to make sure our Reltio tenant can access the Lambda by configuring the AWS credentials. Unfortunately, at the time of writing this article (31 July 2023) there is no self-service portal for this step of the process. However, Reltio’s support tend to answer quickly to such requests so you shouldn’t worry too much about your timelines.

Make sure you provide the following information when opening your ticket to the Reltio support:

Reltio Environment
Reltio Tenant ID
AWS credentials (Access & Secret keys)
- I would not recommend sharing the secret key directly on the ticket, so do not do that initially. The support will walk you through a secure way to share this information with them so make sure you ask for that.

Step 3: Configuring the LCA in L3

Once the support confirms everything is set up from Reltio’s side, it is time for you to modify the L3 configuration (also known as business configuration) so that an LCA hook is involved in the before merge process. To enable the LCA in your tenant you have to add the following section (specifying the type of LCA hook you want).

"lifecycleActions": {
    "beforeMerge": [
      "Lambda/JSON/MyLambda"
    ]
}

You may notice that the string we put inside the beforeMerge array is somewhat strange. As per Reltio’s documentation it contains three parts – Lambda/Protocol/ActionName

"Lambda" – notifying Reltio that we are going to use Lambda instead of the typical Reltio framework built in LCA
Protocol – either JSON or BinaryJSON
ActionName – the name of our Lambda
- For this article, I have named the Lambda function “MyLambda”

If you face issues the first time triggering the LCA, go back to step 1. (AWS) and make sure your Lambda is created in the correct region that your tenant can work with – you could ask support when providing your credentials. Also note how the beforeMerge property is an array, meaning we could add more than one Lambda to be executed. We are only using one, but it’s good to know what your options are. You could also trigger your LCA conditionally, by providing a filter – something you can read more about on Reltio’s official documentation portal – second link in the Resources section of the article.

Step 4: Experimenting & Exploring the Input Structure

This step is optional, but since you are going to need a sample input to write and test your function with you might as well follow along. Lets deploy a simple Lambda with console.log statement that will provide us with sample input to continue with our development.

export const handler = async (data) => {
  console.log(data);
};

We can now go ahead and trigger our LCA for the first time! Navigate to the Reltio UI and merge two entities of choice. Do not worry that the merge you are trying to perform is going to fail.

Once the merge is done you should navigate to the Lambda logs in your AWS account. You will see there that the function logged the output with a format like the following:

{
  environment: 'https://<environment>.reltio.com/reltio',
  tenant: '<tenant_id>',
  token: '<bearer_token>',
  requests: [ { data: [Object], actions: [Array] } ],
  hook: 'beforeMerge'
}

Lets explore what each property brings to the table:

environment & tenant: Correspond to your Reltio tenant’s information – very helpful once you want to have the same codebase working for multiple Reltio environments – Dev, QA, Production – without storing these as environment variables.
token: A Bearer token that we could use for our subsequent API calls to Reltio.
requests: We will explore this object in detail in the next paragraph.
hook: The name of the hook we are using – helpful since the same Lambda function could be triggered from different hooks.

The requests section of the input has the following structure:

[
	{
        "data": {
            "object": {
                ...
            },
            "type": "configuration/entityTypes/Organization",
            "losers": [
                "URI1",
                "URI2",
                ...
            ]
        },
        "actions": [
            "MyLambda"
        ]
    }
]

A few things you should notice here:

The requests property is an array that contains a single object.
The object inside the array contains two root-level properties – data and actions.
The actions array refers to the name of the Lambda. We could safely ignore this property.
The data object has the properties:
1. object which contains the data exactly the same as if we were doing an API GET call to Reltio to retrieve it.
2. type which helps us quickly identify the type of the object we are working with – remember our Lambda can be configured to be triggered on many occasions for different object types and hooks.
3. losers is an array of Reltio URIs that lost the merge. So in our scenario the winning entity URI will be in data -> object -> uri , while this array would refer to the entities that lost the merge.

Step 5: The Actual Logic

There are several topics that we need to cover before we are able to fully understand the beforeMerge LCA and be able to complete our work:

What conditions we will put on the merge
How do we accept a merge
How do we reject a merge
How to leverage our Bearer token in order to pull entities from Reltio

The Conditions

Going forward, the code may not work on your tenant. Depending on the L3 configuration in your Reltio tenant, you may have different entity types and attribute names. But the principle should be the same and you would be able to replicate and modify this code according to your needs.

Keeping the project small, we are only going to implement these simple conditions:

Accept the merge only if both entities have Salesforce crosswalk
Reject the merge if either entity has Name = “Test” (overrides above condition)

Accepting the Merge

The merge is considered as accepted if we output a non-null value from our Lambda function. But this is not following the best practices and could have an unwanted side effect. So we will provide the following object as output:

[
  {
      "environment": data.environment,
      "tenant": data.tenant,
      "successful": true,
      "type": null
  }
]

Rejecting the Merge

Similarly, the merge is considered rejected if we output a null or error value from our Lambda function. Following the best practices & providing a human-readable error message to the user, we are going to provide an output like the following:

[{
    "environment": "<environment>",
    "tenant": "<tenant>",
    "type": null,
    "errors": {
        "errorDetailMessage": "Conditions for merge not fulfilled!"
    },
    "successful": false
}]

This will result in the following error from the UI:

Naturally, you could provide a more meaningful error message for your implementation.

First Reltio API Call

As you may notice, we only receive the winner entity in the input data. But the checks we want to perform require having information about both entities. To get that, we are going to use the Bearer token we received as input in order to perform API call(s) to Reltio and pull the entities we need based on the loser URIs we have.

We are going to use the axios.js library as HTTP Client, but skip all the details on how to install npm packages to your Lambda instance – you could check the last link in the Resources section at the end of this article to get an understanding on how that’s done.

Lets create a helper function we could use every time we need to pull entities from our Reltio tenant:

const getReltioEntity = async (environment, tenant, uri, token) => {
    const finalUrl = `${environment}/api/${tenant}/${uri}`;
    const axiosResponse = await axios.get(finalUrl, {
        headers: {
            'Authorization': `Bearer ${token}`
        }
    });

    return axiosResponse.data;
}

Putting the Pieces Together

Now that we have an understanding of how the different pieces work, it’s time we put them together and finalize our simple beforeMerge LCA. Lets start by outlining the main logic:

exports.handler = async (data) => {
    const requestData = data.requests[0].data;
    const losers = requestData.losers;

    // Hardcoded to work with just one loser - for simplicity reasons, otherwise a loop over the loser would be required.
    const loserUri = losers[0];
    const loserEntity = await getReltioEntity(data.environment, data.tenant, loserUri, data.token); 
    
    const isValidWinner = validateEntity(requestData.object);
    if(isValidWinner == false || validateEntity(loserEntity) == false) {
        return [{
            "environment": data.environment,
            "tenant": data.tenant,
            "type": null,
            "errors": {
                "errorDetailMessage": "Conditions for merge not fulfilled!"
            },
            "successful": false
        }];
    }
    
    return [{
        "environment": data.environment,
        "tenant": data.tenant,
        "successful": true,
        "type": null
    }];
};

Few things to note here are:

We’re only working with the first loser URI – to make the code even simpler and not focus on coding, but how beforeMerge hooks work instead
a validateEntity helper function was created – so that the code is more readable and we reduce repetitions

An example implementation of the validateEntity function could be:

const validateEntity = (entity) => {
    const hasSalesforceXwalk = entity.crosswalks.some(x => x.type == 'configuration/sources/Salesforce');
    
    let nameIsNotTest = true;
    if(entity.attributes.Name)
        nameIsNotTest = entity.attributes.Name[0].value != 'Test';
    
    return hasSalesforceXwalk && nameIsNotTest;
};

Summary

Leveraging LCA we could enable “hidden” functionality on the Reltio platform. You are now able to confidently start using your first life cycle actions to accept/reject merges based on conditions – by using AWS lambda and your newly gathered knowledge.

Further Improvements

Some of further improvements that I could suggest to you, if you decide to take your code to production level, are:

Automated testing – leveraging any of the JavaScript unit testing frameworks
Refactoring the code to higher level of quality – making it more modular and robust, reducing repetitions, separation of concerns
Implementing CI/CD pipeline
- Pulumi is a great choice of tooling for the job