Enrichment

Last modified by Danniar Firdausy on 2024/09/19 10:19

In this microlearning, we will introduce the relevant stateful components to configure the Enrichment operation in the context of State Generation. In case you want to learn the basics of the State Generation functionality, please check out eMagiz State Generation.

Should you have any questions, please get in touch with academy@emagiz.com.

1. Prerequisites

Basic knowledge of the eMagiz platform
Basic knowledge of the eMagiz State Generation

2. Key concepts

In this microlearning, we will introduce the relevant stateful components to configure the Enrichment operation in the context of State Generation functionality.

By Enrichment, we mean: a form of operation in which we add additional information into the raw message in such a way so that the message has more meaning. This additional information may come from an external source or other kind of sources to provide more context, detail, or value to the original message.

There are some check points to think about beforehand when setting up the Enrichment operation:

What is the raw message that needs to be enriched?
What is the additional information to be used to enrich the raw message?
How to store the past events as states, and for how long these states will be stored?
How to use the states to enrich the raw message?

3. Setting up Enrichment operation

In general, to configure the Enrichment operation, you first need to set up the storage to persist the states and then afterwards set up the enrichment component to retrieve the persisted states.

3.1 Storage Mechanism for State Persistance

As mentioned above, first, we need to think about how to configure the storage in the platform to persist the states, as well as, how to store the states into this storage. The idea here is so that these states later on can be retrieved from somewhere else in your integration landscape for your enrichment operation. For such a purpose, eMagiz provides two support objects to set up this storage mechanism and one flow component to store the states into it:

Infinispan cache manager
Infinispan metadata store
Metadata outbound channel adapter

Please refer to this State Persistence microlearning to learn on how to configure these support objects and flow component.

3.2 Stateful Component for Data Enrichment

Once you have configured the storage mechanism, now you can start with setting up the component to retrieve the stored state data for the enrichment operation. In principle, you can retrieve the stored state data with any flow components that support SpEL Expression (e.g., Standard transformer, Standard filter, etc.). However, to have a visual way of enrichment operation in the sense of adding additional information into the raw payload, you can use a flow component called the Standard Content Enricher that is listed under the "Transformer" category. Here, you have the option to add new fields into the raw payload by specifying the field name as well as the value.

intermediate-state-generation-enrichment-standard-content-enricher.png

To assign a value into the new fields by means of retrieving values from the metadata store that you set up earlier, you can use a SpEL expression such as follows:

@'sttgnrt.connector-infra.support.infinispan-metadata-store'.get(payload.anAttributeToEvaluate) ?: 'N/A'

This SpEL expression does the following things:

It searches for the component named infinispan-metadata-store, which is located in an entry connector flow with the stgnenri message type coming from the sttgnrt system.
It retrieves the value of the entry stored by the infinispan-metadata-store component, which has the key of what is specified in the get function's argument block (in this example, the value of the anAttributeToEvaluate field from the incoming payload).
It returns the string value 'N/A' if no entry is found for the specified key.

Not only to raw payload, with this flow component, you can also retrieve such a state data to be added into a header. As an example, the screenshot above illustrates how retrieving state data can help determine the destination system that should receive the message.

Note that these persisted states can only be retrieved by eMagiz flows running in the same runtime where the support objects are deployed. Thus, we suggest placing these support objects in the infra flow so the stored state data can be retrieved by other flows. For example:

In the connector infra so then they can be accessed by the entry connector flow.
In the gateway infra so then they can be accessed by the entry and exit gate flows.
In the stream infra so then they can be accessed by the event processor flows.

Once you have done so and satisfied with your setup, you can save and close this configuration.

4. Key takeaways

With State Generation, you can enrich your message with pre-stored state data.
To set up a State Generation - Enrichment operation, you need to configure a metadata storage and a flow component that supports SpEL expression, e.g., a standard content enricher.

5. Suggested additional readings

If you are interested in this topic and want more information on it, please read the help text provided by eMagiz and read the following microlearning on the related topic: