Creating a passthrough event processor
In this microlearning, we will explore creating a passthrough event processor in eMagiz. This processor will enable you to efficiently transport data between two topics without needing to transform or filter it. We will guide you through the steps to set up this processor and build on your previous knowledge of event processors.
Should you have any questions, please contact academy@emagiz.com.
1. Prerequisites
- Intermediate knowledge of the eMagiz platform
- An Event Streaming License
- Knowledge of the Event Streaming Pattern
- Followed the crash course on Event Streaming
2. Key concepts
This microlearning centers around designing an event processor.
- By event processor we mean: A flow within eMagiz that consumes data from one topic and transports the data to another topic. In between, you have the option to filter or transform the data (i.e. event)
- By passthrough we mean: Transport data from point A to point B without a need to manipulate (transform and/or filter) data
To create a passthrough event processor we need to have designed an event processor, which we have discussed in the previous microlearning.
3. Creating a passthrough event processor
An Event processor is a flow within eMagiz that retrieves data from one topic (the so-called input topic) and transports the data to another topic (the so-called output topic). During the transport of the data, you can transform the event or filter out events that don't fit your criteria. In a passthrough scenario, however, we implement the scenario that there is no need to transform and/or filter the events.
In this microlearning, we will learn how you can create your passthrough event processor. We will build on this knowledge for the microlearnings that will follow.
3.1 Determine Event Processor Design
The first step of creating your event processor is determining whether you have already designed an event processor. Once you have established which event processor that you have designed and needs to be transferred to the Create phase, then you can continue. For this microlearning, we assume that the event processor is already designed.
3.2 Add integrations
The first step is to add the integration to the Create phase of eMagiz. This process of adding flows to Create is the same for each pattern (Messaging, API Gateway, Event Streaming). So simply navigate to the Create phase and select the button "Add integrations" (which is located in the left bottom of the screen).
Here, you select at least one of the topics that needs to go to Create. eMagiz is smart enough to automatically transfer the other as well. You can always select them both, that will work also.
After you have pressed "Save Selection" and everything went according to plan, you will see a pop-up similar to the one that is shown below:
Press the button called "Go to Create" to get back to the Create overview.
3.3 The Create overview
In the overview of the Stream pattern that is presented, you can zoom in on the event processor to see what the input and output topic of the event processor is.
3.4 Event streaming flow
From here, we can navigate to the flow by: double-clicking on the event processor, or, via right mouse-clicking the event processor to show the context menu and then open it. When you open the flow eMagiz will present you with something as depicted below, a simple flow with an input and an output component.
The only thing this flow does is consume data from the input topic and produce the data on the output topic. The only thing left for you to do is to press "Stop Editing", as eMagiz has already created the complete flow for you and create a new version with a good description, ready to be deployed.
You have now successfully created an event processor that is capable of transporting data from the Input Topic to the Output Topic.
4. Key takeaways
- An event processor in eMagiz facilitates the transportation of data from one topic to another, serving as a conduit between input and output topics.
- In a passthrough case, the event processor simply moves data from the input topic to the output topic without any transformation or filtering.
- Use the "Add integrations" feature to easily transfer your designed event processor to the Create phase in eMagiz. The platform will handle most of the setup automatically.
- eMagiz generates the event processor flow for you. Your main task is to finalize the process by selecting "Stop Editing" and providing a description before deployment.
5. Suggested Additional Readings
If you are interested in this topic and want more information on it please read the help text provided by eMagiz and read the following links: