Design an Event Processor

{{container}}{{container layoutStyle="columns"}}(((

In this microlearning, we will dive into designing an event processor in eMagiz. An event processor helps you to transport, filter, and transform data between topics. We will cover the essentials, from prerequisites to the actual design process, to help you set up your event processor effectively.

3

4

Should you have any questions, please contact [[academy@emagiz.com>>mailto:academy@emagiz.com]].

5

6

== 1. Prerequisites ==

7

8

* Intermediate knowledge of the eMagiz platform

9

* An Event Streaming License

10

* Knowledge of the Event Streaming Pattern

11

* Followed the crash course on Event Streaming

12

13

== 2. Key concepts ==

14

15

This microlearning centers around designing an event processor.

16

* By event processor, we mean: A flow within eMagiz that consumes data from one topic and transports the data to another topic. In between, you have the option to filter or transform the data (i.e., event)

17

18

To create an event processor we need the following:

19

20

* An input topic designed in eMagiz.

21

* An output topic designed in eMagiz.

22

23

== 3. Design an Event Processor ==

24

25

An Event processor is a flow within eMagiz that retrieves data from one topic (the so-called input topic) and transports the data to another topic (the so-called output topic). During the transport of the data, you can transform the event or filter out events that don't fit your criteria.

26

27

In this microlearning, we will learn how you can design your event processor. In the microlearnings that will follow we will build on this knowledge.

28

29

=== 3.1 Input and Output Topic ===

30

31

The first step of setting up your event processor is determining which topics you want to transport the data to. An Event processor needs an input and an output topic to function correctly. When you have established what the input and output topic is you can determine whether these topics are already designed in eMagiz.

32

33

In this microlearning, we assume that the topics are already exists. Please refer to [[this>>doc:Main.eMagiz Academy.Microlearnings.Crash Course.Crash Course Event Streaming.crashcourse-eventstreaming-designing-a-event-stream||target="blank"]] and [[this>>doc:Main.eMagiz Academy.Microlearnings.Crash Course.Crash Course Event Streaming.crashcourse-eventstreaming-create-your-topic||target="blank"]] microlearning for more information on how to create topics in your model.

34

35

[[image:Main.Images.Microlearning.WebHome@intermediate-configuring-emagiz-event-streaming-design-an-event-processor--capture-view.png]]

36

37

Since we already have the input and output topics we can move to the Design phase of eMagiz. In this phase, you can design the event processor.

38

39

[[image:Main.Images.Microlearning.WebHome@intermediate-configuring-emagiz-event-streaming-design-an-event-processor--design-view.png]]

40

41

=== 3.2 Link Input to Output via an Event Processor ===

42

43

As you can see in this picture (and probably remember from one of our earlier microlearnings) there are no systems that produce data on the Output topic.

44

45

This is where eMagiz comes in. Via the event processor, you can configure that all data from the Input topic will be transported towards the output topic. In other words, eMagiz will become the producer of data on the Output topic.

46

47

To design an event processor you simply navigate to one of the lines drawn that represent the input and the output topic and access the context menu via a right-mouse click.

48

49

[[image:Main.Images.Microlearning.WebHome@intermediate-configuring-emagiz-event-streaming-design-an-event-processor--design-add-processor-context-menu.png]]

50

51

When you select the option called Add processor you will see a pop-up similar to the one shown below. Do notice that depending via which line you access the context menu what can be altered changes. When you select the line that has a consuming system you can select the input topic. When you select the line that has a producing system you can select the output topic.

52

53

[[image:Main.Images.Microlearning.WebHome@intermediate-configuring-emagiz-event-streaming-design-an-event-processor--design-add-processor-pop-up.png]]

54

55

Select the correct topic and press Save.

56

57

[[image:Main.Images.Microlearning.WebHome@intermediate-configuring-emagiz-event-streaming-design-an-event-processor--design-add-processor-pop-up-filled-in.png]]

58

59

As a result, your Design overview will change to reflect the choices that you have made

60

61

[[image:Main.Images.Microlearning.WebHome@intermediate-configuring-emagiz-event-streaming-design-an-event-processor--design-view-event-processor.png]]

62

63

You have now successfully designed an event processor that is capable of transporting data from the Input Topic to the Output Topic.

64

65

=== 3.3 Double Lane Consideration ===

66

67

When running your Event Streaming processor(s) in a double lane setup you should be aware that both consumers will become part of the same consumer group and therefore the minimum number of partitions on the topics read by the event processor should be two. For an explanation on how consumers groups work please see this [[fundamental>>doc:Main.eMagiz Academy.Fundamentals.fundamental-event-streaming-introduction||target="blank"]] page.

68

69

== 4. Key takeaways ==

70

71

* An event processor facilitates the movement of data from an input topic to an output topic, enabling both the transport and transformation of data.

72

* In eMagiz, designing an event processor involves selecting the appropriate input and output topics and configuring the processor to manage data flow between them.

73

* In an event processor flow, you have the option to filter or transform events according to your criteria.

74

* Ensure your topics are properly set up before design. If running multiple processors, be aware of consumer group settings and partition requirements for optimal performance.

75

76

== 5. Suggested Additional Readings ==

77

78

If you are interested in this topic and want more information on it please read the release notes provided by eMagiz that accompany the eMagiz Mendix Connector version you have selected and read the following link:

79

80

* [[Fundamentals (Navigation)>>doc:Main.eMagiz Academy.Fundamentals.WebHome||target="blank"]]

81

** [[eMagiz Event Streaming (Explanation)>>doc:Main.eMagiz Academy.Fundamentals.fundamental-event-streaming-introduction||target="blank"]]

82

* [[Crash Courses (Menu)>>doc:Main.eMagiz Academy.Microlearnings.Crash Course.WebHome||target="blank"]]

83

** [[Crash Course Event Streaming (Navigation)>>doc:Main.eMagiz Academy.Microlearnings.Crash Course.Crash Course Event Streaming.WebHome||target="blank"]]

84

*** [[Designing an Event Stream (Explanation)>>doc:Main.eMagiz Academy.Microlearnings.Crash Course.Crash Course Event Streaming.crashcourse-eventstreaming-designing-a-event-stream||target="blank"]]

85

*** [[Create your Topic (Explanation)>>doc:Main.eMagiz Academy.Microlearnings.Crash Course.Crash Course Event Streaming.crashcourse-eventstreaming-create-your-topic||target="blank"]]

86

* [[Event Processor (Search Results)>>url:https://docs.emagiz.com/bin/view/Main/Search?sort=score&sortOrder=desc&highlight=true&facet=true&r=1&f_space_facet=0%2FMain.&l_space_facet=10&f_type=DOCUMENT&f_locale=en&f_locale=&f_locale=en&text=%22event+processor%22||target="blank"]]

87

88

)))((({{toc/}}))){{/container}}{{/container}}

Wiki source code of Design an Event Processor

Navigation