Wiki source code of Designing an Event Stream

Last modified by Eva Torken on 2023/05/11 12:57
Show last authors
1 {{container}}{{container layoutStyle="columns"}}(((
2 In this microlearning, we will focus on how you can design an Event Streaming Solution with the help of the first two phases of eMagiz (i.e Capture and Design).
3
4 Should you have any questions, please contact [[academy@emagiz.com>>mailto:academy@emagiz.com]].
5
6 == 1. Prerequisites ==
7
8 * Basic knowledge of the eMagiz platform
9 * Understanding of Event Streaming concept
10
11 == 2. Key concepts ==
12
13 This microlearning centers around designing an event stream.
14 By event stream we mean: A decoupled service that works best within a microservices architecture with each application having its core function for high volumes.
15
16 To design an event stream you must do the following:
17
18 * Define the system(s) in Capture that you want to connect
19 * Define the data object you want to exchange with the help of an event stream, name this accordingly and draw lines indicating the flow of data between these systems
20 * Check your work in Design via the Design overview.
21
22 Below we will discuss each of these steps separately.
23
24 == 3. Designing an Event Stream ==
25
26 Within the eMagiz platform, you can easily design an event stream.
27 In this section we will discuss how you can design an event stream by following these steps:
28
29 * Define the system(s) in Capture that you want to connect
30 * Define the data object you want to exchange with the help of an event stream, name this accordingly and draw lines indicating the flow of data between these systems
31 * Check your work in Design via the Design overview.
32
33 === 3.1 Define the system(s) in Capture ===
34
35 As with all integration patterns the process starts in Capture. In Capture, you have the option to add systems to the canvas that can connect to a topic.
36 To indicate this ability on the system-level you have the option to select Event Streaming as the default integration pattern on the system level.
37
38 [[image:Main.Images.Microlearning.WebHome@crashcourse-eventstreaming-designing-a-event-stream--capture-es-system-config.png]]
39
40 === 3.2 Define the data object you want to exchange ===
41
42 After you have created the system and filled the details in you can add integrations from and to eMagiz. When looking at Event Streaming within the scope of eMagiz the integration (in messaging) equals a topic (in event streaming).
43 If you draw a line from a system to eMagiz you define that the system from which you have drawn the line is a 'producing' system. A 'producing' system is a system that writes data to one or more topics. If a systems write to more topics you draw more lines.
44
45 [[image:Main.Images.Microlearning.WebHome@crashcourse-eventstreaming-designing-a-event-stream--capture-es-producing-integration.png]]
46
47 If you draw a line from eMagiz to a system you define that the system to which you have drawn the line is a 'consuming' system. A 'consuming' system is a system that reads data from one or more topics. If a system reads from more topics you draw more lines.
48
49 [[image:Main.Images.Microlearning.WebHome@crashcourse-eventstreaming-designing-a-event-stream--capture-es-consuming-integration.png]]
50
51 The moment you have captured these systems you will end up with something like this.
52
53 [[image:Main.Images.Microlearning.WebHome@crashcourse-eventstreaming-designing-a-event-stream--capture-es-result.png]]
54
55 === 3.2 Check your work in Design ===
56
57 Design is the crucial phase when it comes to Event Streaming. More on all of the specific functionalities Design offers you in other microlearnings.
58 In this microlearning, we will focus on the Design overview and how you can use this to verify what you have captured
59
60 ==== 3.2.1 Design overview ====
61
62 In Design, you see an overview of your landscape including all first-class integration patterns (messaging, API gateway, and event streaming).
63
64 [[image:Main.Images.Microlearning.WebHome@crashcourse-eventstreaming-designing-a-event-stream--design-overview.png]]
65
66 For Event Streaming integrations you see one or more numbers per integration. These numbers mean as follows (left to right):
67
68 * Number of 'producers' on a topic
69 * Number of event processors for which this data of the topic (integration) is the input of the event processor
70 * Number of event processors for which this data of the topic (integration) is the output of the event processor
71 * Number of 'consumers' on a topic
72
73 ===== 3.2.1.1 Technical config system =====
74
75 If you have configured your system in such a way that you have specified that this system has a default integration pattern of Event Streaming you only need to fill in the technical name in Design
76 as eMagiz already has set the message format to JSON. This is done because the defacto standard of Event Streaming messages is JSON.
77
78 [[image:Main.Images.Microlearning.WebHome@crashcourse-eventstreaming-designing-a-event-stream--design-technical-config-system.png]]
79
80 ===== 3.2.1.2 Red background in Design on integration level =====
81
82 If you see a red background behind the number this indicates that you have indicated that a system is consuming data from a specific topic (integration)
83 but have not yet indicated how the data will be produced on the topic.
84
85 [[image:Main.Images.Microlearning.WebHome@crashcourse-eventstreaming-designing-a-event-stream--design-consumer-without-producer.png]]
86
87 There are two ways to set this straight:
88
89 1. Go back to Capture and draw a line to indicate that a specific system directly produces messages on that topic. For more information on this revisit the section surrounding Capture in this microlearning
90 2. Add an Event processor. More on this later in the course
91
92 == 4. Key takeaways ==
93
94 * To design an event stream you must do the following:
95 * Define the system(s) in Capture that you want to connect
96 * Define the data object you want to exchange with the help of an event stream, name this accordingly and draw lines indicating the flow of data between these systems
97 * Check your work in Design via the Design overview.
98 * To verify whether there are mismatches you can use the visual queues eMagiz provides you within the Design overview
99 * Based on the Design overview you can see in one view the metadata related to your data flow
100
101 == 5. Suggested Additional Readings ==
102
103 If you are interested in this topic and want more information on it please read the help text provided by eMagiz when executing these actions and read the user guide provided on this topic.
104
105 )))((({{toc/}}))){{/container}}{{/container}}