Skip to Content

Changes for page Data Sink

Last modified by Eva Torken on 2023/09/29 15:45
From version 29.1
edited by Erik Bakker
on 2023/09/29 08:33
Change comment: There is no comment for this version
To version 19.1
edited by Erik Bakker
on 2023/09/11 14:05
Change comment: There is no comment for this version

Summary

Details

Page properties
Title
... ... @@ -1,1 +1,1 @@
1 -Data Sink
1 +Monitoring Partition Size and Consumer Lag
Parent
... ... @@ -1,1 +1,1 @@
1 -Main.eMagiz Academy.Microlearnings.Advanced Level.Data Management.WebHome
1 +Main.eMagiz Academy.Microlearnings.Advanced Level.Advanced monitoring Event Streaming.WebHome
Content
... ... @@ -1,5 +1,5 @@
1 1  {{container}}{{container layoutStyle="columns"}}(((
2 -This exercise will provide practical guidance on constructing a flow that will sink information towards the data sink solution of eMagiz. Furthermore it will give insights on how you can subsequently view this data from the Manage phase of eMagiz for further analysis. To send information to the data sink solution of eMagiz, you can use this [[store component >>doc:Main.eMagiz Store.Accelerators.eMagiz Data Sink.WebHome||target="blank"]].
2 +This exercise will provide practical guidance on interpreting the event streaming statistics overview, specifically around partition size and consumer lag.
3 3  
4 4  Should you have any questions, please contact [[academy@emagiz.com>>mailto:academy@emagiz.com]].
5 5  
... ... @@ -6,18 +6,18 @@
6 6  == 1. Prerequisites ==
7 7  
8 8  * Completed previous exercises and microlearnings
9 +* Access to an eMagiz model with an Event Streaming license
10 +* At least one configured topic on which something produces data and some other thing consumes data
9 9  
10 10  == 2. Exercise ==
11 11  
12 -Create an integration that will on top of its regular process also sink information to the data sink solution of eMagiz. This means you need to execute the following steps at a minimum:
13 -* Draw a line in Capture towards an (existing) system
14 -* Add Integration to Create
15 -* Open the exit flow to build the flow in such a way that you can send out emails
16 -** Don't forget to add the store component to your flow and read its documentation
14 +Navigate to Manage -> Monitoring -> Event Streaming Statistics for the model you selected to execute this exercise. Here, analyze the partition size and consumer lag on at least one topic.
17 17  
16 +{{info}}Given that the implementation of the prerequisites can differ, the results of what you see can also differ. We use example data that is part of our Sales Demo model to feed the Vehicle Event topic. An event processor subsequently determines whether a vehicle alert needs to be generated based on the type of event. This accounts for the difference in partition size between both topics. An event processor is a consumer group, and due to the high volume of messages placed on the Vehicle Event topic, we see that a consumer lag is reported.{{/info}}
17 +
18 18  == 3. Solution ==
19 -In this short movie, you will find the solution for how to configure the flow in Create and subsequently how to view the messages in Manage. Every step needed to get to the Create phase is assumed knowledge and, therefore, not covered in this video. Every step to get the changed version deployed and running is assumed knowledge and, therefore, not covered in this video. Please take a moment to review this one and compare it to your solution.
19 +In this short movie, you will find the workout of this exercise. Please take a moment to review this one and compare it to your solution.
20 20  
21 -{{video attachment="advanced-data-management-data-sink.mp4" reference="Main.Videos.Microlearning.WebHome"/}}
21 +{{video attachment="advanced-monitoring-eventstreaming-partition-size-and-consumer-lag-exercise.mp4" reference="Main.Videos.Microlearning.WebHome"/}}
22 22  
23 23  )))((({{toc/}}))){{/container}}{{/container}}