Wiki source code of Topic Storage

Last modified by Eva Torken on 2023/08/10 11:09
Show last authors
1 {{container}}
2 {{container layoutStyle="columns"}}
3 (((
4 A key element within using the Event Streaming platform is the topic storage. As you have learned in the [[Event Streaming Introduction>>doc:Main.eMagiz Academy.Microlearnings.Crash Course.Crash Course Event Streaming.crashcourse-eventstreaming-event-streaming-introduction||target="blank"]] the event streaming pattern is all about topics and retention. As you can imagine the total amount of GB configured in retention across environments and topics defines the total amount of configured topic storage. In this microlearning, we will learn how you can determine the configured size of the topic storage (per environment). Further, we will learn what you should do in case you exceed your allotted amount of topic storage.
5
6 Should you have any questions, please contact [[academy@emagiz.com>>mailto:academy@emagiz.com]].
7
8 == 1. Prerequisites ==
9
10 * Basic knowledge of the eMagiz platform
11
12 == 2. Key concepts ==
13
14 This microlearning centers on topic storage.
15
16 With topic storage we mean: The total configured amount of retention on all topics (across environments)
17
18 The focal point of this microlearning will be topic storage.
19
20 * The key aspects are:
21 ** eMagiz compares the allotted topic storage (contractually allowed) with the configured topic storage
22 ** eMagiz sells topic storage per integration data model (so no separate GBs for Test, Acceptance, and Production)
23 ** eMagiz shows configured topic storage per environment so you can assign yourself how much of the allotted GB you want to use per environment
24
25 == 3. Topic Storage ==
26
27 A key element within using the Event Streaming platform is the topic storage. As you have learned in the [[Event Streaming Introduction>>doc:Main.eMagiz Academy.Microlearnings.Crash Course.Crash Course Event Streaming.crashcourse-eventstreaming-event-streaming-introduction||target="blank"]] the event streaming pattern is all about topics and retention. As you can imagine the total amount of GB configured in retention across environments and topics defines the total amount of configured topic storage. In this microlearning, we will learn how you can determine the configured size of the topic storage (per environment). Further, we will learn what you should do in case you exceed your allotted amount of topic storage.
28
29 The focal point of this microlearning will be topic storage.
30
31 * The key aspects are:
32 ** eMagiz compares the allotted topic storage (contractually allowed) with the configured topic storage
33 ** eMagiz sells topic storage per integration data model (so no separate GBs for Test, Acceptance, and Production)
34 ** eMagiz shows configured topic storage per environment so you can assign yourself how much of the allotted GB you want to use per environment
35
36 Let us first check how we can determine the configured topic storage per enviroment and how you can compare that to the assigned topic storage per environment. To do so we have to navigate towards Design -> Architecture. Within Design Architecture you will a grey block called topic storage.
37
38 [[image:Main.Images.Microlearning.WebHome@intermediate-solution-architecture-topic-storage--topic-storage-design-architecture.png]]
39
40 In this block you will see the configured size on the left (so the total configured retention across topics on this environment) and the assigned (i.e. allotted) size on the right. As long as the number on the right is higher or equals the number on the left you can continue building and deploying your event streaming solution. However the moment the number on the right is lower compared to the number on the left (see below) you have exceeded the assigned size.
41
42 [[image:Main.Images.Microlearning.WebHome@intermediate-solution-architecture-topic-storage--topic-storage-design-architecture-not-enough-storage.png]]
43
44 You can also see this comparison in the right hand panel on the Design Architecture canvas.
45
46 [[image:Main.Images.Microlearning.WebHome@intermediate-solution-architecture-topic-storage--topic-storage-design-architecture-comparison.png]]
47
48 When this happens you could first take a hard look at how you have configured the retention settings per topic. Maybe you will spot large differences between the configured size and the actual retention size on a topic. If so you could downscale the retention settings (in terms of bytes and hours) in order to reduce the number of configured GBs on that environment. Note that you can vary the retention policy of a topic per environment. This allows you to make small Test topics and large(r) Production topics. To figure out whether the configured retention settings match the actual retention on topic level you can use the Manage phase in eMagiz. If you want to learn more on this please check out this [[Event Streaming Introduction>>doc:Main.eMagiz Academy.Microlearnings.Crash Course.Crash Course Event Streaming.crashcourse-eventstreaming-managing-your-event-streaming-solution||target="blank"]].
49
50 In case you cannot scale down your retention settings you need to contact your partner manager in order to come to an agreement on more allotted topic storage on a contractual basis. Note that just as with cloud approval only an eMagiz cloud administrator can change the assigned configuration. This to prevent misuse of the platform in terms of retention.
51
52 == 4. Key takeaways ==
53
54 * The key aspects are:
55 ** eMagiz compares the allotted topic storage (contractually allowed) with the configured topic storage
56 ** eMagiz sells topic storage per integration data model (so no separate GBs for Test, Acceptance, and Production)
57 ** eMagiz shows configured topic storage per environment so you can assign yourself how much of the allotted GB you want to use per environment
58
59 == 5. Suggested Additional Readings ==
60
61 If you are interested in this topic and want more information on it please read the help texts provided by eMagiz
62 )))
63
64 ((({{toc/}}))){{/container}}
65 {{/container}}