Changes for page eMagiz State Generation

Summary

• Page properties (2 modified, 0 added, 0 removed)

Details

Page properties

Author

...	...	@@ -1,1 +1,1 @@
1		-XWiki.CarlijnKokkeler
	1	+XWiki.ebakker

Content

...	...	@@ -11,7 +11,7 @@
11	11
12	12	In this microlearning, the concept of state generation will be explained.
13	13
14		-== 3. Definition ==
	14	+== 3. State Generation ==
15	15
16	16	State generation concerns the generation of a stateful application or service. This means that, for the application or service to function, data is stored and past states are used to derive information. Such a state may e.g. be a temperature or heartrate measure. Stateful applications or services are opposed to stateless applications or services, which do not require data to function and merely represent the current state.
17	17
...	...	@@ -19,19 +19,19 @@
19	19
20	20	[[image:Main.Images.Fundamental.WebHome@fundamental-stateful-1.png]]
21	21
22		-== 4. Benefits and Operations ==
	22	+=== 3.1 Benefits and Operations ===
23	23
24	24	Benefits of real time state generation are, first of all, that data can be provided immediately and is always up-to-date. Secondly, it is possible to take real time action and take real-time decisions. Lastly, there is no need to store data unnecessarily.
25	25
26	26	Stateful data can be generated through four main operations.
27	27
28		-==== 4.1 Enrichment ====
	28	+==== 3.1.1 Enrichment ====
29	29
30	30	First of all, through enriching data. This means that information should be added to the data, so that it has more meaning. This can be achieved by storing states in a database. For example, when it is known that, if a person works from home, this is stored as A, and if a person works in the office, this is stored as B, the data can be enriched with ‘Home’ and ‘Office’. To ensure this, a database should be present storing the connection between A and Home, and B and Office.
31	31
32	32	[[image:Main.Images.Fundamental.WebHome@fundamental-stategeneration-example-enrich.png]]
33	33
34		-==== 4.2 Aggregation ====
	34	+==== 3.1.2 Aggregation ====
35	35
36	36	Secondly, stateful data can be obtained through aggregating data. Aggregation concerns a computation over a certain range of time, such as an average, minimum or maximum. Moreover, with state generation it is possible to define a trend, and recognize patterns in this trend. An advantage here, is that temporary states can be retained, instead of saving data for a longer period of time.
37	37
...	...	@@ -39,19 +39,19 @@
39	39
40	40	[[image:Main.Images.Fundamental.WebHome@fundamental-stategeneration-example-aggregation.png]]
41	41
42		-==== 4.3 Change Detection ====
	42	+==== 3.1.3 Change Detection ====
43	43
44	44	Thirdly, change detection can be applied. This means that a change in the data can be detected. For example, when the temperature at home rises to more than 30 degrees, this can be detected. It can then be defined that, when the temperature at home is more than 30 degrees and the heating is on, the heating should be turned off. In this example, the data is enriched with the last state of the air conditioning.
45	45
46	46	[[image:Main.Images.Fundamental.WebHome@fundamental-stategeneration-example-change-detection.png]]
47	47
48		-==== 4.4 Duplicate Detection ====
	48	+==== 3.1.4 Duplicate Detection ====
49	49
50	50	Lastly, duplicate detection can be applied, meaning that duplicate data can be detected. For example, if the number of people working from home is recorded twice at a certain moment, this can be detected, and the data can be adapted accordingly.
51	51
52	52	[[image:Main.Images.Fundamental.WebHome@fundamental-stategeneration-example-duplicate-detection.png]]
53	53
54		-== 5. State Generation in eMagiz ==
	54	+=== 3.2 State Generation in eMagiz ===
55	55
56	56	State Generation components can be used in all eMagiz patterns. The enrich and aggregate functionalities are likely more fitting to Messaging and API Gateway as opposed to Event Streaming. For Event Streaming, the other two operations, change and duplicate detection, are more relevant.
57	57
...	...	@@ -59,11 +59,11 @@
59	59
60	60	The state store is implemented inside the eMagiz runtime using the H2 database for now. For time related operations another technology is used (Esper), and the functionality is only available on Docker based runtimes (to be release end Q2 2022). You wil find the specific Stateful components inside the Flow Designer as we use in eMagiz to model the flow. Aligned with the general concept of low-code developnment in eMagiz. For State store models, the current data modeling capabilities will be used.
61	61
62		-== 6. Use Cases ==
	62	+=== 3.3 Use Cases ===
63	63
64	64	In the following, real-life implementations are given on how eMagiz State Generation has been realized through enrichment, aggregation, change detection, and duplicate detection.
65	65
66		-==== 6.1 Enrichment ====
	66	+==== 3.3.1 Enrichment ====
67	67
68	68	Company A works with a backend system based on ID/Name combinations. These are different for each implementation, and can be retrieved through a unique API. A difficulty that arises, is that some APIs initially only return the IDs. To prevent multiple calls to our backend system, the key/value pairs (ID/Name) need to be stored, such that these can be used in subsequent calls.
69	69
...	...	@@ -71,7 +71,7 @@
71	71
72	72	[[image:Main.Images.Fundamental.WebHome@fundamental-stategeneration-usecase-enrich.png]]
73	73
74		-==== 6.2 Aggregation ====
	74	+==== 3.3.2 Aggregation ====
75	75
76	76	Company B provides contracts and files for their customers. Their customers manually upload contracts (one by one) to their files, which may take some time. It is desired by Company B that their customers receive a push notification once their files have been uploaded successfully. However, they do not wish to spam their customers by sending a push notification after each file upload when they are uploading several files. So, the push notification should be retained for some time, and should be sent once it is (almost) certain that no additional file will be uploaded.
77	77
...	...	@@ -79,7 +79,7 @@
79	79
80	80	[[image:Main.Images.Fundamental.WebHome@fundamental-stategeneration-usecase-aggregation.png]]
81	81
82		-==== 6.3 Change Detection ====
	82	+==== 3.3.3 Change Detection ====
83	83
84	84	Company C tracks packages and parcels that have been ordered by their customers. When a package is not received within its expected timeframe, an exception occurs. It can happen that such an exception resolves itself, e.g. after rendering for some time, the package is not ‘stuck’ anymore, and is being processed. Before implementing state generation, such a change was never sent to the Alert Manager, because no new exception was triggered. As a result, the employees from Company C were still working under the assumption that the exception is valid, meaning that they could still be searching for a packet that had already been received.
85	85
...	...	@@ -87,7 +87,7 @@
87	87
88	88	[[image:Main.Images.Fundamental.WebHome@fundamental-stategeneration-usecase-change-detection.png]]
89	89
90		-==== 6.4 Duplicate Detection ====
	90	+==== 3.3.4 Duplicate Detection ====
91	91
92	92	Company D is concerned with receiving meter readings. From an external system, new meter readings, or corrected meter readings are received. To protect underlying applications, it needs to be checked whether a duplicate between the external system and their own internal system exists. This can be achieved through a check for each type of meter reading. When a duplicate is found, the meter reading is marked, such that it can be checked later in the process.
93	93	The eMagiz solution for Company D can be viewed below.
...	...	@@ -94,7 +94,7 @@
94	94
95	95	[[image:Main.Images.Fundamental.WebHome@fundamental-stategeneration-usecase-duplicate-detection.png]]
96	96
97		-== 7. Key Takeaways ==
	97	+== 4. Key Takeaways ==
98	98
99	99	* State generation concerns the generation of a stateful application or service
100	100	* Stateful refers to the concept of storing a state of an object
...	...	@@ -102,7 +102,7 @@
102	102	* Stateful can be applied across all patterns in eMagiz and is embedded into the platform in such a way that it provided the same user experience
103	103	* Stateful data can be generated through four main operations: enrichment, aggregation, change detection, duplicate detection
104	104
105		-== 8. Suggested Additional Readings ==
	105	+== 5. Suggested Additional Readings ==
106	106
107	107	N/A
108	108