Changes for page Communication between models

Summary

• Page properties (1 modified, 0 added, 0 removed)

Details

Page properties

Content

...	...	@@ -1,5 +1,5 @@
1	1	{{container}}{{container layoutStyle="columns"}}(((
2		-In a low-code platform like eMagiz, you want to prevent using custom scripting as much as possible. However, sometimes in practice, you encounter situations that cannot be realized with the standard toolbox provided within the platform. As a result, we offer the option to use a Groovy script in these situations to achieve the desired solution. In this microlearning, we will look at where you can use a Groovy script and discuss the key considerations when implementing a Groovy script within your model.
	2	+In this microlearning explains about sharing data across systems in a multi-model environment using event streaming and web services. It discusses the processes, best practices, and considerations for implementing these concepts. Take a look to gain a deeper understanding of how communication between systems in different models can be achieved effectively.
3	3
4	4	Should you have any questions, please get in touch with [[academy@emagiz.com>>mailto:academy@emagiz.com]].
5	5
...	...	@@ -6,75 +6,83 @@
6	6	== 1. Prerequisites ==
7	7
8	8	* Expert knowledge of the eMagiz platform
	9	+* Fundamental [[eMagiz Multi-model>>doc:Main.eMagiz Academy.Fundamentals.fundamental-emagiz-multi-model-explained.WebHome||target="blank"]]
	10	+* Fundamental [[eMagiz Event Streaming>>doc:Main.eMagiz Academy.Fundamentals.fundamental-event-streaming-introduction||target="blank"]]
	11	+* Fundamental [[eMagiz API Gateway>>doc:Main.eMagiz Academy.Fundamentals.fundamental-api-gateway-introduction||target="blank"]]
	12	+* Fundamental [[eMagiz Messaging>>doc:Main.eMagiz Academy.Fundamentals.fundamental-messaging-introduction||target="blank"]]
9	9
10	10	== 2. Key concepts ==
11	11
12		-This microlearning focuses on groovy scripts. A groovy script is a piece of code written in the groovy language that allows you to extend the platform's functionality beyond the mere use of the components supplied via the flow designer.
	16	+* Model - the integration model of a client that runs on a specific cloud slot of eMagiz
	17	+* Multi-model - several integration models of a single client
13	13
14		-* Groovy scripts should be viewed as an extension that can be used in fringe cases, not as a standard best practice
15		-* When a groovy script becomes so complex that you are programming, you should consider alternatives to the solution
16		-* Groovy scripts can be used in a filter, transformation, and as part of a standard executor
17		-* Groovy scripts should be well documented as they are though to read for most of our community members
18		-* There is no guarantee from eMagiz that Groovy scripts within models keep working in newer versions of the platform
	19	+When selecting a method to communicate between models always consider the following concepts:
	20	+* Security
	21	+* Loose coupling
	22	+* Maintainability
	23	+* Clarity
19	19
20		-== 3. Groovy script ==
	25	+== 3. Main Multi-Model Integration Patterns ==
21	21
22		-In a low-code platform like eMagiz, you want to prevent using custom scripting as much as possible. However, sometimes in practice, you encounter situations that cannot be realized with the standard toolbox provided within the platform. As a result, we offer the option to use a Groovy script in these situations to achieve the desired solution. In this microlearning, we will look at where you can use a Groovy script and discuss the key considerations when implementing a Groovy script within your model.
	27	+Most users would think that communication via queues to connect two separate eMagiz models is a bright idea. However, this is something that we do not encourage. We discourage this for several practical and technical reasons alike.
23	23
24		-=== 3.1 Implementation in Create ===
	29	+When looking at the practical side of things getting it configured correctly is time-consuming and an error-prone action (as is evident by the question). Furthermore, it can lead to unexpected situations where you make a typo to listen to a queue on which no messages are provided. This queue will, however, be registered on the JMS level (when you activate the flow in question with the custom configuration), which can lead to confusing queue statistics and even more troublesome false-positive alerting based on missing queue metrics or missing consumers.
25	25
26		-To implement a groovy script in Create, you first need to consider what your Groovy script should do. Then, within the flow designer, there are five components available that can handle a Groovy script.
27		-
28		-* In case you want the Groovy script to filter out messages, you should implement the Groovy script in a "standard filter" component.
29		-* If you want your Groovy script to always transform your message from a certain format into another format, you should consider using a "standard transformer" component.
30		-* The "standard service activator" is your friend if you need an optional output. So, for example, when you want to do some transformation and check whether a message can continue, you can use the "standard service activator" and have an optional output.
31		-* When you want to split your message based on a Groovy script, you should use the "standard splitter."
32		-* If you want to route messages based on a Groovy script, you should use the "standard router."
	31	+On top of that, when you allow one model to write and read from queues registered in another model, maintaining both models will become very complex. This holds for the ones working on the project at the moment but also for those working on it at a later stage and does that need to provide (incidental) support on the environments.
33	33
34		-{{warning}}Note that in practice, we mainly see use cases for the "standard service activator" and the "standard transformation." {{/warning}}
	33	+Another practical reason for not wanting this is that we do not actively support this use case from eMagiz. This means that when we update our technical infrastructure, we will not consider this scenario. This could lead to additional work in the future and reduced stability of your solution.
35	35
36		-[[image:Main.Images.Microlearning.WebHome@expert-data-handling-groovy-script-available-components.png]]
	35	+From the technical point of view, the consequence of this construction is that both models need to know each other certificates and credentials, which are not considered secure. On top of that, because you, theoretically, can exchange data from any queue to any queue, you could create a situation in which updates in one model trigger changes in the other model (i.e., when using the same data model) that are unexpected (and frankly unwanted).
37	37
38		-Regardless of the component you choose, you always need to select the "Groovy script" option and reference the Groovy resource available on the flow level you want to use within the component.
	37	+At the moment, the two most frequently used ways to provide communication between systems in a multi-model environment are event streaming and web services. This microlearning will discuss the processes of sharing data across systems in a multi-model environment using these concepts, including best practices.
	38	+In the figure below, both processes are illustrated. The process at the top illustrates event streaming, and the process at the bottom illustrates web services. Model A and model B are two different models, where a system in model B should have access to the same data as the system in model A. For example, the systems in both models should be the same.
39	39
40		-[[image:Main.Images.Microlearning.WebHome@expert-data-handling-groovy-script-groovy-configuration.png]]
	40	+[[image:Main.Images.Fundamental.WebHome@fundamental-multi-model-best-practice--event-streaming-and-webservices.png]]
41	41
42		-{{info}}As you can see in the help text when selecting the "resource" attribute, you need to make sure that the Groovy script is uploaded within the context of your flow.{{/info}}
	42	+=== 3.1 Event Streaming ===
43	43
44		-After you have the component in such a way that the correct Groovy script is called, you are done with the component's configuration and can test the flow using flow testing.
	44	+To understand the following section, it is necessary to have some background information about event streaming, which can be found [[here>>doc:Main.eMagiz Academy.Fundamentals.fundamental-event-streaming-introduction||target="blank"]].
	45	+The first option is to decide the alignment of the data models between both models. Whenever data from the first model is transferred to the second model, and the topic is considered input for other systems and integrations in that other model, the data needs to be transformed to the common data model of that second model. This way, the topic can be exposed to any system and pattern in the second model. The picture below denotes this situation. These are the specific considerations when implementing this approach:
45	45
46		-=== 3.2 Key considerations ===
	47	+* Leverage the credentials of the first eMagiz model in the second model
	48	+* Create transparent systems with clear names and ensure to align the message type names
47	47
48		-* To make some groovy scripts work, you need to upload (third-party) Java libraries that are not packaged on default by eMagiz
49		-* To make some groovy scripts work, you need to reference Java libraries via a standard header enricher or as a dummy variable
50		-* The store consists of several groovy scripts that might be of use to you
51		-* Groovy scripts should be viewed as an extension that can be used in fringe cases, not as a standard best practice
52		-* When a groovy script becomes so complex that you are programming, you should consider alternatives to the solution
53		-* Groovy scripts can be used in a filter, transformation, and as part of a standard executor
54		-* Groovy scripts should be well documented as they are though to read for most of our community members
55		-* There is no guarantee from eMagiz that Groovy scripts within models keep working in newer versions of the platform
	50	+It is necessary to transform topics to allow communication between systems in different models using event streaming. This is because it is impossible to retrieve data from a topic that exists in one model through a different model. So, systems in model B cannot immediately access the topic present in model A. The topic needs to be transformed to ensure that systems in model B can use the data on the topic in model A. By changing the topic, all systems in model B can access the transformed topic.
56	56
57		-== 4. Assignment ==
	52	+To ensure that the topic from model A can be transformed, model B needs credentials from model A. So, to allow for this pattern, credentials must be shared between models.
58	58
59		-Try to implement a component that references a Groovy script and see whether the flow can be tested.
60		-This assignment can be completed with the help of the (Academy) project you created/used in the previous assignment.
	54	+An advantage of this pattern is that there is no dependency across models. If the system in model A breaks down, the system in model B can still work. A disadvantage of this pattern is that there may be struggles with the setup since no direct accessibility can be achieved. Moreover, as mentioned above, credentials have to be shared, which may threaten security.
61	61
62		-== 5. Key takeaways ==
	56	+=== 3.2 Webservices ===
63	63
64		-* To make some groovy scripts work, you need to upload (third-party) Java libraries that are not packaged on default by eMagiz
65		-* To make some groovy scripts work, you need to reference Java libraries via a standard header enricher or as a dummy variable
66		-* The store consists of several groovy scripts that might be of use to you
67		-* Groovy scripts should be viewed as an extension that can be used in fringe cases, not as a standard best practice
68		-* When a groovy script becomes so complex that you are programming, you should consider alternatives to the solution
69		-* Groovy scripts can be used in a filter, transformation, and as part of a standard executor
70		-* Groovy scripts should be well documented as they are though to read for most of our community members
71		-* There is no guarantee from eMagiz that Groovy scripts within models keep working in newer versions of the platform
	58	+Web services can also be used to communicate between systems in different models. This means that model A has a web service in place, and model B has a web service in place, between which data can be exchanged.
	59	+An advantage of this pattern is that it is easy to keep an overview. Setting it up does not involve many unclarities. A disadvantage is that there is much dependency across models. If the web service in model A crashes, the system in model B cannot receive data anymore.
72	72
73		-== 6. Suggested Additional Readings ==
	61	+=== 3.3 Naming Conventions ===
74	74
75		-If you are interested in this topic and want more information, please read the help text provided by eMagiz and read the following link:
76		-* https://www.tutorialspoint.com/groovy/index.htm
	63	+For clarity purposes, it is advised to give the systems that are the same across models the same name and the name of the other model in which the system is present. So, as illustrated in the figure, the system names in model A and B are partly the same (‘SYS’) and indicate which model also holds the system (model B in model A, model A in model B). This way, it remains clear which models have the same systems, and systems can be found more easily.
77	77
78		-== 7. Silent demonstration video ==
	65	+== 4. Key takeaways ==
79	79
80		-As this is more of theoretical microlearning, there is no video accompanying the microlearning.)))((({{toc/}}))){{/container}}{{/container}}
	67	+* Event streaming and webervices are mainly used to communicate between systems across models.
	68	+* To allow communication between systems in different models using event streaming, it is needed to transform topics.
	69	+* Communication between systems in different models can also directly be achieved using web services.
	70	+* For clarity purposes, it is advised to give the systems that are the same across models the same name, as well as the name of the other model in which the system is present.
	71	+* Both patterns have their advantages and disadvantages.
	72	+
	73	+* Consider the following when communication between two models
	74	+** Security
	75	+** Loose coupling
	76	+** Maintainability
	77	+** Clarity
	78	+
	79	+== 5. Suggested Additional Readings ==
	80	+
	81	+* [[Fundamentals (Navigation)>>doc:Main.eMagiz Academy.Fundamentals.WebHome||target="blank"]]
	82	+** [[eMagiz Security Guide (Explanation)>>doc:Main.eMagiz Academy.Fundamentals.fundamental-emagiz-security-guide||target="blank"]]
	83	+** [[Multi-Model Explained (Explanation)>>doc:Main.eMagiz Academy.Fundamentals.fundamental-emagiz-multi-model-explained.WebHome||target="blank"]]
	84	+* [[Advanced (Menu)>>doc:Main.eMagiz Academy.Microlearnings.Advanced Level.WebHome||target="blank"]]
	85	+** [[Solution Architecture (Navigation)>>doc:Main.eMagiz Academy.Microlearnings.Advanced Level.Solution Architecture.WebHome||target="blank"]]
	86	+*** [[Checklist for Splitting Models (Explanation)>>doc:Main.eMagiz Academy.Microlearnings.Advanced Level.Solution Architecture.Checklist for Splitting Models.WebHome||target="blank"]]
	87	+* [[Communication between models (Search Result)>>url:https://docs.emagiz.com/bin/view/Main/Search?sort=score&sortOrder=desc&highlight=true&facet=true&r=1&f_space_facet=0%2FMain.&f_type=DOCUMENT&f_locale=en&f_locale=&f_locale=en&text=communication+between+models||target="blank"]]
	88	+)))((({{toc/}}))){{/container}}{{/container}}