Changes for page Multi-Model Explained

Summary

• Page properties (1 modified, 0 added, 0 removed)

Details

Page properties

Content

...	...	@@ -14,71 +14,97 @@
14	14	* There are several reasons for choosing multiple integrations models, i.e. separate eMagiz instances. These reasons can be grouped under the term separation of concerns.
15	15	* Separation of concerns is a design principle in software development and architecture for separating an application in two or more sections, such that each section addresses a particular concern.
16	16
17		-== 3. eMagiz Cloud - Inner Workings ==
	17	+== 3. eMagiz multi-model environment - Definition ==
18	18
19		-When running your eMagiz solution, there are generally two methods of deploying. There is the Cloud and the on-premise (i.e., in your own data center) option. We see a strong development towards Cloud-based functionality across markets. Furthermore, we see that patterns such as API Gateway and Event Streaming are tailored for a cloud only approach. Given all those characteristics, we have an eMagiz Cloud first strategy when further developing our patterns and features across the platform.
	19	+For some companies that work with eMagiz, it is preferable to have multiple integration models. Such an environment in eMagiz, in which there are multiple integration models, is called a multi-model environment. With integration model, an eMagiz platform instance is meant, in which it is possible to build integrations. Within an integration model, three types of integration patterns can be modelled. These are [[Messaging>>doc:Main.eMagiz Academy.Fundamentals.fundamental-messaging-introduction.WebHome||target="blank"]], [[API Gateway>>doc:Main.eMagiz Academy.Fundamentals.fundamental-api-gateway-introduction.WebHome||target="blank"]], and [[Event Streaming>>doc:Main.eMagiz Academy.Fundamentals.fundamental-event-streaming-introduction.WebHome||target="blank"]]. An illustration of an integration model, or platform instance, using the Messaging pattern is the following:
	20	+
	21	+[[image:Main.Images.Fundamental.WebHome@fundamental-emagiz-multi-model--single-platform-instance.png]]
20	20
21		-In this fundamental, we will zoom in on how the eMagiz Cloud is structured and how it operates for you as a customer. We will start our journey at a high level in which we focus on how we support multiple customer environments within the same eMagiz Cloud. From there, we will in on the details of each of those customer environments. Building on that, we show the differences between a single lane and a double lane setup. Last but not least, we will focus on the key benefits deploying in the eMagiz Cloud holds for you.
	23	+An illustration of two platform instances, both modelled with the Messaging pattern, is shown below:
22	22
23		-=== 3.1 High-level overview ===
	25	+[[image:Main.Images.Fundamental.WebHome@fundamental-emagiz-multi-model--two-platform-instances.png]]
24	24
25		-The first perspective on the Cloud we take is how we run multiple customers within the same eMagiz Cloud solution that we offer. The picture below shows how we structured our Cloud to ensure that your data is kept safely within your environment.
	27	+Per integration model, it is possible to decide which integration patterns are available. This may be only one pattern, but it could also be two, or all three.
	28	+
	29	+== 4 Separation of concerns ==
26	26
27		-[[image:Main.Images.Fundamental.WebHome@fundamental-emagiz-cloud-inner-workings--high-overview.png]]
	31	+There are several reasons for choosing multiple integrations models, i.e. separate eMagiz instances. These reasons can be grouped under the term separation of concerns. Separation of concerns is a design principle in software development and architecture for separating an application in two or more sections, such that each section addresses a particular concern. Reasons for applying this separation of concerns principle in the eMagiz environment are given below.
28	28
29		-The most outer line of the picture represents the total eMagiz Cloud. Our support department and cloud admins have access to this level from which they can access each customer environment if need be. When going one level deeper, we see the standard region in which all our customers' data is kept. This default region (eu-central-01 located in Frankfurt) allows us to keep data under European Law and reduces the latency as most of our customer base is located within the European continent. Within this region, we have what we call a Carwash. This carwash is placed in front of each of our customer VPC's to add a layer of security. This layer restricts access to customer endpoints. Behind the carwash, we have one separate VPC per customer model. So when you have multiple models running in eMagiz (as part of your Enterprise license), you will effectively have the same amount of VPCs in the Cloud (assuming all of them run in the Cloud). This allows for the best possible separation of concerns between customers and models.
	33	+=== 4.1 Independent operations ===
	34	+
	35	+Within a company, there may be different business processes that can operate independently. It can be useful to have a clear separation in these business processes, such that the processes can evolve independently. This may include entirely different business processes, such as sales and transport, but processes can also be separated based on different use cases or different positions within a business process. For example, one model might address the business processes down the value chain, while the other model addresses the business processes up the value chain.
	36	+By having an eMagiz model for each distinct business process, it possible to release new integrations and make changes independently, so that the processes can evolve freely. For example, in case there is a different model for sales and transport, there is no dependency on progress of both departments. Below, it is illustrated that, in case a flow that is being worked on for the sales department is still in progress, while a flow that is being worked on for the transport department is done, no new release can be made if the models are not separated. In case there is a separate model for the sales department and the transport department, releases can be made independently, meaning that the improvement on the flow for the transport department can already be released.
30	30
31		-=== 3.2 Customer level overview ===
32	32
33		-Now that we have a conceptual idea of how the various customers within the Cloud are separated from each other, we will zoom in on how a standard single-lane VPC setup looks.
	39	+[[image:Main.Images.Fundamental.WebHome@fundamental-emagiz-multi-model--independent-operations-single-instance.png]]
	40	+
	41	+[[image:Main.Images.Fundamental.WebHome@fundamental-emagiz-multi-model--independent-operations-two-instances.png]]
34	34
35		-[[image:Main.Images.Fundamental.WebHome@fundamental-emagiz-cloud-inner-workings--customer-level-overview.png]]
	43	+=== 4.2 Data model ===
36	36
37		-We once again see the outer layers of the eMagiz Cloud and the region. But in this overview, we zoomed in on one of the customer VPCs we have shown in the previous paragraph. When zooming in, we see several new things emerge within the picture. At first, we have an Internet Gateway that connects your VPC to the internet. This way, the carwash can redirect the traffic to the correct VPC, and the VPC is subsequently able to receive and process the message. Immediately after the gateway, we have a load balancer that determines whether the data is on HTTPS or JMS level. Depending on that, the message will be either sent to the core or the connector machine. This allows each VPC to communicate securely with the outside world and means that HTTPS traffic cannot be sent to the core machine.
	45	+Data models are visual representations that describe data using entities and attributes. Entities are representations of physical objects or well-defined singular concepts. Attributes are details that are specific to an entity. For example, a data model may contain the entity ‘customer’ with attributes ‘id’, ‘name’, ‘address’, etc. Within eMagiz, there are up to 3 data models for a model, namely for each of the integration patterns (Messaging, API Gateway, Event Streaming).
	46	+(Part of) a data model in eMagiz is shown below. As can be seen, a data model can become large, and there is a great risk of unclarity. By splitting business processes, data models can be made more specific to the process. This ensures that there is less ambiguity within one model. Moreover, the data models can be reduced in size, since entities and attributes that belong to the data model of a certain process can be removed from the data model of another process. This provides a better overview of the entities and attributes within a model.
38	38
39		-Below the load balancer, we show our DNS functionality. This ensures that when external parties call an endpoint hosted within one of your flows, they do not have to know the IP address of your VPC but can call the DNS name that you configured partly within the portal. We finished off by replacing the IP with emagizcloud.com within all the endpoints that are hosted by eMagiz. This makes life way easier when allowing external parties to connect to your endpoints.
	48	+[[image:Main.Images.Fundamental.WebHome@fundamental-emagiz-multi-model--data-model.png]]
	49	+
	50	+=== 4.3 Risk mitigation ===
40	40
41		-At the bottom of the picture, we see the EFS (Elastic File System). This file storage system stores meta-information securely for each customer so that only that customer can access it. A benefit of using this solution instead of regular file storage is that it can automatically scale. As a result, our cloud offering becomes more robust in dealing with high surges of traffic. Furthermore, by using EFS, your data is kept separate from the machines and can be re-used if the machines within the VPC need to be spun up in a different availability zone. To review: The EFS is also located on multiple availability zones for redundancy and distaster recovery.
	52	+Within a multi-model environment, the impact on the business might a model break down is less as compared to a single-model environment. The reason for this, is that in a multi-model environment only a part of the integration landscape is affected, instead of the complete landscape. For example, eMagiz models run on cloud slots. In case the cloud slot on which a model runs crashes, no business processes can be executed in case the complete business runs on one model. If there are multiple models within a business that describe different business processes, other business processes can still continue if a cloud slot for one model has crashed. This is illustrated below.
	53	+
	54	+[[image:Main.Images.Fundamental.WebHome@fundamental-emagiz-multi-model--risk-mitigation-single-instance.png]]
	55	+
	56	+[[image:Main.Images.Fundamental.WebHome@fundamental-emagiz-multi-model--risk-mitigation-two-instances.png]]
	57	+
	58	+=== 4.4 Manageability & maintainability ===
	59	+
	60	+By having multiple models, there is a clearer overview of the models, and less complexity. This ensures that the models are better manageable and maintainable. Manageable here means the capability of being controlled or dealt without difficulty. Maintainable concerns the ability to preserve or retain a certain state. In case there is only one model for a complete organization, the data models, data flows, and overall architecture contain much information, which can cause complexity. With multiple models, it is clearer what processes models describe, and it is better possible to have an overview of the overall architecture. This way, it is more easily discoverable if a process is not working effectively or if something needs improvement.
	61	+
	62	+=== 4.5 Deployment architecture ===
	63	+
	64	+The deployment architecture in eMagiz shows the machines, containers, and other architectural components that are needed to deploy a release. An example of what such an environment could look like is the following:
	65	+
	66	+[[image:Main.Images.Fundamental.WebHome@fundamental-emagiz-multi-model--deployment-architecture.png]]
	67	+
	68	+Each model has its own deployment architecture. As can be seen, there are several of the same types of components within the environment. In case there is only one model for an organization, the deployment architecture contains all architectural components that are needed to deploy the model. Separate environments ensure that there are less container runtimes and connector machines within one environment. This means that there is less dependency and a better organization within the deployment environments. For example, if one machine hinders the deployment plan, this will cause no issues if that machine is in a different model.
	69	+
	70	+=== 4.6 Specialization & effectiveness within teams ===
	71	+
	72	+When there is one large model, there is more complexity and more diverging concepts have to be described. With multiple models, concepts that describe a model can be more clearly defined, since less diverging information needs to be processed. For example, a team can specialize in one domain of a company to develop a model that is very specific to that domain in terms of data structure. It also ensures that more domain-specific language and conventions can be applied, e.g. in naming entities.
	73	+Furthermore, developers and users can specialize in one model, such that models can be worked on more effectively. In case there is one model for a complete organization, developers will have to understand all concepts. When there are multiple models, developers can study the concepts of one model more closely, and understand the process better. This will ensure that integrations are built more quickly, and potential improvements are discovered and resolved more easily.
	74	+Lastly, different models can be worked on in parallel, also increasing the effectiveness of development and deployment. An illustration of the above mentioned arguments is given below.
	75	+
	76	+[[image:Main.Images.Fundamental.WebHome@fundamental-emagiz-multi-model--specialization-and-effectivenss-within-teams.png]]
	77	+
	78	+=== 4.7 Strategy ===
42	42
43		-More to the right of the picture, we see the monitoring capabilities on the eMagiz Cloud level. Here we depict our most noteworthy monitoring functionality that will be triggered when your VPC or part of your VPC runs into trouble. Apart from the trigger, we also keep the log information for 30 days for analysis purposes if an RCA needs to be performed by eMagiz Support. This information is stored within the Systems Manager and CloudWatch.
	80	+With multiple models, there is a clearer separation of the business process. This means that the progress per business process can be better monitored. For example, if there is a separate model for the sales business process, it can be better discoverable whether more integrations have been set up. If the number of integrations has stopped growing, it might be the case that no new partners have joined the company. By having a better overview of the business processes, strategic decisions can be made more easily. Moreover, more specific KPIs and goals can be formulated and monitored. Alerts can be put in place per model to check upon the progress and achievement of these KPIs and goals, and support can be provided.
	81	+
	82	+=== 4.8 Funding ===
	83	+
	84	+There may be a difference in funding of different departments or business units. For example, the sales department of a company might have a bigger budget for improving their model compared to the transport department. If the sales model is separate from the transport model, investments could be made by the sales department to add additional features or make other improvements. If the transport and sales model are combined, either the sales department will have to make up for all the costs, or less improvements can be made as compared to when the models are split. Thus, by splitting models, the financing can be distributed more fairly.
	85	+
	86	+[[image:Main.Images.Fundamental.WebHome@fundamental-emagiz-multi-model--funding.png]]
	87	+
	88	+=== 4.9 Contractual and legal aspects ===
	89	+
	90	+Having multiple integrations models can be beneficial due to contractual and legal arrangements as well. For example, it might be the case that it is not allowed to share data between departments within one organization. If there is one model for the complete organization, this is a complex issue, since all data is available within one model. In the case that there is a different model for the departments between which no data may be shared, the issue is solved.
	91	+Moreover, it may be that there is a difference in contractual arrangements between different departments within a company. It might then be better to develop separate models. For example, it might be that the sales department has a contractual arrangement that does not allow for a certain type of integration. In case the contractual arrangements of the transport department do allow for that type of integration, and it is beneficial for their model, it is better to have separate models.
44	44
45		-Some of the monitoring triggers lead to an auto-healing action that will restore the state of your environment to normal without anyone having to take action. This means that the downtime in case of an outage is significantly reduced in these cases.
46		-
47		-Moving over to the last portion of the overview, we see some of the features we offer on the eMagiz Cloud. For example, you can define a fixed IP on outbound traffic for cases where the external party uses IP whitelisting to verify traffic. Another feature is the data sink capability that stores sunk messages in a bucket to be retrieved from the portal.
48		-
49		-Please check out the suggested additional readings section for applied knowledge on how you can control the eMagiz Cloud from the portal and utilize some of these functionalities from the eMagiz portal.
50		-
51		-=== 3.3 Single lane vs. Double Lane ===
52		-
53		-In the previous overview, we showed a single-lane setup. In the outline below, we deliver what we call a double lane setup. The most fundamental difference between the two is that you have a mirror image of each piece of functionality you are running with the double lane. Having a mirror image of everything reduces the downtime of the environment during maintenance and unexpected outings of your environment.
54		-
55		-[[image:Main.Images.Fundamental.WebHome@fundamental-emagiz-cloud-inner-workings--customer-level-overview-double-lane.png]]
56		-
57		-In this double lane setup, the backup JMS is dormant until activated. All processing components running in the Cloud will run at the same. As a result, you will see the number of consumers double across all your queues.
58		-
59		-=== 3.4 Key benefits ===
60		-
61		-Now that we have explained how our Cloud is configured, we will wrap up this fundamental by looking at the key benefits the Cloud holds for you when building your models with the help of the eMagiz platform. Below we have summarized these key benefits:
62		-
63		-* Each model has its VPC
64		-* Meta information is stored on EFS for auto-scaling purposes
65		-* Meta information is stored on EFS to guarantee a quick recovery in case of an outage
66		-* Monitoring capabilities provide auto-healing options
67		-* The eMagiz Cloud can be fully controlled via the eMagiz platform (check out our microlearnings under suggested additional readings)
68		-* A carwash is placed in front of all VPCs to add a layer of security
69		-* You can add additional features to your specific VPC
70		-
71	71	== 4. Key takeaways ==
72	72
73		-* Each eMagiz model result in a separate VPC in the eMagiz Cloud
74		-* eMagiz models are deployed in the AWS eu-central-01 zone by default - other regions are possible upon request
75		-* A carwash is placed in front of all VPCs to add a layer of security
76		-* Each VPC has DNS functionality to ensure that external parties don't have to call an IP address directly
77		-* Each VPC is automatically monitored
78		-* You can add additional features to your specific VPC
79		-* Setting up a double lane is a safeguard against downtime
80		-* The eMagiz Cloud can be controlled via the eMagiz platform (check out our microlearnings under suggested additional readings)
81		-
	95	+* An environment in eMagiz in which there are multiple integration models is called a multi-model environment.
	96	+* With integration model, an eMagiz platform instance is meant, in which it is possible to build integrations.
	97	+* There are several reasons for choosing multiple integrations models, i.e. separate eMagiz instances. These reasons can be grouped under the term separation of concerns.
	98	+* Separation of concerns is a design principle in software development and architecture for separating an application in two or more sections, such that each section addresses a particular concern.
	99	+* It can be useful to have a clear separation in these business processes, such that the processes can evolve independently.
	100	+* It can be beneficial to split large data models.
	101	+* Within a multi-model environment, the impact on the business might a model break down is less as compared to a single-model environment.
	102	+* By having multiple models, there is a clearer overview of the models, and less complexity. This ensures that the models are better manageable and maintainable.
	103	+* In case there is only one model for an organization, the deployment architecture contains all architectural components that are needed to deploy the model. Separate environments ensure that there are less container runtimes and connector machines within one environment.
	104	+* With multiple models, concepts that describe a model can be more clearly defined, since less diverging information needs to be processed.
	105	+* By splitting models, the financing can be distributed more fairly.
	106	+* Having multiple integrations models can be beneficial due to contractual and legal arrangements.
	107	+
82	82	== 5. Suggested Additional Readings ==
83	83
84	84	If you are interested in this topic and want to learn how you can control your Cloud with the help of the eMagiz platform, please check out our microlearnings offering on eMagiz Cloud Management: