Changes for page Multi-Model Explained

Last modified by Erik Bakker on 2024/08/08 11:57

From 2.1 to 1.1 From 20.1 to 19.1

From version 19.1

edited by Erik Bakker
on 2024/08/08 11:33

Change comment: There is no comment for this version

To version 2.1

edited by Carlijn Kokkeler
on 2022/10/14 09:12

Change comment: There is no comment for this version

Raw
Rendered

Summary

Page properties (2 modified, 0 added, 0 removed)

Details

Page properties

Author

@@ -1,1 +1,1 @@
--XWiki.ebakker
++XWiki.CarlijnKokkeler

Content

@@ -1,5 +1,5 @@
  {{container}}{{container layoutStyle="columns"}}(((
--In this fundamental we share some valuable insights about working with eMagiz in a multi-model environment. This information covers integration models, separation of concerns, and the benefits of managing different business processes independently.
++In this fundamental, the eMagiz multi-model environment will be explained. Moreover, reasons will be given for having multiple eMagiz models within an organization.
  Should you have any questions, please get in touch with academy@emagiz.com.
@@ -7,113 +7,96 @@
  * Some context on cloud functionality will be helpful.
--== 2. Key Concepts ==
++== 2. Key concepts ==
--* An environment in eMagiz in which there are multiple integration models is called a multi-model environment.
++* An environment in eMagiz in which there are multiple integration models is called a multi-model environment.
  * With integration model, an eMagiz platform instance is meant, in which it is possible to build integrations.
  * There are several reasons for choosing multiple integrations models, i.e. separate eMagiz instances. These reasons can be grouped under the term separation of concerns.
--* Separation of concerns is a design principle in software development and architecture for separating an application in two or more sections, such that each section addresses a particular concern.
++* Separation of concerns is a design principle in software development and architecture for separating an application in two or more sections, such that each section addresses a particular concern.
--== 3. eMagiz Multi-Model Environment - Definition ==
++== 3. eMagiz multi-model environment - Definition ==
--For some companies that work with eMagiz, multiple integration models are preferable. Such an eMagiz environment with multiple integration models is called a multi-model environment. With the integration model, an eMagiz platform instance is meant to make it possible to build integrations. Within an integration model, three types of integration patterns can be modeled. These are [[Messaging>>doc:Main.eMagiz Academy.Fundamentals.fundamental-messaging-introduction||target="blank"]], [[API Gateway>>doc:Main.eMagiz Academy.Fundamentals.fundamental-api-gateway-introduction||target="blank"]], and [[Event Streaming>>doc:Main.eMagiz Academy.Fundamentals.fundamental-event-streaming-introduction||target="blank"]]. An illustration of an integration model, or platform instance, with the Messaging integration pattern, is the following:
--
++For some companies that work with eMagiz, it is preferable to have multiple integration models. Such an environment in eMagiz, in which there are multiple integration models, is called a multi-model environment. With integration model, an eMagiz platform instance is meant, in which it is possible to build integrations. Within an integration model, three types of integration patterns can be modelled. These are Messaging, API Gateway, and Event Streaming, as described in link. An illustration of an integration model, or platform instance, using the Messaging pattern is the following:
++
  [[image:Main.Images.Fundamental.WebHome@fundamental-emagiz-multi-model--single-platform-instance.png]]
--An illustration of two platform instances, both modeled with the Messaging pattern, is shown below:
++An illustration of two platform instances, both modelled with the Messaging pattern, is shown below:
  [[image:Main.Images.Fundamental.WebHome@fundamental-emagiz-multi-model--two-platform-instances.png]]
--Per the integration model, deciding which integration patterns are available is possible. This may be only one pattern, but it could also be two or all three.
++Per integration model, it is possible to decide which integration patterns are available. This may be only one pattern, but it could also be two, or all three.
++
++== 4 Separation of concerns ==
--== 4. Separation of Concerns ==
++There are several reasons for choosing multiple integrations models, i.e. separate eMagiz instances. These reasons can be grouped under the term separation of concerns. Separation of concerns is a design principle in software development and architecture for separating an application in two or more sections, such that each section addresses a particular concern. Reasons for applying this separation of concerns principle in the eMagiz environment are given below.
--There are several reasons for choosing multiple integration models, i.e., separate eMagiz instances. These reasons can be grouped under the term separation of concerns. Separation of concerns is a design principle in software development and architecture that separates an application into two or more sections, each addressing a particular concern. The reasons for applying this separation of concerns principle in the eMagiz environment are below.
++=== 4.1 Independent operations ===
++
++Within a company, there may be different business processes that can operate independently. It can be useful to have a clear separation in these business processes, such that the processes can evolve independently. This may include entirely different business processes, such as sales and transport, but processes can also be separated based on different use cases or different positions within a business process. For example, one model might address the business processes down the value chain, while the other model addresses the business processes up the value chain.
++By having an eMagiz model for each distinct business process, it possible to release new integrations and make changes independently, so that the processes can evolve freely. For example, in case there is a different model for sales and transport, there is no dependency on progress of both departments. Below, it is illustrated that, in case a flow that is being worked on for the sales department is still in progress, while a flow that is being worked on for the transport department is done, no new release can be made if the models are not separated. In case there is a separate model for the sales department and the transport department, releases can be made independently, meaning that the improvement on the flow for the transport department can already be released.
--=== 4.1 Independent Operations ===
--Within a company, different business processes may operate independently. It can be useful to have a clear separation between these business processes so that they can evolve independently. This may include entirely different business processes, such as sales and transport, but processes can also be separated based on different use cases or different positions within a business process. For example, one model might address the business processes down the value chain, while the other model addresses the business processes up the value chain.
--Having an eMagiz model for each distinct business process makes it possible to release new integrations and make changes independently so that the processes can evolve freely. For example, if there is a different model for sales and transport, there is no dependency on the progress of both departments. Below, it is illustrated that, in this case, a flow being worked on for the sales department is still in progress. In contrast, if the flow being worked on for the transport department is done, it is easier to make a new release if the models are separated. In case there is a separate model for the sales department and the transport department, releases can be made independently, meaning that the improvement on the flow for the transport department can be released without having to take into account flow changes from the sales department that may have an impact on it.
++[[image:Main.Images.Fundamental.WebHome@fundamental-emagiz-cloud-inner-workings--high-overview.png]]
++The most outer line of the picture represents the total eMagiz Cloud. Our support department and cloud admins have access to this level from which they can access each customer environment if need be. When going one level deeper, we see the standard region in which all our customers' data is kept. This default region (eu-central-01 located in Frankfurt) allows us to keep data under European Law and reduces the latency as most of our customer base is located within the European continent. Within this region, we have what we call a Carwash. This carwash is placed in front of each of our customer VPC's to add a layer of security. This layer restricts access to customer endpoints. Behind the carwash, we have one separate VPC per customer model. So when you have multiple models running in eMagiz (as part of your Enterprise license), you will effectively have the same amount of VPCs in the Cloud (assuming all of them run in the Cloud). This allows for the best possible separation of concerns between customers and models.
--[[image:Main.Images.Fundamental.WebHome@fundamental-emagiz-multi-model--independent-operations-single-instance.png]]
++=== 3.2 Customer level overview ===
--[[image:Main.Images.Fundamental.WebHome@fundamental-emagiz-multi-model--independent-operations-two-instances.png]]
++Now that we have a conceptual idea of how the various customers within the Cloud are separated from each other, we will zoom in on how a standard single-lane VPC setup looks.
--=== 4.2 Data Model ===
++[[image:Main.Images.Fundamental.WebHome@fundamental-emagiz-cloud-inner-workings--customer-level-overview.png]]
--Data models are visual representations that describe data using entities and attributes. Entities are representations of physical objects or well-defined singular concepts. Attributes are details that are specific to an entity. For example, a data model may contain the entity ‘customer’ with attributes ‘id,’ ‘name,’ ‘address,’ and more. Within eMagiz, there are up to 3 data models for a model, namely for each integration pattern (Messaging, API Gateway, Event Streaming).
--(Part of) a data model in eMagiz is shown below. As can be seen, a data model can become large, and there is an excellent risk of unclarity. By splitting business processes, data models can be made more specific to the process. This ensures that there is less ambiguity within one model. Moreover, the data models can be reduced in size since entities and attributes that belong to the data model of a particular process can be removed from the data model of another process. This provides a better overview of the entities and attributes within a model.
++We once again see the outer layers of the eMagiz Cloud and the region. But in this overview, we zoomed in on one of the customer VPCs we have shown in the previous paragraph. When zooming in, we see several new things emerge within the picture. At first, we have an Internet Gateway that connects your VPC to the internet. This way, the carwash can redirect the traffic to the correct VPC, and the VPC is subsequently able to receive and process the message. Immediately after the gateway, we have a load balancer that determines whether the data is on HTTPS or JMS level. Depending on that, the message will be either sent to the core or the connector machine. This allows each VPC to communicate securely with the outside world and means that HTTPS traffic cannot be sent to the core machine.
--[[image:Main.Images.Fundamental.WebHome@fundamental-emagiz-multi-model--data-model.png]]
++Below the load balancer, we show our DNS functionality. This ensures that when external parties call an endpoint hosted within one of your flows, they do not have to know the IP address of your VPC but can call the DNS name that you configured partly within the portal. We finished off by replacing the IP with emagizcloud.com within all the endpoints that are hosted by eMagiz. This makes life way easier when allowing external parties to connect to your endpoints.
--=== 4.3 Risk Mitigation ===
++At the bottom of the picture, we see the EFS (Elastic File System). This file storage system stores meta-information securely for each customer so that only that customer can access it. A benefit of using this solution instead of regular file storage is that it can automatically scale. As a result, our cloud offering becomes more robust in dealing with high surges of traffic. Furthermore, by using EFS, your data is kept separate from the machines and can be re-used if the machines within the VPC need to be spun up in a different availability zone. To review: The EFS is also located on multiple availability zones for redundancy and distaster recovery.
--Within a multi-model environment, the impact on the business might be less than if a model breaks down compared to a single-model climate. This is because in a multi-model environment, only a part of the integration landscape is affected, instead of the complete landscape. For example, eMagiz models run on cloud slots. In case the cloud slot on which a model runs crashes, no business processes can be executed if the complete business runs on one model. If multiple models within a business describe different business processes, other business processes can still continue if a cloud slot for one model has crashed. This is illustrated below.
++More to the right of the picture, we see the monitoring capabilities on the eMagiz Cloud level. Here we depict our most noteworthy monitoring functionality that will be triggered when your VPC or part of your VPC runs into trouble. Apart from the trigger, we also keep the log information for 30 days for analysis purposes if an RCA needs to be performed by eMagiz Support. This information is stored within the Systems Manager and CloudWatch.
--[[image:Main.Images.Fundamental.WebHome@fundamental-emagiz-multi-model--risk-mitigation-single-instance.png]]
++Some of the monitoring triggers lead to an auto-healing action that will restore the state of your environment to normal without anyone having to take action. This means that the downtime in case of an outage is significantly reduced in these cases.
--[[image:Main.Images.Fundamental.WebHome@fundamental-emagiz-multi-model--risk-mitigation-two-instances.png]]
++Moving over to the last portion of the overview, we see some of the features we offer on the eMagiz Cloud. For example, you can define a fixed IP on outbound traffic for cases where the external party uses IP whitelisting to verify traffic. Another feature is the data sink capability that stores sunk messages in a bucket to be retrieved from the portal.
--=== 4.4 Manageability & Maintainability ===
++Please check out the suggested additional readings section for applied knowledge on how you can control the eMagiz Cloud from the portal and utilize some of these functionalities from the eMagiz portal.
--Having multiple models gives a more straightforward overview of the models and less complexity. This ensures that the models are better manageable and maintainable. Manageable here means the capability of being controlled or dealt with without difficulty. Maintainable concerns the ability to preserve or retain a particular state. If there is more than one model for a complete organization, the data models, data flows, and overall architecture contain much information, which can cause complexity. With multiple models, it is clearer what processes models describe, and it is better possible to have an overview of the overall architecture. This way, it is more easily discoverable if a process is not working effectively or something needs improvement.
++=== 3.3 Single lane vs. Double Lane ===
--=== 4.5 Deployment Architecture ===
++In the previous overview, we showed a single-lane setup. In the outline below, we deliver what we call a double lane setup. The most fundamental difference between the two is that you have a mirror image of each piece of functionality you are running with the double lane. Having a mirror image of everything reduces the downtime of the environment during maintenance and unexpected outings of your environment.
--The deployment architecture in eMagiz shows the machines, containers, and other architectural components that are needed to deploy a release. An example of what such an environment could look like is the following:
++[[image:Main.Images.Fundamental.WebHome@fundamental-emagiz-cloud-inner-workings--customer-level-overview-double-lane.png]]
--[[image:Main.Images.Fundamental.WebHome@fundamental-emagiz-multi-model--deployment-architecture.png]]
++In this double lane setup, the backup JMS is dormant until activated. All processing components running in the Cloud will run at the same. As a result, you will see the number of consumers double across all your queues.
--Each model has its deployment architecture. As can be seen, several of the same types of components exist within the environment. If there is only one model for an organization, the deployment architecture contains all the architectural components needed to deploy the model. Separate environments ensure fewer container runtimes and connector machines within one environment. This means less dependency and a better organization within the deployment environments. For example, if one machine hinders the deployment plan, there will be no issues if that machine is in a different model.
++=== 3.4 Key benefits ===
--Moreover, with multiple instances, the deployment architecture can be scaled more accurately according to what is desired for the model environment. For example, if 1 or 2 interfaces need much capacity, it is only sometimes necessary to scale according to the most significant component. In other words, resources can be used more effectively.
++Now that we have explained how our Cloud is configured, we will wrap up this fundamental by looking at the key benefits the Cloud holds for you when building your models with the help of the eMagiz platform. Below we have summarized these key benefits:
--=== 4.6 Specialization & Effectiveness within Teams ===
++* Each model has its VPC
++* Meta information is stored on EFS for auto-scaling purposes
++* Meta information is stored on EFS to guarantee a quick recovery in case of an outage
++* Monitoring capabilities provide auto-healing options
++* The eMagiz Cloud can be fully controlled via the eMagiz platform (check out our microlearnings under suggested additional readings)
++* A carwash is placed in front of all VPCs to add a layer of security
++* You can add additional features to your specific VPC
--There is more complexity when there is one large model, and more diverging concepts must be described. With multiple models, concepts that describe a model can be more clearly defined since less diverging information needs to be processed. For example, a team can specialize in one company's domain to develop a model that is very specific to that domain in terms of data structure. This also ensures that more domain-specific language and conventions can be applied, e.g., in naming entities.
--Furthermore, developers and users can specialize in one model so that models can be worked on more effectively. If there is one model for a complete organization, developers will have to understand all concepts. When there are multiple models, developers can study the concepts of one model more closely and understand the process better. This will ensure that integrations are built quickly and potential improvements are discovered and resolved more efficiently.
--Lastly, different models can be worked on in parallel, increasing the effectiveness of development and deployment. An illustration of the above-mentioned arguments is given below.
++== 4. Key takeaways ==
--[[image:Main.Images.Fundamental.WebHome@fundamental-emagiz-multi-model--specialization-and-effectivenss-within-teams.png]]
++* Each eMagiz model result in a separate VPC in the eMagiz Cloud
++* eMagiz models are deployed in the AWS eu-central-01 zone by default - other regions are possible upon request
++* A carwash is placed in front of all VPCs to add a layer of security
++* Each VPC has DNS functionality to ensure that external parties don't have to call an IP address directly
++* Each VPC is automatically monitored
++* You can add additional features to your specific VPC
++* Setting up a double lane is a safeguard against downtime
++* The eMagiz Cloud can be controlled via the eMagiz platform (check out our microlearnings under suggested additional readings)
--=== 4.7 Strategy ===
--
--With multiple models, there is a more apparent separation of the business process. This means the progress per business process can be better monitored regarding strategy. For example, if there is a separate model for the sales business process, it can be better discoverable whether more integrations have been set up. If the number of integrations has stopped growing, it might be the case that no new partners have joined the company. By having a better overview of the business processes, strategic decisions can be made more quickly. Moreover, more specific KPIs and goals can be formulated and monitored. Alerts can be put in place per model to check up on the progress and achievement of these KPIs and goals, and support can be provided.
--
--=== 4.8 Funding ===
--
--There may be a difference in funding of different departments or business units. For example, the sales department of a company might have a bigger budget for improving their model compared to the transport department. If the sales model is separate from the transport model, investments could be made by the sales department to add additional features or make other improvements. If the transport and sales models are combined, the sales department must compensate for all the costs, or fewer improvements can be made compared to when the models are split. Thus, by splitting models, the financing can be distributed more fairly.
--
--[[image:Main.Images.Fundamental.WebHome@fundamental-emagiz-multi-model--funding.png]]
--
--=== 4.9 Contractual and legal aspects ===
--
--Multiple integration models can also be beneficial due to contractual and legal arrangements. For example, sharing data between departments within one organization is not allowed. If there is one model for the complete organization, this is a complex issue since all data is available within one model. The problem is solved if there is a different model for the departments between which no data may be shared.
--Moreover, contractual arrangements between different departments within a company may differ. It is better to develop separate models. For example, the sales department might have a contractual arrangement that does not allow for a specific type of integration. In case the contractual arrangements of the transport department allow for that type of integration and it is beneficial for their model, it is better to have separate models.
--
--== 4. Key Takeaways ==
--
--* An environment in eMagiz in which there are multiple integration models is called a multi-model environment.
--* With integration model, an eMagiz platform instance is meant, in which it is possible to build integrations.
--* There are several reasons for choosing multiple integrations models, i.e. separate eMagiz instances. These reasons can be grouped under the term separation of concerns.
--* Separation of concerns is a design principle in software development and architecture for separating an application in two or more sections, such that each section addresses a particular concern.
--* It can be useful to have a clear separation in these business processes, such that the processes can evolve independently.
--* It can be beneficial to split large data models.
--* Within a multi-model environment, the impact on the business might a model break down is less as compared to a single-model environment.
--* By having multiple models, there is a clearer overview of the models, and less complexity. This ensures that the models are better manageable and maintainable.
--* In case there is only one model for an organization, the deployment architecture contains all architectural components that are needed to deploy the model. Separate environments ensure that there are less container runtimes and connector machines within one environment.
--* With multiple models, concepts that describe a model can be more clearly defined, since less diverging information needs to be processed.
--* By splitting models, the financing can be distributed more fairly.
--* Having multiple integrations models can be beneficial due to contractual and legal arrangements.
--
  == 5. Suggested Additional Readings ==
++If you are interested in this topic and want to learn how you can control your Cloud with the help of the eMagiz platform, please check out our microlearnings offering on eMagiz Cloud Management:
--* [[Messaging>>doc:Main.eMagiz Academy.Fundamentals.fundamental-messaging-introduction||target="blank"]]
--* [[API Gateway>>doc:Main.eMagiz Academy.Fundamentals.fundamental-api-gateway-introduction||target="blank"]]
--* [[Event Streaming>>doc:Main.eMagiz Academy.Fundamentals.fundamental-event-streaming-introduction||target="blank"]]
--* [[Data Models >>doc:Main.eMagiz Academy.Fundamentals.fundamental-data-models||target="blank"]]
--* [[Understanding Design Architecture - Basic>>doc:Main.eMagiz Academy.Microlearnings.Crash Course.Crash Course Platform.crashcourse-platform-design-understanding-design-architecture-basic||target="blank"]]
++* [[Novice - eMagiz Cloud Management>>doc:Main.eMagiz Academy.Microlearnings.Novice.eMagiz Cloud Management.WebHome||target="blank"]]
++* [[Intermediate - eMagiz Cloud Management>>doc:Main.eMagiz Academy.Microlearnings.Intermediate Level.eMagiz Cloud Management.WebHome||target="blank"]]
++* [[Advanced - eMagiz Cloud Management>>doc:Main.eMagiz Academy.Microlearnings.Advanced Level.eMagiz Cloud Management.WebHome||target="blank"]]
  )))((({{toc/}}))){{/container}}{{/container}}

Changes for page Multi-Model Explained

Summary

Details

Navigation