Wiki source code of Traceability in eMagiz
Last modified by Erik Bakker on 2024/02/08 16:08
Show last authors
| author | version | line-number | content |
|---|---|---|---|
| 1 | {{container}}{{container layoutStyle="columns"}}((( | ||
| 2 | This fundamental will zoom in on which traceability concepts we have incorporated within the eMagiz portal to ensure that data passes from A to B. We will focus on traceability from three perspectives. At first, we zoom in on whether a message has been sent from A to B. Secondly, we determine whether a specific message has been dispatched from A to B. Thirdly, we focus on proving that the message was indeed sent years after a message has been sent. After this journey, you should have a solid understanding of which options eMagiz offers to trace messages between various systems across the integration platform. | ||
| 3 | |||
| 4 | All these concepts are used for Traceability & Compliancy business requirements... | ||
| 5 | |||
| 6 | Should you have any questions, please get in touch with academy@emagiz.com. | ||
| 7 | |||
| 8 | == 1. Prerequisites == | ||
| 9 | |||
| 10 | * An overall view of the capabilities of the eMagiz platform will be helpful | ||
| 11 | |||
| 12 | == 2. Key concepts == | ||
| 13 | |||
| 14 | eMagiz offers various options to trace messages across the platform | ||
| 15 | * Statistics | ||
| 16 | * Explore | ||
| 17 | * Long Term Archiving | ||
| 18 | |||
| 19 | == 3. Traceability in eMagiz == | ||
| 20 | |||
| 21 | This fundamental will zoom in on which traceability concepts we have incorporated within the eMagiz portal to ensure that data travels from A to B. We will focus on traceability from three perspectives. At first, we zoom in on whether a message has been sent from A to B. Secondly, we determine whether a specific message has been dispatched from A to B. Thirdly, we focus on proving that the message was indeed sent years after a message has been sent. After this journey, you should have a solid understanding of which options eMagiz offers to trace messages between various systems across the integration platform. | ||
| 22 | |||
| 23 | === 3.1 Statistics === | ||
| 24 | |||
| 25 | ===== 3.1.1 Queue Statistics ==== | ||
| 26 | |||
| 27 | In the Manage phase of eMagiz, we collect statistics to determine whether a message has been sent from one system to another. For example, we have the queue statistics for messaging. In this overview, you can zoom in on a specific queue and see the number of messages sent within that timeframe, the number of messages in the queue at a particular time, and whether or not messages are consumed at a specific moment in time. See below for an illustration of this concept. | ||
| 28 | |||
| 29 | [[image:Main.Images.Fundamental.WebHome@fundamental-traceability-in-emagiz--queue-statistics.png]] | ||
| 30 | |||
| 31 | With the help of this overview, you can determine that a queue has processed a message at a specific moment. However, the statistics do not tell you anything about the content of the data. For most use cases, this is sufficient as the fact that a message has been sent is proof that the receiving application has received the data and can be found in that application. | ||
| 32 | |||
| 33 | For more information on queue statistics please check out the following [[Crashcourse Messaging>>doc:Main.eMagiz Academy.Microlearnings.Crash Course.Crash Course Messaging.crashcourse-messaging-interpreting-queue-statistics-gen3||target="blank"]]. | ||
| 34 | |||
| 35 | ===== 3.1.2 HTTP Statistics ==== | ||
| 36 | |||
| 37 | Basides the queue statistics, there are the HTTP statistics used for the API Gateway. Here, an overview is shown for all runtimes which contain APIs. For each of these runtimes the performace statistics are shown as seen in the illustration below. | ||
| 38 | |||
| 39 | [[image:Main.Images.Fundamental.WebHome@fundamental-traceability-in-emagiz-http-statistics.png]] | ||
| 40 | |||
| 41 | It is possible to zoom in on each of the runtimes such that the information about solely that runtime is shown. Here, the information is focused on the details around the requests. Such as the request method, the HTTP response and the requests per user. | ||
| 42 | |||
| 43 | For more information on HTTP statistics please check out the following [[HTTP Statistics>>doc:Main.eMagiz Academy.Microlearnings.Advanced Level.Advanced monitoring.advanced-monitoring-http-statistics.WebHome||target="blank"]]. | ||
| 44 | |||
| 45 | ===== 3.1.3 Runtime Statistics ==== | ||
| 46 | |||
| 47 | The overview for the runtime statistics provides us with similar information as the overview of the HTTP statistics. However, when zooming in on the runtimes, specific information is shown on the performance of that particular runtime instead of information about messages. This overview is shown in the illustration below. | ||
| 48 | |||
| 49 | [[image:Main.Images.Fundamental.WebHome@fundamental-traceability-in-emagiz-runtime-statistics.png]] | ||
| 50 | |||
| 51 | For more information on runtime statistics please check out the following [[Runtime Statistics>>doc:Main.eMagiz Academy.Microlearnings.Advanced Level.Advanced monitoring.advanced-monitoring-runtime-statistics-key-metrics.WebHome||target="blank"]]. | ||
| 52 | |||
| 53 | In some cases, however, more traceability is needed for Ops work. For those use cases, we have additional functionality within eMagiz called data sink. | ||
| 54 | |||
| 55 | === 3.2 Explore === | ||
| 56 | |||
| 57 | ===== 3.2.1 Message Redelivery ===== | ||
| 58 | |||
| 59 | In the 'Explore' overview, one of the traceability components which can be found is the 'Redelivery'. Here, the messages waiting for redelivery are shown. With the different buttons the user can manage the messages. This overview is shown in the illustration below. | ||
| 60 | |||
| 61 | [[image:Main.Images.Microlearning.WebHome@intermediate-message-redelivery-redelivery-in-manage-gen3--message-redelivery-overview.png]] | ||
| 62 | |||
| 63 | For more information on message redelivery please check out the following [[Message Redelivery>>doc:Main.eMagiz Academy.Microlearnings.Intermediate Level.Message Redelivery.intermediate-message-redelivery-redelivery-in-manage-gen3.WebHome||target="blank"]]. | ||
| 64 | |||
| 65 | ===== 3.2.3 Queue Browser ===== | ||
| 66 | |||
| 67 | The second option within the explore functionality is the queue browser. The queue browser provides the oppurtunity to 'watch' your flows. This can be done in two different manners. The first is via the 'Explore' button. Which is used to inspect messages that remained in the flow and have therefore not been processed. The second is via the 'Wiretap' button. Which is used to live trace messages moving through the flow. The queue browser can be seen in the illustration below. | ||
| 68 | |||
| 69 | [[image:Main.Images.Fundamental.WebHome@fundamental-traceability-in-emagiz-queue-browser.png]] | ||
| 70 | |||
| 71 | For more information on the queue browser please check out the following [[Queue browser>>doc:Main.eMagiz Academy.Microlearnings.Intermediate Level.Testing in eMagiz.intermediate-testing-in-emagiz-queue-browser.WebHome||target="blank"]]. | ||
| 72 | |||
| 73 | ===== 3.2.3 Data Sink ===== | ||
| 74 | |||
| 75 | On top of the standard Manage functionality in eMagiz, you can acquire additional functionality that allows you to sink data at any given point in the flow into a bucket hosted by eMagiz. We advise you to do this twice per integration. Once when the message enters the eMagiz platform and once the message leaves the eMagiz platform. By providing a unique identifier (i.e., an order number), you can search through this data for maximum 180 days (minimum 30 days) to determine whether a specific message with a particular identifier is indeed received and/or delivered by eMagiz in the Manage phase of eMagiz. See below for an illustration of this functionality. | ||
| 76 | |||
| 77 | [[image:Main.Images.Fundamental.WebHome@fundamental-traceability-in-emagiz--data-sink-view-manage.png]] | ||
| 78 | |||
| 79 | [[image:Main.Images.Fundamental.WebHome@fundamental-traceability-in-emagiz--data-sink-search-options.png]] | ||
| 80 | |||
| 81 | [[image:Main.Images.Fundamental.WebHome@fundamental-traceability-in-emagiz--data-sink-search-results.png]] | ||
| 82 | |||
| 83 | For more information on data sink please check out the following [[Data sink>>doc:Main.eMagiz Academy.Microlearnings.Advanced Level.Data Management.advanced-data-management-data-sink||target="blank"]]. | ||
| 84 | |||
| 85 | However, in some cases, there are additional legal requirements you need to meet. These legal requirements require you to prove for an extended period (i.e., seven years) that a specific message was sent at a particular moment in time. For this requirement, we have the long-term archiving functionality in eMagiz. | ||
| 86 | |||
| 87 | ==== 3.5 Long Term Archiving ==== | ||
| 88 | |||
| 89 | The long-term archiving functionality is an expansion of the data sink functionality. By adding a specific tag to the logic within the flow, the data will be placed in the data sink and placed in our long-term archiving solution. In this solution, we will keep the data for a standard period of seven years before deleting the data from the long-term archiving solution. This allows you to retrieve chunks of data from the long-term archiving via a ticket request in our support portal. As a result, we will provide you with the requested data promptly. | ||
| 90 | |||
| 91 | For more information on data sink please check out the following [[Long term archiving>>doc:Main.eMagiz Academy.Microlearnings.Advanced Level.Data Management.advanced-data-management-long-term-archiving||target="blank"]]. | ||
| 92 | |||
| 93 | == 4. Key takeaways == | ||
| 94 | |||
| 95 | * Each eMagiz model has a standard Manage phase in which traceability elements are available for your environment | ||
| 96 | * The main elements are the statistics and explore | ||
| 97 | * Thhe statistics are used for the different integration patterns | ||
| 98 | * The message redelivery and the queue browser are used to explore messages on a queue | ||
| 99 | * eMagiz offers Data Sink and long term Archiving as additional licensed features | ||
| 100 | * Data sink is helpful for an Ops scenario to check whether a specific message is processed by eMagiz (Traceability) | ||
| 101 | * Long term archiving is helpful for legal purposes as it gives you the option to prove for seven years that a specific message is sent at a specific time (Compliancy) | ||
| 102 | |||
| 103 | == 5. Suggested Additional Readings == | ||
| 104 | |||
| 105 | If you are interested in this topic and want to learn how you can control your Cloud with the help of the eMagiz platform, please check out our microlearnings offering on eMagiz Cloud Management: | ||
| 106 | |||
| 107 | * [[Crashcourse Messaging>>doc:Main.eMagiz Academy.Microlearnings.Crash Course.Crash Course Messaging.crashcourse-messaging-interpreting-queue-statistics-gen3||target="blank"]] | ||
| 108 | * [[HTTP Statistics>>doc:Main.eMagiz Academy.Microlearnings.Advanced Level.Advanced monitoring.advanced-monitoring-http-statistics.WebHome||target="blank"]] | ||
| 109 | * [[Runtime Statistics>>doc:Main.eMagiz Academy.Microlearnings.Advanced Level.Advanced monitoring.advanced-monitoring-runtime-statistics-key-metrics.WebHome||target="blank"]] | ||
| 110 | * [[Message Redelivery>>doc:Main.eMagiz Academy.Microlearnings.Intermediate Level.Message Redelivery.intermediate-message-redelivery-redelivery-in-manage-gen3.WebHome||target="blank"]] | ||
| 111 | * [[Queue browser>>doc:Main.eMagiz Academy.Microlearnings.Intermediate Level.Testing in eMagiz.intermediate-testing-in-emagiz-queue-browser.WebHome||target="blank"]] | ||
| 112 | * [[Data sink>>doc:Main.eMagiz Academy.Microlearnings.Advanced Level.Data Management.advanced-data-management-data-sink||target="blank"]] | ||
| 113 | * [[Long term archiving>>doc:Main.eMagiz Academy.Microlearnings.Advanced Level.Data Management.advanced-data-management-long-term-archiving||target="blank"]] | ||
| 114 | |||
| 115 | )))((({{toc/}}))){{/container}}{{/container}} |