Skip to Content

Traceability in eMagiz

Last modified by Bouke Reitsma on 2024/08/21 08:46

In this document, we will delve into the traceability concepts incorporated within the eMagiz portal to ensure data travels seamlessly across systems. We will explore various statistics and functionalities, such as queue statistics, HTTP statistics, runtime statistics, message redelivery, queue browser, and data sink, to understand how eMagiz facilitates message tracing and monitoring across the integration platform.

Should you have any questions, please get in touch with academy@emagiz.com.

1. Prerequisites

  • An overall view of the capabilities of the eMagiz platform will be helpful

2. Key concepts

eMagiz offers various options to trace messages across the platform

  • Statistics
  • Explore
  • Long Term Archiving

3. Traceability in eMagiz

This fundamental will investigate which traceability concepts we have incorporated within the eMagiz portal to ensure data travels from A to B. We will focus on traceability from three perspectives. At first, we zoom in to see whether a message has been sent from A to B. Secondly, we determine whether a specific message has been dispatched from A to B. Thirdly, we focus on proving that the message was indeed sent years after a message had been sent. After this journey, you should understand which options eMagiz offers to trace messages between various systems across the integration platform.

3.1 Statistics

3.1.1 Queue Statistics

In the Manage phase of eMagiz, we collect statistics to determine whether a message has been sent from one system to another. For example, we have the queue statistics for messaging. In this overview, you can zoom in on a specific queue and see the number of messages sent within that timeframe, the number of messages in the queue at a particular time, and whether or not messages are consumed at a specific time. See below for an illustration of this concept.

fundamental-traceability-in-emagiz--queue-statistics.png

Warning

Using browser extensions, mainly adblockers, can reduce the experience or even prevent showing the graphs on the dashboard. Therefore, we advise disabling extensions on the http://my.emagiz.com domain when you are experiencing issues.

This overview helps you determine whether a queue has processed a message at a specific moment. However, the statistics do not tell you anything about the content of the data. For most use cases, this is sufficient, as the fact that a message has been sent is proof that the receiving application has received the data and can be found in that application.

For more information on queue statistics, please check out the following Crashcourse Messaging.

3.1.2 HTTP Statistics

In addition to the queue statistics, there are the HTTP statistics used for the API Gateway. Here, an overview is shown for all runtimes that contain APIs. For each of these runtimes, the performance statistics are shown, as seen in the illustration below. 

fundamental-traceability-in-emagiz-http-statistics.png

It is possible to zoom in on each of the runtimes so that information about solely that runtime is shown. Here, the information is focused on the details around the requests, such as the request method, the HTTP response, and the requests per user.
 
For more information on HTTP statistics, please check out the following HTTP Statistics.

3.1.3 Runtime Statistics

The overview of the runtime statistics provides similar information to the summary of the HTTP statistics. However, when zooming in on the runtimes, specific information is shown on the performance of that particular runtime instead of information about messages. This overview is shown in the illustration below.

fundamental-traceability-in-emagiz-runtime-statistics.png
  
For more information on runtime statistics, please check out the following Runtime Statistics.

However, in some cases, more traceability is needed for Ops work. We have additional functionality within eMagiz called data sink for those use cases.

3.2 Explore

3.2.1 Message Redelivery

One of the traceability components in the 'Explore' overview is 'Redelivery.' Here, messages waiting for redelivery are shown. The user can manage the messages with the different buttons. This overview is illustrated below.

intermediate-message-redelivery-redelivery-in-manage-gen3--message-redelivery-overview.png

For more information on message redelivery, please check out the following Message Redelivery.

3.2.3 Queue Browser

The second option within the explore functionality is the queue browser. The queue browser provides the opportunity to 'watch' your flows. This can be done in two different manners. The first is via the 'Explore' button, which is used to inspect messages that remained in the flow and have, therefore, not been processed. The second is via the 'Wiretap' button, which is used to live trace messages moving through the flow. The queue browser can be seen in the illustration below.

fundamental-traceability-in-emagiz-queue-browser.png

For more information on the queue browser, please check out the following Queue browser.

3.2.3 Data Sink

On top of the standard Manage functionality in eMagiz, you can acquire additional functionality that allows you to sink data at any given point in the flow into a bucket hosted by eMagiz. We advise you to do this twice per integration. Once the message enters the eMagiz platform and once the message leaves the eMagiz platform, By providing a unique identifier (i.e., an order number), you can search through this data for a maximum of 180 days (minimum 30 days) to determine whether a specific message with a particular identifier is indeed received and delivered by eMagiz in the Manage phase of eMagiz. See below for an illustration of this functionality.

fundamental-traceability-in-emagiz--data-sink-view-manage.png

fundamental-traceability-in-emagiz--data-sink-search-options.png

fundamental-traceability-in-emagiz--data-sink-search-results.png

For more information on data sink, please check out the following Data sink.

However, in some cases, you need to meet additional legal requirements. These legal requirements require you to prove for an extended period (i.e., seven years) that a specific message was sent at a particular moment. For this requirement, we have the long-term archiving functionality in eMagiz.

3.5 Long-Term Archiving

The long-term archiving functionality expands the data sink functionality. Adding a specific tag to the logic within the flow will place the data in the data sink and in our long-term archiving solution. In this solution, we will keep the data for a standard period of seven years before deleting it. This allows you to retrieve chunks of data from the long-term archiving via a ticket request in our support portal. As a result, we will promptly provide you with the requested data.

For more information on this functionality, please check out the following microlearning.

4. Key takeaways

  • Each eMagiz model has a standard Manage phase in which traceability elements are available for your environment
  • The main elements are the statistics and explore
  • Thhe statistics are used for the different integration patterns
  • The message redelivery and the queue browser are used to explore messages on a queue
  • eMagiz offers Data Sink and long term Archiving as additional licensed features
  • Data sink is helpful for an Ops scenario to check whether a specific message is processed by eMagiz (Traceability)
  • Long term archiving is helpful for legal purposes as it gives you the option to prove for seven years that a specific message is sent at a specific time (Compliancy)

5. Suggested Additional Readings

If you are interested in this topic and want to learn how you can control your Cloud with the help of the eMagiz platform, please check out our microlearnings offering on eMagiz Cloud Management: