Wiki source code of Data sink
Last modified by Erik Bakker on 2023/09/29 08:10
Show last authors
| author | version | line-number | content |
|---|---|---|---|
| 1 | {{container}}{{container layoutStyle="columns"}}((( | ||
| 2 | {{info}} | ||
| 3 | Please note that you can only use this functionality after obtaining an additional license to the platform. | ||
| 4 | {{/info}} | ||
| 5 | |||
| 6 | In this microlearning, we will learn how to implement data sink functionality on flow level in eMagiz. On top of that, we will learn conceptually how this data sink solution works, so you get a feeling of what happens underwater to safely and securely store your data. | ||
| 7 | |||
| 8 | Should you have any questions, please get in touch with [[academy@emagiz.com>>mailto:academy@emagiz.com]]. | ||
| 9 | |||
| 10 | == 1. Prerequisites == | ||
| 11 | |||
| 12 | * Advanced knowledge of the eMagiz platform | ||
| 13 | * A valid license to use data sink | ||
| 14 | |||
| 15 | == 2. Key concepts == | ||
| 16 | |||
| 17 | * With this functionality, you can determine on flow level whether data needs to be placed in the data sink | ||
| 18 | * A working example is provided via the Store of how to implement the functionality in your flow | ||
| 19 | * A additional license is needed to enable this functionality | ||
| 20 | * Retrieving data can be executed via the Data Sink functionality in Manage | ||
| 21 | |||
| 22 | == 3. Data sink via eMagiz == | ||
| 23 | |||
| 24 | In this microlearning, we will learn how to implement data sink functionality on flow level in eMagiz. On top of that, we will learn conceptually how this data sink solution works, so you get a feeling of what happens underwater to safely and securely store your data. | ||
| 25 | |||
| 26 | Before you can utilize this functionality, you need to secure an additional license on the platform. If you want more information or set up talks to require such an additional license, please get in touch with us at productmanagement@emagiz.com. | ||
| 27 | |||
| 28 | Once you have secured the additional license, we will configure your AWS cloud slot to enable data sink. Afterward, you can implement the functionality in the flow that will send the data to AWS to be archived. | ||
| 29 | |||
| 30 | === 3.1 Implementation in the flow === | ||
| 31 | |||
| 32 | To implement the functionality in the flow, you can use the store component called "eMagiz Data Sink" This [[store component>>doc:Main.eMagiz Store.Accelerators.eMagiz Data Sink.WebHome||target="blank"]] will provide you with all the necessary components, including the correct configuration needed to set up the connection. Furthermore, you can only use this [[store component>>doc:Main.eMagiz Store.Accelerators.eMagiz Data Sink.WebHome||target="blank"]] after acquiring the proper license. We will automatically fill in all relevant connection details needed to store the data correctly. Finally, follow the import wizard provided with the store component to configure it properly. | ||
| 33 | |||
| 34 | The only thing left for you is determining when you want to send the data to the archive. Typically we see that the point of entry on the model and the point of exit of the model are the most desirable candidates for archiving. This way, you can prove when OPS questions arise that you received and subsequently send a specific message. | ||
| 35 | |||
| 36 | === 3.2 Data sink inner workings === | ||
| 37 | |||
| 38 | Once your message is sent to the data sink solution, it will be sent to what we call a bucket. The data will remain in this bucket for a specific period determined based on the agreed-upon license. Typical values are ninety or one hundred eighty days. Within this timeframe, you can view the messages within the bucket to help analyze operational problems. | ||
| 39 | |||
| 40 | The whole process from beginning to end is monitored, and the model owner, alongside eMagiz support, will be notified if things go wrong within the solution. Furthermore, they will also be informed of the amount of data in various stages of the process. See below for a simplified version of how this works for you. | ||
| 41 | |||
| 42 | [[image:Main.Images.Microlearning.WebHome@advanced-data-management-data-sink-solution.png]] | ||
| 43 | |||
| 44 | === 3.3 View the data via eMagiz === | ||
| 45 | |||
| 46 | When you navigate to the Manage phase of your model in eMagiz, you will see a tab called "Data sink" once the license has been activated on your model. Here, you will see all messages sent within the time frame. | ||
| 47 | |||
| 48 | [[image:Main.Images.Microlearning.WebHome@advanced-data-management-data-sink-data-sink-view-manage.png]] | ||
| 49 | |||
| 50 | Here you can search on the Unique Identifier you defined in your flow or list the last 1000 objects within the bucket. | ||
| 51 | |||
| 52 | [[image:Main.Images.Microlearning.WebHome@advanced-data-management-data-sink-data-sink-search-options.png]] | ||
| 53 | |||
| 54 | Once you have the view you want, you can select a specific message from the list and press Show Left or Show Right. This way, you can compare two messages (input and output, for example) to each other. An example of how this can look is shown below. | ||
| 55 | |||
| 56 | [[image:Main.Images.Microlearning.WebHome@advanced-data-management-data-sink-data-sink-search-results.png]] | ||
| 57 | |||
| 58 | == 4. Key takeaways == | ||
| 59 | |||
| 60 | * With this functionality, you can determine on flow level whether data needs to be placed in the data sink | ||
| 61 | * A working example is provided via the Store of how to implement the functionality in your flow | ||
| 62 | * A additional license is needed to enable this functionality | ||
| 63 | * Retrieving data can be executed via the Data Sink functionality in Manage | ||
| 64 | |||
| 65 | == 5. Suggested Additional Readings == | ||
| 66 | |||
| 67 | If you are interested in this topic and want more information, please read the release notes and help texts provided by eMagiz.)))((({{toc/}}))){{/container}}{{/container}} |