Skip to Content

Changes for page Pick up Files

Last modified by Danniar Firdausy on 2024/08/21 16:48
From version 22.6
edited by Danniar Firdausy
on 2024/08/21 16:47
Change comment: There is no comment for this version
To version 15.1
edited by eMagiz
on 2022/05/10 11:16
Change comment: There is no comment for this version

Summary

Details

Page properties
Title
... ... @@ -1,1 +1,0 @@
1 -Pick up Files
Parent
... ... @@ -1,1 +1,0 @@
1 -WebHome
Author
... ... @@ -1,1 +1,1 @@
1 -XWiki.dfirdausy
1 +XWiki.marijn
Default language
... ... @@ -1,1 +1,0 @@
1 -en
Content
... ... @@ -1,8 +1,20 @@
1 -{{container}}{{container layoutStyle="columns"}}(((
2 -In this microlearning, we will explain the basics of retrieving files from a local directory using eMagiz. We will cover the essentials for setting up file retrieval from a directory that is directly accessible within the network where the eMagiz process is running. We will walk you through key concepts, considerations, and best practices to ensure a smooth file handling process.
1 +{{html wiki="true"}}
2 +<div class="ez-academy">
3 + <div class="ez-academy_body">
3 3  
4 -If you have any questions along the way, feel free to reach out to us at [[academy@emagiz.com>>mailto:academy@emagiz.com]].
5 +<div class="doc">
5 5  
7 +
8 +
9 += Pick up Files =
10 +
11 +In this microlearning, we will explain the basics of retrieving files from a local directory (i.e. a directory that is directly accessible within the network where the eMagiz process is running)
12 +
13 +Should you have any questions, please contact academy@emagiz.com.
14 +
15 +* Last update: February 25th, 2021
16 +* Required reading time: 8 minutes
17 +
6 6  == 1. Prerequisites ==
7 7  
8 8  * Basic knowledge of the eMagiz platform
... ... @@ -10,9 +10,18 @@
10 10  == 2. Key concepts ==
11 11  
12 12  This microlearning centers around retrieving files from a local directory via eMagiz.
13 -* With retrieving we mean: Checking with a certain interval if new files are ready to be pulled from the directory by emagiz
14 -* With local directory we mean: A directory that is directly accessible within the network where the eMagiz process is running
25 +By retrieving we mean: Checking with a certain interval if new files are ready to be pulled from the directory by emagiz
26 +By local directory we mean: A directory that is directly accessible within the network where the eMagiz process is running
15 15  
28 +Key considerations when picking up files are:
29 +
30 +* What are the determining criteria to define when a file is "ready" to be picked up (age, pattern, size, etc.)
31 +* With what frequency is eMagiz going to check for new files
32 +* Has eMagiz the appropriate rights to read from the directory where the messages are to be retrieved from
33 +* What is the format (i.e. XML, JSON, EDI, TXT, CSV, etc.) of the files that eMagiz needs to retrieve
34 +
35 +
36 +
16 16  == 3. Pick up Files ==
17 17  
18 18  One of the easiest connectivity methods to implement within eMagiz is the retrieval of files from a local directory.
... ... @@ -27,7 +27,7 @@
27 27  
28 28  By default, eMagiz has a filter that will only accept each file once. This option is called prevent duplicates
29 29  
30 -[[image:Main.Images.Microlearning.WebHome@crashcourse-messaging-pick-up-files--prevent-duplicates.png]]
51 +<p align="center">[[image:crashcourse-messaging-pick-up-files--prevent-duplicates.png||]]</p>
31 31  
32 32  As the eMagiz help text clearly states this option will ensure that eMagiz will keep a list in memory and files are only to be passed once.
33 33  This check is done before any other filtering is done. So be aware when mixing these settings.
... ... @@ -43,7 +43,7 @@
43 43  
44 44  To filter which messages need to be retrieved from a local directory we use a composite file list filter.
45 45  
46 -[[image:Main.Images.Microlearning.WebHome@crashcourse-messaging-pick-up-files--composite-file-list-filter-component.png]]
67 +<p align="center">[[image:crashcourse-messaging-pick-up-files--composite-file-list-filter-component.png||]]</p>
47 47  
48 48  In this filter, we use the following four criteria to filter our messages:
49 49  
... ... @@ -54,12 +54,12 @@
54 54  
55 55  The result of this configuration should be:
56 56  
57 -[[image:Main.Images.Microlearning.WebHome@crashcourse-messaging-pick-up-files--composite-file-list-filter.png]]
78 +<p align="center">[[image:crashcourse-messaging-pick-up-files--composite-file-list-filter.png||]]</p>
58 58  
59 59  The next step would be to link this support object to the file inbound channel adapter (the starting point of our flow). To do so open the component and navigate to the Advanced tab.
60 60  Here you need to select the correct filter to reference your support object.
61 61  
62 -[[image:Main.Images.Microlearning.WebHome@crashcourse-messaging-pick-up-files--composite-file-list-filter-linked.png]]
83 +<p align="center">[[image:crashcourse-messaging-pick-up-files--composite-file-list-filter-linked.png||]]</p>
63 63  
64 64  === 3.2 Frequency ===
65 65  
... ... @@ -72,7 +72,7 @@
72 72  * Fixed-Rate
73 73  * Cron Trigger
74 74  
75 -[[image:Main.Images.Microlearning.WebHome@crashcourse-messaging-pick-up-files--frequency-options.png]]
96 +<p align="center">[[image:crashcourse-messaging-pick-up-files--frequency-options.png||]]</p>
76 76  
77 77  Both the Fixed Delay and the Fixed Rate are static. This means that when you want to change the frequency with which you look you will have to make a new version of the flow.
78 78  
... ... @@ -95,35 +95,52 @@
95 95  Files come in many different forms, shapes, and sizes. Due to the wide variety of options, you will need to ensure that the data is processed correctly.
96 96  How to process each message format (i.e. XML, JSON, EDI, CSV, etc.) correctly is explained in other microlearnings where we zoom in on each of the message formats.
97 97  
98 -For learning about picking up files we need to know that in order for eMagiz to process the content of the file in the onramp and further downstream the contents of the file need to be transformed to string.
119 +For learning about picking up files we need to know that in other for eMagiz to
120 +process the content of the file in the onramp and further downstream the contents of the file need to be transformed to string.
99 99  
100 100  There is a component in eMagiz called the file to string transformer. You should place this directly after receiving the file from the external source
101 101  
102 -[[image:Main.Images.Microlearning.WebHome@crashcourse-messaging-pick-up-files--file-to-string-component.png]]
124 +<p align="center">[[image:crashcourse-messaging-pick-up-files--file-to-string-component.png||]]</p>
103 103  
104 104  Configuration wise you need to select the option Delete files to ensure that the files are deleted
105 105  
106 -[[image:Main.Images.Microlearning.WebHome@crashcourse-messaging-pick-up-files--file-to-string-configuration.png]]
128 +<p align="center">[[image:crashcourse-messaging-pick-up-files--file-to-string-configuration.png||]]</p>
107 107  
108 108  The result of linking the components together will lead to the following solution
109 109  
110 -[[image:Main.Images.Microlearning.WebHome@crashcourse-messaging-pick-up-files--simple-solution.png]]
132 +<p align="center">[[image:crashcourse-messaging-pick-up-files--simple-solution.png||]]</p>
111 111  
112 112  With all this in mind, you have successfully learned how to set up your first connectivity via a messaging flow in eMagiz!
113 113  
114 -== 4. Key takeaways ==
136 +===== Practice =====
115 115  
116 -* Criteria for File Retrieval: Determine the criteria for identifying when a file is "ready" to be picked up, such as age, pattern, or size. Ensure these criteria are properly configured to avoid issues with file processing.
117 -* File Formats: Ensure you handle various file formats (e.g., XML, JSON, EDI, CSV) correctly. Use the file to string transformer to process file contents as needed.
118 -* Check Frequency: Decide how frequently eMagiz will check the local directory for new files. You can choose between options like Fixed Delay, Fixed Rate, or Cron Trigger, based on your needs and expected changes in frequency.
119 -* Access Rights: Verify that eMagiz has the necessary permissions to read from the directory and any parent directories. Proper access rights are crucial for successful file retrieval.
138 +== 4. Assignment ==
120 120  
121 -== 5. Suggested Additional Readings ==
140 +Build an entry that retrieves files from an external source and ensures that the data is delivered on the onramp queue.
141 +This assignment can be completed with the help of your (Academy) project you have created/used in the previous assignment.
122 122  
123 -If you are interested in this topic and want more information on it please read the help text provided by eMagiz and read the following link:
143 +== 5. Key takeaways ==
124 124  
125 -* [[Store (Menu)>>doc:Main.eMagiz Store.WebHome||target="blank"]]
126 -** [[Accelerators (Navigation)>>doc:Main.eMagiz Store.Accelerators.WebHome||target="blank"]]
127 -*** [[File Inbound (Store Item)>>doc:Main.eMagiz Store.Accelerators.File Inbound.WebHome||target="blank"]]
128 -* [[Pick up Files (Search Results)>>url:https://docs.emagiz.com/bin/view/Main/Search?sort=score&sortOrder=desc&highlight=true&facet=true&r=1&f_space_facet=0%2FMain.&l_space_facet=10&f_type=DOCUMENT&f_locale=en&f_locale=&f_locale=en&text=%22pick+up+files%22||target="blank"]]
129 -)))((({{toc/}}))){{/container}}{{/container}}
145 +* Key considerations when picking up files are:
146 + * What are the determining criteria to define when a file is "ready" to be picked up (age, pattern, size, etc.)
147 + * With what frequency is eMagiz going to check for new files
148 + * Has eMagiz the appropriate rights to read from the directory where the messages are to be retrieved from
149 + * What is the format (i.e. XML, JSON, EDI, TXT, CSV, etc.) of the files that eMagiz needs to retrieve
150 +
151 +
152 +
153 +== 6. Suggested Additional Readings ==
154 +
155 +If you are interested in this topic and want more information on it please read the help text provided by eMagiz.
156 +
157 +== 7. Silent demonstration video ==
158 +
159 +This video demonstrates how you could have handled the assignment and gives you some context on what you have just learned.
160 +
161 +<iframe width="1280" height="720" src="../../vid/microlearning/crashcourse-messaging-pick-up-files.mp4" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
162 +
163 +</div>
164 +</div>
165 +</div>
166 +
167 +{{/html}}