Skip to Content

Changes for page State Generation

Last modified by Erik Bakker on 2025/06/10 09:58
From version 60.1
edited by Danniar Firdausy
on 2024/06/06 11:44
Change comment: There is no comment for this version
To version 36.1
edited by Erik Bakker
on 2022/05/03 13:21
Change comment: There is no comment for this version

Summary

Details

Page properties
Title
... ... @@ -1,1 +1,1 @@
1 -State Generation
1 +Data Pipelines
Author
... ... @@ -1,1 +1,1 @@
1 -XWiki.dfirdausy
1 +XWiki.ebakker
Content
... ... @@ -1,8 +1,9 @@
1 -{{velocity}}
2 -#set ($discard = $xwiki.ssx.use('Main.Extensions.ContainerSizingCoursePage.WebHome'))
3 -{{/velocity}}
1 +In this course we will focus on data pipelines. Data Pipelines are one of the lesser known integration patterns that eMagiz supports.
2 +eMagiz Data Pipelines are good in transporting large quantities of data with no need for transformation between source and sink.
3 +Within eMagiz, we support several combinations of those source and sink systems.
4 +For the combinations that are used most, we have made separate microlearnings to explain how they work.
4 4  
5 -{{container}}{{container layoutStyle="columns"}}(((In this course, we will focus on the concept of state generation and how to properly implement it with the help of the eMagiz platform. This course is focusses four operations: enrichment, change detection, duplication detection, and aggregation.)))((({{html}}<div class="strip__attribute" style="text-align:center"><table><tr><td><img class="strip__attribute-icon strip__attribute-icon--duration" src="/bin/download/Main/Images/General/WebHome/icon-duration32.svg"/></td></tr><tr><td><div class="strip__attribute-label">20-30 minutes</div></td></tr></table></div>{{/html}}))){{/container}}{{/container}}
6 +* **Total time:** between 30 and 45 minutes
6 6  
7 7  
8 8  (% style="border-color:#10137e" %)
... ... @@ -13,17 +13,14 @@
13 13   <main class="master">
14 14   <ul class="strip-container">
15 15   <li class="strip">
16 - <a href="/bin/view/Main/eMagiz%20Academy/Microlearnings/Intermediate%20Level/Database%20connectivity/intermediate-database-connectivity-available-options" target="_blank" class="strip__link">
17 - <div class="strip__attribute__new__stack">
18 - <img class="strip__attribute-icon strip__attribute-icon--duration" src="/bin/download/Main/Images/General/WebHome/icon-reading32.svg"/>
19 - </div>
20 - <label for="" class="strip__label">
21 - <span>226</span>
22 - Available Options
17 + <a href="../../docs/microlearning/intermediate-data-pipelines-understanding-data-pipelines" class="strip__link">
18 + <label for="" class="strip__label">
19 + <span>208</span>
20 + Understanding Data Pipelines
23 23   </label>
24 24   <div class="strip__attribute">
25 25   <img class="strip__attribute-icon strip__attribute-icon--duration" src="/bin/download/Main/Images/General/WebHome/icon-duration32.svg"/>
26 - <div class="strip__attribute-label">4 minutes</div>
24 + <div class="strip__attribute-label">5 minutes</div>
27 27   </div>
28 28   <div class="strip__attribute">
29 29   <img class="strip__attribute-icon strip__attribute-icon--roles" src="/bin/download/Main/Images/General/WebHome/icon-roles32.svg"/>
... ... @@ -31,18 +31,15 @@
31 31   </div>
32 32   </a>
33 33   </li>
34 - <li class="strip">
35 - <a href="/bin/view/Main/eMagiz%20Academy/Microlearnings/Intermediate%20Level/Database%20connectivity/intermediate-database-connectivity-driver" target="_blank" class="strip__link">
36 - <div class="strip__attribute__new__stack">
37 - <img class="strip__attribute-icon strip__attribute-icon--duration" src="/bin/download/Main/Images/General/WebHome/icon-reading32.svg"/>
38 - </div>
39 - <label for="" class="strip__label">
40 - <span>226</span>
41 - Driver
32 + <li class="strip">
33 + <a href="../../docs/microlearning/intermediate-data-pipelines-job-dashboard-for-data-pipeline" class="strip__link">
34 + <label for="" class="strip__label">
35 + <span>208</span>
36 + Data pipeline - Job Dashboard
42 42   </label>
43 43   <div class="strip__attribute">
44 44   <img class="strip__attribute-icon strip__attribute-icon--duration" src="/bin/download/Main/Images/General/WebHome/icon-duration32.svg"/>
45 - <div class="strip__attribute-label">4 minutes</div>
40 + <div class="strip__attribute-label">5 minutes</div>
46 46   </div>
47 47   <div class="strip__attribute">
48 48   <img class="strip__attribute-icon strip__attribute-icon--roles" src="/bin/download/Main/Images/General/WebHome/icon-roles32.svg"/>
... ... @@ -50,37 +50,15 @@
50 50   </div>
51 51   </a>
52 52   </li>
53 - <li class="strip">
54 - <a href="/bin/view/Main/eMagiz%20Academy/Microlearnings/Intermediate%20Level/Database%20connectivity/intermediate-database-connectivity-sql-query" target="_blank" class="strip__link">
55 - <div class="strip__attribute__new__stack">
56 - <img class="strip__attribute-icon strip__attribute-icon--duration" src="/bin/download/Main/Images/General/WebHome/icon-reading32.svg"/>
57 - </div>
58 - <label for="" class="strip__label">
59 - <span>226</span>
60 - SQL Query
61 - </label>
62 - <div class="strip__attribute">
63 - <img class="strip__attribute-icon strip__attribute-icon--duration" src="/bin/download/Main/Images/General/WebHome/icon-duration32.svg"/>
64 - <div class="strip__attribute-label">6 minutes</div>
65 - </div>
66 - <div class="strip__attribute">
67 - <img class="strip__attribute-icon strip__attribute-icon--roles" src="/bin/download/Main/Images/General/WebHome/icon-roles32.svg"/>
68 - <div class="strip__attribute-label">Developer</div>
69 - </div>
70 - </a>
71 - </li>
72 72   <li class="strip">
73 - <a href="/bin/view/Main/eMagiz%20Academy/Microlearnings/Intermediate%20Level/Database%20connectivity/intermediate-database-connectivity-update-database-after-retrieval" target="_blank" class="strip__link">
74 - <div class="strip__attribute__new__stack">
75 - <img class="strip__attribute-icon strip__attribute-icon--duration" src="/bin/download/Main/Images/General/WebHome/icon-reading32.svg"/>
76 - </div>
77 - <label for="" class="strip__label">
78 - <span>226</span>
79 - Update database after retrieval
49 + <a href="../../docs/microlearning/intermediate-data-pipelines-datapipeline-mendix-to-aws-redshift" class="strip__link">
50 + <label for="" class="strip__label">
51 + <span>208</span>
52 + Data pipeline - Mendix to AWS Redshift
80 80   </label>
81 81   <div class="strip__attribute">
82 82   <img class="strip__attribute-icon strip__attribute-icon--duration" src="/bin/download/Main/Images/General/WebHome/icon-duration32.svg"/>
83 - <div class="strip__attribute-label">4 minutes</div>
56 + <div class="strip__attribute-label">10 minutes</div>
84 84   </div>
85 85   <div class="strip__attribute">
86 86   <img class="strip__attribute-icon strip__attribute-icon--roles" src="/bin/download/Main/Images/General/WebHome/icon-roles32.svg"/>
... ... @@ -89,17 +89,14 @@
89 89   </a>
90 90   </li>
91 91   <li class="strip">
92 - <a href="/bin/view/Main/eMagiz%20Academy/Microlearnings/Intermediate%20Level/Database%20connectivity/intermediate-database-connectivity-database-operations" target="_blank" class="strip__link">
93 - <div class="strip__attribute__new__stack">
94 - <img class="strip__attribute-icon strip__attribute-icon--duration" src="/bin/download/Main/Images/General/WebHome/icon-exercises32.svg"/>
95 - </div>
96 - <label for="" class="strip__label">
97 - <span>226</span>
98 - Database operations
65 + <a href="../../docs/microlearning/intermediate-data-pipelines-datapipeline-mendix-to-sftp" class="strip__link">
66 + <label for="" class="strip__label">
67 + <span>208</span>
68 + Data pipeline - Mendix to SFTP
99 99   </label>
100 100   <div class="strip__attribute">
101 101   <img class="strip__attribute-icon strip__attribute-icon--duration" src="/bin/download/Main/Images/General/WebHome/icon-duration32.svg"/>
102 - <div class="strip__attribute-label">6 minutes</div>
72 + <div class="strip__attribute-label">7 minutes</div>
103 103   </div>
104 104   <div class="strip__attribute">
105 105   <img class="strip__attribute-icon strip__attribute-icon--roles" src="/bin/download/Main/Images/General/WebHome/icon-roles32.svg"/>
... ... @@ -107,7 +107,7 @@
107 107   </div>
108 108   </a>
109 109   </li>
110 - </ul>
80 + </ul>
111 111   </main>
112 112   </div>
113 113  </div>