Skip to Content
Last modified by Erik Bakker on 2024/02/21 21:51
From version 13.1
edited by eMagiz
on 2022/06/09 12:34
Change comment: Renamed from xwiki:Migrated Pages.Understanding Data Pipelines
To version 14.1
edited by eMagiz
on 2022/06/09 12:35
Change comment: There is no comment for this version

Summary

Details

Page properties
Title
... ... @@ -1,0 +1,1 @@
1 +Understanding Data Pipelines
Parent
... ... @@ -1,0 +1,1 @@
1 +WebHome
Author
... ... @@ -1,1 +1,1 @@
1 -XWiki.marijn
1 +XWiki.eMagiz
Content
... ... @@ -1,13 +1,7 @@
1 -{{html wiki="true"}}
2 -<div class="ez-academy">
3 - <div class="ez-academy_body">
1 +{{container}}
2 +{{container layoutStyle="columns"}}
3 +(((
4 4  
5 -<div class="doc">
6 -
7 -
8 -
9 -= Understanding Data Pipelines =
10 -
11 11  In this microlearning, we will introduce the data pipeline pattern to you. After this microlearning, you should be able to identify situations in which a data pipeline solution is the best choice for you.
12 12  With the help of such a data pipeline, you can transfer large volumes of data between a source and a sink.
13 13  
... ... @@ -45,7 +45,7 @@
45 45  
46 46  In the flow designer of eMagiz a data pipeline looks similar to the picture shown below:
47 47  
48 -<p align="center">[[image:intermediate-datapipelines-understanding-data-pipelines--data-pipeline-in-emagiz.png||]]</p>
42 +[[image:Main.Images.Microlearning.WebHome@intermediate-datapipelines-understanding-data-pipelines--data-pipeline-in-emagiz.png]]
49 49  
50 50  === 3.2 Source Systems ===
51 51  
... ... @@ -62,9 +62,7 @@
62 62  * Azure Event Hub
63 63  * Database
64 64  * Remote directory (FTP or SFTP) in flat-file (CSV) format
65 -
66 -===== Practice =====
67 -
59 +
68 68  == 4. Assignment ==
69 69  
70 70  Browse the public store of eMagiz to see which combinations of source and sink systems are used frequently and are therefore standardized.
... ... @@ -86,11 +86,10 @@
86 86  
87 87  This video demonstrates a working solution and how you can validate whether the refresh has worked in AWS Redshift.
88 88  
89 -<iframe width="1280" height="720" src="../../vid/microlearning/intermediate-datapipelines-understanding-data-pipelines.mp4" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
90 90  
91 -</div>
82 +{{video attachment="intermediate-datapipelines-understanding-data-pipelines.mp4" reference="Main.Videos.Microlearning.WebHome"/}}
92 92  
93 -</div>
94 -</div>
84 + )))
95 95  
96 -{{/html}}
86 +((({{toc/}}))){{/container}}
87 +{{/container}}