Skip to Content
Last modified by Erik Bakker on 2024/02/21 21:51
From version 3.1
edited by Erik Bakker
on 2022/04/29 15:21
Change comment: There is no comment for this version
To version 24.1
edited by Erik Bakker
on 2022/05/02 11:44
Change comment: There is no comment for this version

Summary

Details

Page properties
Title
... ... @@ -1,1 +1,1 @@
1 -REST - Request & Response
1 +Mendix -> Redshift - Data Pipeline
Content
... ... @@ -1,34 +1,41 @@
1 1  {{container}}
2 2  {{container layoutStyle="columns"}}
3 -(((**~1. How to use**
4 -With this store component, you can easily set up a connection to a REST endpoint using various authentication mechanisms while taking the response into account.
3 +(((
4 +== About ==
5 +Data pipeline configuration inside an entry connector to transfer Mendix App object data to an Amazon Redshift table. This flow can also be used as starting point for other data pipeline integrations.
5 5  
6 -**2. Keynotes & restrictions **
7 +== Documentation ==
7 7  
8 -a. Note that you need to select the valid authentication based on the requirement of the external system.
9 +==== 1. How to use====
10 +With the help of this store item, you have a data pipeline configuration inside an entry connector to transfer Mendix App object data to an Amazon Redshift table. This allows for the following:
11 +
12 +* Reading data from Mendix using OData v3.
13 +* Writing data to Amazon Redshift using Amazon S3 ([[https:~~/~~/docs.aws.amazon.com/redshift/latest/dg/t_loading-tables-from-s3.html>>url:https://docs.aws.amazon.com/redshift/latest/dg/t_loading-tables-from-s3.html]]).
14 +* Merge behavior using staging tables (documentation).
15 +* Job dashboard (See Deploy -> Runtime Dashboard -> [Select Runtime] - > [Select Flow] -> Jobs ).
16 +* Refresh an AWS Materialized View a minute after the job completes successfully.
9 9  
10 -**3. License Information**
18 +==== 2. Keynotes & restrictions====
19 +
20 +* Before use, please rename job 'system.message.job' and step 'system.message.step1' to match your system and integration name. Make sure to update support.job-launch-request accordingly.
21 +* When using multiple pipeline flows on one runtime, an h2 'already in use' exception might occur. To prevent these errors please keep the name of the h2 database different for each flow on one runtime by renaming the 'dp.h2.message.database' deploy property in the "support.h2-database" component.
22 +* When you want to import the flow multiple times for one system, you should change the names of the following properties by replacing 'message' with the name of the message type of each integration:
23 + ** dp.odata.message.cron (component: receive.cron)
24 + ** dp.odata.message.url (component: system.message.job -> system.message.step1 -> Item reader)
25 + ** dp.h2.message.database (component: support.h2-database)
26 + ** dp.sftp.message.filename.prefix (component: transform.job-launch-request - > Expression)
27 +
28 +==== 3. License Information====
11 11  Part of the standard license agreement as agreed upon when using the store for the first time.
12 12  
13 -**4. Relevant eMagiz Academy Microlearnings**
14 -[[https:~~/~~/emagiz.github.io/docs/microlearning/intermediate-rest-webservice-connectivity-index>>url:https://emagiz.github.io/docs/microlearning/intermediate-rest-webservice-connectivity-index]]
31 +==== 4. Relevant eMagiz Academy Microlearnings====
15 15  
33 +[[https:~~/~~/emagiz.github.io/docs/microlearning/intermediate-data-pipelines-index>>url:https://emagiz.github.io/docs/microlearning/intermediate-data-pipelines-index]]
16 16  
17 -Would you please consult the privacy policy of eMagiz at the following link: [[https:~~/~~/www.emagiz.com/privacy-policy/?>>url:https://www.emagiz.com/privacy-policy/?]])))
35 +//Would you please consult the privacy policy of eMagiz at the following link:// [[https:~~/~~/www.emagiz.com/privacy-policy/?>>url:https://www.emagiz.com/privacy-policy/?]])))
18 18  
19 19  
20 20  
21 -((((% border="2" cellpadding="10" cellspacing="10" style="width:292px" %)
22 -|=(% style="width: 45px;" %)#|=(% style="width: 241px;" %)Option
23 -|(% style="width:45px" %) a|(% style="width:241px" %) OAuth 2.0 - Authorization Code
24 -|(% style="width:45px" %) b|(% style="width:241px" %) REST - Basic Auth
25 -|(% style="width:45px" %) c|(% style="width:241px" %) REST - Client Certificate
26 -|(% style="width:45px" %) d|(% style="width:241px" %) REST - Header Mapper
27 -|(% style="width:45px" %) e|(% style="width:241px" %) OAuth 2.0 - Client Credentials
28 -|(% style="width:45px" %) f|(% style="width:241px" %) REST - SSL
29 -|(% style="width:45px" %) g|(% style="width:241px" %) Dynamic Token
30 -|(% style="width:45px" %) h|(% style="width:241px" %) Token
31 -|(% style="width:45px" %) i|(% style="width:241px" %) No Authentication (Fire & Forget)
32 -)))
39 +((()))
33 33  {{/container}}
34 34  {{/container}}