207 - Aligned State

Hi there, eMagiz developers! We have done much work for state generation, so that you can derive more information from your data! Examples are enrichment, aggregation, change detection and duplicate detection. Next to state generation, several improvements regarding the alignment of components have been made. Moreover, performance improvements and bug fixes have been implemented. Lastly, a change in the metrics storage duration has been performed.

State Generation**

New components
The following components have been added:

• Aggregator: reverse of the splitter, used to combine multiple messages into a single one.
• Infinispan metadata outbound channel adapter: used to (temporarily) store metadata, which can be used to enrich messages, or make decisions in filters or scripts.
• Content enricher: used to add properties to the payload of messages, similar to what the header enricher can do for headers.
• Duplicate detector: support object used to detect duplicate messages for a certain key on an inbound channel of a flow component. Duplicates can be marked or discarded.
• Infinispan support objects, supporting the aggregator, metadata outbound channel adapter and content enricher.

SpEL functions & JSON
Added SpEL functions for:

• Encoding & decoding Base64, Hex and hmac.
• Date retrieval (currentDateTime, currentEpochMilli, currentEpochSecond).
• Date conversion (DateTime to Epoch, Epoch to DateTime, DateTime formatter and DateTime parser).

Added support for SpEL property accessors for XML and JSON, meaning that you can read message payloads in SpEL expressions easily. For example, to retrieve the value for 'id' in the following JSON String: `{"id":"123"}`, this SpEL expression suffices: `payload.id`.

Added support for `#JsonPath` usage in SpEL expressions. With this functionality, you can access your JSON payloads in a similar way as you would access XML payloads using xPath.

Metrics Storage Duration

We will change the retention of monitoring data of different environments. For minute level data, it will be kept for test, acceptance and production environments for 7, 14 and 30 days respectively. Then the minute level data will be reduced to hour level data then will be stored for test, acceptance and production for 3, 6 and 12 months respectively.

Feedback Items

Import from store
The button 'Import from Store' in Design will directly open the store instead of showing a pop up asking whether the store should be opened.

Save and cancel buttons placement
The save and cancel buttons have been placed slightly further apart from each other to avoid misclicking.

User session times
The user session times for next generation architecture dashboards have been extended, so that the user is not thrown out of the model when using the Manage phase for longer than an hour.

Cancel and next buttons order
The order of the cancel and next buttons when creating a new trigger has been changed to increase alignment across the platform. Now, all cancel buttons are placed on the right of a 'next' or 'save' button.

Data sink page
The data sink page has been moved to the “Explore” tab.

Unused images removal
When a release is removed, the related unused images in the on-premises machines will be removed as well.

Flow Designer performance
Performance improvements for the Flow Designer have been implemented.

Properties with special characters
When a user would migrate to the next generation architecture and deploy their release containing properties with special characters, certain flows could not find the correct values to properties anymore or could not find the properties altogether. With this release, special characters such as a backslash do not break properties after deployment.

Hidden flow fragements
When importing items from the store, flow fragments that are hidden in design will be listed under the versions in create.

Resources disappearances
Resources would disappear in the flow designer when pressing Cancel on selecting a new resource. With this release, resources without a name will not trigger error messages when users select components.

Error channel for all inbounds
If there is no custom error handling, next generation migration would set the error channel to “errorChannel” only for the first inbound in an entry flow. We added a migration step, such that the error channel is set to “errorChannel” of all inbounds in a flow if the custom error handling is set to false.

Highlighting resources
The performance of highlighting resources of selected components is improved in Read only mode.

Bug Fixes

Topic and event-processor names
The styling of the event streaming canvas in the Create and Deploy phases has been altered slightly to make topic and event-processor names more readable.

System alignment
The positioning of Systems across Design, Create and Deploy phases are now synchronized to the position of their respective System in the Capture phase. So, systems are aligned regarding positioning across all phases. This will be done for existing systems as well.

Error alerting
The error trigger is now configured to check if an error message contains a certain term, besides checking the error header.

Alerting
For the next generation architecture models, several adaptations to triggers have been made.

• For the next generation architecture models, you will no longer receive alert messages for every error message, every log message of 'level' ERROR, and for memory usage above 80%.
• For the next generation architecture models with event streaming, the threshold to receive the 'Topic approaching maximum size' alert has been increased from 95% of configured size used to 110%.
• The next generation architecture configurable trigger for error messages has been extended with the option to check whether the 'last-exception' message contained a certain text. This can be combined with the header-value match.

Unused containers
Containers are set inactive when they are unused and removed in Design.

Error popups when promoting a release version
We fixed an issue that a lot of error popups were shown when you promoted a release version to a next environment.

Fancy Forum Answers

As always, this is a gentle reminder to ask questions via the Q&A forum. The Q&A forum is available in the eMagiz iPaaS portal, so we can all benefit from the knowledge within the community. For some inspiration, take a look at these forum answers:

• Message Redelivery blank error message
• Timeout error kafka batch creation
• SFTP Connectivity: Transfering a whole folder structure
• Next generation architecture with dynamic file pickup / filedrop

Key Takeaways

Thanks to all who helped build and those who tested and gave feedback to improve the product. To end this post, here are some key takeaways for you:

• If you have questions surrounding our Program Increment Planning, please get in touch with productmanagement@emagiz.com
• If you have feedback or ideas for us, talk to the Platypus
• Please inform us of new additions to the store (productmanagement@emagiz.com) so we can all benefit from these.
• Clear your browser cache (Ctrl + Shift + Del)
• Check out the release notes here
• Start thinking about how the license tracker can aid your development
• Start thinking about major, minor, and patch
• Upgrade to the latest build number
• Keep making great integrations

Let's stay in touch and till next time!