Product Update - v22.38

Details of the product update v22.38 from 22nd of September 2022.

Improvements

Credentials Page: Search by component name

In our quest to improve every aspect of our User Interface, we added a search function to the Credentials page. When you navigate to Integrate - Credentials you can now Search by component name to find the credential card you need instead of scrolling to find it.

WebHook Step: Send sample request

When you configure a webhook component step in your integration flow, you can either Send sample request or Add sample manually. When you select Send sample request a WebHook address is provided which you can use to send your example sample.

We also highlight that this address to be used only one-time to send a representative sample which you can use to configure subsequent steps in your integration flow.

REST-API-V2 component POST and PUT body type

To improve the user experience and prevent an accidental JSON data structure pasting into the Body field while in an Integrator mode, we changed the User Interface behaviour to default into JSONata mode when an application/json content type is selected for the POST and PUT HTTP calls.

This change avoids possible errors when users forget to change the input field settings deom Integrator to JSONata mode.

Run-time environment variables

Components exchange information with various platform services using run-time environment variables during execution of each step of an integration flow.

We are constantly working to standardise and add new environment variables to streamline the information exchange. The following environment variables were added to the run-time:

WEBHOOK_URI - This environment variable will be available to each step in the flow which exchanges data with the webhook service, eg. WebHook or Flow-linking. This environment variable from the Tenant attributes webhooks_domain or from the HELM3 environment variable WEBHOOKS_BASE_URI if the Tenant record is missing.
ELASTICIO_CONTRACT_ID - To help with troubleshooting of errors in multi-contract cases this environment variable will be available during the run-time of every execution including:
- Verify credentials
- Select model
- Dynamic metadata
- Generated samples and flow execution

Updates

Node.js Sailor version `2.7.0`

A new version of Node.JS Sailor library 2.7.0 is released. This update is preparatory groundwork for the message persistence feature.

Java Sailor version `3.4.0`

A new version of Java Sailor library 3.4.0 is released. This update is preparatory groundwork for the message persistence feature.

HELM3

We constantly improve different aspects of HELM3 deployments in collaboration with our partners. This section lists updates and improvements included in this release.

Iron-bank service Scalability

Iron-bank service collects statistics from the running integration flows. The service is now running as Kubernetes DeamonSet together with the cadvisor. This reduces the load on the system significantly and increases the scalability of Iron Bank.

Externally Build Docker Components

Information in this section is intended for our customers who intend to extend the automatic component delivery to include custom developed components.

API changes

Added new endpoint to PATCH /v2/teams to add docker registry to a team. Check the API documentation for the details.

Components Pusher changes

HELM3 configuration changes

To extend the automatic component delivery to include custom developed components, we modified the componentsConfiguration section of HELM3 charts in the following way:

componentsConfiguration:
  developmentTeam: teamName
  components:
    componentName:
      # Image tag for pulling into the internal docker registry. Required, can be "latest"
      targetTag: latest
      # Image name by which to pull repo from registry. Optional, `team_name/repo_name` by default
      dockerRepoName: "orgName/repoName"
      # Team where create RepoBuild. Optional. Will override root `developmentTeam` on the `components` level
      developmentTeam: teamName
      # Access for the component in the platform context. Optional, `global` by default
      access: team

Please Note: These are breaking changes and you should carefully revise your setup if you intend to use this feature.

developmentTeam - unchanged
componentName - unchanged
registry, url, repo and tag got changed into dockerRepoName, developmentTeam, targetTag and access. Please review these changes carefully.

Along with these changes the following new required variables were introduced:

DOCKER_REGISTRIES_ENCRYPTION_KEY used to encrypt credentials for docker registries
COMPONENTS_PUSHER_AUTH_USERNAME used to authenticate to components-pusher server
COMPONENTS_PUSHER_AUTH_PASSWORD used to authenticate to components-pusher server

The components-pusher expects to find these variables to deploy the component docker images into your default team.

The new approach expects to also find a new optional TEAM_DOCKER_REGISTRY environment variable in case of custom developed components. It is defined in the following way:

TEAM_DOCKER_REGISTRY:
   uri: "https://index.docker.io/v1/"
   credentials:
       username: ""
       password: ""

For the uri system expects string containg the address of docker registry. The credentials object can have the following properties:

authFile - path of the authentication file
registryToken - bearer token for accessing the source registry
username and password

If none of the above are used - components-pusher will try to access the registry anonymously.

Behavior Changes

The components-pusher is not longer a cronjob but a fully integrated service of the platform. It runs in Kubernetes cluster in a single instance (replicas non-configurable).

The service runs following way:

Regularly checks for new builds for repositories with dockerTargetTag fields in the repo documents.
When new builds of repository are detected they are copied into internal docker registry.
The frequency of checks is configurable with the .Values.componentsPusher.processLoopInterval parameter of the component-pusher service. The frequency of checks is set in millisecond(ms). The default is 86400000 ms (1 day).
The component-pusher uses the newly added internal endpoint (/copy/:component_id?version=:version) to pull of the specified component and version from docker registry. Uses basic authentication with COMPONENTS_PUSHER_AUTH_USERNAME and COMPONENTS_PUSHER_AUTH_PASSWORD values defined in the HELM3 charts.

Gendry changes

createTeam task now creates/updates default team with docker registry data from the TEAM_DOCKER_REGISTRY environment variable
New createDefaultRepos task creates/updates components listed in componentsConfiguration. Teams, in which these repositories are created, require to have docker registry data associated with them. If default team used - docker registry data will be taken from TEAM_DOCKER_REGISTRY variable. If any other team used - it needs to be patched with correct docker registry data using API endpoint PATCH /v2/teams.

Fixed bugs

Fixed the User Interface bug where the execution logs could not be copied.
Fixed unexpected behavior of the Time Picker when filtering in the Threads and Execution pages.
Addressed the limitation when credentials fields (oauth and additional fields, baseUrl, etc ) could not be retrieved in the OAuthFieldView. Now all fields can be accessed directly from the cfg object.

Components

Microsoft Dynamics CRM V2 component `1.0.0`

Introducing new component for Microsoft Dynamics CRM.

ADDED Make Raw Request action

Marketo component `2.2.5`

FIXED the bug when non required field would show as required and block configuration of the step.

Shopware 6 component `1.1.0`

ADDED new authentication method using CLIENT_ID and CLIENT_SECRET credentials
ADDED Lookup Object (at most one) action
ADDED Lookup Objects (plural) action
ADDED Upsert Object action
ADDED Delete Object By ID action
ADDED Get New and Updated Objects Polling trigger

HJSON component `1.0.3`

UPDATED the Sailor library version to 2.6.29
UPDATED the component-commons library version to 3.0.2
FIXED the vulnerabilities in dependencies

Zendesk component `1.0.2`

UPDATED the Sailor library version to 2.6.29
UPDATED the component-commons library version to 3.0.2
FIXED the vulnerabilities in dependencies

VTEX component `1.4.3`

FIXED output metadata for the Lookup Object (at Most 1) action for Object Type - Orders
FIXED incorrect behavior of the Allow ID to be omitted option in the Lookup Object (at Most 1) action
UPDATED the Sailor library version to 2.6.29

Shopify component `1.6.5`

ADDED retry on 429 error (respecting the Retry-After header)

Hubspot component `1.5.3`

FIXED the Verify credentials