Released: 2022-04-11
We have improved the sample data retrieval to provide a more unified user experience during flow design. The look and feel is now more in line with our new approach.

The above images show the new Sample data retrieval view. This release
adds new functionality, allowing users to skip sample generation by clicking on
the Skip Sample button. When this function is used the platform will add an
empty JSON {} instead of the Sample data.
When you retrieve the Sample the platform UI will show it only in JSON format. In we removed the Integrator view of Data Sample.

Here you can view the Sample, collapse different sections of the JSON data structure and edit it.
To show the RAM quota overuse we improved the quota Usage History histogram. If you go over the set RAM quota value the overused part will show in red.

To support the newest SSH versions we deprecated the old ssh-dss signature and
we added ssh-ed25519 instead. Now when you upload your SSH key to our platform
you will see the information about the supported types of signatures.

Please Note the old ssh-rsa keys are still supported. If you want to use
them you have two options:
8.8+ client and generate your SSH Key using the -oPubkeyAcceptedKeyTypes=+ssh-rsa parameter. It will generate an RSA+SHA-2 type SSH key which provides a high level of security and works with the OpenSSH 8.8+ clients.<=8.7 client to generate your SSH Key.The platform includes new capability to hard-delete the flows after an expiration time set by the platform management. Every time you try to delete the flow a warning message will show with the following text:
Are you sure you want to delete “FLOW NAME” flow permanently? Deleting this flow is an irreversible process, we cannot undo a deletion if you complete the process by accident. Please note that the credentials linked with the flow will not be deleted.
We constantly improve different aspects of HELM3 deployments in collaboration with our partners. This section lists updates and improvements done in this round.
cadvisor and fluentd demons (v22.14)To enable better coexistence of platform microservices within the multi-service
Kubernetes clusters, we modified the logic used to run cadvisor and fluentd services.
The platform will run these services when the node has uniquely defined selectors.
To achieve this we introduced a new HELM3 configuration for the node labels for the Kubernetes pod allocation.
global:
nodeSelectors:
platform: "platform"
longRunningTask: "longrunning"
ordinaryTask: "ordinary"
For these changes to take affect we deprecate the following HELM3 configurations:
KUBERNETES_ORDINARY_LABEL_VALUEKUBERNETES_LONG_RUNNING_LABEL_VALUEThese variables ware removed from the secrets HELM3 chart and are no longer used
by the admiral service starting from release. Please use
Values.global.nodeSelectors platform HELM3 chart configuration instead.
cadvisor service will run on the nodes with longRunningTask and ordinaryTask labels.fluentd service will run on the nodes with platform, longRunningTask and ordinaryTask labels.To extend the Ingress setup capability we made changes in the Handmaiden microservice
to provide 2 new ways to provide annotations to the Ingress records. With this new
capability you can set unique Ingress values per each tenant in your cluster.
You can set the Ingress values directly in your HELM3 chart like:
global:
services:
handmaiden:
annotations:
kubernetes.io/ingress.class: 'nginx'
testAnnotation: 'testValue'
If you set the above configuration the platform passes it to the new environment
variable INGRESS_OPERATOR_ANNOTATIONS used by the Handmaiden service. This adds
Ingress annotations to all tenants in your multi-tenant cluster.
You can set unique Ingress annotations for each tenant using the new configuration while creating the tenant via an API call. A typical call body could look like:
{
"data": {
"type": "tenant",
"attributes": {
"ingress_config" : {
"annotations": {
"annotationName": "value"
}
}
}
}
}
Our platform will serves these annotations based on priority of annotation the configurations (from low to high):
Introducing a new cron job called clear-deleted-flows which permanently deletes
flows from the MongoDB with the DELETED status. Every user trying to delete a
flow will see a warning message informing that this is a
permanent and irreversible action.
We introduce new environment variables which you can use to control the
clear-deleted-flows job:
WIPER_CLEAR_DELETED_FLOWS_AGE_SECONDS - Time in seconds the job must wait before deleting the flow permanently after it is marked as DELETED in MongoDB. We set the default value to 86400 seconds (1 day).WIPER_CLEAR_DELETED_FLOWS_LIMIT - Maximum number of flows the clear-deleted-flows job will permanently delete each time it runs.This release process applies if you already have a cluster running previous versions of the platform code. The release will start an automatic migration.
WIPER_CLEAR_DELETED_FLOWS_LIMIT environment variable to an applicable number for your case to perform this operation faster of you have large number of flows in your DB.After the migration the
marathonevents,requestbinsandtaskstartlogsdeprecated collection will be removed.
New Environment Variables:
IRON_BANK_CLICKHOUSE_NODES (required) - array of objects with host (required), port, user and password valuesIRON_BANK_CLICKHOUSE_DATABASE (optional) - ClickHouse database name (default iron_bank)IRON_BANK_CLICKHOUSE_NODES:
- host: "10.0.0.1"
port: "8123"
user: "default"
password: "123"
- host: "10.0.0.2"
port: "8123"
user: "user"
password: "1234"
IRON_BANK_CLICKHOUSE_DATABASE: "iron_bank"
New Iron Bank provision logic
IRON_BANK_CLICKHOUSE_NODES are not available, an error is thrown and iron bank shuts downIRON_BANK_CLICKHOUSE_NODESRemoved Environment Variables:
IRON_BANK_CLICKHOUSE_URI
500 status code in case when the /refresh/[id] Oauth server would return an error. Now the service reports 4xx indicating the third party problem.401 in case of API-Key regeneration.1.5.1 (v22.14)Get New and Updated Objects Polling preventing it from extracting more than 10K records.2.6.27 version.Circle.ci configuration.2.2.2 (v22.14)2.6.27 version.Circle.ci configuration.1.3.0 (v22.14)Lookup Object (at Most One) action1.1.0 and 1.2.0 (v22.14)Get New and Updated Objects Polling trigger,Lookup Object (at Most One) action,Make a Payment action.1.2.9 (v22.14)3.3.9 version.1.4.6 (v22.14)2.5.2 (v22.14)3.3.9 version.1.0.4 (v22.14)3.3.9 version.Released: 2022-04-21
If you are one of our many power users you might have noticed from time to time a pop-up warning with a message Not enough quota to start task. These messages were not related to your RAM quota limit. Instead these messages are triggered by real-time flows with more than 25 steps. Because the platform initiates each step sequentially, these errors are triggered when all flow steps cannot be initiated within the allowed time.
We have resolved this issue by ensuring that all steps (Kubernetes pods) are initiated in parallel, thus significantly shortening the time required to start the flow. We also changed the warning message to The flow FLOW NAME (FLOW ID) is taking longer than expected to transition.

News in this section are for our customers who use OEM version of the elastic.io platform.
Tenant administration can now enforce the 2FA during the login. We introduce a
tenant configuration flag called forced_totp to switch this feature on.
forced_totp you must enable the enabled_totp as well.enabled_totp feature flag, system will remove forced_totp as well.You can set these parameters either during the tenant creation or you can update the parameters of an existing tenant via an API call. An example of such an API call is shown below.
{
"data": {
"type": "tenant",
"attributes":{
"feature_flags": {
"enabled_totp" : true,
"forced_totp" : true
}
}
}
}
From version, the platform checks status of the forced_totp
parameter during your navigation of every page in the UI. If tenant administration
enforced the 2FA and you have not enabled the 2FA for your account then you will
be redirected to your profile page and presented a pop-up form to enable the 2FA.
1.5.2 (v22.16)2.2.3 (v22.16)1.4.7 (v22.16)1.0.1 (v22.16)2.6.27Circle.ci configuration.component-commons-library version to 2.0.21.0.14 (v22.16)2.6.27Circle.ci configuration.3.1.2 (v22.16)component-commons-library to read and upload attachments through the Maester service2.6.271.2.3 (v22.16)2.6.27Circle.ci configuration.Released: 2022-05-05
We constantly improve different aspects of HELM3 deployments in collaboration with our partners. This section lists updates and improvements done in this round.
You can now configure the number of failed login attempts allowed before users are locked out of the system.
We added a new variable MAX_LOGIN_ATTEMPTS to the HELM3 secrets chart. You may set this variable according to your own requirements. The default value is set to five (5). Here is how we calculate this number:
MAX_LOGIN_ATTEMPTS then your user will be locked out of the system. To unlock such users the tenant administration must first disable 2FA for this user and instruct users to navigate /forgot address of the tenant to reset their password. This will reset the counter and user can login again.Information in this section is intended for our customers who use OEM version of the elastic.io platform.
The platform builds a component docker image every time you push a new version of your component. When you remove the component or a particular version of it from the platform, an associated docker build will remain until you would remove it at later stage directly from the docker registry.
With this update we improved the situation in the following way:
remove-deleted-repos-from-docker) to help cleanup the remnant builds. Contact us to get a copy of this tool.3.1.3 (v22.18.1)Emit Batch behavior2.0.1 (v22.18.1)Raw Request actionUpsert Object actionWebhook triggerCreate Object and Update Object actions removed in favor the new Upsert Object action.1.3.0 (v22.18.1)Webhook triggerCreate Object actionUpdate Object actionMake a Payment action1.4.0 (v22.18.1)Place Order action1.0.3 (v22.18.1)Ansprache field for Add new Subscriber actionAs a part of our on-going improvements of integration components, we
2.6.27Circle.ci configuration.Here is the list of components for release.
1.2.81.0.111.0.01.1.61.4.11.2.111.0.01.1.31.2.19 and 2.0.12. Both deprecated and V2 were updated.2.0.11.0.3Released: 2022-05-19
While working on the platform interface it is possible that users can accidentally change or delete important or sensitive information.
For such cases we introduce re-authentication for sensitive actions. After for the following actions you will be asked to enter your password to authenticate:
Please Note: You will be asked to re-authenticate once every 6 hours.
To prevent erroneous and bot registrations we introduce email verification step to our trial registration process. When you register first time with our platform, an email will be sent to your given email address. Click on the link in your received email to finish your registration. There are certain rules and restrictions to this process as well:
Please Note: If you are invited to the platform directly, no email verification will be required.
We constantly improve different aspects of HELM3 deployments in collaboration with our partners. This section lists updates and improvements included in this release.
When your integration flow needs to transfer big objects or attachments, one of the platform microservices called Maester temporarily stores them in MongoDB while passing only the object IDs through the integration flow steps.
Starting from , you can use a dedicated database for storing Maester
objects and the run-time attachments. To configure, add MAESTER_MONGO_URI
environment variable to HELM3 secrets charts and set it to target this new database.
Note: You will need to migrate the grids to new MongoDB as well to ensure the Maester object availability after implementation.
Organisations who do not need a separate MongoDB instance to store their Maester
objects should set the MAESTER_MONGO_URI variable to the same value as the
MONGO_URI variable. This will save objects in your main database.
To configure the re-authentication for sensitive actions:
SENSITIVE_ACTION_AUTH_LIFETIME environment variable to HELM3 secrets chart and set it to specify how long user authentications are valid. The system expect the value of this parameter in milliseconds. The default is set to 21600000 (6 hours).tenant.featureFlags.sensitiveActionsReauth feature flag (set it to true).Information in this section is intended for our customers who use OEM version of the elastic.io platform.
Developing integrations requires a certain level of access to business systems. In cases where the integrators are not responsible for these systems it is critical that only secure access is allowed and that system admins can control credentials used To solve the dilemma we introduce White-labeled credential management feature.
As an integrator you provide a unique URL to the credential holder to enter their access credentials without registering or logging-in to the system. The access holder is presented an interface to enter their credentials, verify them and save.
In addition they can modify the existing credentials as well.
For both cases they are presented with a simple screen to work with only the specified credentials.
There are certain preconditions before you can enable this feature:
autoCreateUsers set to false.Once the above conditions are met you can take the following steps to generate the URL to request credentials from their owner:
The initial URL has a form /embedded-credentials/repoId where the repoId
would be the ID of component for which you need to set the credential. Then you
would need to add the following query parameters to this URL:
workspaceId (required) - this is the ID of the workspace where integration is set.ssoProviderType (optional) - values can be SAMLProvider or OIDCProvider.ssoProviderId (optional) - this is the ID of your SSO provider.Let us construct the URL:
TENANT_URL/embedded-credentials/[repoId]?workspaceId=[workspaceId]&ssoProviderType=[ssoProviderType]&ssoProviderId=[ssoProviderId]
If you do not specify ssoProviderType and ssoProviderId then they will be
automatically taken from the tenant.loginRedirectSsoProvider property but only if this is defined.
To set the email verification process
in your tenant the following must be done using an API call to update your tenant records using tenantAdmin privileges:
attributes.hide_register parameter to false.signin_v_2 type - add the attributes.signin_v_2 attribute containing the URL of the provider logo using the attributes.signin_v_2.logo_url tenant parameter.email_verification feature flag (NEW) - set attributes.feature_flags.email_verification to true.Cannot add property token, object is not extensible appears in the platform user interface when trying to evaluate JSONata expressions with "$count()".1.3.0 (v22.20)Receive Instant Payment Notification triggerReceive Payment Data Transfer triggerWebhook triggerCreate Object actionUpdate Object actionMake a Payment action1.0.5 (v22.20)component-commons-library to read and upload attachments through the Maester2.6.271.6.0 (v22.20)HMAC signature for the Webhook subscription trigger1.0.6 (v22.20)2.5.4 (v22.20)1.4.1 (v22.20)Place Order actionCircle.ci configuration.3.1.4 (v22.20)Emit Batch behavior for the Read CSV attachment actionAs a part of our on-going improvements of integration components, we
2.6.27Circle.ci configuration.Here is the list of components for release.
2.2.41.4.31.0.11.3.21.1.32.0.21.3.92.2.2 and 2.2.31.1.62.0.31.4.11.6.61.1.21.0.81.5.92.2.11.1.6 a.k.a Lookup Table Component1.1.51.3.51.0.1Released: 2022-06-02
We are ISO 27001 certified! Check our dedicated page for more details and to download the copy of our certificate.
As part of ongoing improvements we have refined our system architecture around executions to decrease latency when loading the executions page. As part of these changes there is a temporary duplication of the number of executions recorded in your executions page. No data is lost, there is no influence on quota consumption, and these duplications will disappear as we end the transition phase in approximately 30 days. If you have any questions please contact our support directly.
We are changing policies for our retention services. As platform traffic grows the resource burden of retaining files becomes significant.
lightweight objects) retention period is reduced to 3 days (72 hours)To provide more vertical space during the flow design process we changed the flow designer header behaviour. Now when the left-hand-side menu is closed the action buttons and statuses merge into the same line with flow title and description.
When your flow title or description is larger than the available space the visible text is cropped. By hovering over these fields you can reveal the full text.
Reusing mappings or code snippets from published flows can accelerate flow development. You can now select and copy such text from your published flows without first making a draft of the flow. This feature includes:
Example shows copied code from the Node.js code component configuration view:
Now you can see the IDs of any credentials while browsing through the list of credentials in the Integrate > Credentials > Component Name view:
Or while configuring a step in your integration flow:
We now obfuscate authentication header values for the REST-API-V2 component. You
will notice this in the Configuration view when basic or api-key authentication is used.
This is part 2 of our initiative to prevent accidental or malicious changes to sensitive data and follows on from release 22.20.
This time we extend the list of actions for which you will be asked to enter your password to authenticate:
Again, please note: You will be asked to re-authenticate once every 6 hours.
We constantly improve different aspects of HELM3 deployments in collaboration with our partners. This section lists updates and improvements included in this release.
We introduce a new variable (MAESTER_MAX_SIZE_PER_OBJECT in bytes) you can use to control
the maximum object size accepted by the Maester service. The default maximum value
is set 1GB. The recommended range is from 100MB to 1GB.
Information in this section is intended for our customers who use OEM version of the elastic.io platform.
If your tenant has an an open registration page then you can block registration
from any domain by adding it into the domain blacklist. For this to work in your
tenant, make an API call
to update your tenant records using tenantAdmin privileges:
attributes.hide_register parameter to false.attributes.email_domains_blacklist record to contain comma separated domain names in an array object like ["example.com", "foo.bar"].During the 20.20 platform release we introduced email verification process which worked only when the signin_v_2 were used.
Now you can use the email verification process even if you are using a regular
signup page in your tenant. To enable this feature in your tenant the following
steps must be done using an API call
to update your tenant records using tenantAdmin privileges:
attributes.hide_register parameter to false.email_verification feature flag - set attributes.feature_flags.email_verification to true.[Deleted User] instead."Cannot perform 'get' on a proxy that has been revoked" error was shown after evaluation of the JSONata mapping expression.1.0.2 (v22.22)2.1.0 (v22.22)Write Entire Bucket Contents actionRead Entire Bucket Contents action1.4.2 (v22.22)Network Diagnostics action1.4.0 (v22.22)Lookup Flow Hydrates Component Version Info- making this view dynamicUpsert Flow Handles Matching on Semantic Versions - making this view dynamicLookup Flow Hydrates Pub-Sub Topic Info - making this view dynamic1.3.4 (v22.22)16.x (addresses error with Verify Credentials)MAX_FILE_SIZE to control attachment size limit1.6.7 (v22.22)2.4.41.0.0 (v22.22)Get Ukraine Alerts Polling triggerWebhook trigger2.0.3 (v22.22)Quota exceeded errors1.1.1 (v22.22)2.5.5 (v22.22)connectionTimeZone now defaults to SERVER for MySQL connections1.3.1 (v22.22)1.4.8 (v22.22)Download Files/File by name actions and Read Files trigger2.2.4 (v22.22)Released: 2022-06-21
To extend vertical space of the flow designer screen we moved the Footer to the bottom of side-menu. You can now see the platform version, Privacy Policy and ToS links by opening the menu.
This is the 3rd and final part of our initiative to prevent accidental or malicious changes to sensitive data and follows on from releases 22.20 and 22.22.
In this final part we extend re-authentication for sensitive actions to support cases when a user logs in using OIDC/SAML. The user will be redirected to OIDC/SAML provider for authentication.
Again, please note: You will be asked to re-authenticate once every 6 hours.
This feature will work on all modern web browsers versions from:
66+63+To improve component deployment and build process we modernised the appbuilder
and apprunner docker images to address potential vulnerabilities in the Node.js
packages. All platform services, as well as the Node.js based integration components
are built with these images.
The modernisation brought some important changes which might affect you when you deploy the new version of your component code.
Dockerfile to define the build process in the Node.js components. It must be in the root directory of your component code.Dockerfile you might have noticed that it was not used in the build process. Now the build process will check for the existence of it and use it. Which means if you have not updated your Dockerfile recently, chances are your component build might fail or the newly built component might not function as expected. Check your Dockerfile before deploying or remove it from the component code if custom build parameters are not necessary.Node.js version 18.Dockerfile is not yet supported for Java components.*) would not show after the metadata reload in case the mandatory fields were controlled by a checkbox.2.2.4 and 2.3.0 (v22.24)Type Of Search - External IDs to Upsert Object actionUpsert Object action (metadata needs to find fields that contain attachment)Upsert Object action2.0.13 (v22.24)1.4.2 (v22.24)Last interaction option to Time stamp field to poll on in Get New And Updated Objects Polling triggerGet New And Updated Objects Polling trigger to use scroll instead of search to collect all records1.0.2 (v22.24)2.1.0 (v22.24)Read Entire Bucket Contents actionWrite Entire Bucket Contents action1.5.0 (v22.24)Network Diagnostics action1.4.0 (v22.24)pubSubTopicName at actions Lookup Object By Unique Criteria and Lookup Objects (plural)Upsert Object By Unique CriteriacomponentSemanticVersion at actions Lookup Object By Unique Criteria and Lookup Objects (plural)Upsert Object By Unique Criteria1.0.0 and 1.0.1 (v22.24)Get Ukraine Alerts Polling TriggerWebhook Trigger1.3.3 and 1.3.4 (v22.24)version field to the component.json fileMAX_FILE_SIZE to control attachment size limitcomponent-commons-library to v2.1.01.6.7 (v22.24)1.6.1 and 1.6.2 (v22.24)1.1.0 (v22.24)Write Entire Bucket Contents ActionRead Entire Bucket Contents ActionUpsert Object input metadata1.4.9 (v22.24)Poll Files trigger1.0.0 (v22.24)Make Raw Request action1.2.0 (v22.24)TypeScript.Released: 2022-06-30
2.6.28 (v22.26)A new version of the Node.JS Sailor library 2.6.28 is released. For Users who
develop their own components using our SDKs this update provides improved handling
of any errors generated by the Maester service during upload of lightweight messages.
With this release it is now possible to to set and edit snapshots
while the flow is in Draft status - flows no longer need to have the status
Published or have been executed.
Information in this section is intended for our customers who use OEM version of the elastic.io platform.
This is further improvement of email domain blacklist feature from 22.22
release. Now you can use a wildcard character (*) with a domain name to exclude
all variants, for example gmail.*, yahoo.*, etc.