Jump to
Product Updates
v v v v
Updated on

Product Updates in 2019 Q4

Product Updates Archive for 2019 Q4 period.

Product update - v3.21.0

Released: 2019-10-30

Features (v3.21.0)

Recipes (v3.21.0)

You can now manage Recipe visibility with the new API endpoint PATCH v2/:recipe_id/visibility. This action is only available to users with proper permissions, and can be applied to components visible to them.

Also, you can now use Recipe details page (replace variables with corresponding data):

app.elastic.io/c/:contract_id#/w/:workspace_id/recipes/:recipe_id

Recipe descriptions support markdown syntax, except for check-boxes. Images for the page are hosted by Recipe author.

Another page for Recipes UI is Recipe activation page (replace variables with corresponding data):

app.elastic.io/c/:contract_id#/w/:workspace_id/recipes/:recipe_id/activate

It contains Recipe variables fields and Component sections listed with Credentials. Here you can choose existing Credentials, create new or edit existing ones during Recipe activation. Some data will require manual configuration: Credentials, Fields, Flow Variables.

You can find the corresponding API doc here. POST v2/recipes and PATCH v2/recipes/:id do not have attributes.visibility anymore. Instead, new Recipes are created with the default visibility set for the Workspace. See more details here.

Changing Recipe visibility requires certain permissions, depending on visibility level: Workspace, Contract, Tenant, Global.

Quota (v3.21.0)

You can now see monthly quota history and quota usage of a Contract on the corresponding page:

Quota Overview Page

See details on quota history here.

Manage Snapshots via API (v3.21.0)

You can now manage Snapshots via the API.

API endpoints for Snapshots:

  • GET /v2/flows/$flowId/snapshots
  • GET /v2/flows/$flowId/snapshots/$stepId
  • POST /v2/flows/$flowId/snapshots/
  • PATCH /v2/flows/$flowId/snapshosts/$stepId
  • DELETE /v2/flows/$flowId/snapshots/$stepId

You can find the corresponding API docs here.

Reference by ID in component metadata

Also, we have added reference by ID in Component metadata JSON Schema.

As for JSON Schema reference by ID:

{
  “definitions”: {
    “address”: {
      “$id”: “#address”,
      “type”: “object”,
      “properties”: {
        “street_address”: { “type”: “string” },
        “city”:           { “type”: “string” },
        “state”:          { “type”: “string” }
      },
      “required”: [“street_address”, “city”, “state”]
    }
  },
  “type”: “object”,
  “properties”: {
    “billing_address”: { “$ref”: “#address” },
    “shipping_address”: { “$ref”: “#address” }
  }
}

Docker Registry Migration (v3.21.0)

Another IMPORTANT thing - Docker Registry service doesn’t run under the root user any more and requires migration. Please note that this is crucial, and without migration the Platform will fail.

The migration has to be run AFTER deploying the release, where the docker-registry is being run as non-root. If the old version of the docker-registry (with root) was run, just to deploy the new version, remove the old migration job and run the migration again.

The official registry (registry:2.7.1 or mirror elasticio/registry:2.7.1) is being run as a root, elasticio/registry:2.7.1-non-root is being run as docker. Our migration contains an update of ownership of files according to changed user (root->docker), and is implemented as kubernetes job (i.e. it has to be run only once).

Specify the YOUR_CONTEXT below and execute:

cat << EOM | kubectl create --context YOUR_CONTEXT --namespace platform  -f -
{
    "apiVersion": "batch/v1",
    "kind": "Job",
    "metadata": {
        "labels": {
            "app": "docker-registry-migration-for-non-root"
        },
        "name": "docker-registry-migration-for-non-root",
        "namespace": "platform"
    },
    "spec": {
        "backoffLimit": 0,
        "completions": 1,
        "parallelism": 1,
        "template": {
            "metadata": {
                "labels": {
                    "app": "docker-registry-migration-for-non-root",
                    "job-name": "docker-registry-migration-for-non-root"
                },
                "name": "docker-registry-migration-for-non-root"
            },
            "spec": {
                "containers": [
                    {
                        "command": [
                            "sh",
                            "-c",
                            "chown -R docker:docker /var/lib/registry"
                        ],
                        "image": "elasticio/registry:2.7.1-non-root",
                        "imagePullPolicy": "Always",
                        "name": "docker-registry-migration-for-non-root",
                        "securityContext": {
                            "runAsUser": 0
                        },
                        "terminationMessagePath": "/dev/termination-log",
                        "terminationMessagePolicy": "File",
                        "volumeMounts": [
                            {
                                "mountPath": "/var/lib/registry",
                                "name": "docker-storage",
                                "subPath": "docker"
                            }
                        ]
                    }
                ],
                "dnsPolicy": "ClusterFirst",
                "imagePullSecrets": [
                    {
                        "name": "elasticiodevops"
                    }
                ],
                "nodeSelector": {
                    "elasticio-role": "platform"
                },
                "restartPolicy": "Never",
                "schedulerName": "default-scheduler",
                "volumes": [
                    {
                        "name": "docker-storage",
                        "persistentVolumeClaim": {
                            "claimName": "platform-storage-slugs-volume-claim"
                        }
                    }
                ]
            }
        }
    }
}
EOM

To check the success, create new build and publish it:

shell
kubectl --context YOUR_CONTEXT --namespace platform exec  -it  docker-registry-INSERT_CERTAIN_POD_POSTFIX -- sh -c "ls -la /var/lib/registry"

It must contain docker docker:

drwxr-xr-x    3 docker   docker        4096 Aug  7 19:01 .
drwxr-xr-x    1 root     root          4096 Oct  4 15:18 ..
drwxr-xr-x    3 docker   docker        4096 Aug  7 19:01 docker

Fixed Bugs (v3.21.0)

  • FIXED GET Flows-Versions and CRUD with Flow draft are now available only to Workspace members.
  • FIXED - Drop-down lists of numbers in Components are now displayed correctly.
  • FIXED - Now errors are grouped by input message and error message.

Components (v3.21.0)

Includes

  • v25 - 2019-10-10
  • v26 - 2019-10-24

Google Spreadsheet Component (v3.21.0)

  • NEW - component retrieves actual data without formatting and reads spreadsheets as numbers/values without formatting

Shopify admin Component (v3.21.0)

  • NEW - component has been fully refactored in order to comply OIH standards
  • NEW - introducing new action called Lookup Objects

JDBC Component (v25)

  • NEW - introducing new action called Execute custom query
  • IMPROVED - Rebound (retry) mechanism for Transaction Lock error has been added
  • IMPROVED - Get Rows Polling Trigger has better datetime field type support
  • FIXED - bug when no schemas in drop-down list for call procedures action are displayed has been fixed

CSV Component (v3.21.0)

  • FIXED - bug when Error: write after end message had been thrown in Write CSV attachment action in runtime in some cases

Magento2 Component (v3.21.0)

  • MEW - Component completeness matrix has been added to the component documentation

SFTP Component (v3.21.0)

  • NEW - introducing support for custom ports

Product update - v3.22.0

Released : 2019-11-21

Features (v3.22.0)

You can now download execution logs to a file by clicking the corresponding button for:

  • the entire Flow

Download Flow logs

  • a certain Step

Download Step Logs

Also, you can now see the Workspace quota usage information per Flow. On this page a user comes from the Contract quota details page by clicking on one of the workspaces:

Quota

Fixed Bugs (v3.22.0)

  • FIXED - Now your Flow drafts will be saved much more frequently.

Components (v3.22.0)

Includes

  • v27 - 2019-11-07
  • v28 - 2019-11-21

Docusign Component (v3.22.0)

  • NEW - introducing OAuth client support

Magento2 Component (v3.22.0)

  • NEW - introducing action Lookup Object (at most 1)
  • NEW - introducing action Delete Object
  • NEW - introducing action Set tiered prices
  • NEW - introducing action Upsert customer/company

Salesforce Component (v3.22.0)

  • NEW - introducing OAuth client support

SAP Business byDesign Component (v3.22.0)

  • NEW - added an optional rebound mechanism in case of SOAP fault

Shopware Component (v3.22.0)

  • IMPROVED - Upsert object action now supports external IDs along with internal ones

Shopify Admin Component (v3.22.0)

  • NEW - introducing action Create object
  • NEW - introducing action Delete Object
  • NEW - introducing action Lookup Object (at most 1)
  • NEW - introducing action Upsert object
  • NEW - introducing action Get New and Updated Objects
  • NEW - introducing action Webhook subscription

Product update - v19.51

Released : 2019-12-19

NOTE: From this release we have adopted new release numbering format based on year and week of the release - YY.WW. YY are the last two digits of the year, and WW is the number of the week in that year. Following this template, our current release is 19.51.

Features (v19.51)

Retry failed messages (v19.51)

You can now retry failed messages using the new feature called Retry. Any error emitted by a component can be retried if the flow is active. Error or success entry appears in the current Runlog execution. Feature details can be found here:

Retry feature in action

Also, now you can filter mapping view to see mandatory fields and hide optional fields:

Hiding optional fields

Disable passthrough API (v19.51)

You can now disable the passthrough for a selected flow step using an API call to the following endpoint:

PATCH /v2/flows/:id

NOTE When the passthrough is disabled for a step, the platform would reset all the data from the previous steps and start again collecting the data from the steps from that step only.

This feature is available via an API call only and here is the schema of the passthrough toggling request body:

{
    "data": {
        "type": "flow",
        "id": "5de7d63a944b750013fb3f09",
        "attributes": {
            "nodes_config": {
                "step_2": {
                    "passthrough": {
                        "disabled": true
                    }
                }
            }
        }
    }
}

Improvements (v19.51)

Order of fields in component descriptor (v19.51)

Now you can configure the order of fields in component.json using the new numeric key order for:

  • output metadata
  • credentials
  • action/trigger
  • action/trigger order
  • action/trigger on recipe activation page
  • credentials on recipe activation page

Ordering fields in component.json

For more details read the component descriptor ordering fields document.

Custom scripts for individual tenants (v19.51)

Additionally, custom scripts for individual tenants can run on every page before authorization:

  • Login pages
  • Forgot password pages
  • Register pages
  • Reset password pages
  • Invite page

Little bits (v19.51)

  • Component environment variables can have global visibility again.
  • The deprecated links links.documentation and tenant.links in the tenant description body are no longer supported.

Components (v19.51)

Includes

  • v29 - 2019-12-05
  • v30 - 2019-12-19

Salesforce Component (v19.51)

  • NEW - Adding support for Bulk operations feature (Create/Update/Delete and Query)
  • NEW - introducing action Delete Object
  • NEW - introducing action Lookup Objects
  • IMPROVED - Create object action: can utilize binary data attachment from previous step
  • IMPROVED - Upsert object action: can utilize binary data attachment from previous step
  • IMPROVED - Lookup Object (at most 1) action: can pass binary data (if found object has it) to the next component as a binary attachment
  • IMPROVED - Query action: can query deleted objects

SAP byDesign Component (v19.51)

  • FIXED - bug where some objects may be missed in the Get New And Updated Objects Polling Trigger for Material entity type.

SAP R3 Component (v19.51)

  • FIXED - Component failed to start Error shown during flow run.

SFTP Component (v19.51)

The component has been refactored and created from scratch in an OIH-compatible way. The following actions/triggers have been introduced:

  • NEW - introducing action Delete File
  • NEW - introducing action Lookup File (At most 1) By Filename
  • NEW - introducing action Download files
  • NEW - introducing action Upload File From URL
  • NEW - introducing trigger Get new and updated files polling
  • NEW - Adding max value to file size and environment variable to configure it
  • NEW - introducing a new authentication type: Key-based auth mechanism
  • IMPROVED - The following actions are renamed:
    • Upload files -> Upload files From Attachments Header
    • Lookup file by name -> Download file by name
    • Get new and updated files -> Poll files
  • FIXED - It was not possible to get 0-size files for Get new and updated trigger before

AWS S3 Component (v19.51)

The component has been fully reworked and refactored. So it’s become more effective and reliable.

  • NEW - introducing action Rename File
  • FIXED - Buckets created in Regions launched after March 20, 2019 were not reachable via the existing naming scheme.

Lookup Table Component (v19.51)

A new component which can be used to convert from different tables that parsed as a CSV.

Quickbooks Component (v19.51)

  • FIXED - bug when connecting to some production systems

Shopware Component (v19.51)

  • NEW - Rebound feature has been implemented for the component