Product Updates in 2019 Q3

2019-09-25 - v3.20.0

Frontend (v3.20.0)

There is the new page with quota usages on the Navigation Panel, section Contract Settings:

Recipes (v3.20.0)

You can now export a Flow to a Recipe via the following API endpoint:

POST /v2/flows/:flow_id/export-to-recipe/

Find more info about it here.

To list all available Recipes, use the GET /v2/recipes endpoint. For the first iteration (until private/public Recipes will be introduced) the URL should contain the contract_id: /v2/recipes?contract_id=. You can read more about it here.

We changed the structure of POST/PATCH /v2/recipes, and the response structure of GET /v2/recipes. Check this section for details.

IMPORTANT: Recipes are still beta, so we don’t recommend using them with critical data or production.

Other (v3.20.0)

There are new endpoints you can use to see quota statistics for all Workspaces in your Contract. Additionally, there are endpoints for per-Flow and per-Workspace resource usage control, and Workspace or Contract usage history display. See details here.

Also, you can now use &ref in JSONschema. However, reference by $id is still not supported.

List Of Fixed Bugs (v3.20.0)

• Now POST,PUT,PATCH,DELETE /v2/flows/:id/draft are only allowed with WORKSPACE.FLOW.EDIT permission.

• Component data sample maximum provision time before timeout is 1 minute now.

• Also, we fixed the issue with wrong URL in the developer team invitation letter.

Technical Details (v3.20.0)

Other Stuff

Quota statistics endpoints:

GET /v2/quotas/per_workspace_mem_usage_monthly_limit/tenantId.contractId.workspaceId/find_the_most_specific_limit=true

GET /v2/quotas/per_workspace_cpu_usage_monthly_limit/tenantId.contractId.workspaceId/?find_the_most_specific_limit=true

GET /v2/quota-usages/workspaces/workspaceId?period=week

GET /v2/quota-usages/workspaces/workspaceId?period=2019-09

Per-Flow resource usage, for Workspace members only:

GET /v2/quota-usages/workspaces/{WORKSPACE_ID}/flows?period=week (see details here)

Per-Workspace limits, available to users with CONTRACT.EDIT_WORKSPACE_LIMITS permission:

PUT /v2/quotas/:quota_type/:limit_context

Contract and Workspace usage history:

GET /v2/quota-usages/contracts/:id/history?from=yyyy-mm-dd&to=yyyy-mm-dd

GET /v2/quota-usages/workspaces/:id/history?from=yyyy-mm-dd&to=yyyy-mm-dd

2019-09-26 - v24

SOAP Component (v24)

• NEW - component can generate SOAP responses given a WSDL file

PostgreSQL Component (v24)

• FIXED - incorrect error response handling

JDBC Component (v24)

• NEW - new action Insert action

Sugar CRM Component (v24)

• FIXED - Action / Upsert Account field postal address is now not limited to 20 characters

XML Component (v24)

• FIXED - bug with getting ‘File’s metadata is not found’ message when trying to handle attachments
• FIXED - bug with filtering files by name

2019-09-12 - v23

SAP byDesign Component (v23)

• NEW - introducing Trigger: Get New And Updated Objects Polling which supports only the following object types:
  • Query Materials
  • Query Sales Orders
  • Query Accounts
• FIXED - Password input field should be PasswordFieldView (input is hidden)
• FIXED - Authorization failed due to specific symbols in Username or Password

WebHook Component (v23)

• IMPROVED - WebHook includes additional information in response message:
  • msg.headers
  • msg.url
  • msg.method

REST-API Component (v23)

Code was refactored: new methods are implemented

CSV Component (v23)

In order to keep the component updated and to follow best practices, it was migrated from Travis CI to CircleCI Component code was refactored (dependencies updated)

2019-08-28 - v22

SAP ECC (R/3, ERP) Component (v22)

• NEW - introducing action Call RFC Service which enables a call with an RFC function on the SAP ECC (R/3, ERP) platform

Code Component (v22)

• FIXED - Incorrect message consuming in Code-component

JDBC component (v22)

• IMPROVED - Field for configuration of the connection custom parameters was added to component credentials.

Email Component (v22)

• NEW - Retry in case of problems with retrieving attachments

Shopware Component (v22)

• NEW - introducing action Upsert Object Action, which creates a new object or update object which already exists by provided ID. It makes
  • POST request when get message body is without id to create new entity and
  • PUT request when get message body includes id to update existing object.

REST-API-component (v22)

• NEW - introducing rebound on REST HTTP timeouts. Retry on failure option enables rebound feature for following HTTP status codes:
  • 408: Request Timeout
  • 423: Locked
  • 429: Too Many Requests
  • 500: Internal Server Error
  • 502: Bad Gateway
  • 503: Service Unavailable
  • 504: Gateway Timeout
  • DNS lookup timeout

2019-08-28 - v3.19.0

Frontend (v3.19.0)

We are in a constant state of UI honing, to make it even more convenient for you. First of all, we fixed the navigational menu, so it does not hide if you move the cursor away from it. To close the menu, click “hamburger” icon again.

IMPORTANT: Please don’t forget that from now on your company logo will be moved to the navigation panel. For proper display, please make sure that your logo complies with the following requirements:

• Logo shape - square

• Logo size = 40x40 pixels (smaller logos will be centered with whitespace around them)

• Logo format - .PNG or .SVG

Recipes (v3.19.0)

You can now use Recipes to simplify Flow creation. Basically, a Recipe is a template of a working integration Flow. It has the following conditions:

• A Recipe may not reference any credentials

• A Component in a Recipe must always be versioned

• Mapping expressions may contain variables to be replaced with values during Recipe activation. You can perform CRUD operations with Recipes and create a flow from a recipe using API. You can find examples here. Please note that the section is an experimental API.

IMPORTANT: Recipes are beta right now, so we don’t recommend using them with critical data or production.

Alerts (v3.19.0)

Not to let you waste any time, we have added handy alerts that will notify you of any important changes in your flows statuses:

• All Contract owners will receive emails upon exceeding 80%, 85%, 90% and 95% of resource usage quota

• All users that subscribed to errors on a particular integration Flow will receive an email if the Flow gets suspended

Other (v3.19.0)

You can now send a sample request to Webhook. Here is the new process of Webhook setup:

1. Generate a unique URL for the Webhook to post sample data to and present it in the UI

2. Send a sample request to the given URL. Once the request has been sent, it is stored in the DB.

3. The sample is retrieved by ID and is used for mapping. Sample URL is active for 10 minutes.

List Of Fixed Bugs (v3.19.0)

We were totally dedicated to improvements this month, so all the discovered bugs were immediately fixed before going into production.

Frontend stuff (v3.19.0)

Introduced FRONTEND_NO_EXTERNAL_RESOURCES config variable. If set to "true", then frontend will use scripts, styles and fonts from the same domain, without external CDN usage. If you want to use external CDNs, do not specify this variable.

To customize the link that leads to Component creation instructions, when creating a new Repository, PATCH your Tenant with:

{
  "attributes":{
    "docs_base_url":"{URL_TO_DOCUMENTATION}"
  }
}

If docs_base_url is not specified the repository page docs link will set to default value.

Dockerized Slugs (v3.19.0)

This is very cool, but quite new and experimental, so use at your own risk!

You can now build components as docker images via Gitreceiver. Gitreceiver pushes images into docker-registry, then docker at k8s nodes pulls images from that docker registry to run them.

We introduced ksonnet component (docker) and 4 new parameters:

1. docker_registry_replicas Just number of replicas of docker registry deployment. Clearly it should be greater then 1. You can change this value any time you want.

2. docker_registry_secret_name. Admiral will use this secret while creating pods as authorization secret in docker registry. This secret should be defined in tasks namespace, it should contain login/pass and server (see docker_registry_uri) for authorization in docker registry (docker-registry type). Probably elasticiotasks value is good enough. Generally you can change this value, but:

*a) be sure than proper secret exists in tasks namespace.

b) be sure that you’ve restarted admiral after change.

c) It’s better to remove previous secret only when all pods, that was defined with it will die. Otherwise that won’t restart in case of failure.*

3. docker_registry_uri. Value is used by gitreceiver to find out where to push newly built images and by admiral to create fully-qualified image name of docker image for pod. Typically value looks like this: “http://$LOGIN:$PASSWORD@127.0.0.1:31000/elasticio”. $LOGIN and $PASSWORD should be same as secret mentioned above. Secret also should reference to IP address and port. Port is arbitrary from NodePort range (30000-32000 in default K8S installation). We don’t recommend changing it during installation runtime. URl’s path – anything you like. elasticio seems quite reasonable and I don’t see any reasons to change it. Also, it is better not to edit this value during lifetime of installation. Host MUST be 127.0.0.1 for all installations.

4. docker_registry_http_secret. Just a good random string of any length. Used by docker registry for its own crypto magic. Changes requires restart of all docker-registry pods.

Ksonnet scripts install everything and maintain all constraints between ksonnet entities. To push component as docker it’s required to add "buildType":"docker" into component.json file in component repository.

Other Stuff (v3.19.0)

We replaced custom_links field for Tenant with custom_nav_menu_items. PATCH your Tenant to display the “Quick Help” menu:

"custom_nav_menu_items":[
 {
   "title":"Quick Help",
   "icon":"help",
   "custom_class":"custom_class",
   "links":[
     {
       "url":"https://docs.elastic.io/",
       "title":"Documentation",
       "icon":"description"
     },
     {
       "title":"Help Center",
       "icon":"forum",
       "custom_class":"intercom-launcher"
     }
   ]
 }
]

URLs in custom_nav_menu_items now supports url parameters (workspaceId, contractId, tenantId, etc.). When a URL has placeholders http://custom-url?workspaceId={workspaceId}&contractId={contractId}, those placeholders are substituted with real parameters. custom_nav_menu_items supports a 2-level structure.

IMPORTANT: The following Tenant configurations are deprecated now:

• connectorCatalogUri
• flowCatalogCurrentProjectSpaceUri
• infoSupportUri
• infoVideoUri
• infoDocumentationUri

2019-08-08 - v21

Netsuite Component (v21)

• IMPROVED - Error messages have become more user-friendly and some additional information is moved from errors thrown to logs.

2019-08-01 - v3.18.0

Frontend (v3.18.0)

You can now enjoy a better, more convenient UI navigation:

The Quick Help button opens a menu, which allows you to visit our product documentation page, or open a support chat:

The button with your initials opens your profile menu, where you can manage profile and SSH keys:

What’s even better, you can now enjoy breadcrumbs navigation:

All in all, we improved the navigation to make it more intuitive. For example, you can now go directly to Flow Designer from Flow execution page. Also you can now see the mappings of a read-only Flow without the need to create a draft.

Additionally, you can now multiselect options in certain components, as we added MultiSelectView class for component fields. Details here.

IMPORTANT: As of version 3.19.0, your company logo will be moved to the navigation panel. For proper display, please make sure that your logo complies with the following requirements:

• Logo size = 40x40 pixels

• Logo format - .PNG or .SVG

Other (v3.18.0)

This is important. There is now custom_links instead of links.documentation in Tenant. Check out the details here.

List Of Former Bugs (v3.18.0)

• Scrolling now works properly on Profile menu.

• handlebars mapper type for Debug task is officially gone, and won’t confuse you anymore.

• Flows overview page does not automatically set the user filter when you are navigating to the page with the query parameter in URL

Technical Details (v3.18.0)

Frontend stuff (v3.18.0)

To use the new viewClass you need to add the class to component.json file, as “viewClass”:”MultiSelectView” for certain field. Then you can select multiple items on the Frontend. Values will be passed in the configuration as an array.

EXAMPLE:

"triggers": {
    "getPetsByStatusWithGenerators": {
      "main": "./lib/triggers/getPetsByStatusWithGenerators.js",
      "type": "polling",
      "title": "Get Pets By Status With Generators",
      "fields": {
        "status": {
          "label": "Pet Status",
          "required": true,
          "viewClass": "MultiSelectView",
          "model": {
            "available": "Available",
            "pending": "Pending",
            "sold": "Sold"
          },
          "prompt": "Select Pet Status"
        }

Other Stuff (v3.18.0)

We introduced custom_links field for Tenant, with the following format:

[
  {
      "url": "https://docs.example.com",
      "title": "Documentation",
      "icon": "description",
      "custom_class": "custom_class"
  },
 ...
]

NOTE: links.documentation is deprecated from now on. Support of links.documentation will be removed from the Platform in 3.23.0 release. Please be sure to PATCH your tenant with "links": null and move links.documentation to custom_links.

2019-07-29 - v20

Google Spreadsheets component (v20)

• FIXED - Remove fields spreadsheetId, spreadsheetUrl from Create new spreadsheet action, as theirs inputs are not affect on new spreadsheet creation.

Filter Component (v20)

• IMPROVED - Updated Action Simple JSONata Filter. Assertion Checkbox added: If checked, the checkbox adds Assertion functionality. Instead of doing nothing, the component will throw an error when the condition is not met

JDBC Component (v20)

• NEW - introducing action Execute stored procedure which calls stored procedure from selected DB Schema and Stored procedure name.

Postgresql Component (v20)

• IMPROVED - Updated action SQL Injection by adding Number of retries in case of deadlock transaction field.

You can specify the maximum number of retries, that is intended to help to solve lock’s issues in case of a deadlock transaction. The delay between retries is 1 second. Default value for this configuration field is 0, it means, that such behavior is switched off (by default) and no any retry will be performed in case of deadlock transaction.

Netsuite Component (v20)

• NEW - introducing Get New and Updated Objects Polling generic trigger that polls NetSuite instance for new and/or updated objects (of any type available in the NetSuite).
• NEW - introducing action Lookup Objects which looks for objects available in NetSuite with a given criteria.
• NEW - introducing action Lookup Object By Id which looks for an object by the provided Id.
• NEW - introducing action Delete Object By Id which deletes an object by the provided Id.
• NEW - introducing action Upsert Object By Id which either updates an object in NetSuite by a provided Id or inserts it as a new object if it does not exist.

All previously existed triggers and actions marked as deprecated, as they are not match with OIH standards:

Deprecated Trigger:

• Search entity (Please use Get New and Updated Objects Polling Trigger instead)

Deprecated Actions:

• Lookup Customer (Please use Lookup Object By ID Action instead)
• Lookup Invoice (Please use Lookup Object By ID Action instead)
• Upsert Contact (Please use Upsert Object By Id Action instead)
• Upsert Customer (Please use Upsert Object By Id Action instead)
• Upsert Invoice (Please use Upsert Object By Id Action instead)
• Upsert Sales Order (Please use Upsert Object By Id Action instead)
• Upsert Vendor(Please use Upsert Object By Id Action instead)

2019-07-11 -v19

JDBC Component (V19)

• FIXED - Incorrect parsing of variables
• FIXED - Upsert By Primary Key Action now supports null values

Google Spreadsheets component (V19)

• NEW - introducing trigger Get Spreadsheet Row to read the data in each row of a given Google Spreadsheet and passes it to the next step of your integration flow.
• NEW - introducing action Create new Spreadsheet to create a new Google spreadsheet.
• NEW - introducing action Add Spreadsheet Row to create a new Google spreadsheet row

CSV Component (V19)

• IMPROVED - Updated action Read CSV attachment by adding a new option Emit all messages to emit all rows in one message as an array.

Salesforce Component (V19)

• FIXED - Attachment Object type can not be sent to Salesforce

2019-07-01 - v3.17.0

Frontend (v3.17.0)

Credentials management is now way better due to a few tweaks. First of all, you can now edit or delete credentials in the Designer. Secondly, you don’t have to verify credentials any more. Of course, verification protects Flows from invalid credentials. However, we thought that the step should be optional. So now you don’t have to spend extra time on credentials verification, if you don’t want to. Check the screenshots below to see how this looks in the UI.

• When creating new Credentials:

• when adding new credentials to a Component in a Flow:

Also, we changed the order in step configuration process. Now it goes as follows:

1. Select the Component

2. Choose version (optional)

3. Choose Trigger/Action

3. Choose credential (or choose Agent, if the Component does not have credentials)

Other (v3.17.0)

Now all Flows will start and stop faster, taking less time between clicking start or stop in the UI, than they did earlier. If you are interested in details, you can find them here.

You can now change metadata for your Platform HTML page, like this.

Additionally, we tweaked scaling a bit, adjusting the mechanism of graceful platform application scaling. This means you won’t get the old deployment/scaling errors.

Fixed Bugs (v3.17.0)

• We were totally dedicated to improvements this month, so all the discovered bugs were immediately fixed before going into production.

Technical Stuff (v3.17.0)

WARNING! Boring technical stuff ahead. Read only if you have to.

Frontend stuff (v3.17.0)

“Please, configure all steps in order to save the flow” message replaced with alert. It shows in case of non-configured components in a flow.

Other Stuff (v3.17.0)

So, now the API call is made immediately after you’ve clicked start/stop. Obviously, this improvement boosts “reaction” time. Additionally, we reduced latency between message emitted in first step by scheduler or Webhooks, and the actual pod start.

To change metadata add the following attributes to POST /v2/tenants and PATCH /v2/tenants:

{
  "data":{
    "type":"tenant",
    "attributes":{
      "html_meta":{
        "description":"I am a description",
        "keywords":[ "hotdog", "burger", "sandwich" ],
        "author":"Acme Corporation"
      }
    }
  }
}