Fields vs Metadata

Prerequisites

The article assumes that you are familiar with the following topics:

• What is component and action in it.
• Flow creation, configuration and data Mapping.
• Basic understanding of component development.

Introduction

The component has the following predefined entities that it consumes on execution start:

• Message:
  • Body - a result of the mapping of the emitted message from the previous step into input metadata.
  • Attachments - an array of attachments provided by the previous step.
• Configuration:
  • Credentials - chosen during step configuration.
  • Input field values.
• Snapshot - 5KB storage shared between executions.

In this article, we will talk about the differences of Input fields and input metadata as it often confuses new Elastic.io platform users.

Semantics

Despite some technical and UX similarities that you will see later in this article there is a fundamental difference in the purpose of existence both Input fields and input metadata.

Input fields

Exists to fine-tune component action behavior as a step in a particular flow. Some parameters can be configured for the step that is predefined by the platform as passthrough behavior, pre-fetch count, or custom RAM limit. But at the same time, each component solves its integration task (e.g. Salesforce API communication) and may require the ability to tune behavior to work best in a given use-case.

So, Input fields can be compared to buttons on any electrical device, let’s say copy machine, Input fields for copy machine component would be:

• Number of copies
• B&W or Color
• Image quality
• Contrast and so on.

Or another example is oven, then Input fields here are:

• Temperature
• Timer

Input metadata

On the other hand, input metadata is a predefined structure of the message that the component is designed to process. This structure is needed to show the integrator what message needs to be formed using UI (integrator mode) or manual (developer mode) mapping. Structure of useful payload, data itself.

If we continue the real-world example with a copy machine then input metadata would be a frame for scanned objects. And integrator task here would be to place an object that needs to be scanned (payload from previous steps or/and literals) in the scan frame. In case of oven input metadata would be cooking forms that needs to be filled (mapped) with ingredients (payload from previous steps or/and literals)

Input metadata can be static or dynamic. We will look further at the differences and how they are implemented in UI and developer mode. The main reason for having two types of metadata is the fact that it is often necessary to configure metadata for a particular scenario, which can vary depending on the configuration of a particular component.

component.json

As you may already know, how component feels and looks described by a component developer in component.json file.

Details on how to describe input fields and metadata in component.json can be taken from articles:

• General position
• input fields description structure
• input fields types explained

The main difference between Input fields and input metadata in the scope of component.json would be that Input fields structure is static and will remain the same for defined action. While metadata can be both static or dynamically generated by component code during flow configuration.

Input fields values available to the code which generates dynamic metadata, this is underlined purposes of both. As a component developer, you can change dynamic metadata according to input fields parameters.

A good example of the last sentence would be the general architecture of upsert action (you can meet such action in lots of different components, e.g. Salesforce). If the target system supports more then one object type, then you would need to render different input metadata according to the upsert object type chosen in the input field “Object type”.

UI

The Input fields are part of the Configuration Step and are always static except for dropdown menus, where sometimes objects are generated dynamically,depending on the credentials and other configuration fields. If you are working with dynamic Input fields as in example below, you always have the reload option:

Input metadata structure visible immediate if it is static and rendered after required Input fields will be filled if metadata is dynamic.

Another difference is that input metadata (rendered representation) can be filled with mapped values from the payload (from previous steps or/and literals). While Input fields should be filled manually with a value that will be static in the scope of the flow and won’t change in the runtime.

Please note: Input fields and Input metadata depend on what kind of credentials you use and what permission you have. In some cases, this may change how the fields and metadata look.

Component development

From the component developer perspective you can face metadata and input fields in the following cases:

• Action development
• Dynamic drop-down list population for input fields
• Dynamic metadata generation

Let’s look into each of these cases.

Action development

The action method itself has nothing to do with metadata or input fields structure or generation. But it works with values put into both. Here is the sample of typical action code (Node.js):

module.exports.process = async function process(msg, cfg, snapshot) {
        const body = msg.body;
        const {field1, field2, field3, } = cfg;
        ...
}

Where cfg containing credentials and input fields and msg contains mapping result into input metadata structure (plus attachments info).

Dynamic drop-down list population for input fields

If an action is designed to have a drop-down list with dynamic values then the component developer needs to implement logic that will serve this function. Here is the sample of component.json that identifies dynamic nature of the drop-down list input field:

"actions": {
    "action1": {
        "title": "Action title",
        "main": "./lib/actions/action1.js",  --Path to file where method shouldbe implemented
        "description": "Action desciption",
        "fields": {
            "dynList": {
                "viewClass": "SelectView",
                "label": "Dynamic List",
                "model": "dynList",  --Method name that should provide object with enteties for drop-down list
                "prompt": "Please select item"
            }
        }
    }
}

Sample of such method implementation:

module.exports.dynList = async function dynamicList(cfg) {
        const {cred1, cred2, cred3, } = cfg;
        let list = {
		"FieldID1": "FieldName1",
		"FieldID2": "FieldName2",
		"FieldID3": "FieldName3"
	};
        ...
	return list;
}

Dynamic metadata generation

While for input fields dynamic population developer should specify the method name, for dynamic metadata generation method name is predefined, it is getMetaModel. In component.json instead of metadata parameter dynamicMetadata should be specified:

"dynamicMetadata": true

getMetaModel implementation sample:

module.exports.getMetaModel = async function getMeta(cfg) {
        const {cred1, cred2, cred3, field1, field2, } = cfg;
	const meta = {
	    "in": {
	        "type": "object",
	        "properties": {}
	    },
	    "out": {
	        "type": "object",
	        "properties": {}
	    },
	};
	...
	meta.in.properties = getInMeta(cfg);
	meta.in.properties = getOutMeta(cfg);
	return meta;
}

Where getInMeta() and getOutMeta() does not required, just as sample. Method getMetaModel should return an object with in and out fields that contain conventional JSON-schema as if it would be static.