Revise Write Xml (#31)

* Update dependencies
* Create new JSON to XML action
* Add help links

Author: jhorbulyk

CHANGELOG.md

@@ -1,6 +1,11 @@
+## 1.3.0 (April 23, 2020)
+* Update dependencies
+* Create new JSON to XML action
+* Add help links
+
 ## 1.2.1 (March 30, 2020)
 
-* Minor logs impovements in "XML to JSON" action
+* Minor logs improvements in "XML to JSON" action
 
 ## 1.2.0 (January 30, 2020)
 

README.md

@@ -1,19 +1,45 @@
 # XML Component [![CircleCI](https://circleci.com/gh/elasticio/xml-component.svg?style=svg)](https://circleci.com/gh/elasticio/xml-component)
 
 ## Description
-elastic.io iPaaS component to convert between XML and JSON data. 
+iPaaS component to convert between XML and JSON data. 
 
 ### Purpose
-Allows users to convert XML attachments and strings to and from JSON
-This component has 3 actions allowing users to pass in either generic but well format XML/JSON string or XML attachment 
-and produces a generic string of the other file type. The output then can be maped and used in other components.
+Allows users to convert XML attachments and strings to and from JSON.
+This component has 3 actions allowing users to pass in either generic but well formatted XML/JSON strings or XML attachments 
+and produces a generic string or attachment of the other file type. The output then can be mapped and used in other components.
 
-### Requirements
+### Requirements and Conversion Behavior
 Provided XML document (for `XML to JSON`) should be [well-formed](https://en.wikipedia.org/wiki/Well-formed_document) 
-in order to be parsed correctly. You will get an error otherwise.
+in order to be parsed correctly. You will get an error otherwise. 
+
+JSON inputs must be objects with exactly one field as XML documents must be contained in a single 'root' tag. 
+[JSON inputs can not have any field names which are not valid as XML tag names:](https://www.w3schools.com/xml/xml_elements.asp) 
+* They must start with a letter or underscore
+* They cannot start with the letters xml (or XML, or Xml, etc)
+* They must only contain letters, digits, hyphens, underscores, and periods
+
+XML attributes on a tag can be read and set by setting an `_attr` sub-object in the JSON.  
+The inner-text of an XML element can also be controlled with `#` sub-object.
+
+For example:
+```json
+{
+  "someTag": {
+    "_attr": {
+      "id": "my id"
+    },
+    "_": "my inner text"
+  }
+}
+```
+is equivalent to
+```xml
+<someTag id="my id">my inner text</someTag>
+```
 
 #### Environment variables 
-No environment variables need to be set.
+* `MAX_FILE_SIZE`: *optional* - Controls the maximum size of an attachment to be written in MB. 
+Defaults to 10 MB where 1 MB = 1024 * 1024 bytes.
 
 ## Actions
 
@@ -26,13 +52,36 @@ and produces one outbound message per matching attachment. As input, the user ca
 files by name or leave this field empty for processing all incoming *.xml files.  
 
 ### JSON to XML 
-Treats incoming message body as JSON and converts it to a generic XML string.
+Provides an input where a user provides a JSONata expression that should evaluate to an object to convert to JSON. 
+See [Requirements & Conversion Behavior](#requirements-and-conversion-behavior) for details on conversion logic.
+The following options are supported:
+* **Upload XML as file to attachments**: When checked, the resulting XML will be placed directly into an attachment.
+The attachment information will be provided in both the message's attachments section as well as `attachmentUrl` and `attachmentSize`
+will be populated. The attachment size will be described in bytes.  
+When this box is not checked, the resulting XML will be provided in the `xmlString` field.
+* **Exclude XML Header/Description**: When checked, no XML header of the form `<?xml version="1.0" encoding="UTF-8" standalone="no"?>` will be prepended to the XML output.
+* **Is the XML file standalone**: When checked, the xml header/description will have a value of `yes` for standalone. Otherwise, the value will be `no`. Has no effect when XML header/description is excluded.
+
+The incoming message should have a single field `input`. When using integrator mode, this appears as the input **JSON to convert** When building mappings in developper mode, one must set the `input` property. E.g.:
+```
+{
+  "input": {
+             "someTag": {
+               "_attr": {
+                 "id": "my id"
+               },
+               "_": "my inner text"
+             }
+           }
+}
+```
 
 ## Known limitations
  - The maximum size of incoming file for processing is 5 MiB. If the size of incoming file will be more than 5 MiB, 
  action will throw error `Attachment *.xml is to large to be processed by XML component. File limit is: 5242880 byte, 
  file given was: * byte.`. 
- - `XML Attachemnt to JSON` action does not support local agents due to current platform limitations.
+ - All actions involving attachments are not supported on local agents due to current platform limitations.
+ - When creating XML files with invalid XML tags, the name of the potentially invalid tag will not be reported.
 
 ## Additional Info
 Icon made by Freepik from www.flaticon.com 

component.json

@@ -1,6 +1,10 @@
 {
   "title": "XML",
-  "description": "Component to work with XML files",
+  "help": {
+    "link":  "/components/xml/",
+    "description": "Component to convert between XML and JSON data"
+  },
+  "description": "Component to convert between XML and JSON data",
   "buildType": "docker",
   "actions": {
     "xmlToJson": {
@@ -14,11 +18,69 @@
     },
     "jsonToXml": {
       "title": "JSON to XML",
-      "main": "./lib/actions/jsonToXml.js",
+      "main": "./lib/actions/jsonToXmlOld.js",
+      "deprecated": true,
       "description": "Takes the body of message passed into the component and converts to generic XML string",
       "metadata": {
         "in": {},
-        "out": "./lib/schemas/jsonToXml.out.json"
+        "out": "./lib/schemas/jsonToXmlOld.out.json"
+      }
+    },
+    "jsonToXmlV2": {
+      "title": "JSON to XML",
+      "main": "./lib/actions/jsonToXml.js",
+      "help": {
+        "link":  "#json-to-xml",
+        "description": "Takes the result of a JSONata expression and creates corresponding XML as either a string or an attachment"
+      },
+      "fields": {
+        "uploadToAttachment": {
+          "order": 3,
+          "label": "Upload XML as file to attachments",
+          "viewClass": "CheckBoxView"
+        },
+        "excludeXmlHeader": {
+          "order": 2,
+          "label": "Exclude XML Header/Description",
+          "viewClass": "CheckBoxView"
+        },
+        "headerStandalone": {
+          "order": 1,
+          "label": "Is the XML file standalone",
+          "viewClass": "CheckBoxView"
+        }
+      },
+      "metadata": {
+        "in": {
+          "type": "object",
+          "properties": {
+            "input": {
+              "title": "JSON to convert",
+              "type": "object",
+              "required": true
+            }
+          }
+        },
+        "out": {
+          "type": "object",
+          "properties": {
+            "xmlString": {
+              "type": "string",
+              "required": false,
+              "title": "XML String"
+            },
+            "attachmentUrl": {
+              "title": "Attachment URL",
+              "type": "string",
+              "required": false
+            },
+            "attachmentSize": {
+              "title": "Attachment Size (in bytes)",
+              "type": "number",
+              "required": false
+            }
+          }
+        }
       }
     },
     "attachmentToJson": {

lib/actions/jsonToXml.js

@@ -1,69 +1,69 @@
-/* eslint no-invalid-this: 0 no-console: 0 */
-const eioUtils = require('elasticio-node').messages;
+const { AttachmentProcessor } = require('@elastic.io/component-commons-library');
+const { messages } = require('elasticio-node');
 const xml2js = require('xml2js');
 const _ = require('lodash');
 
-const ERROR = 'Prop name is invalid for XML tag';
+const MB_TO_BYTES = 1024 * 1024;
+const MAX_FILE_SIZE = process.env.MAX_FILE_SIZE * MB_TO_BYTES || 10 * MB_TO_BYTES;
 
-/**
- * Checks whether property name is valid
- * @param {String} key - propName
- * @returns {Boolean} - valid prop or not
- */
-const propNameIsInvalid = (key) => /^\d/.test(key);
+module.exports.process = async function process(msg, cfg) {
+  const { input } = msg.body;
+  const { uploadToAttachment, excludeXmlHeader, headerStandalone } = cfg;
 
-/**
- * Checks whether object contains properties
- * that startsWith number
- * @see https://github.com/elasticio/xml-component/issues/1
- * @param {Object|Number|String} value
- * @param {String} key
- */
-function validateJsonPropNames(value, key) {
-  if (propNameIsInvalid(key)) {
-    const message = 'Can\'t create XML element from prop that starts with digit.'
-      + 'See XML naming rules https://www.w3schools.com/xml/xml_elements.asp';
-    throw new Error(`${ERROR}: ${key}. ${message}`);
-  }
-
-  if (!_.isPlainObject(value)) {
-    return;
-  }
+  this.logger.info('Message received.');
 
-  Object.keys(value).forEach((prop) => {
-    validateJsonPropNames(value[prop], prop);
-  });
-}
-
-/**
- * This method will be called from elastic.io platform providing following data
- *
- * @param msg incoming message object that contains ``body`` with payload
- * @param cfg configuration that is account information and configuration field values
- */
-function processAction(msg, cfg) {
-  this.logger.debug('Action started, message=%j cfg=%j', msg, cfg);
   const options = {
     trim: false,
     normalize: false,
     explicitArray: false,
     normalizeTags: false,
     attrkey: '_attr',
-    tagNameProcessors: [
-      (name) => name.replace(':', '-'),
-    ],
+    explicitRoot: false,
+    xmldec: {
+      standalone: headerStandalone,
+      encoding: 'UTF-8',
+    },
+    headless: excludeXmlHeader,
   };
   const builder = new xml2js.Builder(options);
 
-  const jsonToTransform = msg.body;
+  // Check to make sure that input has at most one key
+  // https://github.com/Leonidas-from-XIV/node-xml2js/issues/564
+  if (!_.isPlainObject(input) || Object.keys(input).length !== 1) {
+    throw new Error('Input must be an object with exactly one key.');
+  }
 
-  validateJsonPropNames(jsonToTransform);
+  const xmlString = builder.buildObject(input);
 
-  const result = builder.buildObject(jsonToTransform);
-  this.logger.debug('Successfully converted body to XML result=%s', result);
-  return eioUtils.newMessageWithBody({
-    xmlString: result,
-  });
-}
+  if (!uploadToAttachment) {
+    this.logger.info('Sending XML data in message.');
+    await this.emit('data', messages.newMessageWithBody({
+      xmlString,
+    }));
+    return;
+  }
+
+  const attachmentSize = Buffer.byteLength(xmlString);
+  if (attachmentSize > MAX_FILE_SIZE) {
+    throw new Error(`XML data is ${attachmentSize} bytes, and is too large to upload as an attachment. Max attachment size is ${MAX_FILE_SIZE} bytes`);
+  }
+  this.logger.info(`Will create XML attachment of size ${attachmentSize} byte(s)`);
 
-module.exports.process = processAction;
+  const attachmentProcessor = new AttachmentProcessor();
+  const uploadResult = await attachmentProcessor.uploadAttachment(xmlString);
+  const attachmentUrl = uploadResult.config.url;
+  this.logger.info(`Successfully created attachment at ${attachmentUrl}`);
+
+  const outboundMessage = messages.newEmptyMessage();
+  outboundMessage.attachments = {
+    'jsonToXml.xml': {
+      url: attachmentUrl,
+      size: attachmentSize,
+    },
+  };
+  outboundMessage.body = {
+    attachmentUrl,
+    attachmentSize,
+  };
+  await this.emit('data', outboundMessage);
+};

lib/actions/jsonToXmlOld.js

@@ -0,0 +1,69 @@
+/* eslint no-invalid-this: 0 no-console: 0 */
+const eioUtils = require('elasticio-node').messages;
+const xml2js = require('xml2js');
+const _ = require('lodash');
+
+const ERROR = 'Prop name is invalid for XML tag';
+
+/**
+ * Checks whether property name is valid
+ * @param {String} key - propName
+ * @returns {Boolean} - valid prop or not
+ */
+const propNameIsInvalid = (key) => /^\d/.test(key);
+
+/**
+ * Checks whether object contains properties
+ * that startsWith number
+ * @see https://github.com/elasticio/xml-component/issues/1
+ * @param {Object|Number|String} value
+ * @param {String} key
+ */
+function validateJsonPropNames(value, key) {
+  if (propNameIsInvalid(key)) {
+    const message = 'Can\'t create XML element from prop that starts with digit.'
+      + 'See XML naming rules https://www.w3schools.com/xml/xml_elements.asp';
+    throw new Error(`${ERROR}: ${key}. ${message}`);
+  }
+
+  if (!_.isPlainObject(value)) {
+    return;
+  }
+
+  Object.keys(value).forEach((prop) => {
+    validateJsonPropNames(value[prop], prop);
+  });
+}
+
+/**
+ * This method will be called from elastic.io platform providing following data
+ *
+ * @param msg incoming message object that contains ``body`` with payload
+ * @param cfg configuration that is account information and configuration field values
+ */
+function processAction(msg, cfg) {
+  this.logger.debug('Action started, message=%j cfg=%j', msg, cfg);
+  const options = {
+    trim: false,
+    normalize: false,
+    explicitArray: false,
+    normalizeTags: false,
+    attrkey: '_attr',
+    tagNameProcessors: [
+      (name) => name.replace(':', '-'),
+    ],
+  };
+  const builder = new xml2js.Builder(options);
+
+  const jsonToTransform = msg.body;
+
+  validateJsonPropNames(jsonToTransform);
+
+  const result = builder.buildObject(jsonToTransform);
+  this.logger.debug('Successfully converted body to XML result=%s', result);
+  return eioUtils.newMessageWithBody({
+    xmlString: result,
+  });
+}
+
+module.exports.process = processAction;