Mapping Kit

Mapping Kit is a library for mapping and transforming JSON payloads. It exposes a function that accepts a mapping configuration object and a payload object and outputs a mapped and transformed payload. A mapping configuration is a mixture of raw values (values that appear in the output payload as they appear in the mapping configuration) and directives, which can fetch and transform data from the input payload.

For example:

Mapping:

{
  "name": "Mr. Rogers",
  "neighborhood": { "@path": "$.properties.neighborhood" },
  "greeting": { "@template": "Won't you be my {{properties.noun}}?" }
}

Input:

{
  "type": "track",
  "event": "Sweater On",
  "context": {
    "library": {
      "name": "analytics.js",
      "version": "2.11.1"
    }
  },
  "properties": {
    "neighborhood": "Latrobe",
    "noun": "neighbor",
    "sweaterColor": "red"
  }
}

Output:

{
  "name": "Mr. Rogers",
  "neighborhood": "Latrobe",
  "greeting": "Won't you be my neighbor?"
}

Usage
Terms
Mixing raw values and directives
Validation
Options
- merge
Removing values from object
Directives
- @if
- @path
- @template
- @literal
- @arrayPath
- @case
- @replace
- @merge
- @transform
- @flatten
- @json
- @liquid
- @excludeWhenNull

Usage

import { transform } from '../mapping-kit'

const mapping = { '@path': '$.foo.bar' }
const input = { foo: { bar: 'Hello!' } }

const output = transform(mapping, input)
// => "Hello!"

Terms

In Mapping Kit, there are only two kinds of values: raw values and directives. Raw values can be any JSON value and Mapping Kit will return them in the output payload untouched:

42

"Hello, world!"

{ "foo": "bar" }

["product123", "product456"]

Directives are objects with a single @-prefixed key that tell Mapping Kit to fetch data from the input payload or transform some data:

{ "@path": "$.properties.name" }

{ "@template": "Hello there, {{traits.name}}" }

In this document, the act of converting a directive to its final raw value is called "resolving" the directive.

Mixing raw values and directives

Directives and raw values can be mixed to create complex mappings. For example:

Mapping:

{
  "action": "create",
  "userId": {
    "@path": "$.traits.email"
  },
  "userProperties": {
    "@path": "$.traits"
  }
}

Input:

{
  "traits": {
    "name": "Peter Gibbons",
    "email": "[email protected]",
    "plan": "premium",
    "logins": 5,
    "address": {
      "street": "6th St",
      "city": "San Francisco",
      "state": "CA",
      "postalCode": "94103",
      "country": "USA"
    }
  }
}


Output:

{
  "action": "create",
  "userId": "[email protected]",
  "userProperties": {
    "name": "Peter Gibbons",
    "email": "[email protected]",
    "plan": "premium",
    "logins": 5,
    "address": {
      "street": "6th St",
      "city": "San Francisco",
      "state": "CA",
      "postalCode": "94103",
      "country": "USA"
    }
  }
}

A directive may not, however, be mixed in at the same level as a raw value:

Invalid:

{
  "foo": "bar",
  "@path": "$.properties.biz"
}

Valid:

{
  "foo": "bar",
  "baz": { "@path": "$.properties.biz" }
}

And a directive may only have one @-prefixed directive in it:

Invalid:

{
  "@path": "$.foo.bar",
  "@template": "{{biz.baz}"
}

Valid:

{
  "foo": { "@path": "$.foo.bar" },
  "baz": {
    "@template": "{{biz.baz}}"
  }
}

Validation

Mapping configurations can be validated using JSON Schema. The test suite is a good source-of-truth for current implementation behavior.

Removing values from object

undefined values in objects are removed from the mapped output while null is not:

Input:

{
  "a": 1
}

Mappings:

{
  "foo": {
    "@path": "$.a"
  },
  "bar": {
    "@path": "$.b"
  },
  "baz": null
}
=>
{
  "foo": 1,
  "baz": null
}

Directives

@if

The @if directive resolves to different values based on a given conditional. It must have at least one conditional (see below) and one branch ("then" or "else").

The supported conditional values are:

"exists": If the given value is not undefined or null, the @if directive resolves to the "then" value. Otherwise, the "else" value is used.
"blank": If the given value is not undefined, not null, and does not loosely equal an empty string ('') (i.e., is not blank), the @if directive resolves to the "then" value. Otherwise, the "else" value is used. Because this check uses JavaScript loose equality semantics, values such as 0 and false are currently treated as blank and will use the "else" branch.

Input:

{
  "a": "cool",
  "b": true
}

Mappings:

{
  "@if": {
    "exists": { "@path": "$.a" },
    "then": "yep",
    "else": "nope"
  }
}
=>
"yep"

{
  "@if": {
    "exists": { "@path": "$.nope" },
    "then": "yep",
    "else": "nope"
  }
}
=>
"nope"

{
  "@if": {
    "blank": { "@path": "$.c" },
    "then": "yep",
    "else": "nope"
  }
}
=>
"nope"

If "then" or "else" are not defined and the conditional indicates that their value should be used, the field will not appear in the resolved output. This is useful for including a field only if it (or some other field) exists:

Input:

{
  "a": "cool"
}

Mappings:

{
  "foo-exists": {
    "@if": {
      "exists": { "@path": "$.foo" },
      "then": true
    }
  }
}
=>
{}

{
  "a": {
    "@if": {
      "exists": { "@path": "$.oops" },
      "then": { "@path": "$.a" }
    }
  }
}
=>
{}

@path

The @path directive resolves to the value at the given path. @path supports basic dot notation. Like JSONPath, you can include or omit the leading $.

Input:

{
  "foo": {
    "bar": 42,
    "baz": [{ "num": 1 }, { "num": 2 }]
  },
  "hello": "world"
}

Mappings:

{ "@path": "$.hello" } => "world"

{ "@path": "$.foo.bar" } => 42

{ "@path": "$.foo.baz[0].num" } => 1

@template

The @template directive resolves to a string replacing curly brace {{}} placeholders.

Input:

{
  "traits": {
    "name": "Mr. Rogers"
  },
  "userId": "abc123"
}

Mappings:

{ "@template": "Hello, {{traits.name}}!" } => "Hello, Mr. Rogers!"

{ "@template": "Hello, {{traits.fullName}}!" } => "Hello, !"

{ "@template": "{{traits.name}} ({{userId}})" } => "Mr.Rogers (abc123)"

@literal

The @literal directive resolves to the value with no modification. This is needed primarily to work around literal values being interpreted incorrectly as invalid templates.

Input:
n/a

Mappings:
{ "@literal": true } => true

@arrayPath

The @arrayPath directive resolves a value at a given path (much like @path), but allows you to specify the shape of each item in the resulting array. You can use directives for each key in the given shape, relative to the root object.

Typically, the root object is expected to be an array, which will be iterated to produce the resulting array from the specified item shape. It is not required that the root object be an array.

For the item shape to be respected, the root object must be either an array of plain objects OR a singular plain object. If the root object is a singular plain object, it will be arrified into an array of 1.

Input:

{
  "properties": {
    "products": [{ "productId": 1 }, { "productId": 2 }]
  }
}

Mapping:

{
  "@arrayPath": ["$.properties.products"]
}

Result:

[
  {
    "productId": 1
  },
  {
    "productId": 2
  }
]

Mappings with item shape:

{
  "@arrayPath": [
    "$.properties.products",
    {
      "some_other_key": { "@path": "$.productId" }
    }
  ]
}

Result:

[
  {
    "some_other_key": 1
  },
  {
    "some_other_key": 2
  }
]

@case

The @case directive changes a string value at a given path to its respective lowercase() or uppercase() representation.

While this directive does expect a string value at the given path, it can handle other types and will simply resolve to whatever is found if it is not a string.

Input:

{
  "properties": {
    "message": "THIS STRING IS IN ALL CAPS"
  }
}

Mapping:

{
  "@case": {
    "operator": "lower",
    "value": { "@path": "$.properties.message" }
  }
}

Result:

"this is a string in all caps"

@replace

The @replace directive replaces occurrences of a pattern in a string with a replacement string. The value field specifies the input string (may be a directive or raw value). The pattern field is required; replacement defaults to an empty string if omitted.

By default, replacement is global (all occurrences) and case-sensitive. Use global: false to replace only the first occurrence, and ignorecase: true for case-insensitive matching.

Input:

{
  "a": "cool-story"
}

Mappings:

{
  "@replace": {
    "value": { "@path": "$.a" },
    "pattern": "-",
    "replacement": ""
  }
}
=>
"coolstory"

{
  "@replace": {
    "value": { "@path": "$.a" },
    "pattern": "-",
    "replacement": "nice"
  }
}
=>
"coolnicestory"

Input:

{
  "a": "cWWl-story-ww"
}

Mappings:

{
  "@replace": {
    "value": { "@path": "$.a" },
    "pattern": "WW",
    "replacement": "oo",
    "ignorecase": false
  }
}
=>
"cool-story-ww"

Input:

{
  "a": "just-the-first"
}

Mappings:

{
  "@replace": {
    "value": { "@path": "$.a" },
    "pattern": "-",
    "replacement": "@",
    "global": false
  }
}
=>
"just@the-first"

A second pattern/replacement pair (pattern2 / replacement2) can be provided to apply a second substitution on the result of the first:

Input:

{
  "a": "something-great!"
}

Mapping:

{
  "@replace": {
    "value": { "@path": "$.a" },
    "pattern": "-",
    "replacement": " ",
    "pattern2": "great",
    "replacement2": "awesome"
  }
}

Output:

"something awesome!"

@merge

The @merge directive resolves a list of objects to a single object. It accepts a list of one or more objects (either raw objects or directives that resolve to objects), and a direction that determines how overwrites will be applied for duplicate keys. The resolved object is built by combining each object in turn, moving in the specified direction, overwriting any duplicate keys.

Input:

{
  "traits": {
    "name": "Mr. Rogers",
    "greeting": "Neighbor",
    "neighborhood": "Latrobe"

  },
  "properties": {
    "neighborhood": "Make Believe"
  }
}

Mappings:

{
  "@merge": {
    "objects": [
      { "@path": "traits" },
      { "@path": "properties" }
    ],
    "direction": "right"
  }
}
=>
{
  "name": "Mr. Rogers",
  "greeting": "Neighbor",
  "neighborhood": "Make Believe"
}

{
  "@merge": {
    "objects": [
      { "@path": "properties" },
      { "@path": "traits" }
    ],
    "direction": "right"
  }
}
=>
{
  "name": "Mr. Rogers",
  "greeting": "Neighbor",
  "neighborhood": "Latrobe"
}

The @merge directive is especially useful for providing default values:

Input:

{
  "traits": {
    "name": "Mr. Rogers"
  }
}

Mapping:

{
  "@merge": {
    "objects": [
      {
        "name": "Missing name",
        "neighborhood": "Missing neighborhood"
      },
      { "@path": "traits" }
    ],
    "direction": "right"
  }
}

Output:

{
  "name": "Mr. Rogers",
  "neighborhood": "Missing neighborhood"
}

@transform

The @transform directive allows you to operate on the result of a mapping-kit transformation. It accepts an apply parameter, which is the mapping to apply to the original payload, and a mapping parameter, which will be run on the resulting payload. The @transform directive is useful when you need to run mappings in sequence.

Input:

{
  "a": 1,
  "b": 2
}

Mappings:
{
  "@transform": {
    "apply": {
      "foo": {
        "@path": "$.a"
      }
    },
    "mapping": {
      "newValue": { "@path": "$.foo" }
    }
  }
}
=>
{
  "newValue": 1
}

@flatten

The @flatten directive flattens a nested object into a single-level object using a separator string for the keys. The value field specifies the object to flatten and the separator field (required) specifies the string used to join nested keys. Set omitArrays: true to leave array values un-flattened.

Input:

{
  "foo": {
    "bar": "baz",
    "aces": { "a": 1, "b": 2 }
  }
}

Mapping:

{
  "@flatten": {
    "value": { "@path": "$.foo" },
    "separator": "."
  }
}

Output:

{
  "bar": "baz",
  "aces.a": 1,
  "aces.b": 2
}

With omitArrays: true, array values are preserved as-is instead of being flattened:

Input:

{
  "foo": {
    "bar": "baz",
    "tags": [1, 2]
  }
}

Mapping:

{
  "@flatten": {
    "value": { "@path": "$.foo" },
    "separator": ".",
    "omitArrays": true
  }
}

Output:

{
  "bar": "baz",
  "tags": [1, 2]
}

@json

The @json directive encodes a value to a JSON string or decodes a JSON string to a value. The mode field must be either "encode" or "decode", and the value field specifies the input.

Input:

{
  "foo": { "bar": "baz" }
}

Mappings:

{ "@json": { "mode": "encode", "value": { "@path": "$.foo" } } }
=>
"{\"bar\":\"baz\"}"

Input:

{
  "foo": "[\"bar\",\"baz\"]"
}

Mappings:

{ "@json": { "mode": "decode", "value": { "@path": "$.foo" } } }
=>
["bar", "baz"]

If mode is "decode" and the value is not valid JSON, the original string is returned unchanged.

@liquid

The @liquid directive evaluates a Liquid template string against the input payload and returns the rendered result. The directive value must be a string of at most 1000 characters.

Input:

{
  "properties": {
    "name": "SpongeBob",
    "world": "Bikini Bottom"
  }
}

Mappings:

{ "@liquid": "Hello, {{ properties.name }}!" }
=>
"Hello, SpongeBob!"

{ "@liquid": "{% if properties.world == \"Bikini Bottom\" %}Under the sea{% endif %}" }
=>
"Under the sea"

{ "@liquid": "{{ properties.name | upcase }}" }
=>
"SPONGEBOB"

Restrictions: The following Liquid tags are disabled: case, for, include, layout, render, tablerow. Several array-manipulation filters (e.g. sort, map, reverse) are also disabled.

@excludeWhenNull

The @excludeWhenNull directive will exclude the field from the output if the resolved value is null.

Input:

{
  "a": null,
  "b": "hello"
}

Mappings:

{
  "a": {
    "@excludeWhenNull": {
      "@path": "$.a"
    }
  },
  "b": {
    "@excludeWhenNull": {
      "@path": "$.b"
    }
  }
}
=>
{
  "b": "hello"
}

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Mapping Kit

Table of contents

Usage

Terms

Mixing raw values and directives

Validation

Removing values from object

Directives

@if

@path

@template

@literal

@arrayPath

@case

@replace

@merge

@transform

@flatten

@json

@liquid

@excludeWhenNull

FilesExpand file tree

README.md

Latest commit

History

README.md

File metadata and controls

Mapping Kit

Table of contents

Usage

Terms

Mixing raw values and directives

Validation

Removing values from object

Directives

@if

@path

@template

@literal

@arrayPath

@case

@replace

@merge

@transform

@flatten

@json

@liquid

@excludeWhenNull