Guide for building a .json-based language
Retrieve a pre-built language to understand how they are constructed.
If the language is given in .csv format, which is more straightforward and can be built by a business analyst, Translator converts the .csv file into the more technical .json format.
JSONPath syntax
More info and examples can be found here
Also, you can specify constant values instead of JSONPath, for example:
{
"name": "new_language",
"mapping": {
"$.value": "Some constant value"
}
}
Sometimes, you might need to set dyanmic JSONPath, that will be generated based on data, that you have passed,
for this purpose you can use {{ <jsonPath> }} syntax, so <jsonPath> will be executed first against your data,
f.e you create language like this:
{
"mapping": {
"$.foo": "$.data.{{ $.target_field }}"
}
}
So when translation begins, $.target_field will be queried, and result of this query will
replace {{ $.target_field }}.
For next input:
{
"target_field": "bar",
"data": {
"bar": "baz"
}
}
You will get next result:
{
"foo": "baz"
}
You may also need to set up a mapping to only happen in a single direction. For example, only when translating to staircase or from staircase.
In order to do this place the mapping into the appropriate section in "single_direction_mappings". The mapping itself should look the same.
{
"single_direction_mappings": {
"to_staircase": {
"$.foo_only_to": "$.data.{{ $.target_field }}"
},
"from_staircase": {
"$.foo_only_from": "$.data.{{ $.target_field }}"
}
}
}
Lastly you may want to pre-process your data before translation or post-process after translation. This is possible by setting up hooks.
In order to use hooks you will need to create endpoints inside of your staircase environment and then link the paths to those endpoints in the mapping hooks property.
For example, if your environment is you will set up an endpoint at and then set
mapping_hooks to {"pre": "my/pre/hook"}. The domain will be added automatically.
The pre hook will be sent the exact request payload that was sent to the "/translate" endpoint.
The post hook will be sent the translated data as json with the result under the key "translation_result".
{
"mapping_hooks": {
"pre": "my/pre/hook",
"post": "my/post/hook"
}
}
Also, we have some predefined utils that we can use in our mappings, just using $utils.<util_name>(*args)
List of currently supported utils:
Join many elements to one string
$utils.join([*elements], character)
Note: Constants are items not starting with $ - are joined as-is.
Return value: Joined string
Example:
{
"name": "util_language",
"mapping": {
"$utils.join(['$.address.line_text', '$.address.city', '$.address.state_code', '$.postal_code', 'USA'], ' ')": "$.address"
}
}
Split one element to many
$utils.split(element, separator)
Return value: array of string
Example:
{
"name": "util_language",
"mapping": {
"$.deal_sets[0].collaterals[0].address.line_text": "$utils.split('$.address', ' ')[0]",
"$.deal_sets[0].collaterals[0].address.city": "$utils.split('$.address', ' ')[1]",
"$.deal_sets[0].collaterals[0].address.state_code": "$utils.split('$.address', ' ')[2]",
"$.deal_sets[0].collaterals[0].address.postal_code": "$utils.split('$.address', ' ')[3]"
}
}
Slice String
$utils.slice(element, beginIndex, endIndex)
Use first element that has a value defined
$utils.first_defined(elements)
Return value: value of first defined element or null
Example:
{
"name": "util_language",
"mapping": {
"$.name": "$utils.first_defined(['$.official_name', '$.first_name', '$.nickname'])",
}
}
String
$utils.if_value_in(element, values)
Return value: value of element if it's deemed valid or null
Example:
{
"name": "util_language",
"mapping": {
"$.deal_type": "$utils.if_value_in('$.type', ['type_a', 'type_b', 'type_c'])"
}
}
Transform values
Specify values_mapping to transform values. Example:
Use the keyword default for no match.
Note: Feature is useful for one-way mapping - use caution when reverse-mapping from staircase back to language.
{
"name": "values_mapping_language",
"values_mapping": [
{
"element": "$.deal_sets[0].collaterals[0].property_valuations[0].avms[0].low_value_range",
"mapping": [
{"staircase_value": "HelloWorld", "language_value": "FooBar"},
{"staircase_value": "BarBaz", "language_value": "WorldHello"},
{"staircase_value": "CatchAll", "language_value": "_default_"},
],
}
]
}
Where element is JSONPath in staircase language, and mapping is list of values mapping for specified element
Date handling
Specify normalize_date to transform dates into standard yyyy-MM-dd format
or denormalize_date to transform dates from this format.
{
"mapping": {},
"single_direction_mappings": {
"to_staircase": {
"$.borrower_birth_date": "$utils.normalize_date('$.birthday', 'yyyy/MM/dd')"
},
"from_staircase": {
"$utils.denormalize_date('$.borrower_birth_date', 'yyyy/MM/dd')": "$.birthday"
}
}
}
Date to be in in standard ISO:
year: yyyy, month: MM, day: dd, hour: hh, hour24:HH, minute: mm, second: ss, ms: SSS etc.
If it is a timestamp, specify "epoch" for date_format argument
If it should parsed from any format (for normalize_date), specify "any" for date_format argument
Letters T, Z and UTC to be stripped out prior to conversion.
Example: yyyy-MM-dd HH:mm:ss.SSS for 2019-01-02T15:26:02.123Z UTC
DateTime handling
Specify normalize_datetime to transform dates/times into standard yyyy-MM-ddTHH.mm.ss.SSS-0X:00 format
or denormalize_datetime to transform dates/times from this format.
{
"mapping": {},
"single_direction_mappings": {
"to_staircase": {
"$.sample_datetime_field": "$utils.normalize_datetime('$.datetime_in', 'yyyy-MM-dd HH:mm:ss.SSS', 'UTC', 'ET')",
"$.sample_datetime_field2": "$utils.normalize_datetime('$.datetime_in_epoch', 'epoch', 'UTC', 'ET')"
},
"from_staircase": {
"$utils.denormalize_datetime('$.sample_datetime_field', 'yyyy-MM-dd HH:mm:ss.SSS', 'ET', 'UTC')": "$.datetime_in"
"$utils.denormalize_datetime('$.sample_datetime_field2', 'epoch', 'ET', 'UTC')": "$.datetime_in_epoch"
}
}
}
$utils.normalize_datetime(target_field, input_datetime_format, fromTimezone, toTimezone)
$utils.denormalize_datetime(target_field, output_datetime_format, fromTimezone, toTimezone)
Date to be in in standard ISO or epoch
If it should parsed from any format, specify "any" for input_datetime_format/output_datetime_format argument
fromTimezone and toTimezone must be specified as "UTC","ET","CT","PT"
Daylight Savings Time is calcuated based on target date, and will be returned based on target date
year: yyyy, month: MM, day: dd, hour: hh, hour24:HH, minute: mm, second: ss, ms: SSS etc.
Output date/time in ISO standard: yyyy-MM-ddTHH:mm:ss.SSS-0X:00
X = offset from UTC (i.e. 5 for EST / 4 for EDT)
Example: ('$.datetime_in', 'yyyy-MM-dd HH:mm:ss.SSS','UTC','ET') - converts from UTC to ET
Example: ($.datetime_in_epoch', 'epoch','UTC','ET') - converts from epoch - in UTC by definition - to ET
Parse Float
Specify parse_float to transform floats into standard format. Second value - number of digits past decimal.
{
"name": "util_language",
"mapping": {
"$.float_value_result": "$utils.parse_float('$.float_value_input', '2')"
},
}
to use with wildcards:
{
"name": "util_language",
"mapping": {
"$.deal_sets.deal_set[0].deals.deal[0].assets.asset[*].cash_or_market_value_amount": "$utils.map(\'$.report.items[0].accounts[*].balances.current\',\'$utils.parse_float(item,2)\')"
},
}
note: item is a keyword - and must be used as-is
Parses float and trims digits past decimal to integer given - rounds as appropriate.
Incoming float values can come in as either string ("234.556") or float (234.556).
Outgoing values are numeric only. JSON does not differentiate between integer and float, as such, trailing zeros are trimmed.
If a float is un-translatable, will be returned as null/None.
Examples:
Sum Numbers
Specify sum_numbers to sum numbers and truncate/round as above.
{
"name": "util_language",
"mapping": {
"$.sum_numbers_result": "$utils.sum_numbers('$.sum_number_in';'$.sum_number_in2';'sum_number_in3', '2')"
},
}
Items are truncated/rounded prior to summation. Final sum is also truncated/rounded as specified.
Multiply Numbers
Specify multiply_numbers to multiply numbers and truncate/round as above.
{
"name": "util_language",
"mapping": {
"$.multiply_numbers_result": "$utils.multiply_numbers('$.number_in';12, '2')"
},
}
Items are truncated/rounded prior to summation. Final sum is also truncated/rounded as specified.
Parse location
You can parse street address and street intersection for the United States.
Util knows about directional prefixes and suffixes, fractional building numbers, building units,
grid-based addresses (such as those used in parts of Utah), 5 and 9 digit ZIP codes, and all of the official
USPS abbreviations for street types, state names and secondary unit designators.
$utils.parse_location(element)
Return value:
{
"title": "Address"
"type": "object",
"properties": {
"number": {
"type": "string",
"description": "House or street number."
},
"prefix": {
"type": "string",
"description": "Directional prefix for the street, such as N, NE, E, etc. A given prefix should be one to two characters long."
},
"street": {
"type": "string",
"description": "Name of the street, without directional or type qualifiers."
},
"type": {
"type": "string",
"description": "Abbreviated street type, e.g. Rd, St, Ave, etc. See the USPS official type abbreviations at for a list of abbreviations used."
},
"suffix": {
"type": "string",
"description": "Directional suffix for the street, as above."
},
"city": {
"type": "string",
"description": "Name of the city, town, or other locale that the address is situated in."
},
"state": {
"type": "string",
"description": "The state which the address is situated in, given as its two-letter postal abbreviation. for a list of abbreviations used."
},
"zip": {
"type": "string",
"description": "Five digit ZIP postal code for the address, including leading zero, if needed."
},
"sec_unit_type": {
"type": "string",
"description": "If the address includes a Secondary Unit Designator, such as a room, suite or appartment, the sec_unit_type field will indicate the type of unit."
},
"sec_unit_num": {
"type": "string",
"description": "If the address includes a Secondary Unit Designator, such as a room, suite or appartment, the sec_unit_num field will indicate the number of the unit (which may not be numeric)."
}
}
}
None of his properties is guaranteed and depends on adress that is provided
Example:
{
"mapping": {
"$.number": "$utils.parse_location('$.address')['number']",
"$.city_state": "$utils.join([$utils.parse_location('$.address')['city'], $utils.parse_location('$.address')['state']], ' ')"
},
"name": "address_location"
}
SetNull
Specify set_null to hardcode a null to the field - will show in JSON as 'None'. Cannot be used with wildcards.
Example:
{
"name": "util_language",
"mapping": {
"$.null_value_to": "set_null",
"set_null": "$.null_value_from"
},
}
Generate ID
You have ability to generate ULID by using $utlis.ulid
$utils.ulid(element, many)
Example:
Language:
{
"mapping": {
"$.elements[*].id": "$utils.ulid('$.input[*].foo')",
"$.data[*].id": "$utils.ulid('$.input[*].bar')"
}
}
Input:
{
"foo": [1, 2, "hello"],
"bar": [1, 2, "hello"]
}
Output:
{
"elements": [
{"id": "01F4EYZ82K7M46DQAZKZT4Y7QT"},
{"id": "01F4EYZVVM3BTAYHBHV1AJDQZC"},
{"id": "01F4EZ05VWZZ3BM3EFP6FYQ28H"}
],
"data": [
{"id": "01F4EYZ82K7M46DQAZKZT4Y7QT"},
{"id": "01F4EYZVVM3BTAYHBHV1AJDQZC"},
{"id": "01F4EZ05VWZZ3BM3EFP6FYQ28H"}
]
}
As you can see, we generated id's, but we have same id's in elements and data arrays, because input contains
same values in objects by provided in language JSONPathes
Frame
If your languge is in application/ld+json you can apply frame algorithm
to your payload, and query data from it
$utils.frame(element, frameObject)
Language example:
{
"mapping": {
"$.book_title": "$utils.frame('$[\'ex:contains\'][\'dc11:title\']', { '@type': 'ex:Library', 'ex:contains': { '@type': 'ex:Book' } })"
},
"jsonld_context": {
"dc11": "",
"ex": "",
"xsd": "",
"ex:contains": {
"@type": "@id"
}
},
"content_type": "application/ld+json"
}
Input:
{
"@graph": [
{
"@id": "",
"@type": "ex:Library",
"ex:contains": ""
},
{
"@id": "",
"@type": "ex:Book",
"dc11:creator": "Plato",
"dc11:title": "The Republic",
"ex:contains": ""
},
{
"@id": "",
"@type": "ex:Chapter",
"dc11:description": "An introductory chapter on The Republic.",
"dc11:title": "The Introduction"
}
]
}
Output:
{
"book_title": "The Republic"
}
Convert string case
$utils.convert_case(element, case)
Return value: Converted string
Example:
{
"name": "util_language",
"mapping": {
"$utils.convert_case('$.address.state_code', 'upper')": "$.state"
}
}
Extract text with regex
$utils.regex_extract(element, regex)
Return value: Extracted string
Example:
{
"name": "regex_language",
"mapping": {
"$.income": "$utils.regex_extract('$.borrower_income', '\\d+')"
}
}
Set current date
$utils.set_current_date
Example:
{
"name": "current_date_language",
"mapping": {
"$.current_date": "$utils.set_current_date"
}
}
Return value: string with current date in normalized format "yyyy-MM-dd" by 'America/New_York' timezone.
Format field
$utils.format(element, type)
Return value: Formatted string
Example:
{
"name": "format-ssn",
"mapping": {
"$.ssn": "$utils.format('$.incoming_ssn', 'ssn')"
}
}
Relationship
$utils.relationship(element, query, key, relName)
Map XLINK relationships (1:1 and 1:M) used in XML-like schemas
The following is an example mapping (note: mappings for this operation are single-direction):
{
"single_direction_mappings": {
"to_staircase": {
"$.relationships.relationship[*]|ASSET_LIABILITY-$.assets[?(@.@type=='real_estate_owned')]-with_liability[*]": "$utils.relationship('$.assets[?(@.@type==\'real_estate_owned\')]', 'with_liability[*]', '@id', 'ASSET_LIABILITY')"
}
}
}
The above mapping uses the JSON Path query $.assets[?(@.@type==\'real_estate_owned\')] to get the relationship's
left-side (or from-) entity. The value in its @id field will be use to mark this first entity in the created
output. The JSON Path with_liability[*] marks the value of right-side (or to-) entity in the relationship.
The relationship's name is
{
"people": [
{
"@id": "BORROWER_1",
"@type": "borrower",
"owns_asset": [
"ASSET_1",
"ASSET_2"
]
}
],
"assets": [
{
"@id": "ASSET_1",
"@type": "retirement_account",
"has_account_identifier": {
"has_value": "4444"
},
"has_asset_description": {
"has_value": "Roth IRA"
},
"has_market_value_amount": {
"has_value": 923.27
}
},
{
"@id": "ASSET_2",
"@type": "real_estate_owned",
"has_market_value_amount": {
"has_value": 25632.87
},
"with_liability": [
"LIABILITY_1"
]
}
],
"liabilities": [
{
"@id": "LIABILITY_1",
"@type": "liability",
"has_liability_payment_amount": {
"has_value": 923.27
}
}
]
}
When the language mappings are applied to the above example input, the following output is created:
{
"relationships": {
"relationship": [
{
"relationship_link": "ASSET_LIABILITY",
"link_from": "ASSET_2",
"link_to": "LIABILITY_1"
}
]
}
}
