# Connection
# Connection
Vendor connectors declared as state machines: one repository per vendor endpoint, each versioned, tested and monitored on its own.
A connector is a JSON document, not a service. A flow declares states — an HTTP call in JSON or XML, an inline transformation, a branch — with input selectors pulling values out of the request payload and the credential set in scope, and a callback address for asynchronous completion.
One repository per vendor endpoint, with a test for each status code that endpoint can return. Webhook and browser-widget variants get their own repositories on the same shape.
## How it works
Declaring the transport rather than coding it makes a vendor integration reviewable by someone who does not read the language it would otherwise be written in, and it makes the unit small enough to version honestly: one endpoint changes, one repository changes.
Credentials are profiles, selected at run time. A vendor's stored credential set holds a default plus named profiles, and the first state of the flow picks the profile the request asked for and merges it over the default. The same connector therefore serves a lender using its own vendor account and a lender using the shared one, without a second flow.
Overhead measurement excludes the vendor's own elapsed time. Four connector API identifiers are excluded by name — the three generic outbound request types and the webhook notification — because their duration measures the vendor rather than the platform. Cold starts are reported separately, as invocation code `7000` with status `REQUEST_MADE`.
## Operations
### Transformations
`PUT` `/transformations/html-to-pdf`
#### HTML to PDF
`transform_html_to_pdf`
Performs synchronous format transformation. Transforms HTML to PDF.
Downloads data from the URL and uploads the transformed data to PDF.
Upload is sent with "PUT" with HTTP method.
An example of how to integrate the conversion in the flow.
```
{
"version": "2",
"authorization": {
"mode": "pass_through",
"auth_type": "custom",
"credentials": "dynamic"
},
"definition": {
"StartAt": "DownloadHTML",
"States": {
"DownloadHTML": {
"Next": "InstantiateBlob",
"Type": "Task",
"ResourceDefinition": {
"Type": "HTTP_ANY",
"HttpMethod": "GET",
"Headers": {
"Accept": "text/html"
},
"VendorEndpointUrlPath": "$.request_payload.source_url",
"SaveResponseToFile": true,
"Extract": {
"Save": {
"download_from_url": "$.response_file_url"
}
}
}
},
"InstantiateBlob": {
"Type": "Task",
"Next": "SaveAsPDF",
"ResourceDefinition": {
"Type": "HTTP_JSON",
"HttpMethod": "POST",
"VendorEndpointUrl": "",
"Headers": {
"Content-Type": "application/json",
"x-api-key": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
},
"PathParametersMapping": {
"request_host": "$.request_host"
},
"RequestPayload": {
"extension": ".pdf",
"presign_url_ttl": 3600
},
"Extract": {
"Save": {
"blob_id": "$.blob_id",
"upload_to_url": "$.presigned_urls.upload.url"
}
}
}
},
"SaveAsPDF": {
"Type": "Task",
"Next": "ComposeResponse",
"ResourceDefinition": {
"Type": "HTTP_JSON",
"HttpMethod": "PUT",
"Headers": {
"Content-Type": "application/json",
"x-api-key": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
},
"RequestPayload": {
"urls": {
"download_from_url": "$.download_from_url",
"upload_to_url": "$.upload_to_url"
}
},
"VendorEndpointUrl": "",
"PathParametersMapping": {
"request_host": "$.request_host"
}
}
},
"ComposeResponse": {
"End": true,
"Type": "Task",
"ResourceDefinition": {
"Type": "CODE",
"Source": "loopback = lambda response: response",
"Runtime": "python3.8",
"InputDataSelector": {
"blob_id": "$.blob_id"
},
"SaveOutputTo": "response_payload",
"FunctionName": "loopback"
}
}
}
}
}
```
#### Note
Subscribe to `connection-html-to-pdf` component to access this feature.
Show the rest
##### Request
application/json Copy
```
{
"urls": {
"download_from_url": "https://staircase.co/html-is-downloaded-from-here",
"upload_to_url": "https://staircase.co/pdf-is-upload-to-here"
}
}
```
##### Response
400403404422
application/json Copy Bad request syntax. Request payload does not exist or malformed.
```
{
"error": {
"message": "Bad Request",
"reason": [
"\"\" is less than 1 character."
]
}
}
```
application/json Copy Request is forbidden, API key is not valid
```
{
"message": "Your API key is not valid, please refresh it."
}
```
application/json Copy Resource not found. Resource you were looking for was not found.
```
{
"error": {
"message": "Not Found",
"reason": "Nothing matches the given URI."
}
}
```
application/json Copy A 422 status code occurs when a request is well-formed, however, due to semantic errors it is unable to be processed.
```
{
"error": {
"message": "Unprocessable entity.",
"reason": "Domain name could not be validated."
}
}
```
##### Request body`application/json`
1 fields
| Field | Type | Description |
| --- | --- | --- |
| `urls`required | `object` | Container for download and upload URLs. |
| `download_from_url`required | `string (uri)` | The URL from where connector pulls the HTML data. Must have `https://` in the URLs part. |
| `upload_to_url`required | `string (uri)` | The URL where connector puts the PDF data. Must have `https://` in the URLs scheme. |
##### Response `403``application/json`
1 fields
Request is forbidden, API key is not valid
| Field | Type | Description |
| --- | --- | --- |
| `message` | `string` | Forbidden error messageExample `Your API key is not valid, please refresh it.` |
##### Other responses
`204``400``404``422`
`PUT` `/transformations/xml-to-json`
#### XML to JSON
`transform_xml_to_json`
Performs synchronous format transformation. Transforms XML to JSON.
Downloads data from the input URL and uploads the transformed data to output URL.
Upload is sent with "PUT" HTTP method.
An example of how to integrate the transformation in the flow definition.
```
{
"version": "2",
"authorization": {
"mode": "pass_through",
"auth_type": "custom",
"credentials": "dynamic"
},
"definition": {
"StartAt": "DownloadXML",
"States": {
"DownloadXML": {
"Next": "InstantiateBlob",
"Type": "Task",
"ResourceDefinition": {
"Type": "HTTP_ANY",
"HttpMethod": "GET",
"Headers": {
"Content-Type": "$.request_payload.content_type"
},
"VendorEndpointUrlPath": "$.request_payload.source_url",
"SaveResponseToFile": true,
"Extract": {
"Save": {
"download_from_url": "$.response_file_url"
}
}
}
},
"InstantiateBlob": {
"Type": "Task",
"Next": "SaveAsJSON",
"ResourceDefinition": {
"Type": "HTTP_JSON",
"HttpMethod": "POST",
"VendorEndpointUrl": "",
"Headers": {
"Content-Type": "application/json",
"x-api-key": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
},
"PathParametersMapping": {
"request_host": "$.request_host"
},
"RequestPayload": {
"extension": ".json",
"presign_url_ttl": 3600
},
"Extract": {
"Save": {
"blob_id": "$.blob_id",
"upload_to_url": "$.presigned_urls.upload.url"
}
}
}
},
"SaveAsJSON": {
"Type": "Task",
"Next": "ComposeResponse",
"ResourceDefinition": {
"Type": "HTTP_JSON",
"HttpMethod": "PUT",
"Headers": {
"Content-Type": "application/json",
"x-api-key": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
},
"RequestPayload": {
"transaction_id": "$.transaction_id",
"urls": {
"download_from_url": "$.download_from_url",
"upload_to_url": "$.upload_to_url"
}
},
"VendorEndpointUrl": "",
"PathParametersMapping": {
"request_host": "$.request_host"
}
}
},
"ComposeResponse": {
"End": true,
"Type": "Task",
"ResourceDefinition": {
"Type": "CODE",
"Source": "loopback = lambda response: response",
"Runtime": "python3.8",
"InputDataSelector": {
"blob_id": "$.blob_id"
},
"SaveOutputTo": "response_payload",
"FunctionName": "loopback"
}
}
}
}
}
```
#### Transformation Modes
User can specify either `data` to be transformed directly or `urls` from which the data will be pulled and uploaded to (only one can be used a a time).
Show the rest
#### Warning
If `data` mode is used but the supplied XML is too large, request will not be evaluated. Consider switching to `url` based transformation.
#### Transformation Configuration
- `force_values_into_array` (Optional) Boolean or array of strings. Forces array creation for every child in an XML document at any given depth if boolean is used. If array of strings is supplied, only the nodes named by those strings will be forced to be arrays. By default, `force_values_into_array` is `true`.
- `attribute_prefix` (Optional) String. Prefix for tag attributes that will be added to differentiate attributes from text. By default, `attribute_prefix` is `"@"`. User can set it to empty string to not append prefix to attributes. Access pattern to a free text of an element will depend on the format of the tag.
- If an element has at least one attribute, free text will be available under `#text`
- If an element has no attributes defined, free text will be the next following element
Transformation Samples Input XML:
```
It's me, Connection!
eyJwaem.ei2u.2e03rinmALDd
```
Suppose `attribute_prefix` is `_` and `force_values_into_array` is `true`
```
{
"dialogs": [
{
"_filterTopic": "E2P",
"message": [
{
"_receiver": "124191486",
"_sender": "1",
"text": [
{
"_version": "0.1.1",
"_engine": "CHTML",
"entities": [
{
"entity": [
{
"_offset": "9",
"_length": "9",
"_transform": "bold"
},
{
"_offset": "5",
"_length": "2",
"_transform": "link",
"_link": ""
}
]
}
],
"raw": [
"It's me, Connection!"
]
}
],
"#text": [
"eyJwaem.ei2u.2e03rinmALDd"
]
}
]
}
]
}
```
Changing `force_values_into_array` to `false`
```
{
"dialogs": {
"_filterTopic": "E2P",
"message": {
"_receiver": "124191486",
"_sender": "1",
"text": {
"_version": "0.1.1",
"_engine": "CHTML",
"entities": {
"entity": [
{
"_offset": "9",
"_length": "9",
"_transform": "bold"
},
{
"_offset": "5",
"_length": "2",
"_transform": "link",
"_link": ""
}
]
},
"raw": "It's me, Connection!"
},
"#text": "eyJwaem.ei2u.2e03rinmALDd"
}
}
}
```
Specification for interpreting XML as JSON is partially described here. However, parameters like `force_values_into_array` and `attribute_prefix` can be controlled.
##### Note
Subscribe to `connection-xml-to-json` component to access this feature.
##### Request
URL based transformationDirect transformationSpecific array nodes
application/json Copy
```
{
"transaction_id": "01F7S0N7CQ31DZQYVSVPN66GNK",
"urls": {
"download_from_url": "https://staircase.co/xml-is-downloaded-from-here",
"upload_to_url": "https://staircase.co/json-is-uploaded-here"
},
"configuration": {
"attribute_prefix": "__"
},
"product_configuration_id": "345a814d-432e-4801-96b4-74dc03ef1047"
}
```
application/json Copy
```
{
"transaction_id": "01F7S0N7CQ31DZQYVSVPN66GNK",
"data": "Staircase",
"configuration": {
"attribute_prefix": "__"
},
"product_configuration_id": "345a814d-432e-4801-96b4-74dc03ef1047"
}
```
application/json Copy
```
{
"transaction_id": "01F7S0N7CQ31DZQYVSVPN66GNK",
"data": "JohnNothing",
"configuration": {
"force_values_into_array": [
"people"
]
}
}
```
##### Response
400403404422
application/json Copy Bad request syntax. Request payload does not exist or malformed.
```
{
"error": {
"message": "Bad Request",
"reason": "\"\" is less than 1 character."
}
}
```
application/json Copy Request is forbidden, API key is not valid
```
{
"message": "Your API key is not valid, please refresh it."
}
```
application/json Copy Resource not found. Resource you were looking for was not found.
```
{
"error": {
"message": "Not Found",
"reason": "Nothing matches the given URI."
}
}
```
application/json Copy A 422 status code occurs when a request is well-formed, however, due to semantic errors it is unable to be processed.
```
{
"error": {
"message": "Unprocessable entity.",
"reason": "Domain name could not be validated."
}
}
```
##### Request body`application/json`
5 fields
| Field | Type | Description |
| --- | --- | --- |
| `configuration` | `object` | Appends a specified value as attribute prefix, '@' by default. |
| `attribute_prefix` | `string` | Appends a specified value as attribute prefix, '@' by default. |
| `force_values_into_array` | `one of` | Forces all values into array. Can be array or boolean. |
| `json_schema` | `object` | JSON schema of the output data. Output data will be converted to types specified in the schema. If conversion is not possible, transformation will fail. If the schema is not specified, the output data will not be converted. If the schema is not a valid JSON Schema, transformation will fail. |
| `data` | `string` | XML data to be transformed. Should not exceed 5242880 bytes. |
| `product_configuration_id` | `string` | Configuration ID that references a Configuration registered in Marketplace. |
| `transaction_id` | `string` | Transaction ID |
| `urls` | `object` | Container for download and upload URLs. |
| `download_from_url`required | `string (uri)` | The URL from which XML data is to be pulled. Must have `https://` in the URLs scheme. |
| `upload_to_url`required | `string (uri)` | The URL to which converted to JSON data will be uploaded. Must have `https://` in the URLs scheme. |
##### Response `200``application/json`
1 fields
Transformation was successful.
| Field | Type | Description |
| --- | --- | --- |
| `transformed_data` | `object` | Transformed data |
##### Response `400``application/json`
1 fields
Bad request syntax. Request payload does not exist or malformed.
| Field | Type | Description |
| --- | --- | --- |
| `error` | `object` | Contains error information |
| `message` | `string` | Contains error message |
| `reason` | `string` | Contains the cause of the error |
##### Response `403``application/json`
1 fields
Request is forbidden, API key is not valid
| Field | Type | Description |
| --- | --- | --- |
| `message` | `string` | Forbidden error messageExample `Your API key is not valid, please refresh it.` |
##### Response `404``application/json`
1 fields
Resource not found. Resource you were looking for was not found.
| Field | Type | Description |
| --- | --- | --- |
| `error` | `object` | Contains error information |
| `message` | `string` | Contains error message |
| `reason` | `string` | Contains the cause of the error |
##### Other responses
`204``422`
`PUT` `/transformations/json-to-xml`
#### JSON to XML
`transform_json_to_xml`
Performs synchronous format transformation. Transforms JSON to XML.
Downloads data from the input URL and uploads the transformed data to output URL.
Upload is sent with "PUT" HTTP method.
An example of how to integrate the transformation in the flow definition.
```
{
"version": "2",
"authorization": {
"mode": "pass_through",
"auth_type": "custom",
"credentials": "dynamic"
},
"definition": {
"StartAt": "DownloadJSON",
"States": {
"DownloadJSON": {
"Next": "InstantiateBlob",
"Type": "Task",
"ResourceDefinition": {
"Type": "HTTP_ANY",
"HttpMethod": "GET",
"Headers": {
"Content-Type": "$.request_payload.content_type"
},
"VendorEndpointUrlPath": "$.request_payload.source_url",
"SaveResponseToFile": true,
"Extract": {
"Save": {
"download_from_url": "$.response_file_url"
}
}
}
},
"InstantiateBlob": {
"Type": "Task",
"Next": "SaveAsXML",
"ResourceDefinition": {
"Type": "HTTP_JSON",
"HttpMethod": "POST",
"VendorEndpointUrl": "",
"Headers": {
"Content-Type": "application/json",
"x-api-key": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
},
"PathParametersMapping": {
"request_host": "$.request_host"
},
"RequestPayload": {
"extension": ".xml",
"presign_url_ttl": 3600
},
"Extract": {
"Save": {
"blob_id": "$.blob_id",
"upload_to_url": "$.presigned_urls.upload.url"
}
}
}
},
"SaveAsXML": {
"Type": "Task",
"Next": "ComposeResponse",
"ResourceDefinition": {
"Type": "HTTP_JSON",
"HttpMethod": "PUT",
"Headers": {
"Content-Type": "application/json",
"x-api-key": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
},
"RequestPayload": {
"transaction_id": "$.transaction_id",
"urls": {
"download_from_url": "$.download_from_url",
"upload_to_url": "$.upload_to_url"
}
},
"VendorEndpointUrl": "",
"PathParametersMapping": {
"request_host": "$.request_host"
}
}
},
"ComposeResponse": {
"End": true,
"Type": "Task",
"ResourceDefinition": {
"Type": "CODE",
"Source": "loopback = lambda response: response",
"Runtime": "python3.8",
"InputDataSelector": {
"blob_id": "$.blob_id"
},
"SaveOutputTo": "response_payload",
"FunctionName": "loopback"
}
}
}
}
}
```
#### Transformation Modes
User can specify either `data` to be transformed directly or `urls` from which the data will be pulled and uploaded to (only one can be used a a time).
Show the rest
#### Warning
If `data` mode is used but the supplied JSON is too large, request will not be evaluated. Consider switching to `url` based transformation.
#### Transformation Details
User can set the values of transformation variables, such as `attribute_prefix`, `wrapper`, or `array_enumeration`.
##### `attribute_prefix`
Some JSON objects that are to be transformed to XML have previously been derived from XML or have been meant to be transformed to XML later, preserving elements' attribute values with a specific attribute prefix. User can pass this attribute prefix to construct XML with respective attribute definitions located in correct places. Default value for `attribute_prefix` is `"@"`. If text is specified using `"#text": "my_text"`, the resulting element will contain my_text as text enclosed directly by parent element - ` my_text`, instead of usual `my_text` resulting from `"element": "my_text"`
##### `preserve_prefixes`
User can set this to `true` if respective attribute prefixes should not be deleted from the resulting attributes upon transformation.
##### `wrapper`
Some JSON objects are problematic to transform to XML, especially those that are enclosed into the array at the top level. To avoid multi-root documents (not proper XML) to be produced, the result is wrapped into the root element using the value of `wrapper`, which is `"root"` by default. Additionally, in case JSON object is an array at the highest level, to construct proper XML it is wrapped around with `"items"` element as a child of `wrapper` element.
Wrapper is customizable i.e. allows user to provide the tag attributes using relevant schema. Note that the input JSON is wrapped before transformation, thus all other configuration settings apply to the provided wrapper tag/attributes during the transformation.
##### `array_enumeration`
Allows defining special variables - `sequence_start` and `attribute_name` - enabling the transformation to enumerate array items in the generated XML output. The index for the item in the array is stored as offset from `sequence_start` in the attribute of the item tag named as `attribute_name`.
##### `omit_wrapper`
Allows to omit enforced wrapper in case the JSON data is suitable for direct conversion to XML.
Transformation Examples Transforming:
```
{
"Developer": {
"@age": 32,
"@language": "c++",
"#text": "my_little_json"
}
}
```
produces:
```
my_little_json
```
Changing #text key to text:
```
{
"Developer": {
"@age": 32,
"@language": "c++",
"text": "my_little_json"
}
}
```
results into:
```
my_little_json
```
Finally, following JSON:
```
[
{
"shape": "square",
"size": 50
},
{
"shape": "circle",
"size": 25
}
]
```
yields:
```
square
50
circle
25
```
Array Enumeration Examples Sample Input JSON:
```
{
"people": [
{
"name": "John",
"age": 30
},
{
"name": "Jane",
"age": 25
},
{
"name": "Jack",
"age": 40
}
]
}
```
Sample Output XML with array enumeration enabled:
```
John
30
Jane
25
Jack
40
```
Customized Wrapper Example For the following JSON:
```
{
"company": "EvilMegaCorp",
"owner": {
"SuperEvil": "confirmed",
"@country": "KTR"
}
}
```
with the following configuration parameters:
```
{
"wrapper": {
"value": "root",
"attributes": {
"origin_source": "json"
}
},
"attribute_prefix": "@"
}
```
the output XML looks like:
```
EvilMegaCorp
confirmed
```
##### Note
Subscribe to `connection-json-to-xml` component to access this feature.
##### Request
URL based transformationDirect transformationArray Enumeration Enabled
application/json Copy
```
{
"transaction_id": "01F7S0N7CQ31DZQYVSVPN66GNK",
"urls": {
"download_from_url": "https://staircase.co/json-is-downloaded-from-here",
"upload_to_url": "https://staircase.co/xml-is-uploaded-here"
},
"configuration": {
"wrapper": "my_wrapper",
"attribute_prefix": "__"
},
"product_configuration_id": "21194609-9634-40a3-a044-979682e58a37"
}
```
application/json Copy
```
{
"transaction_id": "01F7S0N7CQ31DZQYVSVPN66GNK",
"data": "{\"key_1\": \"value_1\"}",
"product_configuration_id": "345a814d-432e-4801-96b4-74dc03ef1047"
}
```
application/json Copy
```
{
"transaction_id": "01F7S0N7CQ31DZQYVSVPN66GNK",
"data": "{\"people\": [{\"name\": \"John\", \"age\": 30}, {\"name\": \"Jane\", \"age\": 25}, {\"name\": \"Jack\", \"age\": 40}]}",
"configuration": {
"array_enumeration": {
"sequence_start": 0,
"attribute_name": "ItemIndex"
}
}
}
```
##### Response
200400403404422
application/json Copy Transformation was successful.
```
{
"transformed_data": "John30Jane25Jack40"
}
```
application/json Copy Bad request syntax. Request payload does not exist or malformed.
```
{
"error": {
"message": "Bad Request",
"reason": [
"\"\" is less than 1 character."
]
}
}
```
application/json Copy Request is forbidden, API key is not valid
```
{
"message": "Your API key is not valid, please refresh it."
}
```
application/json Copy Resource not found. Resource you were looking for was not found.
```
{
"error": {
"message": "Not Found",
"reason": "Nothing matches the given URI."
}
}
```
application/json Copy A 422 status code occurs when a request is well-formed, however, due to semantic errors it is unable to be processed.
```
{
"error": {
"message": "Unprocessable entity.",
"reason": {
"remote": "[422, error_message]"
}
}
}
```
##### Response `200``application/json`
1 fields
Transformation was successful.
| Field | Type | Description |
| --- | --- | --- |
| `transformed_data` | `string` | Transformed data |
##### Response `400``application/json`
1 fields
Bad request syntax. Request payload does not exist or malformed.
| Field | Type | Description |
| --- | --- | --- |
| `error` | `object` | error |
| `message`required | `string` | message |
| `errors` | `array` | errors |
##### Response `403``application/json`
1 fields
Request is forbidden, API key is not valid
| Field | Type | Description |
| --- | --- | --- |
| `message` | `string` | Forbidden error messageExample `Your API key is not valid, please refresh it.` |
##### Response `404``application/json`
1 fields
Resource not found. Resource you were looking for was not found.
| Field | Type | Description |
| --- | --- | --- |
| `error` | `object` | error |
| `message`required | `string` | message |
| `reason`required | `string` | reason |
##### Response `422``application/json`
1 fields
A 422 status code occurs when a request is well-formed, however, due to semantic errors it is unable to be processed.
| Field | Type | Description |
| --- | --- | --- |
| `error` | `object` | error |
| `message`required | `string` | message |
| `reason` | `one of` | reason |
##### Other responses
`204`
`PUT` `/transformations/html-to-json`
#### HTML to JSON
`transform_html_to_json`
Performs synchronous format transformation. Transforms HTML to JSON.
Downloads data from the input URL and uploads the transformed data to output URL.
Upload is sent with "PUT" HTTP method.
An example of how to integrate the transformation in the flow definition.
```
{
"version": "2",
"authorization": {
"mode": "pass_through",
"auth_type": "custom",
"credentials": "dynamic"
},
"definition": {
"StartAt": "DownloadHTML",
"States": {
"DownloadHTML": {
"Next": "InstantiateBlob",
"Type": "Task",
"ResourceDefinition": {
"Type": "HTTP_ANY",
"HttpMethod": "GET",
"Headers": {
"Content-Type": "$.request_payload.content_type"
},
"VendorEndpointUrlPath": "$.request_payload.source_url",
"SaveResponseToFile": true,
"Extract": {
"Save": {
"download_from_url": "$.response_file_url"
}
}
}
},
"InstantiateBlob": {
"Type": "Task",
"Next": "SaveAsJSON",
"ResourceDefinition": {
"Type": "HTTP_JSON",
"HttpMethod": "POST",
"VendorEndpointUrl": "",
"Headers": {
"Content-Type": "application/json",
"x-api-key": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
},
"PathParametersMapping": {
"request_host": "$.request_host"
},
"RequestPayload": {
"extension": ".json",
"presign_url_ttl": 3600
},
"Extract": {
"Save": {
"blob_id": "$.blob_id",
"upload_to_url": "$.presigned_urls.upload.url"
}
}
}
},
"SaveAsJSON": {
"Type": "Task",
"Next": "ComposeResponse",
"ResourceDefinition": {
"Type": "HTTP_JSON",
"HttpMethod": "PUT",
"Headers": {
"Content-Type": "application/json",
"x-api-key": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
},
"RequestPayload": {
"transaction_id": "$.transaction_id",
"urls": {
"download_from_url": "$.download_from_url",
"upload_to_url": "$.upload_to_url"
}
},
"VendorEndpointUrl": "",
"PathParametersMapping": {
"request_host": "$.request_host"
}
}
},
"ComposeResponse": {
"End": true,
"Type": "Task",
"ResourceDefinition": {
"Type": "CODE",
"Source": "loopback = lambda response: response",
"Runtime": "python3.8",
"InputDataSelector": {
"blob_id": "$.blob_id"
},
"SaveOutputTo": "response_payload",
"FunctionName": "loopback"
}
}
}
}
}
```
#### Transformation Modes
User can specify either `data` to be transformed directly or `urls` from which the data will be pulled and uploaded to (only one can be used a a time).
Show the rest
#### Warning
If `data` mode is used but the supplied HTML is too large, request will not be evaluated. Consider switching to `url` based transformation.
Transformation Example Applying Transformation to:
```
| Company |
Country |
| Staircase |
USA |
```
produces:
```
{
"div": [
{
"_attributes": {
"align": "center"
},
"table": [
{
"_attributes": {
"border": "0",
"width": "95%"
},
"tr": [
{
"th": [
{
"_value": "Company"
},
{
"_value": "Country"
}
]
},
{
"td": [
{
"_value": "Staircase"
},
{
"_value": "USA"
}
]
}
]
}
]
}
]
}
```
##### Note
Subscribe to `connection-html-to-json` component to access this feature.
##### Request
URL based transformationDirect transformation
application/json Copy
```
{
"transaction_id": "01F7S0N7CQ31DZQYVSVPN66GNK",
"urls": {
"download_from_url": "https://staircase.co/html-is-downloaded-from-here",
"upload_to_url": "https://staircase.co/json-is-uploaded-here"
}
}
```
application/json Copy
```
{
"transaction_id": "01F7S0N7CQ31DZQYVSVPN66GNK",
"data": "Staircase
",
"product_configuration_id": "345a814d-432e-4801-96b4-74dc03ef1047"
}
```
##### Response
400403404422
application/json Copy Bad request syntax. Request payload does not exist or malformed.
```
{
"error": {
"message": "Bad Request",
"reason": [
"\"\" is less than 1 character."
]
}
}
```
application/json Copy Request is forbidden, API key is not valid
```
{
"message": "Your API key is not valid, please refresh it."
}
```
application/json Copy Resource not found. Resource you were looking for was not found.
```
{
"error": {
"message": "Not Found",
"reason": "Nothing matches the given URI."
}
}
```
application/json Copy A 422 status code occurs when a request is well-formed, however, due to semantic errors it is unable to be processed.
```
{
"error": {
"message": "Unprocessable entity.",
"reason": {
"remote": "[422, error_message]"
}
}
}
```
##### Response `200``application/json`
1 fields
Transformation was successful.
| Field | Type | Description |
| --- | --- | --- |
| `transformed_data` | `object` | Transformed data |
##### Response `400``application/json`
1 fields
Bad request syntax. Request payload does not exist or malformed.
| Field | Type | Description |
| --- | --- | --- |
| `error` | `object` | error |
| `message`required | `string` | message |
| `errors` | `array` | errors |
##### Response `403``application/json`
1 fields
Request is forbidden, API key is not valid
| Field | Type | Description |
| --- | --- | --- |
| `message` | `string` | Forbidden error messageExample `Your API key is not valid, please refresh it.` |
##### Response `404``application/json`
1 fields
Resource not found. Resource you were looking for was not found.
| Field | Type | Description |
| --- | --- | --- |
| `error` | `object` | error |
| `message`required | `string` | message |
| `reason`required | `string` | reason |
##### Response `422``application/json`
1 fields
A 422 status code occurs when a request is well-formed, however, due to semantic errors it is unable to be processed.
| Field | Type | Description |
| --- | --- | --- |
| `error` | `object` | error |
| `message`required | `string` | message |
| `reason` | `object` | reason |
##### Other responses
`204`
`PUT` `/transformations/text-to-json`
#### Formatted Text to JSON
`transform_text_to_json`
Performs synchronous format transformation. Transforms Formatted Text to JSON.
Downloads data from the input URL and uploads the transformed data to output URL.
Upload is sent with "PUT" HTTP method.
An example of how to integrate the transformation in the flow definition.
```
{
"version": "2",
"flow_name": "my_lovely_flow",
"authorization": {
"mode": "pass_through",
"auth_type": "custom",
"credentials": "dynamic"
},
"definition": {
"StartAt": "DownloadFormattedText",
"States": {
"DownloadFormattedText": {
"Next": "InstantiateBlob",
"Type": "Task",
"ResourceDefinition": {
"Type": "HTTP_ANY",
"HttpMethod": "GET",
"Headers": {
"Content-Type": "$.request_payload.content_type"
},
"VendorEndpointUrlPath": "$.request_payload.source_url",
"SaveResponseToFile": true,
"Extract": {
"Save": {
"download_from_url": "$.response_file_url"
}
}
}
},
"InstantiateBlob": {
"Type": "Task",
"Next": "SaveAsJSON",
"ResourceDefinition": {
"Type": "HTTP_JSON",
"HttpMethod": "POST",
"VendorEndpointUrl": "",
"Headers": {
"Content-Type": "application/json",
"x-api-key": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
},
"PathParametersMapping": {
"request_host": "$.request_host"
},
"RequestPayload": {
"extension": ".json",
"presign_url_ttl": 3600
},
"Extract": {
"Save": {
"blob_id": "$.blob_id",
"upload_to_url": "$.presigned_urls.upload.url"
}
}
}
},
"SaveAsJSON": {
"Type": "Task",
"Next": "ComposeResponse",
"ResourceDefinition": {
"Type": "HTTP_JSON",
"HttpMethod": "PUT",
"Headers": {
"Content-Type": "application/json",
"x-api-key": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
},
"RequestPayload": {
"transaction_id": "$.transaction_id",
"urls": {
"download_from_url": "$.download_from_url",
"upload_to_url": "$.upload_to_url"
},
"template_name": "$.request_payload.template_name",
"multipart_boundary_header": "$.request_payload.multipart_boundary_header"
},
"VendorEndpointUrl": "",
"PathParametersMapping": {
"request_host": "$.request_host"
}
}
},
"ComposeResponse": {
"End": true,
"Type": "Task",
"ResourceDefinition": {
"Type": "CODE",
"Source": "loopback = lambda response: response",
"Runtime": "python3.8",
"InputDataSelector": {
"blob_id": "$.blob_id"
},
"SaveOutputTo": "response_payload",
"FunctionName": "loopback"
}
}
}
}
}
```
#### Transformation Modes
User can specify either `data` to be transformed directly or `urls` from which the data will be pulled and uploaded to (only one can be used at a time).
Show the rest
#### Warning
If `data` mode is used but the supplied JSON is too large, request will not be evaluated. Consider switching to `url` based transformation.
#### Templates
This endpoint requires usage of a template that allows to match the necessary data fields with their values to compose a JSON object as a result. Check the Formatted Text Templates documentation for the necessary operations and information on how to build the Template. This endpoint attempts to apply the specified Template to the given Text to produce the result, to the best possible extent.
`template_name` requested in the payload should be a valid name of already existing template in the environment.
User has to specify `multipart_boundary_header` which sets the multipart boundary value for the file separator.
##### Note
Subscribe to `connection-plain-text-to-json` component to access this feature.
##### Request
URL based transformationData based transformation
application/json Copy
```
{
"transaction_id": "01F7S0N7CQ31DZQYVSVPN66GNK",
"urls": {
"download_from_url": "https://staircase.co/text-is-downloaded-from-here",
"upload_to_url": "https://staircase.co/json-is-uploaded-here"
},
"template_name": "my_special_template",
"multipart_boundary_header": "----------ieoau._._+2_8_GoodLuck8.3-ds0d0J0S0Kl234324jfLdsjfdAuaoei-----"
}
```
application/json Copy
```
{
"transaction_id": "01F7S0N7CQ31DZQYVSVPN66GNK",
"data": "company: Staircase\ncountry: US\n",
"template_name": "my_special_template",
"multipart_boundary_header": "----------ieoau._._+2_8_GoodLuck8.3-ds0d0J0S0Kl234324jfLdsjfdAuaoei-----",
"product_configuration_id": "605d8623-262f-4d64-a490-4963d73db8f0"
}
```
##### Response
400403404422
application/json Copy Bad request syntax. Request payload does not exist or malformed.
```
{
"error": {
"message": "Bad Request",
"reason": [
"\"\" is less than 1 character."
]
}
}
```
application/json Copy Request is forbidden, API key is not valid
```
{
"message": "Your API key is not valid, please refresh it."
}
```
application/json Copy Resource not found. Resource you were looking for was not found.
```
{
"error": {
"message": "Not Found",
"reason": "Nothing matches the given URI."
}
}
```
application/json Copy A 422 status code occurs when a request is well-formed, however, due to semantic errors it is unable to be processed.
```
{
"error": {
"message": "Unprocessable entity.",
"reason": {
"remote": "[422, error_message]"
}
}
}
```
##### Response `200``application/json`
1 fields
Transformation was successful.
| Field | Type | Description |
| --- | --- | --- |
| `transformed_data` | `object` | Transformed data |
##### Response `400``application/json`
1 fields
Bad request syntax. Request payload does not exist or malformed.
| Field | Type | Description |
| --- | --- | --- |
| `error` | `object` | error |
| `message`required | `string` | message |
| `errors` | `array` | errors |
##### Response `403``application/json`
1 fields
Request is forbidden, API key is not valid
| Field | Type | Description |
| --- | --- | --- |
| `message` | `string` | Forbidden error messageExample `Your API key is not valid, please refresh it.` |
##### Response `404``application/json`
1 fields
Resource not found. Resource you were looking for was not found.
| Field | Type | Description |
| --- | --- | --- |
| `error` | `object` | error |
| `message`required | `string` | message |
| `reason`required | `string` | reason |
##### Response `422``application/json`
1 fields
A 422 status code occurs when a request is well-formed, however, due to semantic errors it is unable to be processed.
| Field | Type | Description |
| --- | --- | --- |
| `error` | `object` | error |
| `message`required | `string` | message |
| `reason` | `object` | reason |
##### Other responses
`204`
### Transformations[new]
`PUT` `/transformations/csv-to-json`
#### CSV to JSON
`transform_csv_to_json`
Performs synchronous format transformation. Transforms CSV to JSON.
Downloads data from the input URL and uploads the transformed data to output URL, if `url` based transformation has been chosen. Otherwise, if `data` format is picked, accepts the CSV data and attempts to transform it into JSON directly, returning the result in the response payload. Check below for the transformation parameters that user can/should control.
Show the rest Upload is sent with "PUT" HTTP method for the `url` based transformation.
An example of how to integrate the url based transformation in the flow definition.
```
{
"version": "2",
"authorization": {
"mode": "pass_through",
"auth_type": "custom",
"credentials": "dynamic"
},
"definition": {
"StartAt": "DownloadCSV",
"States": {
"DownloadCSV": {
"Next": "InstantiateBlob",
"Type": "Task",
"ResourceDefinition": {
"Type": "HTTP_ANY",
"HttpMethod": "GET",
"Headers": {
"Content-Type": "$.request_payload.content_type"
},
"VendorEndpointUrlPath": "$.request_payload.source_url",
"SaveResponseToFile": true,
"Extract": {
"Save": {
"download_from_url": "$.response_file_url"
}
}
}
},
"InstantiateBlob": {
"Type": "Task",
"Next": "SaveAsJSON",
"ResourceDefinition": {
"Type": "HTTP_JSON",
"HttpMethod": "POST",
"VendorEndpointUrl": "",
"Headers": {
"Content-Type": "application/json",
"x-api-key": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
},
"PathParametersMapping": {
"request_host": "$.request_host"
},
"RequestPayload": {
"extension": ".json",
"presign_url_ttl": 3600
},
"Extract": {
"Save": {
"blob_id": "$.blob_id",
"upload_to_url": "$.presigned_urls.upload.url"
}
}
}
},
"SaveAsJSON": {
"Type": "Task",
"Next": "ComposeResponse",
"ResourceDefinition": {
"Type": "HTTP_JSON",
"HttpMethod": "PUT",
"Headers": {
"Content-Type": "application/json",
"x-api-key": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
},
"RequestPayload": {
"transaction_id": "$.transaction_id",
"urls": {
"download_from_url": "$.download_from_url",
"upload_to_url": "$.upload_to_url"
}
},
"VendorEndpointUrl": "",
"PathParametersMapping": {
"request_host": "$.request_host"
}
}
},
"ComposeResponse": {
"End": true,
"Type": "Task",
"ResourceDefinition": {
"Type": "CODE",
"Source": "loopback = lambda response: response",
"Runtime": "python3.8",
"InputDataSelector": {
"blob_id": "$.blob_id"
},
"SaveOutputTo": "response_payload",
"FunctionName": "loopback"
}
}
}
}
}
```
#### Transformation Modes
User can specify either `data` to be transformed directly or `urls` from which the data will be pulled and uploaded to (only one can be used at a time).
#### Warning
If `data` mode is used but the supplied CSV is too large, request will not be evaluated. Consider switching to `url` based transformation.
If 'urls' mode is used, user should ensure that the download link responds with the `Content-Type` header containing relevant value, such as `text/plain` or `csv`.
#### Transformation Configuration
User can define specific transformation properties to control the process and use custom characters set for the key attributes involved. Below is the list of properties that are under control of the user:
- `delimiter` - defines the character to be used as the delimiter for the CSV record fields. Default value is `,`
- `fieldnames` - this endpoint treats the passed data's 1st row as the header from which it generates relevant field names. If your data is not supposed to have 1st row as header, specify this property with a list of custom fieldnames, 1 per record field (see examples below for better clarity)
- `escape_character` - defines the character that is used to escape other special characters used in CSV or defined by user.
- `include_columns` - defines the set of columns to include in the output JSON (allows to crop out unneeded columns)
- `column_types` - allows to define specific data types which will be cast to the values of the specified columns (see the possible values for the data types below). Note that empty values in the string columns are transformed into `null` instead of empty strings.
#### Data Types
For the `column_types` configuration parameter user can specify column names with the data types to cast upon values of those. The available data types are: `integer`, `float`, and `string`.
Transformation Examples Applying transformation with default configuration to:
```
h1,h2,h3
r1v1,r1v2,r1v3
r2v1,r2v2,r2v3
```
results in:
```
[
{
"h1": "r1v1",
"h2": "r1v2",
"h3": "r1v3"
},
{
"h1": "r2v1",
"h2": "r2v2",
"h3": "r2v3"
}
]
```
In case your data resembles this:
```
h1;h2;h3
r1v1;r1v2;r1v3
r2v1;r2v2;r2v3
```
then, by setting `configuration` with the value below, you can achieve the same result with the previous example:
```
{
"delimiter": ";"
}
```
Assume that you only need several columns from the input CSV, whereas it contains many more. One can specify specific column names to be used to form the result JSON, cropping out all the additional columns. For instance:
```
h1,h2,h3
r1v1,r1v2,r1v3
r2v1,r2v2,r2v3
```
You can exclude column `h3` from the result JSON by specifying following configuration:
```
{
"include_columns": ["h1", "h2"]
}
```
Result:
```
[
{
"h1": "r1v1",
"h2": "r1v2"
},
{
"h1": "r2v1",
"h2": "r2v2"
}
]
```
Finally, user could also benefit from `column_types` configuration parameter, which allows to force-cast data types on specific columns. Below is example:
```
name,surname,age,id
bob,marley,31,456
kurt,cobain,26,298
```
Assuming that user wishes to have `id` column to have string values, one can specify following configuration:
```
{
"column_types": [
{
"column_name": "id",
"column_type": "string"
}
]
}
```
Result:
```
[
{
"name": "bob",
"surname": "marley",
"age": 31,
"id": "456"
},
{
"name": "kurt",
"surname": "cobain",
"age": 26,
"id": "298"
}
]
```
##### Note
Subscribe to `connection-csv-to-json` component to access this feature.
##### Request
URL based transformationData based transformation
application/json Copy
```
{
"transaction_id": "01F7S0N7CQ31DZQYVSVPN66GNK",
"urls": {
"download_from_url": "https://staircase.co/csv-is-downloaded-from-here",
"upload_to_url": "https://staircase.co/csv-is-uploaded-here"
},
"configuration": {
"delimiter": ";"
}
}
```
application/json Copy
```
{
"transaction_id": "01F7S0N7CQ31DZQYVSVPN66GNK",
"data": "h1,h2,h3\nr1v1,r1v2,r1v3\nr2v1,r2v2,r2v3",
"configuration": {
"column_types": [
{
"column_name": "h1"
},
{
"column_type": "string"
}
]
}
}
```
##### Response
400403404422
application/json Copy Bad request syntax. Request payload does not exist or malformed.
```
{
"error": {
"message": "Bad Request",
"reason": [
"\"\" is less than 1 character."
]
}
}
```
application/json Copy Request is forbidden, API key is not valid
```
{
"message": "Your API key is not valid, please refresh it."
}
```
application/json Copy Resource not found. Resource you were looking for was not found.
```
{
"error": {
"message": "Not Found",
"reason": "Nothing matches the given URI."
}
}
```
application/json Copy A 422 status code occurs when a request is well-formed, however, due to semantic errors it is unable to be processed.
```
{
"error": {
"message": "Unprocessable entity.",
"reason": {
"remote": "[422, error_message]"
}
}
}
```
##### Response `200``application/json`
1 fields
Transformation was successful.
| Field | Type | Description |
| --- | --- | --- |
| `transformed_data` | `object[]` | Transformed data consisting of records |
##### Response `400``application/json`
1 fields
Bad request syntax. Request payload does not exist or malformed.
| Field | Type | Description |
| --- | --- | --- |
| `error` | `object` | error |
| `message`required | `string` | message |
| `errors` | `array` | errors |
##### Response `403``application/json`
1 fields
Request is forbidden, API key is not valid
| Field | Type | Description |
| --- | --- | --- |
| `message` | `string` | Forbidden error messageExample `Your API key is not valid, please refresh it.` |
##### Response `404``application/json`
1 fields
Resource not found. Resource you were looking for was not found.
| Field | Type | Description |
| --- | --- | --- |
| `error` | `object` | error |
| `message`required | `string` | message |
| `reason`required | `string` | reason |
##### Response `422``application/json`
1 fields
A 422 status code occurs when a request is well-formed, however, due to semantic errors it is unable to be processed.
| Field | Type | Description |
| --- | --- | --- |
| `error` | `object` | error |
| `message`required | `string` | message |
| `reason` | `one of` | Reason |
##### Other responses
`204`
`PUT` `/transformations/csv-to-json/async`
#### CSV to JSON (Async)[new]
`transform_csv_to_json_async`
CSV to JSON
Performs asynchronous format transformation. Transforms CSV to JSON. This endpoint is designed to handle the processing of CSV files with a high volume of rows and columns.
Each asynchronous transformation is assigned a `transaction_id` and a unique `response_collection_id`. To identify and retrieve the URL to a response, you need to use both `transaction_id` and `response_collection_id`. See Refresh Data URL for more.
Show the rest
#### Synchronous vs Asynchronous
Opting for asynchronous transformation is recommended when processing large CSV files with numerous rows and columns. However, it's important to note that asynchronous transformation has its own drawbacks. This approach doesn't directly upload data to the specified URL or return transformed data in the webhook response payload. Instead, it generates a temporary URL that can be used to access the transformed data once the processing is complete.
For synchronous transformation, you can check out CSV to JSON API.
#### Events
Events are sent to `callback_url` provided by user. At the end of conversion, you will receive a webhook with the following schema:
```
{
"$schema": "",
"type": "object",
"properties": {
"status": {
"type": "string",
"enum": ["Failed", "Completed"]
},
"error": {
"type": "object",
"properties": {
"error_type": {
"type": "string"
},
"context": {
"type": "object",
"additionalProperties": true
}
},
"required": ["error_type", "context"],
"additionalProperties": false
},
"result": {
"type": "object",
"properties": {
"temporary_url": {
"type": "string"
}
},
"required": ["temporary_url"],
"additionalProperties": false
},
"transaction_id": {
"type": "string"
},
"response_collection_id": {
"type": "string"
}
},
"required": ["status", "transaction_id", "response_collection_id"],
"additionalProperties": false
}
```
Please note that the result and error fields cannot appear at the same time. However, both are included in the schema for better understanding and display purposes.
You can refresh the temporary URL in the Refresh Data URL
#### Known limitations
Including type definitions for every column is strongly recommended, as this helps to establish consistent data types across all rows. Without this information, the first row of each column is used to infer the type for subsequent rows, which can lead to inconsistencies and errors. At least one column type definition must be provided.
##### Request
application/json Copy
```
{
"transaction_id": "01F7S0N7CQ31DZQYVSVPN66GNK",
"callback_url": "https://watch.staircaseapi.com/callback-listener/tok89782uomvc2nuyYBw",
"urls": {
"download_from_url": "https://staircase.co/csv-is-downloaded-from-here"
},
"configuration": {
"delimiter": "|",
"column_types": [
{
"column_name": "LoanNumber",
"column_type": "integer"
},
{
"column_name": "AAndHFlagChangeById",
"column_type": "string"
},
{
"column_name": "AccelerationAmount",
"column_type": "float"
},
{
"column_name": "AccelerationReasonCode",
"column_type": "float"
}
]
}
}
```
##### Response
400403422
application/json Copy Bad request syntax. Request payload does not exist or malformed.
```
{
"error": {
"message": "Bad Request",
"reason": [
"\"\" is less than 1 character."
]
}
}
```
application/json Copy Request is forbidden, API key is not valid
```
{
"message": "Your API key is not valid, please refresh it."
}
```
application/json Copy A 422 status code occurs when a request is well-formed, however, due to semantic errors it is unable to be processed.
```
{
"error": {
"message": "Unprocessable entity.",
"reason": {
"remote": "[422, error_message]"
}
}
}
```
##### Request body`application/json`
4 fields
| Field | Type | Description |
| --- | --- | --- |
| `transaction_id`required | `string` | The unique identifier for the transaction. |
| `configuration`required | `object` | An object that specifies the configuration options for the data file. |
| `delimiter` | `string` | The delimiter used to separate fields in the data file. |
| `fieldnames` | `string[]` | An array of field names for the columns in the data file. |
| `escape_character` | `string` | The character used to escape special characters in the data file. |
| `column_types`required | `object[]` | An array of objects that specify the column names and data types in the data file. |
| `column_name` | `string` | The name of the column in the data file. |
| `column_type` | `string` | The type of data in the column.`float``integer``string` |
| `include_columns` | `string[]` | An array of column names to include in the output. |
| `urls`required | `object` | An object that specifies the URLs for the data file. |
| `download_from_url`required | `string` | The URL from which to download the data file. |
| `callback_url`required | `string` | The URL to call with the results of the data processing. |
##### Response `202``application/json`
3 fields
Transformation has been initialized.
| Field | Type | Description |
| --- | --- | --- |
| `transaction_id`required | `string` | The unique identifier for the transaction.Example `01ABDC918DE524F` |
| `response_collection_id`required | `string` | The unique identifier for the response collection.Example `002f8b98-c5-4ac3-b26d-edecba96ef19` |
| `future_temporary_url`required | `string` | The temporary URL that can be used in the future. Note that object may not be immediately.Example `https://urls.staircaseapi.com/early-access` |
##### Response `400``application/json`
1 fields
Bad request syntax. Request payload does not exist or malformed.
| Field | Type | Description |
| --- | --- | --- |
| `error` | `object` | error |
| `message`required | `string` | message |
| `errors` | `array` | errors |
##### Response `403``application/json`
1 fields
Request is forbidden, API key is not valid
| Field | Type | Description |
| --- | --- | --- |
| `message` | `string` | Forbidden error messageExample `Your API key is not valid, please refresh it.` |
##### Response `422``application/json`
1 fields
A 422 status code occurs when a request is well-formed, however, due to semantic errors it is unable to be processed.
| Field | Type | Description |
| --- | --- | --- |
| `error` | `object` | error |
| `message`required | `string` | message |
| `reason` | `one of` | Reason |
`GET` `/transformations/csv-to-json/async/results/{transaction_id}/{response_collection_id}`
#### Refresh Data URL[new]
`refresh_csv_data_url`
CSV to JSON
Refreshes the `URL` of a given transformation.
To identify a response, you need to use both `transaction_id` and `response_collection_id`.
The URL is not included when no transformation for the specified response collection ID. If a transformation is still pending or not found, the `transformation_status` will indicate this by showing `pending_or_not_found`.
##### Response
application/json Copy Request is forbidden, API key is not valid
```
{
"message": "Your API key is not valid, please refresh it."
}
```
##### Parameters
2
| Parameter | Type | Example | Description |
| --- | --- | --- | --- |
| `transaction_id` required | `string` path | `01F7S0N7CQ31DZQYVSVPN66GNK` | The unique identifier for the transaction. |
| `response_collection_id` required | `string` path | `002f8b98-c5-4ac3-b26d-edecba96ef19` | The unique identifier for the response collection. |
##### Response `200``application/json`
2 fields
Transformation metadata.
| Field | Type | Description |
| --- | --- | --- |
| `transformation_status`required | `string` | Status of a data transformation process.`available``pending_or_not_found` |
| `temporary_url`required | `string` | Temporary URL for accessing transformed data. Can be null. |
##### Response `403``application/json`
1 fields
Request is forbidden, API key is not valid
| Field | Type | Description |
| --- | --- | --- |
| `message` | `string` | Forbidden error messageExample `Your API key is not valid, please refresh it.` |
### Formatted Text Templates
`POST` `/transformations/text-to-json/templates`
#### Create Transformation Template
`create_template`
Creates and stores transformation Template that will be used to transform Formatted Text data to JSON objects the way user prefers it. Template can be referenced by the `template_name`, so it should be unique. User can use `template_section_break` value to specify templates for different variations of the same file (`#####formatted_text_template_section_break#####` by default).
Show the rest
#### Template Specification
Templates are written in a plain text data format and do not follow strict rules outside the template statements. The so-called template statements are enclosed in double curly braces - `{{my_template_statement}}`. The statement inside follows a strict syntax that is validated by the endpoint. The template statements are used to match certain user-defined regular expressions to the data inside Formatted Text files. A typical template_statement structure is shown below:
{{ "`regex`", "`object_path`", "`value_structure`", "`multiline`" }}
| Statement Component | Description | Required |
| --- | --- | --- |
| regex | Regular expression | true |
| object_path | JSONPath where the created value should be stored | true |
| value_structure | Allows to configure the object structure created as value | false |
| multiline | Allows regex detection with new line character `\n` | false |
`regex` (required) needs at least one `(regex group)` to extract relevant value from data.
`object_path` (required) - object properties separated by dot (.). Object is created dynamically, if it contains pipe character (|) then this `prop.prop_A|prop.prop_B` becomes:
```
{
"prop":
{
"prop_A": "match group 1 from REGEX value",
"prop_B": "match group 2 from REGEX value"
}
}
```
(scalable to `n` REGEX groups)
`value_structure` (optional) - when this is set instead of adding value to OBJ_PATH, this creates an object from `propA|propB` to:
```
{
"propA": "match group 1 from REGEX value",
"propB": "match group 2 from REGEX value"
}
```
Can be used to add objects as individual elements to arrays.
`multiline` (optional) - forces the Template to read data lines until it reaches an empty line. The combined lines are then matched to regex.
Template Statement Examples Template Statement: `{{ "Dataset:\s(.*)\n\s*(.*)", "Dataset[]", "code|message", "true" }}`
applied to:
Dataset: 1003 Data No Errors/Warnings detected
Dataset: Additional Case Data No Errors/Warnings detected
produces:
```
"Dataset": [
{
"code": "1003 Data",
"message": "No Errors/Warnings detected"
},
{
"code": "Additional Case Data",
"message": "No Errors/Warnings detected"
}
]
```
Template Statement:
`{{ "(.+performance\stime\swas.+)", "Credit Report Retrieval Log.Performance" }}`
applied to:
Credit agency Agency200's performance time was 1 seconds.
produces:
```
"Credit Report Retrieval Log": {
"Performance": "Credit agency Agency200's performance time was 1 seconds."
}
```
##### Note
###### Subscribe to `connection-plain-text-to-json` component to access this feature.
##### Request
application/json Copy
```
{
"template_name": "my_special_template",
"template_content": "my_template_content"
}
```
##### Response
201400403404422
application/json Copy Template was created successfully
```
{
"201 Response Example": {
"value": {
"template_name": "my_special_template"
}
}
}
```
application/json Copy Bad request syntax. Request payload does not exist or malformed.
```
{
"error": {
"message": "Bad Request",
"reason": [
"\"\" is less than 1 character."
]
}
}
```
application/json Copy Request is forbidden, API key is not valid
```
{
"message": "Your API key is not valid, please refresh it."
}
```
application/json Copy Resource not found. Resource you were looking for was not found.
```
{
"error": {
"message": "Not Found",
"reason": "Nothing matches the given URI."
}
}
```
application/json Copy A 422 status code occurs when a request is well-formed, however, due to semantic errors it is unable to be processed.
```
{
"error": {
"message": "Unprocessable entity.",
"reason": {
"remote": "[422, error_message]"
}
}
}
```
##### Request body`application/json`
3 fields
| Field | Type | Description |
| --- | --- | --- |
| `template_name`required | `string` | Template name that can be referenced |
| `template_content`required | `string` | Template content to be validated and saved |
| `template_section_break` | `string` | Allows overriding default value |
##### Response `201``application/json`
1 fields
Template was created successfully
| Field | Type | Description |
| --- | --- | --- |
| `template_name` | `string` | Template name that can be referenced |
##### Response `400``application/json`
1 fields
Bad request syntax. Request payload does not exist or malformed.
| Field | Type | Description |
| --- | --- | --- |
| `error` | `object` | error |
| `message`required | `string` | message |
| `errors` | `array` | errors |
##### Response `403``application/json`
1 fields
Request is forbidden, API key is not valid
| Field | Type | Description |
| --- | --- | --- |
| `message` | `string` | Forbidden error messageExample `Your API key is not valid, please refresh it.` |
##### Response `404``application/json`
1 fields
Resource not found. Resource you were looking for was not found.
| Field | Type | Description |
| --- | --- | --- |
| `error` | `object` | error |
| `message`required | `string` | message |
| `reason`required | `string` | reason |
##### Response `422``application/json`
1 fields
A 422 status code occurs when a request is well-formed, however, due to semantic errors it is unable to be processed.
| Field | Type | Description |
| --- | --- | --- |
| `error` | `object` | error |
| `message`required | `string` | message |
| `reason` | `object` | reason |
`GET` `/transformations/text-to-json/templates`
#### Retrieve All Templates
`retrieve_templates`
Retrieves all Templates created and stored by the user. Since the response can become large, the number of items returned is limited and can be configured by the user.
#### Pagination by Token
This endpoint supports pagination by token. If you receive a non-null value for 'next_token' under the "page" collection in response payload, to get the next page, you will have to pass 'next_token' in the following request as query parameter
#### Note
###### Subscribe to `connection-plain-text-to-json` component to access this feature.
##### Response
200400403404422
application/json Copy Retrieved all Transformation Templates
```
{
"templates": [],
"page": {
"count": 2,
"next_token": "cGs9VEVNUExBVEUlMjNteV90ZW1wbGF0ZV8yJnNrPVRFTVBMQVRFJTIzbXlfdGVtcGxhdGVfMg=="
}
}
```
application/json Copy Bad request syntax. Request payload does not exist or malformed.
```
{
"error": {
"message": "Bad Request",
"reason": [
"\"\" is less than 1 character."
]
}
}
```
application/json Copy Request is forbidden, API key is not valid
```
{
"message": "Your API key is not valid, please refresh it."
}
```
application/json Copy Resource not found. Resource you were looking for was not found.
```
{
"error": {
"message": "Not Found",
"reason": "Nothing matches the given URI."
}
}
```
application/json Copy A 422 status code occurs when a request is well-formed, however, due to semantic errors it is unable to be processed.
```
{
"error": {
"message": "Unprocessable entity.",
"reason": {
"remote": "[422, error_message]"
}
}
}
```
##### Parameters
2
| Parameter | Type | Example | Description |
| --- | --- | --- | --- |
| `limit` | `integer` query | `15` | Limits number of items returned |
| `next_token` | `string` query | `cGs9VEVNUExBVEUlMjNteV90ZW1wbGF0ZV8yJnNrPVRFTVBMQVRFJTIzbXlfdGVtcGxhdGVfMg==` | Next token can be used to retrieve next page of results |
##### Response `200``application/json`
2 fields
Retrieved all Transformation Templates
| Field | Type | Description |
| --- | --- | --- |
| `templates` | `object[]` | Array of Transformation Templates created earlier |
| `template_name` | `string` | Template name |
| `template_content` | `string` | Template content |
| `page` | `object` | Contains information about pagination |
| `count` | `integer` | Count of items returned |
| `next_token` | `string` | The next token for further requests |
##### Response `400``application/json`
1 fields
Bad request syntax. Request payload does not exist or malformed.
| Field | Type | Description |
| --- | --- | --- |
| `error` | `object` | error |
| `message`required | `string` | message |
| `errors` | `array` | errors |
##### Response `403``application/json`
1 fields
Request is forbidden, API key is not valid
| Field | Type | Description |
| --- | --- | --- |
| `message` | `string` | Forbidden error messageExample `Your API key is not valid, please refresh it.` |
##### Response `404``application/json`
1 fields
Resource not found. Resource you were looking for was not found.
| Field | Type | Description |
| --- | --- | --- |
| `error` | `object` | error |
| `message`required | `string` | message |
| `reason`required | `string` | reason |
##### Response `422``application/json`
1 fields
A 422 status code occurs when a request is well-formed, however, due to semantic errors it is unable to be processed.
| Field | Type | Description |
| --- | --- | --- |
| `error` | `object` | error |
| `message`required | `string` | message |
| `reason` | `object` | reason |
`GET` `/transformations/text-to-json/templates/{template_name}`
#### Retrieve Transformation Template
`retrieve_template`
Retrieves an individual Template referenced by its name.
#### Note
###### Subscribe to `connection-plain-text-to-json` component to access this feature.
##### Response
200400403404422
application/json Copy Retrieved all Transformation Templates
```
{
"template_name": "my_special_template",
"template_content": "template content"
}
```
application/json Copy Bad request syntax. Request payload does not exist or malformed.
```
{
"error": {
"message": "Bad Request",
"reason": [
"\"\" is less than 1 character."
]
}
}
```
application/json Copy Request is forbidden, API key is not valid
```
{
"message": "Your API key is not valid, please refresh it."
}
```
application/json Copy Resource not found. Resource you were looking for was not found.
```
{
"error": {
"message": "Not Found",
"reason": "Nothing matches the given URI."
}
}
```
application/json Copy A 422 status code occurs when a request is well-formed, however, due to semantic errors it is unable to be processed.
```
{
"error": {
"message": "Unprocessable entity.",
"reason": {
"remote": "[422, error_message]"
}
}
}
```
##### Parameters
1
| Parameter | Type | Example | Description |
| --- | --- | --- | --- |
| `template_name` required | `string` path | `my_special_template` | Template Name |
##### Response `200``application/json`
2 fields
Retrieved all Transformation Templates
| Field | Type | Description |
| --- | --- | --- |
| `template_name` | `string` | Name of the template |
| `template_content` | `string` | Content of the template |
##### Response `400``application/json`
1 fields
Bad request syntax. Request payload does not exist or malformed.
| Field | Type | Description |
| --- | --- | --- |
| `error` | `object` | error |
| `message`required | `string` | message |
| `errors` | `array` | errors |
##### Response `403``application/json`
1 fields
Request is forbidden, API key is not valid
| Field | Type | Description |
| --- | --- | --- |
| `message` | `string` | Forbidden error messageExample `Your API key is not valid, please refresh it.` |
##### Response `404``application/json`
1 fields
Resource not found. Resource you were looking for was not found.
| Field | Type | Description |
| --- | --- | --- |
| `error` | `object` | error |
| `message`required | `string` | message |
| `reason`required | `string` | reason |
##### Response `422``application/json`
1 fields
A 422 status code occurs when a request is well-formed, however, due to semantic errors it is unable to be processed.
| Field | Type | Description |
| --- | --- | --- |
| `error` | `object` | error |
| `message`required | `string` | message |
| `reason` | `object` | reason |
`PATCH` `/transformations/text-to-json/templates/{template_name}`
#### Update Transformation Template
`update_template`
Update the Template content referenced by its name.
#### Note
###### Subscribe to `connection-plain-text-to-json` component to access this feature.
##### Request
application/json Copy
```
{
"template_content": "my_template_content"
}
```
##### Response
200400403404422
application/json Copy Template was updated successfully
```
{
"200 Response Example": {
"message": "The template `template_name` has been updated successfully."
}
}
```
application/json Copy Bad request syntax. Request payload does not exist or malformed.
```
{
"error": {
"message": "Bad Request",
"reason": [
"\"\" is less than 1 character."
]
}
}
```
application/json Copy Request is forbidden, API key is not valid
```
{
"message": "Your API key is not valid, please refresh it."
}
```
application/json Copy Resource not found. Resource you were looking for was not found.
```
{
"error": {
"message": "Not Found",
"reason": "Nothing matches the given URI."
}
}
```
application/json Copy A 422 status code occurs when a request is well-formed, however, due to semantic errors it is unable to be processed.
```
{
"error": {
"message": "Unprocessable entity.",
"reason": {
"remote": "[422, error_message]"
}
}
}
```
##### Parameters
1
| Parameter | Type | Example | Description |
| --- | --- | --- | --- |
| `template_name` required | `string` path | `my_special_template` | Template Name |
##### Request body`application/json`
1 fields
| Field | Type | Description |
| --- | --- | --- |
| `template_content`required | `string` | Template content to be validated and updated |
##### Response `200``application/json`
1 fields
Template was updated successfully
| Field | Type | Description |
| --- | --- | --- |
| `message` | `string` | Request fulfilled successfully message |
##### Response `400``application/json`
1 fields
Bad request syntax. Request payload does not exist or malformed.
| Field | Type | Description |
| --- | --- | --- |
| `error` | `object` | error |
| `message`required | `string` | message |
| `errors` | `array` | errors |
##### Response `403``application/json`
1 fields
Request is forbidden, API key is not valid
| Field | Type | Description |
| --- | --- | --- |
| `message` | `string` | Forbidden error messageExample `Your API key is not valid, please refresh it.` |
##### Response `404``application/json`
1 fields
Resource not found. Resource you were looking for was not found.
| Field | Type | Description |
| --- | --- | --- |
| `error` | `object` | error |
| `message`required | `string` | message |
| `reason`required | `string` | reason |
##### Response `422``application/json`
1 fields
A 422 status code occurs when a request is well-formed, however, due to semantic errors it is unable to be processed.
| Field | Type | Description |
| --- | --- | --- |
| `error` | `object` | error |
| `message`required | `string` | message |
| `reason` | `object` | reason |
`DELETE` `/transformations/text-to-json/templates/{template_name}`
#### Delete Transformation Template
`delete_template`
Deletes a Transformation Template referenced by its name.
#### Note
###### Subscribe to `connection-plain-text-to-json` component to access this feature.
##### Response
200400403404422
application/json Copy The template was deleted successfully
```
{
"message": "The template `my_special_template` has been deleted successfully."
}
```
application/json Copy Bad request syntax. Request payload does not exist or malformed.
```
{
"error": {
"message": "Bad Request",
"reason": [
"\"\" is less than 1 character."
]
}
}
```
application/json Copy Request is forbidden, API key is not valid
```
{
"message": "Your API key is not valid, please refresh it."
}
```
application/json Copy Resource not found. Resource you were looking for was not found.
```
{
"error": {
"message": "Not Found",
"reason": "Nothing matches the given URI."
}
}
```
application/json Copy A 422 status code occurs when a request is well-formed, however, due to semantic errors it is unable to be processed.
```
{
"error": {
"message": "Unprocessable entity.",
"reason": {
"remote": "[422, error_message]"
}
}
}
```
##### Parameters
1
| Parameter | Type | Example | Description |
| --- | --- | --- | --- |
| `template_name` required | `string` path | `my_special_template` | Template Name |
##### Response `200``application/json`
1 fields
The template was deleted successfully
| Field | Type | Description |
| --- | --- | --- |
| `message` | `string` | Template deleted successfully message |
##### Response `400``application/json`
1 fields
Bad request syntax. Request payload does not exist or malformed.
| Field | Type | Description |
| --- | --- | --- |
| `error` | `object` | error |
| `message`required | `string` | message |
| `errors` | `array` | errors |
##### Response `403``application/json`
1 fields
Request is forbidden, API key is not valid
| Field | Type | Description |
| --- | --- | --- |
| `message` | `string` | Forbidden error messageExample `Your API key is not valid, please refresh it.` |
##### Response `404``application/json`
1 fields
Resource not found. Resource you were looking for was not found.
| Field | Type | Description |
| --- | --- | --- |
| `error` | `object` | error |
| `message`required | `string` | message |
| `reason`required | `string` | reason |
##### Response `422``application/json`
1 fields
A 422 status code occurs when a request is well-formed, however, due to semantic errors it is unable to be processed.
| Field | Type | Description |
| --- | --- | --- |
| `error` | `object` | error |
| `message`required | `string` | message |
| `reason` | `object` | reason |
## Providers
- Arch MI
- Argyle
- Atomic
- Blend
- Byte Software
- CBC / Factual Data
- Citadel
- DocuSign
- Docutech
- Enact
- Ephesoft
- Equifax
- Essent
- Experian
- Fannie Mae
- Finicity
- Freddie Mac
- ICE Mortgage Technology
- Informative Research
- LendingPad
- MeridianLink
- MGIC
- Mindee
- National MI
- Ocrolus
- Optimal Blue
- Pinwheel
- Plaid
- Prove
- Radian
- RPR
- Sales Boomerang
- Sharper Lending
- SoftworksAI
- Stavvy
- Truework
- Truv
- Valon
- Wage
- Wilqo
- Xactus
## Field mapping
| Provider | Flow | Canonical in | Vendor in | Vendor out | Canonical out |
| --- | --- | --- | --- | --- | --- |
| Argyle | `income` | `staircase-graph` | `income-input` | `income-argyle-output` | `staircase-graph` |
| Essent | `essent-insurance-flow` | `staircase-graph` | `essent-input` | `essent-output` | `staircase-graph` |
| Plaid | `asset` | `staircase-graph` | `asset-input` | `asset-plaid-output` | `staircase-graph` |
| Xactus | `credit-verify` | `staircase-graph` | `credit-input` | `credit-mismo231` | `staircase-graph` |
## Errors
`400``403``404``422`
## More in Integration
- Next product: Job