Destinations

Destination Overview

Destinations are the exit point for data in the Atlas platform. They are configured in the Web Console under the Resources » Destinations heading in the main menu. A Destination is created with a particular type (e.g., HTTP, MQTT, TCP/UDP) and the configuration options are derived from that type. There are a few configuration items common to all Destinations, such as throttle rate, max backoff for retries, max number of retries, etc. When a delivery to any Destination is attempted, it should ideally succeed, but if the receiving system is down, or the throttle rate has been hit, or the destination is paused, that delivery will be queued and tried again later.

Destination Types

There are several types of Destinations implemented currently. If you have a request for a particular type of Destination, send us a note and we will follow up!.

HTTP Destinations

This type of destination takes parameters such as URL, connect timeout and HTTP method to use. A payload of type PAYLOAD_TYPE_HTTP should be sent to this type of destination. Once this destination is created, a signing secret is available to view in the configuration page. This secret will be used to generate an HMAC for each time this destination is targeted, and can be leveraged to verify the authenticity of an incoming HTTP request. The HMAC is generated as follows:

The HMAC and Timestamp are delivered as header values in the HTTP request. X-Atlas-Hmac contains the HMAC value and X-Atlas-Timestamp contains the timestamp. The HMAC depends on the the timestamp, but the timestamp does not neccesarily need to be checked for closeness to delivery time (unless delay attack prevention is desired). If not checking for a valid timestamp, this value can effectively be thought of as a nonce.

To verify the authenticity of an incoming request, the following workflow can be implemented (pseudocode below):

Supported Filter VARS

For HTTP Destinations, whole or part of the URL can be replaced at run time with the $URL$ variable. For example, to modify the subdomain you want a particular request to go to, you could set the URL field in the configuration page to:

Log Only Destinations

This type simply logs the payload as it is received. A payload of any type can be sent to this type of destination. It is not delivered to any endpoint, and therefore cannot fail. This is useful for debugging purposes, and when paired with the CLI tool, becomes a useful endpoint type for developoment.

TCP/UDP Raw Data Destinations

This type of destination is a raw TCP or UDP session to a given host/port combination. A payload of type PAYLOAD_TYPE_RAWBYTES should be sent to this type of destination. When a payload is delivered to this type of destination, the delivery agent will connect to the endpoint, send the payload data, and wait for up to 15 seconds for a response. The response will be logged. Currently, responses are limited to 4kiB in size. The delivery to this type of destination is considered successful if there is no socket-type error (e.g. no route to host).

HL7 Minimal LLP Destinations

This type of destination is a connection to a Health Level 7 (HL7) minimal low-level protocol server. A payload of type PAYLOAD_TYPE_RAWBYTES should be sent to this type of destination. When a payload is delivered to this type of destination, it will connect to the defined host at the defined port, and send the HL7 message to it. The delivery agent will wait for up to 30 seconds for an ACK from the HL7 LLP server. The ACK is recorded. If the ACK is anything other than AA which symbolizes a good acknowledgement, the payload will be queued for retry according to the destination configuration.

MQTT Broker Topic Destinations Roadmap Item

The MQTT Broker Topic destination type is currently a roadmap item. For status on this please see the roadmap page.

This destination type will allow you to publish an MQTT message to the Atlas broker at mqtt.atlasconnex.com where the topic is determined by your Source ID, authorization key, and optionally the authentication secret and timestamp. The body of the MQTT published message will be the payload data.

MQTT Publisher Destinations Roadmap Item

The MQTT Subscriber destination type is currently a roadmap item. For status on this please see the roadmap page.

This destination type will allow you to provide Atlas Connex with a broker host, client credentials (if required) and a topic to publish to. The delivery agent will publish a message with the given QoS. Using QoS 2 will allow Atlas Connex to verify the success of the delivery to the broker. The body of the MQTT published message will be the payload data.

Destination Creation

In the Web Console, navigate to Resources » Destinations. Select the + button to select which type of Destination to create. In the creation modal for your selected type, fill in the relevant details, and click the Create button to completion the creation. You will be redirected to the newly created Destination’s configuration page.

Create a Destination
Create a Destination

Destination Management

In the Console, you can update a Destination, delete it, or change its operational status. For example, if a Destination has reached its throttle rate for the hour window, it will show a Throttled status in the Console. If you want to override the throttle, you can click the play button at the bottom of the configuration page. If a Destination has hit its maximum failures and is now in the Failed state, once you fix the failure (or want to simply try again) you can click the play button as well. If you want to pause a Destination (and thus cause all events which are routed to it to be queued) you can click the pause button, and the Destination will be put into the Paused state.

Destination Configuration

All configuration options can be modified in the Destination’s configuration page.

Common Configuration Items

All Destinations share several common configuration items which inform the Atlas delivery agents how to do their job.

Throttle Rate

This parameter sets a maximum-try-per-hour limit on this destination. This is useful if you have an endpoint that is rate-limited. Set to -1 to disable throttling. If a Destination is throttled, an Alert will be generated for your account.

Max Retries

This parameter sets the maximum number of times the Atlas delivery agents will attempt to deliver your Event payload before it is declared a failure. Set to -1 to disable the maximum. If a Destination fails, a Failure notification will be generated for your account.

Max Retry Backoff

This parameter sets the maximum backoff in seconds that the Atlas delivery agents will use. Atlas delivery agents use an exponential backoff algorithm, so this parameter caps how long the backoff will be for each successive try. The algorithm is roughly:

So for a Max Retry Backoff of 60 and a Max Retries of 10, your backoff times are roughly:

Failure Attempt # Backoff Time (s)
0 1
1 2
2 4
3 8
4 16
5 32
6 60 (max backoff kicks in here)
7 60
8 60
9 60

CLI Enabled

This toggles whether event payloads will be made available to registered CLI sessions rather than the endpoint as configured.

Questions, clarification, or incorrect info? Send us mail!

Auto-generated Thu 14 Apr 2022 08:06:51 PM CDT