Filters

Filter Overview

Filters are customizable blocks of Python code that sit between Sources and Destinations. Filters are used to transform the Payload in a variety of ways, or to make decisions about whether the Payload should continue being processed. For example, if you have an Event with a JSON payload, but you need to consume that Event as XML, you can use a Filter to transform that Payload. If you have a Source that receives many event types, but you are only interested in flowing one of those types on to a Destination, the Filter can be used to stop processing in its branch upon meeting your configured conditions. The Atlas platform also includes a variety of Utility classes for use in Filters which simplify the transformation of payloads.

Filter Basics

Every filter definition starts by defining a method run taking one parameter. For example:


Inside your run method body, you can manipulate the payload as you see fit. If you return None from your filter, the routing for this message will stop in that branch of the Route, otherwise the payload object you return will be passed to the next Filter in the chain as defined by your Route or to the Destination if it’s the final Filter. See Routes for how fanning out multiple branches works.

The Payload Object

The payload object that is passed into your run method is a Payload-type object which contains the payload data. Its structure differs based on the type of the payload.

HTTP Payloads

For HTTP payloads, the object has the following structure:


Raw Bytes Payloads

For Raw Bytes payloads, the object has the following structure:


Filter Creation

In the Web Console, navigate to Resources » Filters. Select the + button to create a new filter.

Create a Filter
Create a Filter


In the creation modal, give the filter a name, description, and select from one our templates to get you started. When finished, click the Create button and you’ll be taken to the Filter’s viewing page.

Filter Configuration

A filter is configured in the Filter Designer. You can access the Filter Designer from the Filter’s viewing page by clicking the Edit button at the bottom. Check out the documentation for the Filter Designer for more info on how to create/edit/debug/update a filter.


Edit a Filter
Edit a Filter


Payload Vars

Every payload has a special variable called vars which can be used to pass info to the eventual destination. For example, if you need to change the URL for and HTTP destination based on some filter input condition, you could set the variable $URL$ in the vars dict.

Vars Example

Filter Utilities

A number of utility classes and constants have been added to make Filter design and use simpler.

Constants

The following constants are available for use inside filters:

time

The time module has the same API as the standard Python time module.

datetime

The datetime module has the same API as the standard Python datetime module.

Example

dateutil

The dateutil module has the same API as the popular Python dateutil package.

Example

json

The json module has the same API as the standard python json module.

urllib

The urllib module is a subset of the standard Python urllib module. It pulls in the urllib.parse subset. See urllib.parse documentation.

hashlib

The hashlib module has the same API as the standard Python hashlib module.

hmac

The hmac module has the same API as the standard Python hmac module.

XMLElement

The XMLElement utility class provides XML tree creation and walking facilities.

Methods

Examples

Miscellaneous

The following non-exhaustive list of miscellaneous items are provided as helpers to building Filters.

print

If you need to access debug statements while designing a filter, you can use the standard print call. The output will be provided back when you validate in the Filter Designer.

Basic Types

The basic types in Python (int, float, str, dict, set, list) are all provided.

Classes

Classes can be defined inside the filter and used as part of the logic.

Questions, clarification, or incorrect info? Send us mail!

Auto-generated Thu 14 Apr 2022 08:06:51 PM CDT