Configuring Filters

<!-- TIER:1 -->

Configuring Filters

A filter is a record gate that decides which records continue through the pipeline and which are silently dropped. Filters are configured directly on exports, imports, and flow router branches -- not as standalone resources.

Concerns when configuring a filter:

• Placement -- where in the pipeline the filter runs (output filter on export, input filter on lookup, pre-import filter, branch routing condition)
• Mechanism -- expression-based rules (declarative, no code) vs. script-based (full JavaScript control)
• Expression syntax -- prefix-notation S-expressions with operators, field access (extract), type coercions, and logical combinators (and/or/not)
• Settings references -- dynamic filter values pulled from flow, export, import, or integration settings via settings accessor
• Script alternative -- when expression rules cannot handle the logic (multi-step conditionals, date math, external lookups), use a filter script instead

Four Places Filters Appear

Output filter (export filter)

Applied after records are retrieved from the source system. Records that match continue through the flow; records that do not match are silently dropped. Available on all export types.

Input filter (lookup export inputFilter)

Applied to incoming records before they trigger a lookup API call. Records that do not match skip the lookup step entirely -- they pass through without enrichment. This reduces unnecessary API calls. Only relevant when isLookup: true.

Import filter (import filter)

Applied to records before they are sent to the destination system. Records that do not match are dropped and never submitted to the destination. Useful for conditional writes (e.g., only import records with a non-empty email).

Branch input filter (router branch inputFilter)

Applied per-branch in a flow router to route records to different processing paths. Each branch has its own filter expression. The last branch can omit a filter to serve as a catch-all. Configured on the flow's routers[].branches[] entries, not on exports or imports.

Quick Reference

Filter Type Decision Matrix

Expression vs. Script

Prefer expressions -- they are simpler, faster, and do not require a separate script resource. Use scripts only when expression operators cannot handle the logic.

Operator Quick Reference

Field access: ["extract", "fieldName"] for record fields, ["settings", "flow.settingName"] for configuration values.

Type coercions: ["number", ...], ["string", ...], ["boolean", ...], ["epochtime", ...].

Transformations: ["lowercase", ...], ["uppercase", ...], ["floor", ...], ["ceiling", ...], ["abs", ...].

Schema Index

Related Skills

• configuring-exports > Quick Reference -- export configuration where output filter and input filter are wired
• configuring-imports > Quick Reference -- import configuration where import filter is wired
• writing-scripts > Record-Level Processors -- script-based filter hook signature and data shapes
• building-flows > Flow Topologies -- branching routers with per-branch input filters
• writing-handlebars > Quick Reference -- Handlebars in dynamic filter values

<!-- TIER:2 -->

How to Configure a Filter

1. Determine filter placement

Where in the pipeline should filtering happen?

• After export retrieval -- use an output filter on the export (filter)
• Before a lookup call -- use an input filter on the lookup export (inputFilter)
• Before import submission -- use a filter on the import (filter)
• For branch routing -- use branch input filters on the flow router

2. Choose expression vs. script

Can the logic be expressed as field comparisons combined with and/or? Use an expression. Need loops, date math, cross-field calculations, or external API calls? Use a script.

3. Build the expression rules

Expression rules use prefix (S-expression) notation. Every rule is an array where the first element is the operator and remaining elements are operands.

Pattern: Single condition

[operator, [extract, fieldName], value]

Pattern: Multiple conditions (all must match)

[and, [condition1], [condition2], ...]

Pattern: Multiple conditions (any must match)

[or, [condition1], [condition2], ...]

Pattern: Field access with type coercion

[operator, [number, [extract, fieldName]], numericValue]

Pattern: Dynamic value from settings

[equals, [string, [extract, fieldName]], [string, [settings, flow.settingName]]]

4. Set the filter on the resource

Use the CLI set command to add or update a filter on an existing export or import. The set command does GET-modify-PUT automatically.

# Set an expression filter on an export (output filter)
celigo exports set <id> filter.type=expression filter.expression.version=1 filter.expression.rules='["notempty",["extract","email"]]'

# Set an expression filter on an import
celigo imports set <id> filter.type=expression filter.expression.version=1 filter.expression.rules='["notequals",["extract","status"],"cancelled"]'

For complex expressions, use JSON input:

# Create or update the full export with filter included
celigo exports update <id> < export-with-filter.json

5. Wire a script filter (when expressions are insufficient)

Create the script resource first, then reference it:

# Create the filter script
celigo scripts create < filter-script.json

# Wire it to the export
celigo exports set <id> filter.type=script filter.script._scriptId=<scriptId> filter.script.function=filterRecords