Configuring Exports

<!-- TIER:1 -->

Configuring Exports

An export is the data source in a Celigo integration. It connects to an external system and pulls data into the pipeline. Exports serve two roles:

• Source -- the starting point that fetches the primary batch of records
• Lookup -- a mid-flow enrichment step (isLookup: true) that fetches additional data per-record during processing

Both roles are used across flows, APIs, and tools.

Beyond fetching data, exports also handle post-retrieval processing before records enter the pipeline:

• Output filter -- expression-based filtering to skip records that don't match criteria
• Transform -- Transformation 2.0 expression rules to reshape/flatten response data before mapping
• preSavePage hook -- JavaScript processing on the full page of records before they enter the pipeline
• One-to-many -- when used as a lookup, fan out child records from a parent. Set oneToMany: true and pathToMany to the child array path so each child triggers a separate lookup. Once fanned out, the array element itself is the record -- see One-to-many fan-out
• Response mapping -- when used as a lookup, extract fields from the lookup response back into the record. Configured on the flow's pageProcessors[] entry, but planned when building the lookup export. The response contains a data array and an errors array. Use data[0].fieldName when you expect a single result (e.g., fetching one order by ID). When multiple results are expected, map the whole array with extract: "data" (downstream steps then read it as a normal JSON array, or fan out over it with one-to-many), or use a lists entry with data[*].fieldName extracts to build a reshaped array. Do not put data[*].fieldName in a top-level fields[].extract -- the wildcard is silently ignored there and nothing is merged. See writing-mappings > Response Mapping Reference. Response mapping uses Transformation 1.0 syntax (extract/generate pairs), not the newer expression-based transforms
• postResponseMap hook -- JavaScript processing after response mapping merges the lookup response back into the record. Configured on the flow's pageProcessors[] entry, but planned when building the lookup export

Export Execution Pipeline

When a flow runs, each export executes this pipeline in strict order:

1. API request / query / file read -- fetches raw data from the external system
2. Response parsing -- resourcePath extracts the record array from the response body or file (e.g., http.response.resourcePath for HTTP, file.json.resourcePath for JSON files, XPath for XML)
3. Transformation (optional) -- transform reshapes individual records after extraction (Transformation 2.0)
4. Output filter (optional) -- discards records that don't match filter expression rules
5. preSavePage hook (optional) -- JavaScript processing on the full page of records

Key distinction: resourcePath tells the export WHERE to find records in the response. Transforms reshape WHAT each record looks like after extraction. When a user says "extract records from X" or "treat each X as a separate record", that's almost always a resourcePath change, not a transform. Use transforms when you need to flatten nested objects, rename fields, or restructure individual records.

Three Categories of Export

Not all exports work the same way. Before building, understand which category you need:

Listeners

Receive data pushed to Celigo from an external system. No polling, no scheduling -- the source system sends data when events happen.

• WebhookExport -- inbound HTTP listener (no connection required)
• AS2Export -- AS2 EDI file reception
• Distributed exports (type: "distributed") -- real-time event-driven push for NetSuite (via SuiteScript) and Salesforce (via streaming API). The platform installs listeners in the source system that fire when records change.
• Change data capture (type: "stream") -- MongoDB change streams that tail the oplog for real-time record changes.

When to use: The source system supports outbound webhooks, push notifications, or change data capture and you want real-time processing.

File Transfers

Read files from a remote location, then either parse them into records or transfer them as blobs.

• FTPExport / S3Export / FileSystemExport -- fetch files from FTP/SFTP, S3, or local filesystem
• HTTPExport with http.type: "file" -- fetch files over HTTP from cloud storage APIs (Google Drive, Box, Dropbox, Azure Blob Storage). The HTTP connector handles auth; the file{} config handles parsing.
• NetSuiteExport with netsuite.type: "file" -- fetch and parse files (CSV, JSON, XLSX, XML, EDI) from the NetSuite file cabinet
• Parsed mode (file.output: "records") -- CSV, XML, JSON, XLSX, EDI files are parsed into individual records
• Blob mode (type: "blob") -- binary files transferred as-is without parsing. Supported on HTTPExport, NetSuiteExport, SalesforceExport, FTPExport, and S3Export.

When to use: The source system drops files (CSV, EDI, XML, etc.) into a directory, bucket, file cabinet, or cloud storage rather than exposing a record-based API.

Record-Based Exports

Actively fetch batches of records from an API or database on a schedule.

• HTTPExport -- REST/GraphQL APIs
• NetSuiteExport -- saved searches, restlets, SuiteQL
• SalesforceExport -- SOQL/Bulk queries
• RDBMSExport -- SQL SELECT queries
• MongodbExport, JDBCExport, DynamodbExport -- other databases
• WrapperExport -- custom stack (Walmart, BigCommerce)

When to use: You need to poll an API or query a database for records on a schedule (full fetch or delta/incremental).

Quick Reference

Adaptor Decision Matrix

Raw HTTP is the fallback, not the default. Pick the most specific match, in order:

1. Native adaptor -- if the application has its own row (NetSuite, Salesforce, databases, FTP/S3), use it. Do not build an HTTPExport against that app's REST API.