Configuring Imports

<!-- TIER:1 -->

Configuring Imports

An import is the data destination in a Celigo integration. It takes records from an upstream step and writes them to an external system -- REST APIs, databases, ERPs, file servers, or AI models. Every import is bound to exactly one connection and one adaptor type.

Imports handle six concerns:

• Field mapping -- transforming source fields into the destination system's expected format (including value resolution via static maps and lookup tables). Uses Mapper 2.0 (mappings[] array) by default; NetSuite and Salesforce imports only support Mapper 1.0 (mapping.fields[] / mapping.lists[])
• Operation logic -- create, update, upsert, delete, attach/detach
• Hooks -- JavaScript pre/post processing at various pipeline stages (preMap, postMap, postSubmit). File-based imports that generate files from records also support postAggregate
• One-to-many -- fan out child records from a parent. Set oneToMany: true and pathToMany to the child array path (e.g., "lineItems") when one source record should create multiple import operations
• Response mapping -- extract fields from the import's API response back into the record for downstream steps. Configured on the flow's pageProcessors[] entry, but planned when building the import. The response is available via _json (the raw API response) and errors. Use _json.fieldName to extract from the response (e.g., _json.id for a created record's ID, _json.output.1.content.0.text for OpenAI responses). Response mapping uses Transformation 1.0 syntax (extract/generate pairs), not the newer expression-based transforms
• postResponseMap hook -- JavaScript processing after response mapping merges the response back into the record. Configured on the flow's pageProcessors[] entry, but planned when building the import. Use to transform or enrich the merged record before downstream steps

Imports are used across flows, APIs, and tools.

Import Execution Pipeline

When records arrive at an import step, this pipeline executes in strict order:

1. Input filter (optional) -- discards records before any processing (configured as an expression on the flow's pageProcessors[] entry)
2. preMap hook (optional) -- JavaScript processing before field mapping
3. Field mapping -- Mapper 2.0 or 1.0 maps source fields to destination fields, including lookups and hardcoded values
4. postMap hook (optional) -- JavaScript processing after field mapping, before submission
5. Submit to destination -- writes the mapped record to the external system
6. postSubmit hook (optional) -- JavaScript processing after the destination responds (access response data, log results, trigger side effects)
7. Response mapping (optional) -- carries data from the destination response back into the record for downstream steps. Configured on the flow's pageProcessors[] entry, not on the import itself
8. postResponseMap hook (optional) -- JavaScript processing after response mapping merges response data back into the record

Key distinction: Response mapping lives on the flow's pageProcessors[] entry, not on the import resource. When building an import that needs to pass data downstream, plan the response mapping at flow design time.

Categories of Import

Record-Based Imports

Submit structured records to APIs, databases, or ERPs. The vast majority of imports.

• NetSuiteDistributedImport -- high-performance SuiteApp writes (add, update, addupdate, delete, attach, detach)
• HTTPImport -- REST/GraphQL APIs (POST, PUT, PATCH, DELETE). Supports connector-assisted (formType: "assistant") and GraphQL (graph_ql) modes
• SalesforceImport -- Salesforce CRUD via SOAP, REST, Bulk, or Composite Record API
• RDBMSImport -- SQL databases (Snowflake, PostgreSQL, MySQL, SQL Server, Oracle). Uses per_record, bulk_insert, or bulk_load query types
• MongodbImport, DynamodbImport, JDBCImport -- other databases

File-Based Imports

Write or upload files to remote storage. Require the file{} configuration block. Two modes:

• Record-to-file -- aggregates incoming records into a file (CSV, JSON, XML, XLSX). The file{} block defines the output format.
• Blob passthrough -- transfers a binary blob as-is from an upstream export. Set the blobKeyPath field to the path in the record that contains the blob key.

Adaptor types:

• HTTPImport with http.type: "file" -- upload files over HTTP to cloud storage APIs (Google Drive, Box, Dropbox, Azure Blob Storage)
• FTPImport -- CSV, XML, JSON, XLSX, EDI files to FTP/SFTP
• S3Import -- objects to Amazon S3
• AS2Import -- AS2 EDI file transmission
• FileSystemImport -- local/on-premise filesystem writes

AI Imports

Invoke AI models for classification, extraction, or safety checks. No _connectionId required unless using BYOK.

• AiAgentImport -- OpenAI or Gemini model invocations with structured output, tool use, and reasoning
• GuardrailImport -- PII detection, content moderation, or custom AI-based validation

Stack and Tool Imports

• WrapperImport -- custom pre-built stack connectors (Walmart, BigCommerce)
• ToolImport -- invoke a Celigo Tool resource

Import Matching (create / update / upsert)

The core decision on most record-based imports is the operation -- what happens to each record at the destination. Create requires no match key; update and delete require one; upsert checks first and does whichever applies. Users describe the intent in business terms ("look up the customer and update them", "match by email and upsert", "skip the ones that already exist") that resolve into five behaviors:

• Always create -- every record is submitted as new. No matching, no checks.
• Create only if missing -- match-check first; submit as create if not found, skip silently if found.
• Update only if exists -- match-check first; submit as update if found, skip silently if not.
• Update only if exists, fail if missing -- strict variant; no-match records error out instead of skipping.
• Upsert -- submit a create when no match is found, an update when matched. The catch-all, and the right default when the user is vague.

When matching applies, decide four things: the matching behavior, the match key field(s) (email, external_id, customer.id -- required for anything other than always-create), the on-match action (update, skip, fail), and the on-no-match action (create, skip, fail).

How a destination implements matching is adaptor-specific -- there is no single "matching mode" field. A destination might expose: a native upsert keyed off an external ID (Salesforce upsert, NetSuite upsert, RDBMS ON CONFLICT); a distinct addupdate operation that handles both paths in one call; an ignoreExisting flag paired with a lookup that probes before writing; two separate create and update endpoints with no upsert variant (see composite imports below); or a lookup endpoint plus separate create and update endpoints, where the lookup runs pre-write to drive the create-vs-update decision. Some destinations have no matching concept at all -- writing a CSV to FTP, sending an email, posting to a webhook -- so every record goes out as-is.

Prefer import-level matching over a separate lookup step. Imports natively support this pre-write check, so a single import that does the matching and the write together means fewer steps, fewer round-trips, and no glue logic to maintain. A standalone lookup earns its place only when the looked-up data has a consumer beyond the write -- a router branching on something other than "does this exist", an AI agent reasoning over the result, or multiple downstream steps reading different fields. If the only consumer is the destination call itself, the work belongs inside the import.

Composite (two-endpoint) imports