Integration Patterns

Integration Patterns

Core Concepts

1. API Design for Financial Systems

Financial APIs serve position data, transaction history, account information, reference data, and order submission. Design conventions differ from general-purpose APIs due to the sensitivity, auditability, and volume of financial data.

REST conventions: Resource-oriented design with nouns for financial entities -- /accounts/{id}/positions, /transactions, /orders/{id}/executions. Use HTTP methods semantically: GET for reads (positions, balances), POST for actions (order submission, transfer initiation), PUT/PATCH for updates (account preferences, model assignments). Return standard HTTP status codes with domain-specific error bodies including error codes, human-readable messages, and correlation IDs for traceability. Financial APIs must distinguish between synchronous operations (position lookup returns immediately) and asynchronous operations (transfer initiation returns a 202 Accepted with a status polling URL or webhook callback). Long-running operations such as bulk rebalancing or batch trade submission should use the async pattern to avoid client-side timeouts.

WebSocket and streaming: For real-time use cases (position updates, order status, market data), WebSocket connections provide server-push capability without polling overhead. Financial WebSocket APIs require heartbeat/ping-pong to detect stale connections, automatic reconnection with state recovery (the client must receive missed updates on reconnect), and back-pressure handling when the consumer cannot keep pace with the producer.

Versioning: URL-based versioning (/v2/positions) is the dominant pattern in financial APIs due to its visibility and cacheability. Breaking changes (field removal, type change, semantic change) require a new version; additive changes (new optional fields) do not. Maintain at least two concurrent versions with a published deprecation timeline (typically 12-18 months in financial services).

Pagination for large datasets: Position and transaction endpoints routinely return thousands of records. Cursor-based pagination (opaque next-page token) is preferred over offset-based for consistency during concurrent writes. Include total count estimates, page size limits, and sorting parameters. For bulk data extraction, offer a separate export/download endpoint returning files rather than paginated API calls.

Rate limiting and throttling: Protect systems from burst traffic during market events (open, close, volatility spikes). Use token bucket or sliding window algorithms. Return 429 Too Many Requests with Retry-After headers. Distinguish rate limits per client, per endpoint, and per operation type (reads vs writes).

Authentication and authorization: OAuth 2.0 client credentials flow for service-to-service. API keys with HMAC request signing for simpler integrations. Mutual TLS (mTLS) for high-security custodian and clearing connections. Role-based access control scoped to accounts, operations (read-only vs read-write), and data sensitivity levels. Token expiration and rotation policies must be automated -- expired credentials are a leading cause of integration outages.

Integration testing: Financial API integrations require dedicated testing environments (UAT/sandbox) that mirror production behavior including realistic data volumes, error scenarios, and latency characteristics. Contract testing (verifying both producer and consumer conform to the API specification) prevents integration regressions during independent deployments. Custodians and data vendors typically provide certification environments; completing certification is a prerequisite for production connectivity. Maintain a suite of integration tests that exercise the happy path, each documented error code, timeout behavior, pagination boundaries, and rate limit responses.

2. FIX Protocol

The Financial Information eXchange (FIX) protocol is the dominant messaging standard for electronic trading, connecting buy-side firms, sell-side firms, exchanges, ECNs, and alternative trading systems.

Protocol structure: FIX messages are sequences of tag=value pairs delimited by SOH (ASCII 0x01). Each message has a header (BeginString, BodyLength, MsgType, SenderCompID, TargetCompID, MsgSeqNum, SendingTime), a body (message-type-specific fields), and a trailer (CheckSum). Tags are numeric (e.g., Tag 35 = MsgType, Tag 55 = Symbol, Tag 44 = Price, Tag 38 = OrderQty).

Session layer: Manages connectivity, sequencing, and recovery. Logon (MsgType=A) establishes the session with sequence number synchronization. Heartbeat (MsgType=0) and TestRequest (MsgType=1) monitor connection health. ResendRequest (MsgType=2) and SequenceReset (MsgType=4) handle gap recovery after disconnection. Logout (MsgType=5) terminates the session. Sequence numbers are strictly monotonic per session per direction; gaps trigger automatic recovery.

Key application messages: NewOrderSingle (MsgType=D) submits an order. ExecutionReport (MsgType=8) reports fills, partial fills, cancellations, and rejects. OrderCancelRequest (MsgType=F) and OrderCancelReplaceRequest (MsgType=G) modify or cancel orders. MarketDataRequest (MsgType=V) subscribes to market data. MarketDataSnapshotFullRefresh (MsgType=W) and MarketDataIncrementalRefresh (MsgType=X) deliver market data.

FIX versions: FIX 4.2 remains widely deployed for equity order routing. FIX 4.4 added improved support for multi-leg instruments, allocations, and position management. FIX 5.0 (with FIXT 1.1 transport) decoupled the session and application layers, enabling transport independence and versioned application messages. Most new implementations target FIX 4.4 or FIX 5.0/FIXT 1.1 depending on counterparty requirements.

FIX engines and libraries: QuickFIX (open-source, C++/Java/.NET/Python), QuickFIX/J (Java), Chronicle FIX (low-latency Java), Cameron FIX (commercial), Onix FIX (commercial, high performance). The engine handles session management, message parsing, sequencing, and persistence. Application logic connects via callbacks (onMessage handlers per message type).

FIX connectivity management: Each counterparty requires a separate FIX session with agreed-upon CompIDs, message versions, custom tags, and testing in UAT before production. Managing 20-50 FIX sessions across brokers, exchanges, and custodians is a material operational burden. Use a FIX hub or order management system to centralize session management.

Custom tags and extensions: Counterparties frequently require custom FIX tags (tag numbers above 5000) for proprietary fields -- internal order IDs, strategy codes, clearing instructions, or regulatory identifiers. Document all custom tags per counterparty in the FIX specification agreement. Validate inbound custom tags against the agreed specification to detect schema drift.

Drop copy sessions: A drop copy (FIX session type) provides a real-time copy of all execution reports to a secondary consumer (risk system, compliance, middle office) without affecting the primary trading session. Drop copies are essential for firms that need real-time trade surveillance or independent position tracking alongside the OMS.

FIX performance considerations: For high-frequency trading, FIX message parsing and serialization latency matters. Binary FIX encodings (SBE -- Simple Binary Encoding, used with FIX 5.0 FIXT) reduce parsing overhead by orders of magnitude compared to text-based FIX. For typical buy-side order routing (hundreds to thousands of orders per day), text-based FIX 4.4 is more than adequate and far simpler to debug -- log files are human-readable.