Market Data

Market Data

Core Concepts

1. Market Data Types

• Trade data: Last sale price, quantity, timestamp, condition codes (regular, odd lot, opening/closing), cumulative volume, VWAP.
• Quote data: NBBO (best bid/offer across all exchanges), bid/ask sizes. Quote updates vastly outnumber trade updates in liquid instruments.
• Depth of book: Multiple price levels beyond the NBBO with resting order quantities. Aggregated depth (size per level) or order-by-order (individual orders visible).
• Index data: Real-time index values, composition, weightings, total return values. Sources include exchange-calculated (S&P 500 via Cboe) and third-party (MSCI, FTSE Russell).
• Fixed income pricing: Dealer quotes, evaluated pricing (ICE, Bloomberg BVAL), TRACE trade reports for corporates, EMMA for municipals. Inherently less standardized than equities.
• Options data: Chains (strikes/expirations), Greeks, implied volatility, volume, open interest. OPRA provides the consolidated options feed.
• Fundamental data: Earnings, financial statements, corporate actions, analyst estimates. Sourced from vendors (Bloomberg, FactSet, S&P Capital IQ) rather than exchange feeds.
• News and events: Headlines, economic calendar (FOMC, employment), corporate events (earnings dates, ex-dates), sentiment scores.

2. Data Levels

Level 1 — Top of Book: NBBO, last sale, volume, daily OHLC. Sufficient for portfolio management, client reporting, and order entry. Lowest cost and bandwidth.

Level 2 — Market Depth: Multiple price levels with aggregate size (top 5-20 levels per side). Reveals liquidity beyond the NBBO. Essential for active trading, market impact assessment, and algorithmic execution (TWAP, VWAP). Higher cost and bandwidth.

Level 3 — Full Order Book: Individual order detail (price, size, order ID) enabling complete book reconstruction and order lifecycle tracking. Provided by direct feeds (Nasdaq ITCH, NYSE Arca). Required for market making, HFT, and queue position modeling. Highest cost — hundreds of thousands of messages per second per exchange.

3. Consolidated Tape vs Direct Feeds

Securities Information Processors (SIPs): CTA/CQS for NYSE-listed (Tape A/B), UTP for Nasdaq-listed (Tape C), OPRA for options. SIPs collect data from all exchanges, compute the NBBO, and disseminate a consolidated stream. Under Reg NMS, the SIP NBBO is the regulatory benchmark for best execution.

Direct exchange feeds: Proprietary feeds from individual exchanges (NYSE Arca, Nasdaq TotalView/ITCH, Cboe PITCH, IEX DEEP) delivering order-by-order data with lower latency than the SIP. A firm must subscribe to multiple feeds and compute NBBO internally. Each exchange uses different protocols requiring per-exchange parsers.

4. Market Data Vendors

Cost figures throughout this skill reflect 2024-2025 list pricing; verify current pricing with vendors before budgeting.

Bloomberg: Terminal ($20K-$25K/user/year), B-PIPE (enterprise real-time feed), Data License (bulk EOD/reference data), BEAP (cloud API).

Refinitiv (LSEG): Eikon (desktop, lower cost than Bloomberg, strong FX/FI), Elektron/ LSEG Real-Time (enterprise feed), DataScope (bulk EOD), Tick History (historical ticks).

ICE Data Services: Consolidated feeds, evaluated fixed income pricing (widely used for NAV and regulatory reporting), ICE Benchmark Administration.

FactSet: Research-oriented, flexible API delivery, competitive pricing for smaller buy-side, strong Excel/portfolio management integration.

S&P Capital IQ / Market Intelligence: Comprehensive fundamentals, credit ratings, company filings. Morningstar: Fund/ETF data, ratings, Morningstar Direct for research.

Free/open sources: Exchange websites and financial portals provide delayed (15-min) quotes. Useful for non-time-sensitive display but limited reliability and coverage.

Vendor selection criteria: Asset class coverage, latency, reliability/uptime SLA, API quality, total licensing cost (including exchange fees), historical data depth, support, data quality handling.

5. Market Data Licensing and Entitlements

License categories: Non-professional (retail, personal use, lower fees) vs professional (business use, significantly higher). Display (human views on screen) vs non-display (automated systems: algorithms, risk engines, pricing — fees based on application type, not per-user). Derived data (substantially transformed; redistribution may be permitted if original data cannot be reverse-engineered; policies vary by exchange).

Licensing models: Per-user/per-device (exact monthly count required), enterprise (flat fee covering a defined entity), usage-based non-display (fees by application category: trading, risk, valuation).

Reporting obligations: Monthly/quarterly subscriber counts submitted to each exchange or via data vendor. Under-reporting triggers back-billing, penalties, and contract termination.

Redistribution: Raw exchange data requires explicit redistribution agreements and additional fees for client-facing display. Vendors typically handle redistribution for data consumed through their platforms.

Cost management: Audit usage periodically to eliminate unused subscriptions. Use delayed data where real-time is unnecessary. Track non-display use — many firms discover unreported non-display obligations only during exchange audits.

6. Market Data Distribution Architecture

Ticker plant: Central ingestion and normalization layer. Parses exchange protocols (ITCH, PITCH, FIX), normalizes to unified schema, maps symbology, caches latest values, applies conflation, and monitors feed health.

Fan-out patterns: Topic-based pub-sub (dominant pattern; middleware: Solace, TIBCO, 29West, Kafka for lower-latency needs), request-reply (REST for on-demand lookups), multicast (network-level fan-out for ultra-low-latency co-located environments).

Conflation: Throttles update rates for slower consumers. Time-based (deliver latest value every N ms), change-based (suppress duplicates), priority-based (never conflate trades; conflate quotes for slower consumers).

APIs: REST for historical/reference data, WebSocket for real-time streaming to web/mobile applications, proprietary binary APIs for ultra-low-latency consumers.

Cloud services: AWS Data Exchange, Google Cloud Marketplace, Azure Data Share. Adds network latency (unsuitable for latency-sensitive trading) but appropriate for analytics, portfolio management, and client-facing applications.

7. Historical Market Data

EOD databases: Daily OHLCV. Sufficient for portfolio analytics and long-horizon backtesting. Tick-level data: Every trade/quote with microsecond timestamps. Required for intraday backtesting and microstructure research. A single day of U.S. equity ticks may exceed 10-20 TB. Providers: Refinitiv Tick History, NYSE TAQ, LOBSTER.