Skip to content

protocol-fix.md

Latest commit

 

History

History
301 lines (211 loc) · 16.3 KB

protocol-fix.md

File metadata and controls

301 lines (211 loc) · 16.3 KB
Raw

FIX Protocol 5.0 in High-Frequency Trading

Overview and Evolution

The Financial Information eXchange (FIX) protocol is the de facto messaging standard for electronic trading across global financial markets. FIX 5.0, introduced alongside the FIXML schema and the FIX Orchestra specification framework, represents a significant architectural departure from earlier versions: it decouples the session layer from the application layer via the FIXT (FIX Transport) 1.1 standard. This separation is architecturally critical — it means the session-level handshaking (logon, heartbeats, sequence number management) is governed by FIXT 1.1, while the business messages (orders, executions, market data) belong to FIX 5.0's application dictionary. In practice, a single FIXT session can multiplex multiple application versions simultaneously, an important feature for firms running hybrid infrastructure.

Session Layer (FIXT 1.1)

Connection Lifecycle

A FIX session follows a strict state machine:

DISCONNECTED → CONNECTED → LOGGED_ON → ACTIVE → LOGGING_OUT → DISCONNECTED

The Logon (MsgType=A) message negotiates:

  • HeartBtInt (tag 108): heartbeat interval in seconds (typically 30s, but often 0 in HFT to suppress unnecessary traffic)
  • EncryptMethod (tag 98): almost universally 0 (None) in modern deployments; TLS is handled at the transport layer
  • ResetOnLogon (tag 553), ResetSeqNumFlag (tag 141): control sequence number reset behavior across sessions

Sequence Numbers

Every message carries a monotonically increasing MsgSeqNum (tag 34). Gaps trigger a ResendRequest (MsgType=2), and the counterparty responds with the missing messages or a SequenceReset (MsgType=4). In HFT, sequence gaps are catastrophic — they force retransmission latency and can trigger risk controls. Engines typically maintain a pre-allocated ring buffer of outbound messages to service resend requests without disk I/O.

Heartbeats and TestRequest

Heartbeat (MsgType=0) messages are sent when no traffic has occurred within HeartBtInt. A TestRequest (MsgType=1) probes liveness; the peer must echo it in a Heartbeat containing the TestReqID. HFT sessions often set HeartBtInt=0 to disable automatic heartbeats entirely, relying on the order flow itself as liveness proof.

Application Layer — Core Message Types

Order Flow

MsgType Name Description
D NewOrderSingle Submit a new order
F OrderCancelRequest Cancel an existing order
G OrderCancelReplaceRequest Modify (amend) an existing order
8 ExecutionReport Ack, fill, cancel confirm, reject
9 OrderCancelReject Reject of cancel/replace
j BusinessMessageReject Application-level reject

The ExecutionReport is the workhorse message. Its ExecType (tag 150) and OrdStatus (tag 39) fields together define the event:

ExecType=0 (New)          → order accepted by exchange
ExecType=1 (PartialFill)  → partial execution
ExecType=2 (Fill)         → full execution
ExecType=4 (Canceled)     → cancel confirmed
ExecType=8 (Rejected)     → order rejected
ExecType=D (Restated)     → order restated (e.g., after corporate action)
ExecType=I (OrderStatus)  → response to OrderStatusRequest

Critical Order Fields

Tag  11  ClOrdID         Client-assigned order ID (must be unique per session)
Tag  37  OrderID         Exchange-assigned order ID
Tag  41  OrigClOrdID     Reference to the order being canceled/replaced
Tag  54  Side            1=Buy, 2=Sell, 5=SellShort
Tag  38  OrderQty        Total order quantity
Tag  44  Price           Limit price (OrdType=2)
Tag  40  OrdType         1=Market, 2=Limit, 3=Stop, 4=StopLimit, ...
Tag  59  TimeInForce     0=Day, 1=GTC, 3=IOC, 4=FOK, 6=GTD
Tag  60  TransactTime    UTC timestamp of the event (microsecond precision in FIX 5.0)
Tag 151  LeavesQty       Remaining open quantity
Tag  14  CumQty          Total filled quantity
Tag   6  AvgPx           Average fill price

ClOrdID management is a major source of bugs in FIX engines. It must be unique per session per day and must be echoed back in all ExecutionReport messages. In HFT, ClOrdID is typically an atomic counter formatted as a fixed-width string to avoid std::to_string overhead on the hot path.

Wire Format and Parsing

Tag=Value Encoding

FIX messages are ASCII text with tag-value pairs delimited by SOH (\x01):

8=FIX.5.0SP2\x019=148\x0135=D\x0149=CLIENT\x0156=EXCHANGE\x0134=1\x01
52=20240420-09:30:00.000000\x0111=ORDER001\x0155=AAPL\x0154=1\x0138=100\x01
40=2\x0144=189.50\x0159=0\x0160=20240420-09:30:00.000000\x0110=042\x01

Key structural rules:

  • Tag 8 (BeginString) must be the first field
  • Tag 9 (BodyLength) must be the second field — it is the byte count from tag 35 to the delimiter before tag 10
  • Tag 10 (CheckSum) must be the last field — it is the sum of all byte values modulo 256, zero-padded to 3 digits
  • Tags 8, 9, and 10 are excluded from the checksum calculation of BodyLength but included in CheckSum

Parsing Strategy in C++

Naïve parsing with std::map<int, std::string_view> is far too slow for HFT. Production engines use:

1. Zero-copy parsing with std::string_view:

// Point directly into the receive buffer — no allocation
struct FixField {
    int tag;
    std::string_view value; // into the raw buffer
};

2. SOH-scan loop with SIMD acceleration:

// Use _mm256_cmpeq_epi8 to find SOH bytes 32 at a time
// Then extract field boundaries with _tzcnt_u32 on the comparison mask

3. Tag dispatch via a perfect hash or jump table:

// Tags are bounded integers — a direct-mapped array indexed by tag number
// is O(1) and cache-friendly for the ~50 tags you actually care about
std::array<FixField*, 10000> tag_index{}; // sparse, but fast

4. Integer and price parsing without atoi/strtod:

// Hand-rolled SWAR (SIMD Within A Register) decimal parsing
// Prices as fixed-point integers (e.g., price * 10000) — never floating point
inline int64_t parse_price(std::string_view s) {
    // Parse mantissa and decimal position in one pass
}

5. Checksum validation as optional: Many HFT engines skip checksum validation on inbound messages entirely on the hot path (trusting TCP integrity) and only validate asynchronously for audit purposes.

Session Management in C++

A production FIX session engine in C++ will exhibit the following design characteristics:

Memory Layout

// Avoid heap allocation on the critical path entirely
struct alignas(64) OutboundMessage {
    char   buf[512];        // pre-allocated, fits in cache lines
    size_t len;
    int64_t seq_num;
    int64_t send_ts_ns;
};

// Ring buffer for retransmission
static constexpr size_t RING_SIZE = 1 << 20; // 1M messages
std::array<OutboundMessage, RING_SIZE> outbound_ring;
std::atomic<uint64_t> ring_head{0}, ring_tail{0};

I/O Architecture

FIX 5.0 in HFT almost never runs over vanilla TCP with kernel sockets. Common approaches:

  • Kernel-bypass networking: Solarflare/Xilinx OpenOnload (ef_vi), Mellanox VMA, or DPDK — delivers single-digit microsecond round-trip times vs. 50–100µs for kernel TCP
  • RDMA over Converged Ethernet (RoCE): Used for ultra-low-latency co-location links to exchange gateways
  • Pre-connected persistent sessions: Sessions stay connected 24/7; logon/logoff only at scheduled maintenance windows. Re-connecting mid-session costs 100s of microseconds — unacceptable
// Non-blocking send with io_uring (Linux 5.1+) for batched async I/O
// Avoids syscall overhead for individual messages
struct io_uring ring;
io_uring_queue_init(4096, &ring, IORING_SETUP_SQPOLL);

Threading Model

┌──────────────────────────────────────────────────────────┐
│  Core 0 (isolated)    │  Core 1 (isolated)               │
│  Network Rx / Parser  │  Strategy / Order Generator       │
│  → lock-free queue →  │  → pre-built FIX message →        │
│                       │  Core 2: FIX Serializer / Tx      │
└──────────────────────────────────────────────────────────┘
  • Rx and Tx run on CPU-pinned, isolated cores (SCHED_FIFO, isolcpus)
  • Communication via lock-free SPSC queues (e.g., boost::lockfree::spsc_queue or a hand-rolled power-of-two ring)
  • Busy-polling rather than epoll — eliminates kernel scheduling latency

FIX 5.0 SP2 — Key Business Features

Market Data (MsgType=V/W/X)

The MarketDataRequest (V) / MarketDataSnapshotFullRefresh (W) / MarketDataIncrementalRefresh (X) trio forms the market data subscription model. In HFT, direct exchange feeds (ITCH, PITCH, OMX MoldUDP64) are always preferred over FIX market data due to lower latency, but FIX MD is used for broker-provided consolidated feeds and for less latency-sensitive instruments.

The MDEntryType (tag 269) field distinguishes bids (0), asks (1), trades (2), settlement prices (6), and others, while NoMDEntries (tag 268) introduces a repeating group of book levels.

Repeating Groups

FIX 5.0's repeating groups are one of its most performance-hostile features. A group is introduced by a delimiter tag (the group count tag, e.g., tag 268) followed by a first-tag-of-group field that marks the start of each repetition. Parsing requires tracking group nesting depth — you cannot skip ahead without understanding the schema's group structure. This is a fundamental reason why FIX is slower to parse than binary protocols like SBE (Simple Binary Encoding) or ITCH.

// Repeating group parsing requires schema awareness
// The "NoXxx" tag tells you the count; the first member tag resets the group
void parse_group(const FixMessage& msg, int no_tag, int first_member_tag,
                 std::function<void(GroupFields&)> callback);

Execution Instructions and Algo Parameters

FIX 5.0 SP2 adds rich support for algorithmic execution through:

  • ExecInst (tag 18): bitmask of execution instructions — f (participate don't initiate), G (all or none), c (cancel on disconnect)
  • PegInstructions (tag group 576/577): peg-to-midpoint, peg-to-primary, etc.
  • StrategyParametersGrp (NoStrategyParameters, tag 957): free-form key-value parameters passed to sell-side algos — e.g., {"MaxParticipationRate": "0.10", "UrgencyLevel": "3"}
  • DisplayInstructions (iceberg orders): DisplayQty (tag 1138), RefreshQty (tag 1088)

Order Types Relevant to HFT

  • Limit (OrdType=2) + IOC (TimeInForce=3): The standard aggressive HFT order — cross the spread, take what's available, leave no resting quantity
  • Limit + Post-Only (encoded via ExecInst=c or venue-specific tags): Passive maker order; critically important for rebate capture strategies
  • Market-to-Limit (OrdType=K): Converts to a limit order at the touch if not immediately filled — used in some latency-arb strategies
  • Pegged orders: Mid-peg, primary-peg orders for market-making

FIX Orchestra and Schema-Driven Development

FIX Orchestra (introduced with FIX 5.0 SP2's ecosystem) provides machine-readable protocol rules in a formal XML/JSON schema. This enables:

  • Code generation for message builders and parsers (e.g., generating struct NewOrderSingle with typed fields directly from the Orchestra spec)
  • Automated conformance testing against a counterparty's published Orchestra file
  • Conditional field validation expressed as conditional expressions (e.g., Price is required when OrdType=Limit)

In a C++ shop, Orchestra specs are typically used as input to a code generator that produces zero-overhead, strongly-typed message structs with inline encode()/decode() methods.

FIX vs. Binary Protocols in HFT

FIX's ASCII encoding is a significant performance liability. A raw comparison:

Metric FIX 5.0 (Tag=Value) SBE (Simple Binary Encoding) ITCH 5.0
Message size (order) ~200–400 bytes ~40–60 bytes ~26 bytes
Parse time (optimized) ~500–1500 ns ~50–150 ns ~20–50 ns
Schema flexibility High Medium Low (fixed)
Human readability Yes No No
Adoption for OMS/EMS Universal Growing Exchange-specific

The typical HFT architecture therefore uses:

  • FIX 5.0 for order entry to prime brokers, dark pools, and many exchange gateways (particularly US equities, FX, and fixed income)
  • Native binary protocols (OUCH, ITCH, Optiq, ETI) for ultra-low-latency direct market access
  • SBE as the Consolidated Audit Trail (CAT) and FIX's own recommended binary encoding layer

Compliance and Risk Features

Drop Copy Sessions

FIX 5.0 formalizes the drop copy pattern: a parallel FIX session, typically pointed at a risk/compliance server, receives a copy of every ExecutionReport in real time. The drop copy session has its own sequence number series and does not affect the trading session's flow. This is mandatory at most prime brokers and many exchanges for regulatory reporting.

Pre-Trade Risk Controls

FIX 5.0's TrdRegTimestamps repeating group (tag 768) carries multiple timestamps per message — order entry time, routing time, exchange receipt time — enabling sub-microsecond latency measurement and audit trail reconstruction. Combined with SenderSubID (tag 50) and TargetSubID (tag 57), fine-grained per-desk or per-strategy routing is achievable within a single session.

Cancel-On-Disconnect (COD)

Many venues implement a proprietary CancelOnDisconnect flag (often via ExecInst or a custom tag) that instructs the exchange to mass-cancel all open orders if the FIX session drops. This is a critical safety mechanism for HFT — an unmonitored resting order book is a major risk exposure.

Practical Implementation Notes for C++

  1. Pre-build messages at startup: Construct the static skeleton of your most common messages (NewOrderSingle, CancelRequest) at session init time. On the hot path, only overwrite the variable fields (ClOrdID, Price, OrderQty, TransactTime) using memcpy into known offsets.

  2. Avoid std::string entirely on the hot path: Use std::array<char, N> with a length field, or std::string_view into pre-allocated buffers.

  3. Timestamp generation: TransactTime (tag 60) requires nanosecond UTC timestamps. Use clock_gettime(CLOCK_REALTIME, ...) with VDSO acceleration, or a hardware TSC-based clock synchronized via PTP/IEEE 1588.

  4. Sequence number persistence: MsgSeqNum must survive process restarts. Use a memory-mapped file for atomic persistence without fsync latency.

  5. Log everything, but asynchronously: All inbound and outbound FIX messages must be logged for regulatory compliance. Use a lock-free MPSC queue draining to a dedicated logger thread writing to a pre-allocated, O_DIRECT file — never fprintf on the hot path.

  6. Test with a FIX acceptor: Tools like QuickFIX/n, Chronicle FIX, or an in-house acceptor simulate exchange behavior during integration testing. Chaos-test sequence number gaps, session resets, and duplicate detection explicitly.

Summary

FIX 5.0 in HFT is an exercise in fighting the protocol's verbosity while retaining its ubiquity. The session/application layer split via FIXT 1.1 provides architectural flexibility; the rich FIX 5.0 SP2 application dictionary covers virtually every order management scenario. The C++ implementation challenge is to strip every unnecessary allocation, copy, and syscall from the hot path while maintaining the strict sequencing, state machine discipline, and auditability that the protocol and regulators demand. Done correctly, a production FIX engine can achieve end-to-end order submission latencies of 2–10 microseconds in a co-located environment, competitive with binary protocol alternatives for many use cases.

Note

Generated by Claude.ai

Model: Sonet 4.6

Prompt: Provide a thorough description of the FIX 5.0 protocol in a high-frequency trading environment. This description is intended for a computer science expert fluent in C++. Describe the important technical and business features of the protocol.