Skip to content

Ordering + Sequencing

How the system guarantees message ordering and how consumers detect and recover from disruptions.

Epochs

An epoch is an incarnation counter that increments when the adapter reconnects to a venue or resets its internal state. Epochs appear in the common message header as a u32 field.

When the epoch changes for a given (venue, msg_type, inst_id) triple:

  • All prior sequence numbers for that triple are invalidated
  • The consumer MUST discard all cached ordering state for that triple
  • The RESET flag is set on the first message of the new epoch

Epochs never decrease. A new epoch means the adapter's view of the venue stream was interrupted — the consumer cannot assume continuity across the boundary.

Sequence numbers

Every message carries a seq field (u64, monotonically increasing) in its common header. Sequence numbers are scoped to an ordering domain — they are only comparable within the same domain.

Ordering domains

The ordering domain is (venue, msg_type, inst_id).

Within a single domain, seq is monotonically increasing and gap-free (unless a GAP or DROP flag is set). Across different domains, sequence numbers are independent and MUST NOT be compared.

Scope Example
Same domain Two L3 messages for inst_id=42 on Binance — seq values are comparable
Different msg_type An L1 and an L3 for the same inst_id — seq values are independent
Different inst_id Two L3 messages for different instruments — seq values are independent
Different venue Any two messages from different venues — seq values are independent

For Status messages, inst_id is 0 — the domain is (venue, STATUS, 0).

Gap detection

A gap occurs when the consumer observes a discontinuity in seq within a single ordering domain, or when the GAP flag is set on an incoming message.

Detection methods:

  1. Seq discontinuity: If the consumer tracks last_seq per domain and receives a message where seq > last_seq + 1, a gap has occurred.
  2. GAP flag: The adapter sets the GAP flag when it detects an upstream discontinuity from the venue.
  3. Ring overflow: The ring consumer detects that the producer has lapped it (has_gap() returns true). This is a transport-level gap that affects all domains on that ring.

On any gap, the consumer MUST mark the affected book as INVALID and initiate recovery.

Reset semantics

A reset occurs when the epoch changes. The adapter sets the RESET flag on the first message of the new epoch.

On reset:

  1. Discard all seq tracking state for the affected (venue, msg_type, inst_id) domain
  2. Mark the affected book as INVALID
  3. Begin tracking seq from the new message's value
  4. Initiate snapshot recovery

A reset is strictly more severe than a gap — it invalidates the entire ordering history, not just a range of missed messages.

Recovery procedure

After a gap or reset, the consumer recovers by requesting a fresh snapshot:

  1. Mark the book as INVALID — do not use it for trading decisions
  2. Request a snapshot via the Control Plane
  3. Continue draining messages from the ring, buffering deltas for the affected instrument
  4. When the SnapshotRef arrives:
    • Read the snapshot payload from the snapshot SHM region
    • Discard any buffered deltas where seq <= snap_seq
    • Apply the snapshot to initialize the book
    • Apply remaining buffered deltas in seq order
    • The book is now VALID
  5. Resume normal processing

The adapter handles all venue-specific reconciliation (e.g., Binance update ID matching) internally. The consumer operates entirely in the adapter's seq / snap_seq space.

Do not skip recovery

Processing deltas over a gap produces a silently incorrect book. There is no way to detect the error after the fact — the only safe action on gap detection is to stop, invalidate, and resync.