Order Routing — Integration Guide¶

Contract for sending orders and consuming execution state via orderd. Two integration tiers, from simplest to most control.

Design principles¶

1. State is mutation lifecycle, not fill state. The state machine tracks what mutation is in-flight. Whether an order has partial fills is filled_qty > 0 on the order record — not a separate state.

2. Every response carries authoritative state. Each order-related response includes order_state. Responses for accepted orders carry filled_qty, so the client can track execution progress from messages without inferring transitions.

3. Price-only modify. Quantity modifications are rejected locally by orderd (reject reason: QTY_MODIFY). For quantity changes, cancel + new. This eliminates all qty/fill interaction edge cases during pending modifications.

4. One in-flight mutation per order. Cannot send modify while cancel is pending, or vice versa. orderd rejects locally (reject reason: MODIFY_IN_FLIGHT).

5. orderd normalizes venue differences. Modify semantics, cancel-all behavior, and ID formats vary by venue — orderd presents a unified protocol. Exchange order ID changes are handled transparently.

6. orderd owns fill deduplication. No duplicate fill_id values on the response ring within a given epoch. Clients do not need to dedup.

OrderManager: in-process convenience¶

OrderManager is a C++ class in the header library that wraps both the request and response rings, tracks order state internally, handles epoch and gap recovery, and fires user-registered callbacks on execution events. It is the order-routing equivalent of market data's BookBuilder.

Use this when you want typed submit methods, automatic state tracking, and recovery without implementing the raw ring protocol yourself.

#include <sorcery/order_manager.h>

sorcery::OrderManager mgr(
    "/sorcery-master-ord-req",
    "/sorcery-master-ord-rsp",
    "/sorcery-master-metadata"
);

mgr.on_fill([](const sorcery::Header& hdr, const sorcery::ord::FillMsg& fill) {
    printf("Fill: order=%lu px=%ld qty=%ld state=%u\n",
           fill.order_id, fill.fill_px, fill.fill_qty, fill.order_state);
});

mgr.new_order(inst_id, venue, /*order_id=*/1, /*px=*/950000, /*qty=*/100000000,
              sorcery::ord::BUY, sorcery::ord::GTC);

while (running) {
    mgr.poll();  // drain response ring, update state, fire callbacks

    auto* order = mgr.find_order(1);
    // order->state, order->px, order->filled_qty, ...
}

Submitting orders¶

Typed methods encode the header, validate against instrument metadata, write to the request ring, and flush. Local validation failures return an error without touching the ring.

// New limit order
mgr.new_order(inst_id, venue, order_id, px, qty, side, tif);
mgr.new_order(inst_id, venue, order_id, px, qty, side, tif, flags);

// Modify price (qty changes are not supported — cancel + new)
mgr.modify(inst_id, venue, order_id, new_px);

// Cancel
mgr.cancel(inst_id, venue, order_id);

// Cancel all — venue-wide or instrument-scoped
mgr.cancel_all(venue);
mgr.cancel_all(venue, inst_id);

All submit methods return bool. true means the request was written to the ring. false means local validation failed (instrument not ACTIVE, price not tick-aligned, qty not step-aligned) — the request was not published and no response callback will fire.

orderd performs its own local validation as well (duplicate order_id, in-flight mutation). Those rejects arrive as normal ORDER_REJECT / MODIFY_REJECT / CANCEL_REJECT callbacks with the LOCAL flag set.

Queries¶

OrderManager manages query correlation and timeout internally. It enforces the one-outstanding-per-scope discipline documented in Message Reference — Query correlation and retries with bounded backoff on timeout.

mgr.query_orders(venue);
mgr.query_orders(venue, inst_id);        // instrument-scoped
mgr.query_balances(venue);
mgr.query_positions(venue);
mgr.query_order(venue, order_id);

Query responses fire the corresponding callbacks: on_orders, on_balances, on_positions, on_order_status.

Callbacks¶

Register callbacks before calling poll(). Each callback receives the common header and the typed payload. Internal order state is already updated when the callback fires — the consumer can read consistent state from within the handler.

All callbacks are optional. Unregistered events are silently consumed after internal state updates.

Driving the loop¶

while (running) {
    mgr.poll();  // drain response ring, update state, fire callbacks
}

poll() drains a batch of frames from the response ring, applies each to the internal state machine, and invokes the registered callback. Callbacks fire in ring order — the response ring is globally monotone, so a partial fill callback always precedes the full fill callback for the same order. Pin one thread, call poll() in a loop.

State views¶

Read-only access to the current order state, available inside or outside callbacks.

// Single order lookup
auto* order = mgr.find_order(order_id);
if (order) {
    // order->state, order->px, order->qty, order->filled_qty, order->side, ...
}

// Iterate all non-terminal orders
for (auto& [oid, view] : mgr.open_orders()) { ... }

// Venue connection health
auto status = mgr.venue_status(venue);
// status.conn_state, status.epoch, status.open_orders

Returned pointers reference process-local storage. They remain valid for the duration of the current poll() call (safe to use inside callbacks and between callbacks within the same poll() batch) and until the next poll() call if accessed outside. poll() may update or remove entries as responses arrive. Do not hold pointers across poll() iterations; copy what you need.

find_order() returns nullptr for unknown order IDs and for terminal orders that have been purged. Terminal orders are retained for a bounded window after reaching their terminal state to allow post-fill processing, then removed.

Automatic recovery¶

Epoch change. When OrderManager detects a new epoch for a venue, it:

1. Marks all non-terminal orders for that venue as uncertain
2. Waits for orderd's reconciliation ORDER_STATUS stream
3. Issues QUERY_ORDERS and QUERY_BALANCES to confirm final state
4. Fires on_epoch_change — the consumer can inspect reconciled state and decide when to resume submissions

During recovery, uncertain orders remain visible in open_orders() and find_order() with their last known state. The consumer can check is_recovering(venue) to know whether these states are confirmed or still being reconciled.

Ring gap. When the response ring overflows:

1. Re-synchronizes the consumer to the current ring position
2. Issues QUERY_ORDERS and QUERY_BALANCES for all active venues
3. Reconciles internal state from query responses
4. Fires on_gap

During recovery, is_recovering(venue) returns true. Submits are not blocked — orderd rejects them with DISCONNECTED if the venue is down. The consumer can check is_recovering() before submitting to avoid unnecessary rejects.

Startup¶

OrderManager starts with empty state. It does not know about orders from previous sessions.

On first poll(), if orderd is already running (or restarting), the consumer will observe epoch-change ORDER_STATUS messages for any open orders orderd discovers on the exchange. OrderManager ingests these and populates its internal state — by the time on_epoch_change fires, open_orders() reflects the reconciled exchange state.

Recommended startup sequence:

1. Construct OrderManager, register callbacks
2. Call poll() in a loop until venue_status(venue).conn_state == CONNECTED and is_recovering(venue) == false
3. Issue query_orders() and query_balances() explicitly to confirm state
4. Begin submitting orders

If the consumer process restarts while orderd continues running, the consumer will not see an epoch change — orderd's connection was never interrupted. In this case the consumer MUST issue explicit queries on startup to rebuild its view of open orders.

Reconciliation orphans¶

Orders discovered on the exchange that have no client-owned order_id (e.g., placed via exchange UI or a different process) are published by orderd with order_id = 0. OrderManager does not add these to its internal state — they are not orders it manages.

order_id = 0 events fire the on_order_status and on_fill callbacks normally, with order_id = 0 in the payload. The consumer MUST handle these in callbacks if reconciliation awareness is required (e.g., unknown position exposure). find_order(0) always returns nullptr.

What OrderManager does not do¶

• Multi-strategy routing. No gateway demux, no order_id namespace management. For multi-strategy, see Deployment topology.
• Position tracking. Use query_positions() and on_positions to query exchange-side positions on demand.
• P&L or risk. No portfolio math. The consumer computes these from fill callbacks and position queries.
• Threading. Single-threaded. If multiple threads need order state, publish from the poll thread via your own concurrency mechanism.

Raw consumer reference¶

The remaining sections document the raw ring protocol — the state machine, venue normalization, sequencing, and recovery semantics that OrderManager handles internally. Read this if you need full control, or if you want to understand the mechanics underneath.

State machine¶

7 states: 4 active + 3 terminal.

                                    ┌──── partial fill ────┐
                                    │  (filled_qty updated) │
                                    ▼                       │
           ┌───────────┐    ack    ┌──────┐    modify     ┌────────────────┐
  NEW ───▶ │PENDING_NEW│ ───────▶ │ LIVE │ ────────────▶ │ PENDING_MODIFY │
           └───────────┘          └──────┘               └────────────────┘
           │    │       │           │   │        ▲          │         │
        reject cancel modify      cancel fill    │        ack/rej   fill
           ▼    ▼       └───────────────────────-┘          ▼         ▼
       REJECTED ┌───────────┐          FILLED             LIVE     FILLED
                │PENDING_CXL│
                └───────────┘
                  │    │    │
                ack  reject  fill
                  ▼    ▼      ▼
             CANCELED LIVE  FILLED

Transition table¶

Rules¶

Fill in any non-terminal state. Update filled_qty += fill.qty. If filled_qty >= order_qty, transition to FILLED regardless of current state. A fill during PENDING_MODIFY or PENDING_CXL does not change the pending state — it only updates filled_qty. If the fill makes the order fully filled, the order transitions to FILLED and any pending modify/cancel response is discarded when it arrives.

One in-flight mutation. orderd rejects modify requests when state is PENDING_MODIFY or PENDING_CXL. Rejects cancel requests when state is PENDING_MODIFY or PENDING_CXL. Reject reason: MODIFY_IN_FLIGHT.

Modify and cancel during PENDING_NEW. Both are allowed — the order was already sent to the exchange and the client may need to react before the ack arrives. Only one mutation at a time: if a modify is sent during PENDING_NEW (→ PENDING_MODIFY), a subsequent cancel is rejected, and vice versa. orderd sends the modify/cancel to the exchange immediately; the exchange processes it after the preceding NEW.

Terminal is final. Once FILLED, REJECTED, or CANCELED, orderd drops all further exchange events for that order and does not publish them.

Late responses. If a modify/cancel response arrives from the exchange after the order reached FILLED via a fill, orderd discards the response silently. The client already saw the FILL.

Mutations during PENDING_NEW¶

Both modify and cancel are allowed during PENDING_NEW. orderd sends the mutation to the exchange immediately — the exchange will process it after the preceding NEW on the same connection.

Cancel during PENDING_NEW. orderd transitions PENDING_NEW → PENDING_CXL. Three outcomes:

1. Exchange never accepted the NEW. orderd publishes ORDER_REJECT (terminal). No CANCEL_REJECT is emitted on this path.
2. Exchange acks the order, then processes the cancel. ORDER_ACK published (state=PENDING_CXL), then CANCEL_ACK → CANCELED.
3. Exchange fills before processing the cancel. FILL → FILLED (terminal, cancel response discarded).

Modify during PENDING_NEW. orderd transitions PENDING_NEW → PENDING_MODIFY. Three outcomes:

1. Exchange rejects the NEW. ORDER_REJECT published (terminal), modify response discarded.
2. Exchange acks the order, then processes the modify. ORDER_ACK published (state=PENDING_MODIFY), then MODIFY_ACK (state=LIVE, new price) or MODIFY_REJECT (state=LIVE, original price).
3. Exchange fills before processing the modify. FILL → FILLED (terminal, modify response discarded).

IOC order lifecycle¶

IOC orders execute immediately and any unfilled remainder is cancelled by the exchange. orderd ensures IOC orders always reach a terminal state: FILLED (if completely filled) or CANCELED (if partially filled or not filled). For partial IOC fills, orderd publishes the FILL(s) and then a synthesized CANCEL_ACK.

Lifecycle examples¶

Happy path: new → ack → fill → filled

Client                       orderd                      Exchange
  │                            │                            │
  │── NEW (oid=1, px=500) ───▶│── submit ─────────────────▶│
  │                            │                            │
  │                            │◀── accepted ───────────────│
  │◀── ORDER_ACK (state=LIVE)──│                            │
  │                            │                            │
  │                            │◀── fill (qty=100) ─────────│
  │◀── FILL (state=FILLED) ───│                            │

Modify flow: new → ack → modify → ack

Client                       orderd                      Exchange
  │                            │                            │
  │── NEW (oid=1, px=500) ───▶│── submit ─────────────────▶│
  │◀── ORDER_ACK ─────────────│◀── accepted ───────────────│
  │                            │                            │
  │── MODIFY (oid=1, px=501)─▶│── amend ──────────────────▶│
  │                            │     (state=PENDING_MODIFY) │
  │                            │◀── amended ────────────────│
  │◀── MODIFY_ACK (px=501) ──│                            │
  │        (state=LIVE)        │                            │

Cancel during modify: rejected

Client                       orderd
  │                            │
  │── MODIFY (oid=1, px=501)─▶│  (state → PENDING_MODIFY)
  │── CANCEL (oid=1) ────────▶│  ✗ REJECTED locally
  │◀── CANCEL_REJECT ─────────│  (reason=MODIFY_IN_FLIGHT)
  │                            │
  │◀── MODIFY_ACK ────────────│  modify completes normally

Modify during PENDING_NEW: price adjustment before ack

Client                       orderd                      Exchange
  │                            │                            │
  │── NEW (oid=1, px=500) ───▶│── submit ─────────────────▶│
  │── MODIFY (oid=1, px=499)─▶│── amend ──────────────────▶│
  │                            │     (state=PENDING_MODIFY) │
  │                            │◀── accepted ───────────────│
  │◀── ORDER_ACK ─────────────│     (state=PENDING_MODIFY) │
  │                            │◀── amended ────────────────│
  │◀── MODIFY_ACK (px=499) ──│                            │
  │        (state=LIVE)        │                            │

Fill during PENDING_MODIFY

Client                       orderd                      Exchange
  │                            │                            │
  │── MODIFY (oid=1, px=501)─▶│── amend ──────────────────▶│
  │                            │                            │
  │                            │◀── fill (partial) ─────────│
  │◀── FILL (state=PENDING_MODIFY, filled_qty updated) ───│
  │                            │                            │
  │                            │◀── amended ────────────────│
  │◀── MODIFY_ACK (state=LIVE)│                            │

Venue normalization¶

orderd presents a unified protocol regardless of venue implementation differences.

Modify normalization¶

All supported venues expose native amend endpoints. orderd sends the amend directly and publishes a single MODIFY_ACK or MODIFY_REJECT.

The client's order_id is always unchanged. On venues where exch_oid changes after an amend, orderd updates the mapping internally.

Cancel-all normalization¶

ID normalization¶

Exchange order IDs that are strings or UUIDs are hashed to u64 using xxHash64 (same as market data's trade_id and order_id hashing). The mapping is deterministic within a session.

Deployment topology¶

Ring ownership¶

The request ring (/sorcery-{stack}-ord-req) is a single-producer stream. In production, exactly one process should own request writes for a given stack.

Recommended pattern for multi-strategy deployments:

1. Run a local execution gateway as the sole request-ring producer
2. Have all strategies send intents to that gateway
3. Let the gateway assign seq, enforce risk policy, and publish to the ring

The response ring is read-only for clients and may be consumed by multiple processes.

Multi-strategy support and routing¶

Multi-strategy operation is supported through external fan-in/fan-out, not through an on-wire strategy field.

• There is no strategy_id/app_id in the order-routing wire header.
• All responses for a stack are published on the same response ring (/sorcery-{stack}-ord-rsp).
• Demultiplexing is done by order_id ownership in your gateway/consumer layer.

If you need hard isolation between strategy groups, run separate stacks (independent SHM namespaces) with separate gateways.

Gateway demux contract (required)¶

For multi-strategy deployments, the gateway contract is part of the integration API even though it is outside the wire header.

Required behavior:

1. Assign each strategy a stable namespace (strategy_ns) inside order_id.
2. Persist order_id -> strategy_ns mapping before publishing NEW to the request ring.
3. Route all non-query responses (ORDER_ACK, ORDER_REJECT, MODIFY_*, CANCEL_*, FILL) strictly by this mapping.
4. Treat order_id = 0 responses as reconciliation-orphan events; never fan them out as normal strategy events.
5. On gateway restart, restore the mapping from durable storage before consuming the response ring.

Recommended order_id layout (u64):

• High 16 bits: strategy_ns
• Low 48 bits: per-strategy monotone counter

This makes ownership derivable from the ID alone and avoids cross-strategy collisions without extra lookup keys.

Response routing rules¶

Apply these rules in the gateway:

If a non-zero order_id arrives without a known owner mapping, the gateway MUST classify it as an internal reconciliation fault and pause submissions on that venue until reconciliation completes.

order_id ownership¶

order_id must be unique within the request stream. If multiple strategy modules share one producer, use deterministic partitioning (for example high bits = strategy ID, low bits = per-strategy counter).

Never reuse an order_id until the previous order is terminal and no replay/recovery flow can surface additional events for that ID.

Sequencing and epochs¶

Response ring ordering¶

The response ring has a single ordering domain: the ring itself. seq is globally monotone across all response msg_types and all instruments. A seq discontinuity means the consumer missed one or more responses.

This differs from market data, where seq is scoped to (venue, msg_type, inst_id). Order routing is simpler because there is only one producer (orderd) writing to the response ring.

Request ring ordering¶

The request ring also has a single ordering domain. seq is monotone within the request-ring producer stream. orderd uses seq to detect if it fell behind the producer (ring overflow on the request side — extremely unlikely given order rates, but handled).

Epoch semantics¶

epoch in the response header is per-venue. It increments when orderd's connection to that venue is interrupted and re-established.

When the client observes a new epoch for venue V:

1. All non-terminal orders for venue V are uncertain — orderd may not know their current exchange-side state
2. orderd proactively reconciles by querying the exchange for open orders
3. orderd publishes ORDER_STATUS for each reconciled order (state may have changed)
4. The RECONNECT flag is set on the first response of the new epoch

PENDING_NEW orders on epoch change. orderd resolves every PENDING_NEW order during reconciliation. If the order appears in the exchange's open-order query, orderd publishes ORDER_ACK. If the order does not appear and no fill was received, orderd publishes ORDER_REJECT with reason DISCONNECTED, RECONNECT set, and LOCAL unset (synthetic reconciliation result). No PENDING_NEW order is left unresolved after epoch reconciliation.

Client recovery on epoch change¶

1. Mark all non-terminal orders for venue V as uncertain
2. Wait for orderd's reconciliation ORDER_STATUS messages (and ORDER_ACK/ORDER_REJECT for PENDING_NEW orders)
3. Issue QUERY_ORDERS and QUERY_BALANCES for venue V before re-enabling submissions

Crash recovery¶

An orderd restart is equivalent to an epoch change for all venues. On startup:

1. orderd increments epoch for every venue
2. orderd queries each exchange for open orders
3. For each discovered open order, orderd publishes ORDER_STATUS on the response ring
4. Orders that were open on the exchange but unknown to orderd (e.g., placed via exchange UI) are published with order_id = 0 and MUST be routed to the reconciliation stream, never to strategy event handlers
5. The RECONNECT flag is set on the first response of the new epoch for each venue

The client treats an orderd restart identically to a venue reconnect. All non-terminal orders are uncertain until reconciliation ORDER_STATUS messages arrive.

Gap recovery¶

On response ring gap (consumer lapped by producer):

1. Re-synchronize ring position via consumer.reset()
2. Issue QUERY_ORDERS and QUERY_BALANCES for all active venues to reconcile order and cash state
3. Resume draining

There is no retransmit from the ring. The authoritative state is always available via queries.

Unknown fills¶

If orderd discovers a fill for an order whose state it has lost track of (e.g., after a disconnect and reconnect), it synthesizes the missing state transitions before publishing the fill:

1. orderd publishes a synthesized ORDER_ACK with the RECONNECT flag, transitioning the order to LIVE
2. orderd publishes the FILL(s)

This ensures the client always observes a valid state sequence — no fill arrives for an order the client hasn't seen acknowledged. The synthesized ORDER_ACK has exch_ts = 0 (orderd doesn't know the original ack time) and the RECONNECT flag set to indicate it was reconstructed.

If the order is completely unknown to orderd (e.g., placed via exchange UI), the fill is published with order_id = 0 and MUST be handled on the reconciliation stream (not strategy callbacks).

Reconciliation truth table¶

The gateway should implement deterministic handling for each recovery trigger:

Fail-open behavior is out of contract for these cases.

Error handling and rejects¶

Local vs exchange rejects¶

orderd validates requests before sending them to the exchange. Local validation failures produce an ORDER_REJECT (or MODIFY_REJECT, CANCEL_REJECT) with the LOCAL flag set in the response header. These are fast — the round trip is ring latency only, no network involved.

Exchange rejects come from the venue after the request was transmitted. They have exch_ts populated and the LOCAL flag is NOT set.

Recoverable vs non-recoverable¶

The reject_reason enum classifies each reason as recoverable or non-recoverable:

• Recoverable (INSUFFICIENT_MARGIN, RATE_LIMITED, POST_ONLY, SELF_TRADE, REDUCE_ONLY, MARKET_CLOSED, MODIFY_IN_FLIGHT, DISCONNECTED): The same request may succeed if retried after the condition resolves. Strategy should back off and retry.

• Non-recoverable (UNKNOWN, INTERNAL, EXCHANGE, INVALID_PRICE, INVALID_QTY, INVALID_INSTRUMENT, TOO_LATE, DUPLICATE_ORDER, ORDER_NOT_FOUND, QTY_MODIFY): The request is fundamentally invalid. Retrying without changes will fail again.

Common scenarios¶

Rate limited. The exchange returned a rate limit error. orderd publishes ORDER_REJECT with RATE_LIMITED. Strategy should throttle and retry after a backoff.

Post-only crossed. A POST_ONLY limit order would have executed as taker. orderd publishes ORDER_REJECT with POST_ONLY. Strategy should adjust price and retry.

Modify during pending. Client sent a modify while another modify or cancel was in flight. orderd publishes MODIFY_REJECT with MODIFY_IN_FLIGHT and the LOCAL flag. Strategy should wait for the pending mutation to resolve, then retry.

Disconnected (submit path). orderd is not connected to the target venue. Publishes ORDER_REJECT with DISCONNECTED and the LOCAL flag. Strategy should wait for STATUS messages to indicate reconnection.

Disconnected (reconciliation path). During epoch reconciliation, unresolved PENDING_NEW orders may emit ORDER_REJECT with DISCONNECTED and RECONNECT set. These are synthetic reconciliation outcomes (LOCAL unset), not submit-time validation rejects.

Versioning and compatibility¶

The schema_ver field in the common header identifies the wire format version. When schema_ver changes:

• New fields may be added to the end of existing payloads (payload_len increases)
• New msg_type values may be added
• Existing field offsets and semantics are never changed

Consumers MUST check schema_ver on ring open and reject unrecognized versions. Within a recognized version, consumers SHOULD use payload_len to determine the actual payload size — if payload_len exceeds the expected size for a msg_type, the extra bytes are new fields that can be safely ignored.

Consumer responsibilities¶

What each tier handles:

Both tiers must poll the metadata region periodically to pick up instrument and asset changes. OrderManager maintains an internal MetadataStore for tick/step validation on submit. Call mgr.reload_metadata() periodically (e.g. every few seconds) to refresh it — same cadence as MetadataStore::reload(). Stale metadata may cause valid orders to fail local validation, or invalid orders to pass through to orderd.