Kaspar

High-Frequency Trading Simulator with Position-Aware Order Book Matching

CME Futures • MDP3 Market Data • iLink 3 • PCAP Replay • Shadow Execution • C++20

Kaspar is a low-latency trading system simulator for CME futures (ES, NQ). It reconstructs full order books from MDP3 market data, simulates fills that respect queue position, and supports live paper trading or historical PCAP replay — all built on a custom C++ actor framework designed for microsecond-level performance.

Unlike toy backtesting engines that assume instant fills at mid, Kaspar models realistic execution: your simulated orders sit in the book at a specific price level and only fill when the market trades through your position in the queue.

Named after Kasprowy Wierch — "a peak of a long crest in the Western Tatras, one of Poland's main winter ski areas."

Author: Vincent Mayeski — [email protected] | GitHub

• Position-aware order book simulator — MBP (market-by-price) book with simulated queue position tracking. Orders fill based on price-time priority, not magical instant execution.
• CME MDP3 v12 market data — Full SBE decoder for incremental book updates, trades, order-by-order (MBO), instrument definitions, and snapshot recovery. Handles sequence gaps automatically.
• Three operating modes — PCAP replay (backtest), live multicast (paper trading), and iLink 3 (live execution). Same codebase, same strategy code, switch with config.
• Shadow execution algorithm — Production-grade execution logic that piggybacks on real market flow. Places orders only when genuine interest appears at a price level. Zero idle quoting.
• Lock-free actor framework — Custom C++20 actor system with O(1) message dispatch, CPU affinity, and sub-microsecond send latency. No shared state, no locks on the hot path.
• iLink 3 reference implementation — Full CME iLink v3 session handler with SBE encoding, HMAC authentication, sequence management, and primary/secondary failover.
• PCAP reader — Replay recorded CME multicast captures for deterministic backtesting. Bit-exact reproduction of market conditions.
• External strategy support — Write strategies in C++ as in-process actors (lowest latency), or in Rust via the in-process C++/Rust FFI interop.
• Rust port of the actor framework — actors/rust (crate actors): a from-scratch Rust port of the actor core — on-stack fast_send, integer-ID O(1) dispatch, the BQueue mailbox, and the per-type object pool — shipping a price-time-FIFO order-book matching engine as an example. In-process (no remoting/registry/groups yet).

CME MDP3 Multicast (or PCAP file)
         |
    SocketReader / PCAPReader
         |
    MessageProcessor ──── RecoveryProcessor (snapshot recovery on gap)
         |
    handler_if (SBE decode + routing)
         |
    ┌────┴────┐
    OB        TachBook          OB = MBP (sim)    TachBook = MBO L3 (live)
    |         |
    EndOfBurst (per instrument)
    |
    light22 (per side)              ← shadow execution lights
    |
    SOM (Simulated Order Manager)   ← queue-position-aware fill simulation
    |
    ┌──┴──┐
    DB    iLink (live only)

Per instrument: buy and sell lights share a QCoord (working order tracking) and PCoord (position state) in shared memory, guarded by fine-grained mutexes. No coordination messages — just shared counters.

# Build (optimized)
KSPRPROJ=$(pwd) make -j6

# Rebuild kaspr executable with MBO L3 order book (for live trading)
cd kaspr/src && KSPRPROJ=~/m2_kaspar USE_TACHBOOK=1 make

# Run with PCAP replay
cd kaspr && ./kaspr config/kaspr.ini

# Run with position reset
cd kaspr && ./kaspr config/kaspr.ini --reset-positions

• C++20 compiler (GCC 12+)
• Boost 1.88+
• ZeroMQ (libzmq)
• nlohmann/json
• GSL (GNU Scientific Library)

kaspar/
├── actors/          C++20 actor framework (messaging, lifecycle) + Rust port + C++/Rust interop
│   └── rust/       Rust port of the actor framework (actors) + matching_engine example
├── kaspr/           Main application (startup, wiring, config)
├── mdp3/            MDP3 market data decoder and recovery
├── mcast_recv/      Multicast UDP receiver and PCAP reader
├── frame_kaspr/     Order book (OB/TachBook), SOM, BFA, Timer
├── light/           Shadow execution algorithm (light22)
├── ilink/           CME iLink 3 session handler
├── ilink_v8/        iLink v8 SBE protocol headers (generated)
├── mktdata_v12/     MDP3 v12 SBE market data headers (generated)
├── chutil/          Core utilities (time, sockets, enums, binary formats)
├── interface/       Factory function headers for actor creation
├── db/              Database actor (stubbed)
├── mtd/             Monitoring and console display
├── mq0/             ZMQ console server
├── logger/          Logging actor
├── positionman/     Per-instrument position tracking
├── oogsl/           GSL math wrappers (stats, matrix, random)
├── genconfig/       CME config generators (MDP3 + iLink)
├── setclassid/      Message ID collision checker
└── mk_kaspr/        Build system templates

Most execution algorithms either cross the spread (expensive) or continuously quote (noisy, adverse selection). Kaspar takes a third path: shadow execution — a percentage-of-volume algorithm that participates in natural market flow rather than fighting it. Shadow algorithms routinely outperform VWAP and TWAP benchmarks because they don't signal intent and they avoid adverse selection by only trading alongside genuine order flow.

Each instrument runs independent execution lights per side. They don't quote. They don't model. They watch — and strike only when the order book reveals genuine intent.

Real market participant places order at 6050.00
    → light22 sees ADD on its side
    → checks: position < target? price in range? level not crowded?
    → throttle gate: deterministic (every Nth) or stochastic (3%)
    → places at 6050.00 — same price, piggybacking on real flow
    → tracks the real order it attached to
    → if that order gets hit or pulled → auto-cancel with delay

Why this works:

The lights coordinate via shared memory — QCoord tracks aggregate working orders, PCoord tracks net position — guarded by fine-grained mutexes rather than passing coordination messages.

See SHADOW_ALGORITHM.md for the full specification, and the write-up "Shadow POV Execution: Trade Where the Market Is Going to Trade".

The actor framework provides the concurrency model for the entire system:

• Lock-free message passing — BQueue mailbox per actor, O(1) dispatch via handler_cache[msg_id]
• Groups for deterministic simulation — A Group runs multiple actors on a single thread with a single message queue. In PCAP replay, the entire pipeline (OB, lights, SOM) goes into one Group — market data, order placement, and fill matching execute in strict message order. No race conditions, no timing artifacts. Bit-exact reproducible backtests.
• Zero-copy fast path — fast_send() executes handler in caller's thread for synchronous queries
• CPU affinity — Pin actors to cores for deterministic latency
• C++/Rust interop — C++ and Rust actors can talk in the same process over a C-ABI FFI bridge (send/fast_send work across the language boundary). This is in-process only — there is no remote/cross-process actor transport.
• Rust port — actors/rust (actors) is a from-scratch Rust port of the actor core (on-stack fast_send, integer-ID O(1) dispatch, BQueue, object pool). It is in-process only (no ZMQ/registry/groups yet) and ships a matching engine as an example — see its README and DEVELOPER_GUIDE.

The design behind the framework is written up here: Low-Latency Actor Systems in C++ and Rust (building this framework in both languages), Actors in C++ and Rust: The Benchmarks, and the Bridge Between Them (the two ports benchmarked head to head, plus the in-process C++/Rust interop), Lock-Free Isn't Free: Cache Pollution, Busy Cores, and Why Kaspar Blocks (why the BQueue blocks instead of spinning, and when lock-free is the slower choice), If a Machine Is Going to Write the Code, Make It Rust (why Rust is the language to have AI generate, and why Kaspar added Rust interop), The Actor Model for Low-Latency Software, A High-Performance Mailbox (the BQueue), and A Custom Memory Allocator (10× improvement) (the object pool).

class MyStrategy : public Actor {
public:
    MyStrategy(actor_ptr ob, actor_ptr som) : Actor("strategy") {
        MESSAGE_HANDLER(frame::ob::msg::EndOfBurst, on_book_update);
        MESSAGE_HANDLER(frame::som::msg::Fill, on_fill);
        this->ob = ob;
        this->som = som;
    }

private:
    void on_book_update(const frame::ob::msg::EndOfBurst* eob) {
        // React to order book changes
        auto* order = new frame::som::msg::Order(
            "ESM6", en::BuySell::BUY, 1, eob->best_bid, en::x::CMEMDFUT);
        som->send(order, this);
    }

    void on_fill(const frame::som::msg::Fill* fill) {
        // Handle execution
    }
};

The same strategy actor in the Rust port (actors/rust) — handlers are plain methods, wired up by the handle_messages! macro; message ids are compile-time constants:

struct MyStrategy {
    ob: ActorRef,
    som: ActorRef,
}

impl MyStrategy {
    fn on_book_update(&mut self, eob: &EndOfBurst, ctx: &mut ActorContext) {
        // React to order book changes
        self.som.send(
            Box::new(Order::new("ESM6", Side::Buy, 1, eob.best_bid, Venue::CmeMdFut)),
            ctx.self_ref(),
        );
    }

    fn on_fill(&mut self, _fill: &Fill, _ctx: &mut ActorContext) {
        // Handle execution
    }
}

handle_messages!(MyStrategy,
    EndOfBurst => on_book_update,
    Fill       => on_fill,
);

See actors/cpp/CLAUDE_AGENT_GUIDE.md for the complete C++ framework reference, and actors/rust/DEVELOPER_GUIDE.md for the Rust API.

Connect via ZMQ (default port 7777) for runtime monitoring:

kaspr {
    general {
        universe config/universe.csv
        mqport 7777
        tachbook false           # true + USE_TACHBOOK=1 for live
    }
    channels {
        chan_310 true             # ES futures
        chan_318 true             # NQ futures
    }
}

• Message dispatch: O(1) vector lookup by message ID — no virtual dispatch, no hash maps
• Actor send: Sub-microsecond enqueue (mutex + condition variable, no allocation on hot path)
• Book update to strategy: Single EndOfBurst message per MDP3 incremental cycle
• Memory: Pool allocators for high-frequency message types, zero GC pauses
• Threading: One thread per actor, CPU affinity pinning, no contention between instruments

Deep-dives on the design behind Kaspar (author's Substack — vincentmayeski.substack.com):

• Low-Latency Actor Systems in C++ and Rust — building the same actor framework in both languages: this repo's C++ core and its Rust port (actors/rust), and what carries over vs. what the borrow checker changes.
• Actors in C++ and Rust: The Benchmarks, and the Bridge Between Them — the two ports benchmarked head to head (fast_send, send, allocation), and the in-process C++/Rust interop bridge, with numbers.
• Lock-Free Isn't Free: Cache Pollution, Busy Cores, and Why Kaspar Blocks — why lock-free can be the slower choice under load (spinning consumers, cache coherence, oversubscription), and why the BQueue blocks and fast_send minimizes thread hops instead.
• If a Machine Is Going to Write the Code, Make It Rust — why Rust is the best language for AI-generated code (compiled, plus the strictest mainstream compiler at catching bugs up front), and why Kaspar added C++/Rust interop.
• The Actor Model for Low-Latency Software — a concurrency model invented for single-CPU machines turned out to be the right one for multicore.
• A High-Performance Mailbox in the Kaspar C++ Actor System — ring buffers are great until they overflow (the BQueue design).
• A Custom Memory Allocator for the Kaspar Actor System Gives 10× Improvement — when you know the size at compile time, almost everything an allocator does becomes unnecessary (the object pool).
• How Message Batching More Than Doubles Actor Model Throughput — draining the whole mailbox under one lock, plus the message pool, for a 2.46× throughput win (and why batching the sender backfires). Code on the experimental sharded-mailbox branch (PR #20), not yet merged.
• Medians Lie, Tails Kill: Why Kaspar Shards Mailbox Locks — per-actor mailboxes shard lock contention to flatten tail-latency jitter (~180× at p99.9). Code on the experimental sharded-mailbox branch, not yet merged.
• The Lock-Free Illusion: Why CAS Storms Kill Actor Queues Under Contention — when a lock-free mailbox wins and when it tails worse than a mutex; the queue-selection matrix. Code on the experimental sharded-mailbox branch, not yet merged.
• Shadow POV Execution: Trade Where the Market Is Going to Trade — a percentage-of-volume algorithm that follows passive flow.

MIT License. Copyright (c) 2026 Vincent Mayeski / M2 Tech (16425640 Canada Inc.). See LICENSE.