Matching engine architecture: what actually powers a low-latency exchange

Most articles on matching engine software are written by people who have read about matching engines. This one is written by the team that ships them. We have been building trading systems for three decades — from FIX gateways into the earliest electronic venues, through FX and crypto exchange builds, market-making and arbitrage stacks, into the matching engines and low-latency infrastructure powering exchanges today. The answers below are the ones we would give to a CTO sitting across the table asking whether to build, buy, or licence.

There is no on-the-one-hand-on-the-other-hand in this piece. If we have shipped it, watched it fail, and shipped the fix, we tell you what worked.

## What a matching engine actually is

A matching engine is a deterministic function. It consumes an ordered stream of commands — new order, cancel, modify, admin — and produces an ordered stream of events: trades, book updates, rejects, acks. Everything else is scaffolding. The order book is state, the WAL is a projection of the input stream, snapshots are checkpoints of the state, and market data is a projection of the output stream. If you keep that model in your head you will not get lost.

A CLOB matching engine (central limit order book) is the specific case where the state is a two-sided book of resting limit orders, matched under price-time priority. Everything in this article is about that case. Auctions, RFQ venues and dark pools have adjacent designs and we build those too, but the CLOB is where the interesting engineering lives.

## Order book data structures: the choice you cannot get wrong

The single most important structural decision is how you represent the resting book. The naive answer — a std::map or BTreeMap keyed by price — will run. It will also collapse the first time a market maker cancels-and-reposts one hundred thousand orders a second around the touch. Tree nodes are pointer-chased, cache-cold and burn a full logarithmic search on every operation that lives on the top of book, which is the operation that dominates.

The other naive answer — a flat array of price buckets covering the entire instrument range — is beautifully cache-friendly and utterly infeasible for an asset that trades from 0.01 to 200,000 in ticks of 0.01. You cannot allocate two billion buckets for BTC.

The answer, and it is the answer we have converged on across every serious engine we have shipped, is a hybrid. A dense array of price buckets covers a window around the touch — typically several thousand ticks either side, sized so it comfortably absorbs any realistic quoting behaviour without spilling. Outside that window a tree (BTreeMap in Rust, a red-black or B+ tree in C++) holds the sparse tail. The window slides as the market moves; the slide is amortised, not per-order.

Inside each price bucket the resting orders form a FIFO queue implemented as an intrusive doubly-linked list. Intrusive because the list pointers live inside the Order struct itself — no separate node allocation, one cache line per order, cancel-in-the-middle in O(1) because you already hold a pointer to the order from the client-order-id hash map. This is the pattern LMAX made famous with the Disruptor and it is the pattern CME Globex, LSE's Millennium and every other production engine of note converges on for the same reason: it is what the hardware wants.

Skip lists are elegant in academic papers and lose to the hybrid layout in every benchmark we have run. If someone tells you their matching engine is built on a skip list, ask them what their touch-side cancel latency looks like under quote-heavy load. Then wait.

## Price-time priority and the boring correctness that isn't

Price-time priority is easy to state — best price wins, ties broken by arrival order — and full of edge cases that separate a real engine from a demo. A partial fill against a resting order does not lose that order's time priority; the remainder stays at the head of the queue. A modify-in-place that improves price loses priority; a modify that reduces size does not. A cross that the taker places against their own resting order must be handled by the self-trade prevention rules the venue publishes — cancel-oldest, cancel-newest, decrement-both — and the rule must be enforced by the engine, not by a downstream service, because a downstream service cannot see the match before it lands in the tape.

Time-in-force flags look like a checklist and behave like a minefield.

- IOC (immediate or cancel) matches what it can, cancels the rest, never rests. If the incoming taker crosses three price levels the engine must walk all three inside a single command, emit N trades, and cancel the residue. All of that lives in one command boundary.

- FOK (fill or kill) is IOC with a pre-check: if the full quantity cannot be filled at limit-or-better against the current book, nothing trades. The pre-check must be atomic with the fill; if you split it you have a race.

- Post-only rejects on any state that would result in a cross. The check is against the current book at the moment of processing, not the book the client saw when they sent it — clients love to blame the engine for this and they are always wrong.

- GTC (good-till-cancel) is the default and the easy one. GTD (good-till-date) needs a time wheel or heap of expiries so the engine can cancel at the tick, not at the next order.

## Complex order types: risk infrastructure disguised as matching

Stop-limit, stop-market, trailing stops and icebergs are described in exchange rulebooks as if they were order types. They are not. They are risk and trigger infrastructure that only pretends to be an order type at the last hop.

A stop order does not sit in the book. It sits in a trigger set indexed by trigger price, watched against the last-trade or mark-price stream. When the trigger fires the engine synthesises a new limit or market order and submits it into the normal matching path. The interesting engineering is: which price stream triggers it (last-trade, mark, index — the venue picks one and lives with it), how you guarantee at-most-once activation across restarts, how you prevent a trigger cascade during a fast market from ordering-inverting against a slower trigger stream. Every venue that has ever had a flash-crash post-mortem has had this conversation.

Trailing stops add a moving trigger price recomputed on every mark update. The mark update rate can be tens of thousands per second on an active symbol; the trailing-stop set can be tens of thousands of orders. That is a nested loop you do not want in your hot path, so the trailing set is organised as a heap keyed on the current trigger price and only the top-of-heap is inspected per mark update.

Icebergs display a slice, replenish from hidden reserve on fill, and re-enter the FIFO queue at the back on each replenish. That last detail is a fairness rule and it is the one home-grown engines routinely get wrong.

## Multi-symbol: shared state was always going to lose

The seductive design for a multi-symbol engine is a single big lock over a HashMap of symbol → OrderBook. It compiles. It benchmarks acceptably on a laptop. It falls off a cliff the moment you have real concurrent traffic across symbols, because every operation on BTC-PERP is contending with every operation on ETH-PERP for the same mutex, and the contention profile is invisible until you are already in production.

We ran that design in our engine. At around eight to ten thousand orders per second across the venue, p99 order-to-ack latency started climbing into the tens of milliseconds and the flame graph was a solid wall of parking_lot::Mutex::lock. We migrated to a typed-command actor per symbol — each symbol owns a single-threaded event loop, commands arrive on a bounded MPSC channel, the matching function runs without any locking at all because there is only one thread that ever touches that book. Cross-symbol coordination (account balances, portfolio margin) happens through a separate actor that the symbol actors talk to over messages, not shared memory. After the migration the same load ran at p99 under two hundred microseconds and throughput scaled linearly per symbol until we saturated cores.

The lesson is not "actors are magic". The lesson is that the matching function is inherently single-threaded per symbol — price-time priority is a serial ordering — and any design that pretends otherwise is fighting the domain. Give each symbol its own thread, its own cache lines, its own event loop, and let the OS scheduler do what it is good at.

## Latency budget: where the microseconds go

On commodity Linux with a well-written engine you can achieve p50 order-to-ack in the tens of microseconds and p99 under two hundred, sitting behind a TCP socket on the loopback interface. End-to-end across a real network, sub-five-millisecond round trip is the realistic target for a cloud-hosted venue and it is what our engine delivers on a DO droplet today. That is enough for retail, for the majority of institutional flow, and for anyone whose latency-sensitivity is measured in "faster than a human can react".

Sub-hundred-microsecond wire-to-wire is a different sport. You need kernel-bypass networking — Solarflare NICs with OpenOnload, or DPDK if you are willing to burn a core polling — because the Linux network stack alone will eat you thirty to fifty microseconds each way. You need NUMA pinning so the matching thread, the NIC interrupt (or the polling thread) and the memory it touches all live on the same socket. You need zero allocations in the hot path — every Order, every trade event, every log line comes from a pre-sized ring buffer or a slab allocator. You need zero syscalls in the matching loop, which in practice means no logging, no metrics increment, no clock read that isn't a vDSO fast path. You need to be very careful about which counters your Prometheus exporter reads, because a naive AtomicU64::fetch_add per order across 32 cores will lock the memory bus and destroy your tail.

The BP engine sits comfortably in the first regime, targeting the second for the LD4 bare-metal cutover. The design does not have to change for that move; the runtime environment does.

## Throughput: the honest numbers

A well-built single-symbol matching loop on modern hardware will process two to five hundred thousand orders per second sustained, single-threaded, on one core. That is the raw matching function with a warm cache. End-to-end throughput — meaning the number your customers actually get, after TCP parse, deserialisation, pre-trade risk, matching, WAL append, event publish, market-data fanout — lands at twenty-five to one hundred thousand orders per second per shard on the same hardware. The two-order-of-magnitude gap is where the honest work lives.

If a vendor quotes you their matching-function throughput and pretends it is their engine throughput, they are either confused or hoping you are.

## The write-ahead log is not optional and it is not an afterthought

The WAL is the engine. Everything else can be rebuilt from it. Design the WAL first, design the engine second.

The WAL is a single append-only file (or set of segmented files) with a monotonic sequence number stamped on every record, a checksum on every record, and an fsync on every commit boundary. Every command that mutates state — new order, cancel, modify, trade, funding, admin — appends to the WAL before its effects are visible to any observer. On restart the engine loads the last snapshot and replays the WAL from the snapshot's sequence number forward. If replay does not produce byte-identical state to the pre-crash engine, the engine is not deterministic and you have a bug you need to find right now, not later.

Determinism is what makes the engine testable, auditable and restorable. Non-determinism is what makes the engine a lawsuit. We fsync. It costs us a couple of microseconds on batched writes. It is not negotiable on a real-money venue.

## Risk on the hot path or you do not have risk

Pre-trade risk — margin check, position limit, self-trade prevention, kill-switch state — must run in-process, in the same thread as the matching function, before the order touches the book. Any design where risk is a separate service reached over a network hop, however fast, has a race window in which an order can be filled that risk would have rejected. On real venues, that race window gets exercised. We have seen it. Do not build it.

The risk engine talks to the same in-memory account state that the matcher's post-trade update path writes to. The update is synchronous. Async settlement to Postgres happens after the fact for durability, but the authoritative live state is in the engine's heap.

## Time is three clocks, not one

Every event in the engine is stamped with three timestamps: receipt time (when the order arrived at the gateway), match time (when the matching function assigned it a sequence number), and market-data time (when the event was published to the tape). They are not the same, they are not interchangeable, and clients will complain about all three.

The clock source is PTP if you are on hardware that supports it and NTP if you are not. NTP will drift you tens of milliseconds; PTP will hold you sub-microsecond. On a cloud VM you will get NTP whether you like it or not; on bare metal at LD4 you will get PTP and your reconciliation gets a lot easier.

## Failure modes and the plan for each

Process crash: WAL replay from the last snapshot. If the snapshot is four seconds to write and the WAL is ninety seconds to replay for a full day, you are back in under two minutes cold-start. We measure this weekly.

Machine loss: hot standby fed by synchronous WAL shipping within the same data centre, asynchronous replication to the geographic standby. Sync inside the DC because the round trip is sub-millisecond and the RPO of zero is worth it; async across regions because five to fifty milliseconds is too much to add to every commit and RPO of a few seconds is acceptable for DR.

Split brain: fencing token in the WAL header, one-writer-at-a-time enforced by an external coordinator (etcd, ZooKeeper, or a simpler HA leader-lock if you know what you're doing). Every serious engine post-mortem in the industry has a split-brain chapter.

## Language choice: Rust or C++, and one hard-won lesson

The core is Rust or C++. Anything with a GC will eat you on the tail. Java engines exist (LMAX) and they work because the team spent years learning to write allocation-free Java that reads like C. That is a valid path and it is not the fast one to get to production.

We build in Rust. The borrow checker is worth the ramp, the ecosystem for async network code is mature, and the ability to reason about ownership without a runtime is the right primitive for a matching engine.

The lesson we paid for: wrapping engine fields in parking_lot::Mutex under a tokio-based network layer will compile, will boot, and will hang every HTTP request in production. parking_lot is a blocking synchronous primitive. Holding it across an await point stalls the whole executor. It is exactly the kind of trap that looks correct in the diff and is fatal at load. Use tokio::sync::Mutex if you must hold a lock across await, migrate to actors if you can, and test every synchronisation change end-to-end under real traffic before you ship.

## Where the 30 years matters

The Basis Points team has been building software for financial markets for three decades. We wrote trading algorithms when "algo" meant a Fortran program on a Sun workstation. We built FIX gateways into the earliest electronic venues, before FIX had a version number anyone remembers. We built FX and crypto exchanges, both the greenfield ones and the ones that had to interoperate with a decade of legacy plumbing. We ran market-making and arbitrage books that had to survive the venues they traded on. And we have built matching engines and low-latency infrastructure for exchanges — the actual core, not the wrapper — on hardware that ranged from RS/6000s to today's NUMA-pinned Xeon boxes with Solarflare NICs.

Most companies selling "a matching engine" today are shipping their first serious build. You can tell from the vocabulary. They talk about "microservices" and "kubernetes" and "eventual consistency" as if those were features of a matching engine rather than symptoms of not having built one before. The vocabulary of engineers who have shipped this class of software is different: sequence number, snapshot cadence, tail latency, cache line, kernel bypass, deterministic replay, self-trade prevention, fencing token. If you are evaluating vendors, listen to which vocabulary they speak.

## What most in-house builds get wrong

We are called in to review in-house engines several times a year. The same mistakes appear every time.

1. **Floating point for prices.** IEEE 754 cannot represent 0.1. Any engine that uses f64 for price or size will accumulate rounding errors that show up as one-satoshi drift in reconciliation and eventually as failed settlements. Use fixed-point integers, always. Store prices as integer ticks.

2. **Out-of-band risk service.** Discussed above. The moment risk crosses a network boundary you have a race and the race will fire.

3. **WAL as afterthought.** Teams build the matching function first, then bolt on a WAL to satisfy audit. The result is a WAL that does not actually let you replay to identical state, which means it is not a WAL, it is a log file. The WAL contract is designed first.

4. **Underestimating operational cost.** A matching engine is not built once and left alone. It needs a monitoring stack, an on-call rota with people who understand it, snapshot and replay drills, a WAL retention policy, a certificate rotation story, a versioned protocol with backwards-compatibility discipline, and a change-management process that treats a config edit like a code deploy. Budget the operations, not just the build.

5. **No self-trade prevention.** Every serious venue has it. Every retail-scale engine we review has forgotten it. When the same account crosses itself the trade is real, the fees are real, and the wash-trading investigation is real.

6. **Containerised production.** Docker is a fine dev-loop tool. It is not what you run your matching engine in. Container network overhead, cgroup CPU throttling and namespace context switches will destroy your tail latency in ways your dev environment will never show. Production runs on a real host, with the engine pinned to isolated cores and the OS tuned for latency, not for average throughput.

## Benchmarks: the BP engine, real numbers

Measured on a 32-core bare-metal box with the workload we ship to production:

- p50 order-to-ack: 45us warm cache

- p99 order-to-ack: 180us warm cache

- Sustained throughput: 55,000 orders per second per shard, full stack (parse, risk, match, WAL, publish)

- Snapshot write: 4 seconds for a full-book snapshot

- WAL replay: 24 hours of production traffic in 90 seconds

- Deterministic replay: byte-identical state hash, verified in CI on every build

These are the numbers we hit today. The numbers we are hitting after the LD4 cutover with kernel bypass and PTP will be materially better and we will publish them when they land.

## Talk to us

The Basis Points matching engine is running our live venue right now. Every design decision in this article is one we made, shipped, and are running against real order flow. If you need a matching engine for your venue, we licence ours. [Talk to us](/contact).

Ready to trade everything, 24/7?