flywheel: high-performance data channels without NIFs

September 1, 2026 · personal · elixir · erlang · otp · realtime · distributed-systems

Recently I was toying with some code related to one of my recurring interests, data structures and algos, specifically histograms in the context of a Radix Sort - and the inspiration for the subject of this post struck when I was reading up on BEAM atomics. Back in 2013 I took Martin Thompson's training on high-performance lock-free concurrent data structures, and it was superb: mechanical sympathy, cache lines, the Disruptor. It has been rattling around in my head ever since. Then OTP 21.2 added the atomics module, a fixed-size array of 64-bit integers with atomic operations over it, and that is an escape hatch out of the higher-level constructs and down much closer to the metal. I wanted to see how far it goes. Out fell flywheel, a high-performance fixed size data channel with back-pressure inspired by the disruptor pattern.

Also note, I am making some assumptions based on what I know and rolling on gut in many areas - many micro-optimisations to be found and the fun is in the details. Also, I need to set up a proper test bench, which means the numbers in this post are more relative than maxed out. But compared to the existing alternatives, this ring buffer goes brrr.

The short version: it moves 64-bit integers between BEAM processes through a fixed-size ring buffer built on :atomics. Payloads live off-heap in a shared array, so nothing is copied, nothing is allocated per message, and no process's garbage collector ever sees them. The mailbox is still involved, but it only carries a wake signal once per batch. And the buffer is bounded, so a producer that outruns its consumer gets back-pressure - which send/2 will never give you.

It is not a general purpose queue, by design. Signed int64 payloads only: packed structs, timestamps, ids, enum tags. No arbitrary terms, no node boundaries, no selective receive.

using it

A ring is a term you make and hand to producers. Capacity is a power of two:

ring = Flywheel.new(1024)
:ok = Flywheel.push(ring, 42)
{:ok, 42} = Flywheel.pop(ring)

Whichever process calls new/2 becomes the consumer and gets woken when a producer publishes, unless you name a different one with :consumer. You can send the ring term itself wherever you like; every copy points at the same shared array, which is what keeps the transport free of copying.

In anything real the consumer is a supervised process, and Flywheel.Channel is that process:

{:ok, ch} = Flywheel.Channel.start_link(capacity: 8192, handler: &IO.inspect/1)
ring = Flywheel.Channel.ring(ch)

# from any number of other processes
Flywheel.push_wait(ring, 42)

Producers hold the ring and write into shared memory directly. They never send the channel a message, so they never show up in its mailbox at all.

the two pushes are the policy

There are two ways to offer an item, and picking between them is most of the design work.

# refuses when full, never blocks
case Flywheel.push(ring, packed) do
  :ok -> :ok
  {:error, :full} -> count_a_drop()
end

# waits for space, up to a timeout you choose
case Flywheel.push_wait(ring, packed, 100) do
  :ok -> :ok
  {:error, :timeout} -> trip_the_breaker()
end

{:error, :full} and {:error, :timeout} are the two return values send/2 cannot produce. A refused push/2 never advanced the cursor, so no sequence is burnt and the consumer never has to wait one out.

draining without allocating

pop_batch/2 is the convenient drain, and it allocates: a cons cell per item plus the reverse inside it, all on the consumer's heap. That puts back the collection pressure the ring was there to remove. In a hot loop, fold straight into an accumulator instead:

{acc, consumed, skipped} =
  Flywheel.fold(ring, 4096, acc, fn item, acc -> apply_update(acc, item) end)

A channel takes the same thing as a handler, {:fold, fun, acc0}, and carries the accumulator across drain passes. In the repo's A/B at 1,000 producers the fold drain went from 3,147 to 4,596 Kmsg/s, a 46% gain, and minor GCs fell from 8,347 to somewhere between 50 and 74. The list drain did not move at all.

The catch is that the fold runs on the channel process. It has to be quick, because a slow one stalls the drain and back-pressures every producer through the bounded capacity. It also has to be total: raise in there and you take down the channel that owns the ring.

the unbounded mailbox

Erlang mailboxes are unbounded. A consumer that falls behind does not fail, it accumulates, and the actual failure turns up much later as the OOM killer. The latency was ruined long before anyone noticed.

Under a flood of 100 producers pushing 500k items unthrottled, that shape looks like this. Median of nine interleaved rounds, i5-13600K, OTP 29, in a container with pinned cores. Relative findings, not portable numbers:

All six rows are the same finding. The mailbox gets its throughput by letting most of the run pile up in a queue with no ceiling, and that backlog is where the 49.6 ms median and the 52.6 MB both come from. The ring sits at about 2,700 items because it is not allowed to hold more, which is why its median is sub-millisecond.

Push the fan-in to 1,000 producers into the same consumer and nothing about the story changes: 5,899 Kmsg/s against 3,803, again winning all nine rounds, medians of 673 µs against 51.2 ms, 5.7 MB against 53.4 MB. So this is not something that only works at low fan-in.

the ring

A ring is capacity slots and two monotonically increasing cursors. The consumer's cursor never passes the producer's; the producer's may never get more than capacity ahead of the consumer's. Slot index is just seq band mask, which is why capacity has to be a power of two.

push_wait/3 yields the scheduler; it never spins. A producer burning its time slice on a spin loop would be starving the consumer it is waiting for. That constraint is why flywheel is a re-derivation of the Disruptor's ideas and not a port of it: the Disruptor's wait strategies assume the waiting thread owns a core, and on the BEAM it does not.

the claim protocol

With one producer, the cursor advance is the publish: write the payload, bump the cursor, done, two atomic operations per item. With many producers you need one more thing, because publication is no longer in order.

The claim is a compare-exchange, not the cheaper fetch-and-add, and that costs something. Fetch-and-add has no way to fail: the cursor moves whether or not there was room. A producer that grabs a sequence and then finds the ring full has already burnt it, and it cannot mark the slot as abandoned either, because the slot still holds a live item from the previous lap.

That leaves the consumer one way out, which is to wait the deadline and skip, one sequence at a time. Under sustained overload that stops being a rare fault and becomes how the thing runs. A CAS claim never gets there.

The stamps exist for one hazard: a producer that dies between claiming its sequence and writing the payload. That is a real risk and it is not eliminated, only bounded. After skip_after_us (50 ms by default) the consumer steps over the sequence and counts a drop.

The rest of the layout is cache mechanics. There are four separate atomics arrays instead of one, because every atomics operation bounds-checks against the array header. Put a hot cell at a low index and it dirties the same line as that header, which every concurrent operation then has to re-fetch. header_slack pushes the hot cells clear of it, and that made more difference than padding the cells apart from each other.

pushed and popped are derived from the cursors instead of being counted. A counter increment per item is roughly 50% overhead on a two-operation push, to record something the cursors already know.

sharding: when one cursor is the problem

Every producer in a single ring compare-exchanges the same word. That word lives in one cache line, and a line can only be written by one core at a time, so past a handful of producers they spend their time passing it between them. You can see it in the sweep further down: a single ring peaks at four senders and falls away from there.

Flywheel.Shards gets rid of the sharing by giving each producer its own ring - the single writer principle, applied to the cursor.

Assignment is a hash of the producer's pid, cached in the process dictionary, because erlang:phash2/2 costs more than the push it is routing. The API is the same one:

shards = Flywheel.Shards.new(4, 4096)

# from any number of producer processes
:ok = Flywheel.Shards.push_wait(shards, 1_234_567)

# in the consumer
{acc, count, _skipped} = Flywheel.Shards.fold(shards, 4096, 0, &(&1 + &2))

what the consumer pays for it

Producers stop contending and the bill lands at the other end: one consumer now has to visit N rings instead of one.

Five hundred thousand messages, one consumer, 65,536 slots total in every configuration so the comparison is at equal memory. Kmsg/s:

Four shards beat or match a single ring at every sender count I measured, by 1.2 to 1.8×. Between eight and 250 senders the single ring has already collapsed to around 11,000 while four shards hold near 19,000. At a thousand producers both fall away and the margin narrows to 1.2×.

Sixty-four shards is where it goes wrong. At four and eight senders it runs at roughly half the four-shard number, because the consumer spends its round walking rings with nothing in them. More shards only pay off once producers genuinely outnumber them. I would default to four.

what you give up

Global ordering. A single ring totally orders every item from every producer. Shards preserve order per producer and nothing more. If you need a global sequence, this is the wrong module.

Fungible capacity. Capacity is shards × capacity_each, but a producer can only draw on its own shard's share. One hot producer gets back-pressure while the other shards sit empty, where a single ring would have pooled the space.

against what you would reach for instead

The contention story on the mailbox side comes down to one operation in ERTS. With the default message_queue_data: :on_heap, every send/2 tries a trylock on one word of the receiver's process struct, and that collapses as senders climb:

Kmsg/s. Look at the third column at one sender: flywheel runs at roughly half the mailbox's speed. That is the honest shape of it. Flywheel is bounded first and quick second, and at low fan-in you are paying for a property you are not using.

I want to be loud about the off_heap flag here. It fixes the cliff outright, and in this isolated, drain-bound probe one process flag beats flywheel by about 1.4 to 1.9× from eight senders up. If raw drain-bound throughput is all you need, Process.flag(:message_queue_data, :off_heap) is one line and you can stop reading.

What it does not give you is a bound. That is why the end-to-end flood table at the top of this post comes out the other way round, with the ring 1.55 to 1.57× ahead while holding a thirtieth of the memory. Once a system saturates, keeping occupancy low is itself a throughput mechanism.

Against the rest of the usual toolbox under the same 100-producer flood. This is explicitly not like-for-like; it measures each tool's overload posture:

Do not quote the drop-on-full row without its fourth column. Under a 50× overload it delivered 19% of what was offered. That is the policy doing exactly what it says on the tin, and it is also why it is not the default.

the worked example: a CME-style feed

Synthetic floods cannot tell you what boundedness is worth, because a synthetic payload costs nothing by turning up late. Market data does. A price from 40 ms ago is history, and a strategy acting on it is putting money on a number already known to be wrong.

So the worked example is a futures feed handler: four decoders publishing CME-shaped book updates into one book builder, across five instruments: E-mini S&P 500, E-mini Nasdaq-100, WTI Crude, COMEX Gold, and the 10-year T-note with its 1/64 tick.

the payload

Each update packs into a single integer. :atomics would give me 63 bits to play with; this uses 59:

# 11 bits instrument | 21 price ticks | 14 size | 4 level | 2 side | 2 action | 5 seq
def pack(inst, ticks, size, level, side, action, seq)
    when inst in 0..2047 and ticks in 0..2_097_151 and size in 0..16_383 and
         level in 0..10 and side in 0..3 and action in 0..3 do
  inst <<< 48 ||| ticks <<< 27 ||| size <<< 13 ||| level <<< 9 |||
    side <<< 7 ||| action <<< 5 ||| (seq &&& 0x1F)
end

Two things in there matter more than the bit-fiddling. Prices are tick indices, not decimals, which is what gets every listed contract inside 21 bits and what keeps the arithmetic exact - the decoder asserts rem(micros, tick) == 0, so a price that is not a whole number of ticks is a decode bug and gets treated as one.

Nothing is clamped, either. Every field is guarded, so if the venue relists a contract outside 21 bits of ticks the decoder raises a FunctionClauseError instead of quietly writing a different price into the book.

capacity is denominated in time

The consumer folds updates straight into an :atomics book, mutated in place, invisible to every collector. Sweeping ring capacity over 500k updates, unthrottled, median of nine rounds:

That last column is capacity multiplied by the per-item drain cost, and nothing else.

The drain ns column is the control here: it stays between 69 and 91 ns across an eightfold capacity sweep. Flat means the metric is measuring what it costs to apply an update, not what it costs to sit around waiting for one, which is what makes the rows comparable. The drift to ~90 ns at the bottom is per-batch fixed cost spread over smaller batches; the drain itself is not slower.

At 32,768 and below, peak outstanding equals capacity exactly. The ring sat on its ceiling for the whole run, which is what the bound column is describing.

Two caveats, because it is easy to read the wrong lesson off that sweep. The throughput knee at 8,192 is real but it is an artefact of the probe, which offers about 14M updates/s, far beyond anything a real channel does. At a sustainable feed rate the small ring never fills and its 93 µs bound is free. And the 32,768-slot row beating the mailbox on throughput, 14,429 against 13,597, is well inside run-to-run spread. I would call that a tie and take the 2.3 ms ceiling as the prize.

two policies, one library

The example is also why one library ships both push functions.

The feed side takes push/2 and never blocks. The exchange is not going to slow down for you, and a late quote is worthless anyway. The five seq_lo bits are what make refusing safe: offer sequences 0..79 into a 64-slot ring and you get 64 accepted, 16 refused, and a consumer whose sequence check reports a gap of exactly 16. That is a counted gap you can take to the venue's recovery channel instead of quietly growing stale. Cheapest field in the layout, most useful work.

The order and fill side takes push_wait/3. Losing a fill is not an option, and neither is a risk process minutes behind the exchange. On a full 8-slot ring:

push_wait(.., 100ms) -> {:error, :timeout} after 99125 us
full? true, still 8 items, nothing lost

That return value is the whole point. send/2 has no way to say it, so the backlog builds up somewhere you cannot see until the node dies. Here it is a value you can alert on, throttle on, or trip a circuit breaker with.

about these numbers

Everything here was measured on one host: an i5-13600K on OTP 29, in a container with pinned cores, because the Windows host clamps the monotonic clock to about 102 µs and that is bigger than several of the things I am trying to measure. It is not a setup that shows what the design can really do, and the absolute figures should not travel. The relative shape held across runs well enough that it seemed worth posting early.

status

I will likely open source it once I have built some confidence in it and jumped through the appropriate hoops to do so. I am still tinkering, and I will write more on the topic when I have more to share, mpmc next up and experiments in pointing to off heap refc binaries for richer data types and comparing the throughput in combination. From this some form of an SBE port and the list goes on...

What exists: 72 correctness tests covering exactly-once delivery under back-pressure, per-producer ordering, the 2^59 sequence wrap including batches that straddle it, producer death mid-claim, the park/wake race, sharded rings, and both channel handler modes. What does not exist: a stable API, a released package, or any production mileage whatsoever.

If you are hitting the failure mode at the top of this post today, the answer is almost certainly Process.flag(:message_queue_data, :off_heap), or NIFs to add some cool data structures, or scaling horizontally using GenStage or one of the many other options. In the meantime I am going to continue pretending I have proper arrays on my favourite platform to see what good stuff will fall out as I extend the capability set.