flywheel: high-performance data channels without NIFs

September 1, 2026 · personal · elixir · erlang · otp · realtime · distributed-systems

I am not sure why the inspiration struck, maybe it was because I was toying with my own recurring interest in data structures and algos, playing about with Radix Sorts for reasons I'll not bother going into here - but the inspiration struck when I was reading up on BEAM atomics. Back in 2013 I took Martin Thompson's training on high-performance lock-free concurrent data structures, and it was superb: mechanical sympathy, cache lines, the Disruptor. It has been rattling around in my head ever since. Then OTP 21.2 added the atomics module, a fixed-size array of 64-bit integers with atomic operations over it, and that is an escape hatch out of the higher-level constructs and down much closer to the metal. I wanted to see how far it goes. Out fell flywheel, a high-performance fixed size data channel with back-pressure inspired by the disruptor pattern.

Also note, I am making some assumptions based on what I know and rolling on gut in many areas - many micro-optmisations to be found and the fun is in the details. Also, I need to setup a proper test bench so the numbers in this post are more relative than maxed out. But compared to the existing alternatives, this ring buffer goes brrr.

The short version: it moves 64-bit integers between BEAM processes through a fixed-size ring buffer built on :atomics. Payloads live off-heap in a shared array, so there is no copying and no per-message allocation, and no process's garbage collector ever sees them. The mailbox is still there, but it carries one wake signal per batch rather than one message per item. And because the buffer is bounded, a producer that outruns its consumer gets back-pressure, the one thing send/2 cannot give you.

It is deliberately not a general queue. Signed int64 payloads only: packed structs, timestamps, ids, enum tags. No arbitrary terms, no node boundaries, no selective receive.

using it

A ring is a term you make and hand to producers. Capacity is a power of two:

ring = Flywheel.new(1024)
:ok = Flywheel.push(ring, 42)
{:ok, 42} = Flywheel.pop(ring)

The process that calls new/2 becomes the consumer, the one woken when a producer publishes, unless :consumer says otherwise. The term itself is safe to send to other processes: they all end up pointing at the same shared array, which is what makes the transport free of copying.

In anything real the consumer is a supervised process, and Flywheel.Channel is that process:

{:ok, ch} = Flywheel.Channel.start_link(capacity: 8192, handler: &IO.inspect/1)
ring = Flywheel.Channel.ring(ch)

# from any number of other processes
Flywheel.push_wait(ring, 42)

Producers hold the ring and write into shared memory directly. They never send the channel a message and never appear in its mailbox, which is the whole point: it carries one wake per batch rather than one message per item.

the two pushes are the policy

There are two ways to offer an item, and choosing between them is the only real design decision the library asks of you.

# refuses when full, never blocks
case Flywheel.push(ring, packed) do
  :ok -> :ok
  {:error, :full} -> count_a_drop()
end

# waits for space, up to a timeout you choose
case Flywheel.push_wait(ring, packed, 100) do
  :ok -> :ok
  {:error, :timeout} -> trip_the_breaker()
end

{:error, :full} and {:error, :timeout} are the return values send/2 has no way to produce, and that is the entire difference between the two transports. A refused push/2 has not advanced the cursor, so no sequence is burnt and the consumer never has to wait one out.

draining without allocating

pop_batch/2 is the convenient drain, and it allocates: one cons cell per item plus the reverse inside it, all on the consumer's heap, which puts back exactly the collection pressure the ring exists to remove. For a hot loop, fold straight into an accumulator instead:

{acc, consumed, skipped} =
  Flywheel.fold(ring, 4096, acc, fn item, acc -> apply_update(acc, item) end)

A channel takes the same thing as a handler, {:fold, fun, acc0}, and carries the accumulator across drain passes. The repo's own A/B at 1,000 producers moved the fold drain from 3,147 to 4,596 Kmsg/s, a 46% gain, with minor GCs collapsing from 8,347 to between 50 and 74, while the list drain did not move at all. The fold runs on the channel process, so it has to be quick and it has to be total: a slow one stalls the drain and back-pressures every producer through the bounded capacity, and one that raises crashes the channel that owns the ring.

the failure mode you already know

Erlang mailboxes are unbounded. A consumer that falls behind does not fail; it accumulates. The failure arrives later, all at once, from the OOM killer, with the latency already ruined long before anyone noticed.

Under a flood of 100 producers pushing 500k items unthrottled, that shape looks like this. Median of nine interleaved rounds, i5-13600K, OTP 29, in a container with pinned cores. Relative findings, not portable numbers:

Those are not six results, they are one. The mailbox reaches its throughput by letting most of the run pile into a queue with no upper bound, and that backlog is the 49.6 ms median and is the 52.6 MB. The ring holds about 2,700 items because it is not allowed to hold more, and that is precisely why its median is sub-millisecond.

Push the fan-in to 1,000 producers into the same consumer and nothing about the story changes: 5,899 Kmsg/s against 3,803, again winning all nine rounds, medians of 673 µs against 51.2 ms, 5.7 MB against 53.4 MB. Boundedness is not a low-fan-in trick.

the ring

A ring is capacity slots and two monotonically increasing cursors. The consumer's cursor never passes the producer's; the producer's may never get more than capacity ahead of the consumer's. Slot index is just seq band mask, which is why capacity has to be a power of two.

The waiting is the part worth dwelling on. push_wait/3 yields the scheduler; it never spins. A producer that burns its time slice waiting would be starving the very consumer it is waiting for, which is the failure this design exists to avoid. That single constraint is why this is a re-derivation of the LMAX Disruptor's ideas rather than a port of it: the Disruptor's wait strategies assume the waiting thread owns a core, and on the BEAM it emphatically does not.

the claim protocol

With one producer, the cursor advance is the publish: write the payload, bump the cursor, done, two atomic operations per item. With many producers you need one more thing, because publication is no longer in order.

The claim is a compare-exchange rather than a fetch-and-add, and that is a deliberate trade of some cost for a property. Fetch-and-add is cheaper but cannot signal its own failure: a producer that claims a sequence and then finds the ring full has already advanced the cursor and cannot mark the slot, because the slot still holds a live item from the previous lap. The consumer could then only recover by waiting out a deadline, one sequence at a time, which under sustained overload becomes the steady state rather than a rare fault. A CAS claim cannot reach that state.

The one hazard the stamps exist to cover is a producer that dies between claiming and writing. That is real, and the cost is bounded rather than eliminated: after skip_after_us (50 ms by default) the consumer steps over the sequence and counts a drop.

Everything else about the layout is cache mechanics. Four separate atomics arrays rather than one, because every atomics operation bounds-checks against the array header, so a hot cell at a low index dirties a line that every concurrent operation must re-fetch; header_slack keeps the hot cells clear of it, and that turned out to matter more than padding between them. The pushed and popped counts are derived from the cursors rather than counted, because a counter increment per item is roughly 50% overhead on a two-operation push to record something the cursors already know.

sharding: when one cursor is the problem

Every producer in a single ring compare-exchanges the same word. That word lives in one cache line, and a line can only be written by one core at a time, so past a handful of producers they spend their time passing it between them. The contention table above shows it: a single ring peaks at four senders and falls away from there.

Flywheel.Shards removes the sharing by giving each producer its own ring. It is the single writer principle, applied to the cursor.

The assignment is a hash of the producer's pid, cached in the process dictionary, because erlang:phash2/2 costs more than the push it is routing. The API is the one you already have:

shards = Flywheel.Shards.new(4, 4096)

# from any number of producer processes
:ok = Flywheel.Shards.push_wait(shards, 1_234_567)

# in the consumer
{acc, count, _skipped} = Flywheel.Shards.fold(shards, 4096, 0, &(&1 + &2))

what the consumer pays for it

Producers stop contending, and the bill lands on the other end. One consumer now has to visit N rings instead of one.

Five hundred thousand messages, one consumer, 65,536 slots total in every configuration so the comparison is at equal memory. Kmsg/s:

Four shards beat or match a single ring at every sender count measured, by 1.2 to 1.8×. The interesting band is eight to 250 senders, where the single ring has already collapsed to around 11,000 and four shards hold near 19,000. At one thousand producers both fall away and the margin narrows to 1.2×.

Sixty-four shards are the cautionary column. At four and eight senders they run at roughly half the four-shard number, because the consumer is spending its round walking rings that have nothing in them. More shards only help once producers genuinely outnumber them. Four is the default I would reach for.

what you give up

Global ordering. A single ring totally orders every item from every producer. Shards preserve order per producer and nothing more. If you need a global sequence, this is the wrong module.

Fungible capacity. Capacity is shards × capacity_each, but a producer can only draw on its own shard's share. One hot producer gets back-pressure while the other shards sit empty, where a single ring would have pooled the space.

where it sits against what you would reach for instead

The contention story is one operation in ERTS. With the default message_queue_data: :on_heap, every send/2 attempts a trylock on one word of the receiver's process struct, and it collapses as senders climb:

Kmsg/s. Note the third column at one sender: flywheel is roughly half the mailbox's speed there, and that is the honest shape of the thing. This is not a faster queue, it is a bounded one, and at low fan-in you are paying for a property you are not using.

The part I want to be loud about is the off_heap flag. It fixes the cliff, and in that isolated, drain-bound probe one process flag beats flywheel by about 1.4 to 1.9× from eight senders up. If raw drain-bound throughput is all you need, Process.flag(:message_queue_data, :off_heap) is one line and you can stop reading. What the flag does not give you is a bound, which is why the end-to-end flood table at the top of this post looks different from this one: there, the ring is 1.55 to 1.57× ahead while holding a thirtieth of the memory, because bounded occupancy is itself a throughput mechanism once a system saturates.

Against the rest of the usual toolbox under the same 100-producer flood. This is explicitly not like-for-like; it measures each tool's overload posture:

The drop-on-full row is the one I would not want quoted without its fourth column. Under a 50× overload it delivered 19% of what was offered. That is the policy working exactly as specified, and it is also why it is not the default.

the worked example: a CME-style feed

Synthetic floods cannot tell you what boundedness is worth, because a synthetic payload costs nothing by arriving late. Market data does. A price from 40 ms ago is not a price, it is history, and a strategy acting on it is betting money on a number already known to be wrong.

So the worked example is a futures feed handler: four decoders publishing CME-shaped book updates into one book builder, across five instruments: E-mini S&P 500, E-mini Nasdaq-100, WTI Crude, COMEX Gold, and the 10-year T-note with its 1/64 tick.

the payload

Each update packs into a single integer. Not 63 bits, which is what :atomics would let you use, but 59:

# 11 bits instrument | 21 price ticks | 14 size | 4 level | 2 side | 2 action | 5 seq
def pack(inst, ticks, size, level, side, action, seq)
    when inst in 0..2047 and ticks in 0..2_097_151 and size in 0..16_383 and
         level in 0..10 and side in 0..3 and action in 0..3 do
  inst <<< 48 ||| ticks <<< 27 ||| size <<< 13 ||| level <<< 9 |||
    side <<< 7 ||| action <<< 5 ||| (seq &&& 0x1F)
end

Two details in there that I like more than the bit-fiddling. Prices are tick indices rather than decimals, which is what makes 21 bits enough for every listed contract and what makes the arithmetic exact: the decoder asserts rem(micros, tick) == 0, so a price that is not a whole number of ticks is a decode bug rather than something to round away. And nothing is clamped: every field is guarded, so a contract the venue relists outside 21 bits of ticks raises a FunctionClauseError in the decoder rather than silently becoming a different price inside the book.

The five seq_lo bits are the cheapest thing in the layout and do the most interesting job.

capacity is denominated in time

The consumer folds updates straight into an :atomics book, mutated in place, invisible to every collector. Sweeping ring capacity over 500k updates, unthrottled, median of nine rounds:

The column that earns the library is the last one, and it comes out of a single multiplication.

Look at the drain ns column in the table above: it stays between 69 and 91 ns across an eightfold capacity sweep. That flatness is the control. It says the metric is measuring the cost of applying an update rather than the cost of waiting around for one, which is what makes the arms comparable at all. The drift to ~90 ns at the bottom of the sweep is per-batch fixed cost amortised over smaller batches, not a slower drain.

And notice that at 32,768 and below, peak outstanding equals capacity exactly. The ring did not approach its ceiling, it sat on it, which is the clearest demonstration of what the bound column means.

Two caveats about that sweep, because it is easy to read the wrong lesson off it. The throughput knee at 8,192 is real but it is an artefact of the probe: this thing offers about 14M updates/s, far beyond any real channel. At a sustainable feed rate the small ring never fills at all, and its 93 µs bound is free. And the 32,768-slot row beating the mailbox on throughput, 14,429 against 13,597, is well inside run-to-run spread. I would call that a tie and take the 2.3 ms ceiling as the entire prize.

two policies, one library

The example is also why one library ships both push functions.

The feed side takes push/2 and never blocks. The exchange will not slow down for you, and a late quote is worthless anyway. What makes the refusal safe is those five seq_lo bits: offer sequences 0..79 into a 64-slot ring and you get 64 accepted, 16 refused, and a consumer whose sequence check reports a gap of exactly 16. That is a counted, recoverable gap you take to the venue's recovery channel, rather than silently growing staleness.

The order and fill side takes push_wait/3. Losing a fill is not an option, and neither is a risk process minutes behind the exchange. On a full 8-slot ring:

push_wait(.., 100ms) -> {:error, :timeout} after 99125 us
full? true, still 8 items, nothing lost

That return value is the entire point. send/2 has no way to say it, so the backlog goes somewhere you cannot see until the node dies. Here it is a value you can alert on, throttle on, or trip a circuit breaker with.

about these numbers

Everything here was measured on one host: an i5-13600K on OTP 29, in a container with pinned cores, because the Windows host clamps the monotonic clock to about 102 µs and that is larger than several of the things being measured. It is not an environment that shows what the design can really do, and the absolute figures should not travel. The relative shape held across runs well enough that it seemed worth sharing early.

status

I will likely open source it once I have built some confidence in it and jumped through the appropriate hoops to do so. I am still tinkering, and I will write more on the topic when I have more to share, mpmc next up and experiments in pointing to off heap refc binaries for richer data types and comparing the throughput in combination. From this some form of an SBE port and the list goes on...

What exists: 72 correctness tests covering exactly-once delivery under back-pressure, per-producer ordering, the 2^59 sequence wrap including batches that straddle it, producer death mid-claim, the park/wake race, sharded rings, and both channel handler modes. What does not exist: a stable API, a released package, or any production mileage whatsoever.

If you are hitting the failure mode at the top of this post today, the answer is almost certainly Process.flag(:message_queue_data, :off_heap), or NIFs to add some cool data structures, or scaling horizontally using GenStage or one of the many other options. In the meantime I am going to continue pretending I have proper arrays on my favorite platform to see what good stuff will fall out as I extend the capability set.