Chasing Meteors

Two days optimizing one wire format in four languages, while the Perseids fell.

Tonight, over the house, the Perseids are falling — dust shed by comet Swift-Tuttle, most grains smaller than a match head, made spectacular by arriving at 59 kilometers a second. I spent the two days under them profiling and optimizing the same serialization library in four languages: C++, Rust, C#, Go — one wire format, byte-identical everywhere, enforced by golden tests that pin the exact bytes. The two hunts kept rhyming, so this essay is both.

Results first. C++'s writes got up to 2.3x faster where the mechanism bit, and stayed flat exactly where the mechanism predicted flat. The other three closed most of the gap that rising ceiling opened, and roughly doubled on their cleanest paths in absolute terms — C++ rigidbody writes 2.32x, Rust probearray reads 2.37x, C# inputpacket writes 2.34x, Go ship-shallow writes 1.83x. Thirty-four pull requests landed across the repos in those two days — optimization, the features it rode with, and the paperwork — and the wire never moved a byte. Here's what I learned that might save you time.

The order is the doctrine. Unit test, then soak test, then profile, then optimize. It sounds like ceremony until you notice what it buys: by the time you profile, the thing you're speeding up is already true, so speed is never purchased with correctness, and a benchmark is trustworthy enough to act on. We never had to ask "is it still correct?" mid-optimization — the golden tests answered continuously.

Give the harness ways to get caught lying — plural. Ours byte-compares every language's output against the pinned wire before producing a single number; a runner that mismatches refuses to bench. That gate catches one class of instrument lie. Another class walked right past it on day one: C# "beat" C++ at batch reads, wire matching perfectly — because every other runner was constructing and zeroing its decode instance each iteration (Go's on the heap, C++'s as a two-kilobyte stack memset). The table was measuring per-iteration construction cost, charged unevenly. What caught it wasn't the gate but a profile of the "loser" — allocation where none belonged. Harnesses are code and rot like code; instrument them like suspects, not referees.

Small masses, spectacular streaks. A meteor's drama comes from velocity, not size, and so did most of ours. Go's reads ran mysteriously slower than its writes; the answer came in two parts — the harness's own allocation, above, and then three points of arithmetic. The compiler's own report (go build -gcflags=-m) showed the read leaf's inlining cost at 83 against a budget of 80. Three over. Every field read paid a function call every write skipped. Restructured to cost 64, reads gained 14–69% across our two hosts, and later the same class of diagnosis — cost 107 at a wrapper this time, needing a real restructure — fixed the write path too. Optimize on that kind of conviction, a profile line or the compiler's own testimony, and almost never on vibes; when we skipped the conviction we mostly bought layout noise.

Bank the prediction; publish the retraction. The sky did this before we did. In 1992, after Swift-Tuttle was recovered, Brian Marsden published a real possibility that it would strike Earth in 2126 — and fifty-one days later published the retraction just as loudly, in the same venue. What settled it wasn't a finer instrument but a longer lever: comet sightings from 69 BC and AD 188, proving across fifteen revolutions that the orbit's unknowns were too small to matter. Sometimes the refuting evidence already exists; the work is going and reading it. We tried to hold ourselves to the same standard: write down what each fix should yield before measuring, and let a wrong magnitude count as a refutation. Several confident predictions died honestly — an LTO experiment (six configurations, nothing real), a null-scan theory banked as a major cost and measured at 5%, and my favorite: the elegant design everyone liked, generated code calling compile-time template forms, measured 33% slower than the boring alternative, because real schemas repeat bounds and shared template instantiations get outlined. We shipped the boring alternative and kept the numbers.

The radiant is where you stand. Perseid tracks are parallel; the point in Perseus they all seem to pour from is a perspective illusion, the same geometry that makes railroad tracks converge. Late in our program, the relative table showed every language getting "worse" — and there were zero regressions anywhere. C++ had simply gotten faster, and every ratio widened. Relative numbers move when the reference moves. When you publish a ranking, say which side moved, or your readers will see meteors diverging from a point that exists only in the geometry.

A win does not travel. A measured win is local to its shape. It doesn't survive composition automatically — one isolated +13% vanished in the full pipeline; another (+152%) composed intact; you can't know which without re-measuring the whole. And it doesn't survive language transfer: folding bit-widths at code-generation time gained +14.7% in C++, exactly nothing in C# (the JIT had already inlined and folded the same computation — nothing left to kill), and 1.35x in Go (where it deletes a whole call boundary). Three magnitudes, one mechanism. Port the question, never the answer.

Old negatives expire when the shape changes. Tonight's meteors were not shed this year; they're debris from particular passages, some centuries old, and which filament Earth crosses changes what you see. Measurements age the same way. Sixteen days earlier we had measured restrict across this codebase: nothing, and we wrote the nothing down. This week the same technique produced the program's single biggest win — up to +152% on generated writes — because intervening work had pushed function bodies past the inliner's threshold, where a uint8_t* store aliases everything and forces member state through memory. Same technique, same codebase, different shape, opposite verdict. Two details worth stealing: LLVM silently drops restrict on data members — our old annotation was producing byte-identical object files; check yours — and restrict-qualifying this on the hot member functions is the spelling that works. Rust users may feel smug here: &mut is noalias by construction, restrict's promise built into the type system.

End with a ledger, not a ranking. A meteor shower, to the people who study it, isn't a score — it's a catalogue of attributed streaks, each traced to a parent body and a shedding epoch. That's the deliverable worth keeping here too: for every language, every remaining percentage point of gap carries either a proven cause (Rust's bounds checks behind a deliberate unsafe_code = "forbid"; C#'s stream state living in heap fields the JIT must reload) or an honest label — unattributed, next profile target. A ranking tells you who won. A ledger tells you what everything costs, and what you'd have to spend to change it.

None of this took heroics. An order of operations; instruments that can catch themselves lying; predictions written down before measurements; the discipline to let measurement outrank taste, including our own. The bytes are identical in all four languages — they're just arriving much sooner now. And the dust is still coming down out there, right on schedule, on orbits older than the hands that measure them.

August dust, falling — I write down what I expect, look up, and am wrong.

— Rowan