Devlog #14 — The Test That Was Wrong for the Right Reason

Date: July 1, 2026

Focus: Closing the one gap between what the validator promised and what the

engine did — and a failing test that turned out to be a compliment.

Every previous devlog in this stretch was about adding something: a contract, a

schema, a preview, a whole app, a folder of daisy-chainable scenes. This one is

about a seam I didn’t know was open until a machine smarter than my own review

pointed at it — and about the good kind of failing test, the kind that fails

because the code is better than the assertion you wrote to check it.

It is a small change. Two functions, one interception, four tests. But it is the

change that made the foundation honest, and I’ve come to think honesty is the

only feature that matters in a tool other people will trust.

1. A Promise the Engine Wasn’t Keeping

When scenes learned to namespace their sequence ids — intro inside a scene named

courtroom becomes courtroom:intro — I gave references a forgiving rule. A bare

intro would resolve to the one sequence with that name, so nobody’s existing

scripts broke and nobody had to type a colon they didn’t want to type. I wrote that

rule as a pure function, tested it six ways, and wired it into the validator. The

validator was happy. I was happy.

The validator was also lying, and I didn’t notice.

Because the validator used the forgiving rule, but the engine — the thing that

actually fires sequences at runtime — still matched ids the old way: exactly, byte

for byte. So a writer could chain two sequences in the same scene with a bare

reference, run validate, get a cheerful green light, ship it — and in the game

that chain would simply never fire. No error. No crash. Just a scene that quietly

refused to continue, and a validator that had promised it would.

That is the worst class of bug, the one that passes every check and fails only the

person who trusted the checks. And I shipped it without seeing it.

2. The Reviewer That Read the Whole Board

What caught it was the final whole-branch review — a pass over the entire feature

at once, on the most capable model I had, precisely because some bugs only exist in

the space between correct-looking pieces. Each individual commit had passed its

own review. The resolver was right. The validator was right. The engine was right.

The gap was in the sentence connecting them: *the validator resolves references one

way, the runtime resolves them another, and nobody had checked that those two ways

agreed.*

It traced the exact call sites — s.id == id, fired.contains(id) — and laid the

contradiction out plainly: the validator promises resolution the engine does not

perform. It even pointed at my own design spec, where the very first example used a

bare same-scene chain, and noted that this example would validate and then not work.

I have written a lot of reviews. I don’t think I would have caught this one by

staring at any single diff, because no single diff was wrong. You had to hold the

whole thing in your head at once and ask whether the promises matched the behavior.

The lesson I keep relearning: the last review should look at the forest, not the

trees, because that is the only place certain bugs live.

3. One Rule, Both Sides

The fix was almost anticlimactic once the problem was named. Give the engine the

same resolver the validator already used. One small method — resolve a reference to

a concrete id using the forgiving three-step rule — and route the three places the

runtime looks up a sequence through it: explicit plays, tick-time plays, and the

on_sequence_finished trigger that powers every chain.

The trigger was the subtle one. It checks a set of finished sequence ids, which

are qualified (courtroom:intro), against a reference that might be bare (intro).

So the reference has to be resolved to its qualified form before the check, not

after. Get that backwards and a bare chain still silently fails — the exact bug,

wearing a slightly different hat. Resolving first, then checking, is the whole trick.

And there was a decision worth making deliberately: when the runtime resolves a bare

reference, should it warn? I decided no. The validator already nudges the writer at

authoring time — “this works, but qualify it before another scene reuses the name.”

The game, at play time, should just do the right thing quietly. The warning belongs

where the writing happens, not where the playing does.

4. The Test That Failed Because It Was Right

Then the part that made me laugh out loud.

I wrote the tests first, the way you’re supposed to. Four of them: a same-scene bare

chain should fire, a cross-scene qualified chain should fire, play should resolve

a bare id, and an ambiguous bare reference should refuse to fire. I built each one

to start a sequence, finish it, then tick once more and check that the chained

sequence had begun.

Two passed immediately. Two failed. And they failed on the assertion before the

one I cared about — the setup, not the payoff. My “the first sequence finished”

check came back with two sequences finished when I expected one.

I stared at that for a moment before it landed. The engine wasn’t waiting a tick to

fire the chained sequence. The moment the first sequence finished, its finish armed

the second, which started and ran to completion in the same tick. The cascade

resolved itself in one frame. My test assumed the chain would amble along one step

per tick; the engine had already sprinted to the end.

The code wasn’t failing my test. My test was failing to keep up with the code.

The fix was to stop asserting when the chain fires and start asserting that it

fires — loop a few ticks and check that the downstream sequence actually ran. That’s

a more honest test anyway: I care that the chain completes, not that it completes on

a schedule I guessed at. But the reason it’s a better test is that a mistake forced

it. I would have shipped the brittle version, the one coupled to a frame count,

if the engine hadn’t embarrassed it first.

5. Why This Is the Good Kind of Mistake

There’s a failure mode where a test fails and you shrug and loosen the assertion

until it passes, and you learn nothing and quietly make the test worthless. This was

the opposite. The test failed, I asked why, and the why taught me something true

about the system: chained sequences cascade within a single tick. That’s a real

property, worth knowing, and I only know it because I guessed wrong out loud and the

compiler-of-truth that is a test suite corrected me.

A green test tells you nothing you didn’t already believe. A red test that surprises

you is the only kind that teaches. The mistake wasn’t a detour from the work — it

was the work, the moment the system told me something I hadn’t known to ask.

6. The Part I Am Proud Of

The foundation was already shipping. Scenes loaded, merged, validated, packaged. By

every green checkmark, it was done. But it wasn’t honest — the tool told writers

one thing and the game did another, and that gap would have found its way to the

first modder who followed the documentation instead of the folklore.

Now the validator and the engine resolve references through the same rule, the same

function, the same three steps. What the editor accepts is exactly what the game

runs. A bare same-scene chain a writer draws in Leitmotif fires in the game, and an

ambiguous one refuses to guess — the same answer, whichever side you ask.

If I reduce it to one sentence: the day the tool and the game started answering the

same question the same way is the day a writer could finally trust both of them at

once.

The scenes were always possible. Then they were authorable. Now they do, in play,

exactly what they promised on the page — and I found that out because a test was

wrong in the most useful way a test can be.