feat(vision-mixer): stinger transitions

[wagenet]

A stinger plays a keyed clip over the program while another transition runs beneath it. It is triggered through the existing transition endpoint as type stinger, naming a Media Player block that declares itself a stinger clip source and is wired to a keyed (DSK) input.

Cut point, the transition beneath, and its duration are Media Player block properties, since where a clip covers follows from the artwork; a take names only its clip. The clip is held on its first frame while idle, so a take only starts playback: 0.5 ms at 4K against 28.4 ms unparked. Declaration is opt-in, so a looping graphic on a keyed input is never parked or unlooped.

The parts worth reviewing

The cut beneath is anchored to the clip, not timed. The clip's stream offset gives the output frame containing the cut point, and a one-shot probe holds the mixer's output on the frame before it until the take is applied. The take waits for the mixer to reach that frame however far its output trails the clock, and gives up only if the output stops advancing for 5 s. Cutting on wall clock instead applies the cut to whichever frame the mixer is on, which on a loaded machine is often before the clip is on air.

A cut the clip may not cover is reported as StingerFailed with still_running: true (a StingerCompleted still follows). What a viewer sees in each case:

• Clip frame late at the cut (checked by a probe on the keyed pad that lives for one take and does one atomic max per clip frame): the mixer composites an earlier clip frame, or none, so the switch shows through whatever the clip does not cover.
• Clip never reaches the mixer by cut point + 500 ms: the keyed input is taken off air and the cut is made plainly, so a late clip cannot play over the new source out of step with it.
• Mixer output stalled for 5 s before the cut frame: the cut is made on wall clock.

A keyed input holds its last frame by default. For a stinger's input that repeat is bounded to two mixer frames: a single late clip frame freezes the graphic rather than blanking it, and the frame a clip ends on is not composited when the input is next revealed. videoaggregator only drops an expired frame on an output frame where the pad has nothing queued, and the next take queues its first frame ahead of time, so a finished stinger waits for the mixer's output to pass the clip's end plus that repeat before hiding the keyed input, re-arming and releasing the mixer.

A take's mixer claim carries a token, so a take whose flow is stopped mid-clip cannot drive the pipeline that replaces it.

Limitations

Straight alpha only: premultiplied alpha composites about 40% too dark and cannot be detected from the file, so a binding that declares it is refused. No fill+key pairs and no audio from the clip.

Following the mixer means a take, and StingerCompleted, run as late as the mixer's output does. On a machine where the mixer ran at about a quarter of real time, completion came several seconds after the clip ended on the clock.

Tests

Run at 3569e429, macOS, local:

• stinger_test (12): 12/12 at normal priority, and 12/12 in 5 runs under taskpolicy -b with four background-QoS CPU burners, which reproduces the macOS runner's flakes (4 of those runs on the same code before the rebase onto main). It covers arming, an undeclared source left alone, a full cycle with re-arm, the refusal paths, clip-failure degradation, a take outliving its flow, composited pixels end to end, cut timing and a repeat take.
• stinger_alpha_test (4) with STROM_REQUIRE_GL=1, so the GL case rendered rather than skipped. openapi_test, pipeline_lifecycle_test, vision_mixer_fx_test, 694 lib tests, cargo clippy --tests --all-features -D warnings, cargo fmt --check.
• stinger_preroll_bench is #[ignore]d as a measurement harness and was not run.

Not run: the rest of the workspace suite, and anything on Linux. Upstream CI sets STROM_REQUIRE_GL only on the macOS job, so the GL alpha case has not run there.

What the timing guards catch:

• Cut timing, twelve takes: a take either reports an uncovered cut, or must land on the named clip frame or read one early (which clip frame is composited near a frame boundary moves with load). At most half may report, so a clip that never reaches the cut still fails. Under much heavier load (eight burners, the suite in parallel, alternating arms run by run), no round put an unreported bad cut on air, against 6 of 6 for the previous head, which cut on wall clock at clip frames 1-11 or on a frame with no clip. Two of those six rounds still failed, on the harness not seeing a program change within 10 s while the mixer ran far below real time; the previous head shows the same timeouts.
• a_repeat_stinger_plays_forward_from_the_start fires the second take as soon as the first completes and judges every program frame by timestamp. Without the end-of-clip wait it failed 10/10 at normal priority when that wait was added; it is weak when starved, because a slow machine fires the second take late. It also fails if the clip leaves the program before it ends.

Not guarded by a deterministic test: the late-clip-frame report. It fired once in the heavy-load runs and never at normal priority or CI-like load; forcing it would need a test-only delay in the clip path.

One segfault in the compositor's blend (blend_pads, in ORC-generated code) appeared in about 80 heavy-load runs across both arms. It could not be reproduced in 40 targeted runs and is not attributed to this change.

🤖 Generated with Claude Code

[srperens]

Noted as a draft, so I am holding off on a full review until it is marked ready for
review. If you want one before then, say so here or request me as a reviewer.

[wagenet]

Written by Claude (AI).

Update this PR once #809 lands. #809 adds a per-DSK dsk_N_alpha_mode property on the vision mixer that composites premultiplied input correctly on both backends. The stinger clip reaches the mixer through the same dsk_in_N input, so the refusal here is no longer needed:

• Remove ALPHA_MODE_PROPERTY / ALPHA_MODE_PREMULTIPLIED and the PremultipliedUnsupported check in backend/src/gst/stinger.rs, along with the test that asserts the rejection. The source-side alpha_mode property is not exposed in the Media Player block's UI anyway.
• Update docs/STINGER_TRANSITIONS.md (lines 54–60 and 75): a premultiplied clip is supported by setting the keyed input's dsk_N_alpha_mode to premultiplied, not refused.

[srperens]

Verdict: Request changes — the mechanism is sound and unusually well argued, but the wall-clock fallback path releases the mixer hold in the wrong order, and the one test covering the GL compositing path cannot run in this repo's CI.

Requested changes

1. Drop the release sender before removing the probe in the fallback path — backend/src/state/stinger.rs, the _ => arm at line 501: insert drop(release_tx); ahead of pad.remove_probe(probe);. As written the sender stays live until the function returns, so if the probe fires as the timeout expires it blocks in recv_timeout while remove_probe waits on it — holding the program output, and a tokio worker thread, for the full 250 ms. The comment directly above already states the intended order.

2. Either install a GL stack in .github/workflows/ci.yml (gstreamer1.0-gl plus a software rasterizer) so alpha_survives_gl_videomixer executes, or drop it and say in the PR body that the GL alpha path is verified by hand only. CLAUDE.md: a test that skips on a missing element "passes green and guards nothing".

Claims

Claim	Verdict	Evidence
Every buffer before the target costs one timestamp compare	CONFIRMED	backend/src/state/stinger.rs:463 — if pts.nseconds() < hold_from_pts short-circuits the || before fired.swap, and the lock/channel at :472 are past that return
Dropping the sender keeps the probe from holding the mixer on the fallback path	CONTRADICTED	backend/src/state/stinger.rs:504 — pad.remove_probe(probe); runs while release_tx is still in scope; it is only dropped at end of function
The stinger tests run in CI	CONFIRMED for stinger_test	backend/tests/stinger_test.rs:389 — panic!("{name} missing — CI installs gst-libav and plugins-good"); — it fails rather than skips
The alpha tests run in CI	CONTRADICTED	backend/tests/stinger_alpha_test.rs:215 — eprintln!("skipping: no usable GL environment"); and .github/workflows/ci.yml:104-107 installs no gstreamer1.0-gl or mesa
remove_probe blocks on a running callback	EXTERNAL	Documented GStreamer behaviour; assumption stated, not settleable here

Diagnosis — Anchoring the cut to the clip's stream offset rather than a wall-clock delay is the right layer: it removes the race with the aggregator instead of timing around it. Coverage is BOUNDED — the hold only applies where the mixer reaches the cut frame within the grace window, and the PR says so and degrades to wall clock. The BUFFER probe respects the CLAUDE.md constraints on the hot path; the blocking, allocation and formatting all sit past the one-shot guard, which is the correct shape for this.

Radius — SHARED. set_dsk_hold_last_frame, mixer_src_pad and mixer_frame_duration_ns are purely additive with no existing call sites, and the last-frame-hold change is gated on opt-in declaration. What is shared is the probe: while armed it sits on the mixer's program output pad, so every take on that mixer pays the compare until it removes itself. A failure would show up as a stalled program output for up to 250 ms, which is change 1.

Tests & CI — Check (Linux), Build (Linux x86_64/ARM64), Check & Build (WASM), API Contract Check all green at 641cd70. Build (macOS) and Build (Windows) report skipping; there is no platform-specific code in this diff, so Linux-green is adequate cover here. The PR body states that a_repeat_stinger_plays_forward_from_the_start does not fail when its fix is reverted — the disclosure CLAUDE.md asks for, taken at face value.

Design record — backend/src/state/mod.rs is 3082 lines after the split that moved stinger state out; worth carving down further, but not this PR's job.

Confidence: HIGH

[wagenet]

Written by Claude (AI).

Both addressed in ed84f67, rebased on main.

1. Release order. Fixed: the fallback arm drops release_tx before pad.remove_probe(probe), so a probe that fired as the wait expired returns immediately instead of holding program output for the full 250 ms. No regression test: the race needs the probe to fire in the instant the wait times out, which cannot be triggered deterministically.

2. GL test in CI. I took the route the repo already uses rather than installing a GL stack. vision_mixer_fx_test documents that headless Linux runners cannot create a GL context even with the plugins present, so adding packages would not make this test run there. Instead it now follows the same rule: it skips where no context exists and fails under STROM_REQUIRE_GL, which CI sets on the macOS job. Checked locally in all three modes: runs and passes with GL required, fails when required and the probe finds no GL, skips when not required.

The macOS job can't run on this PR without the ci:macos label, so I ran it on the fork at ed84f67: https://github.com/wagenet/strom/actions/runs/35233093100. Build (macOS) passed with STROM_REQUIRE_GL: 1 set, and alpha_survives_gl_videomixer ... ok. With the variable set a skip fails, so that pass means the GL path executed. A green fork run doesn't show as a check here, so the label is still needed for it to count upstream.

The PR body previously claimed this test executed on Linux CI. It did not; the body now says what each guard covers.

Also caught in the rebase: main added a MediaPlayerState initializer in a bridge.rs test, which failed to compile against this branch's new field. Textually clean, so only cargo test --lib showed it.

#809 is still open, so the premultiplied refusal stays for now.

[srperens]

Verdict: Comment — supersedes my Request changes of 2026-09-16 (review-5223777646, head 641cd708), which no longer stands: both requested changes are in. What holds it short of Approve is one row, and it is the same GL-coverage gap on a different file.

Claims

Claim	Verdict	Evidence
The probe-teardown race I asked about is fixed	CONFIRMED	backend/src/state/stinger.rs:506 — drop(release_tx); now precedes backend/src/state/stinger.rs:507 — pad.remove_probe(probe);, so a probe that fired as the wait expired is released before remove_probe waits on it
The alpha test no longer skips silently	CONFIRMED	backend/tests/stinger_alpha_test.rs:244 — strom_types::env::var_opt("STROM_REQUIRE_GL").is_none(), — adopts the precedent already on main in vision_mixer_fx_test.rs and shader_validation_test.rs
The four alpha tests executed in this repo's CI	UNVERIFIED	.github/workflows/ci.yml:645 — STROM_REQUIRE_GL: 1 is set on the macOS job alone, and Build (macOS) is skipping on this PR. In Check (Linux) job 105241909348 the binary reports 4 passed ... finished in 0.09s — four GL pipelines cannot negotiate and reach EOS in 90 ms, so all four skipped
The core stinger logic is genuinely covered	CONFIRMED	same job: stinger_test.rs reports 12 passed; 0 failed ... finished in 18.87s

Diagnosis — The cut is anchored to the clip's own stream offset with a one-shot BUFFER probe, and the wall-clock arm is the fallback rather than the design. Coverage of the anchoring mechanism is ABSOLUTE; coverage of the alpha half is BOUNDED to whatever a macOS run would exercise, which is currently nothing.

Repo rules — The BUFFER probe at backend/src/state/stinger.rs:459 respects the hot-path rules: the per-buffer cost is a pts compare plus an AtomicBool swap, and every lock, channel and format sits past that guard, running once per take. The closure captures no Pipeline, Element or Bin — only channels, an atomic and a pre-built String — so it cannot form the CLAUDE.md reference cycle. No new endpoint: this extends the existing transition route with one additive Option<String> field, and API Contract Check passed. The three new StromEvent variants are additions, not modifications.

Radius — SHARED: the probe is installed on the vision mixer's src pad, so a take holds the program output for up to 250 ms (STINGER_MAX_HOLD, backend/src/state/stinger.rs:46). That bound is enforced by recv_timeout, and the fallback arm now releases before removing, so the failure mode is a late cut rather than a held mixer.

What I would ask for before merge — the ci:macos label on this PR, which is the only route that sets STROM_REQUIRE_GL and therefore the only way the four alpha tests have ever run against this diff. I am deliberately not applying it: it fires a paid run, which is a maintainer's call. Merging on Linux-green and letting the post-merge macOS run cover the alpha half is a legitimate alternative — but it is a choice worth making deliberately, not by default.

Confidence: HIGH

Superseded by my review at head ed84f67: both requested changes are in, so this CHANGES_REQUESTED sits at a dead head and misreports the PR.

[wagenet]

Written by Claude (AI).

A correction to the table: only alpha_survives_gl_videomixer skipped on Linux. The other three alpha tests are CPU-only and panic on a missing element. They composite five 64×64 frames, so 90 ms is expected.

The GL test now has a Linux run. #809 installs a GL stack in the Linux job (gstreamer1.0-gl and Mesa's llvmpipe software renderer under Xvfb, with STROM_REQUIRE_GL=1). I ran this PR's head with only that ci.yml change on the fork: https://github.com/wagenet/strom/actions/runs/35359550379. Check (Linux) reports alpha_survives_gl_videomixer ... ok, and with the variable set a skip would have failed. With the earlier macOS fork run, the GL alpha path has executed on both platforms at ed84f67.

If #809 merges first, this PR rebases onto it, drops the premultiplied refusal, and picks up the Linux GL job, so the test runs in this repo's CI without the ci:macos label. If this one merges first, the label is still the only upstream route. PR body updated to match.

[srperens]

Verdict: Comment — supersedes my Comment of 2026-09-18 (head ed84f671), and the verdict stands, for the same reason: the alpha half has still never been compiled or run by this repo's CI. gh api compare/ed84f671...09271e95 reports ahead=2, behind=0 over two files, so the rows below are the two new commits plus the one row that blocks approval, re-read at this head.

Requested change

1. backend/tests/stinger_test.rs:890-893 describes an "unreadable" take as test noise. It is a visible artefact — the program cuts on a frame with no clip on it — so say in the PR body what a viewer sees when it happens and why up to half of takes doing so is acceptable. Nothing else in the diff guards that path: stinger_preroll_bench.rs reports 1 ignored in the same run.

Claims

Claim	Verdict	Evidence
The four alpha tests executed in this repo's CI	UNVERIFIED	Re-read at 09271e95: Check (Linux) job 107403503091 reports stinger_alpha_test 4 passed ... finished in 0.07s, so all four skipped. .github/workflows/ci.yml:645 — ` STROM_REQUIRE_GL: 1` is set on the macOS job alone, and Build (macOS) is skipping. Blocks approval on its own
The new wait does not extend the hold on the mixer's output	CONFIRMED	backend/src/state/stinger.rs:324 — ` drop(guard);` runs before backend/src/state/stinger.rs:371 — ` .wait_for_mixer_to_pass(&flow, &mixer, clip_end_ns, &still_ours)`, so STINGER_MAX_HOLD is unaffected. What the wait extends is the in-flight claim, and a take fired inside it is refused: backend/src/state/stinger.rs:158 — ` return Err(StingerError::AlreadyRunning(block_instance_id.to_string()).into());`
The premise of the wait — a stinger's keyed pad drops its last frame rather than repeating it	CONFIRMED	backend/src/gst/pipeline/effects/mixer_ops.rs:294 — ` pad.set_property("max-last-buffer-repeat", if hold { u64::MAX } else { 0u64 });`, with hold false for a stinger pad, so the last clip frame stays current exactly until the output passes its end
The relaxed cut-frame test still fails a take that never cuts	CONFIRMED	backend/tests/stinger_test.rs:918 — `` ` landed.iter().all(

Diagnosis — 09271e95 closes the gap the re-arm left: re-arming resets the clip's offset while the keyed pad's last frame is still current, so the next take composited it. Waiting on the mixer's own position rather than a wall-clock sleep is the right anchor and matches the cut-point mechanism already in the PR. Coverage is BOUNDED by mixer_frame_duration_ns: caps without a positive framerate return None and the wait is skipped silently — worth a debug!, not a blocker.

Radius — SHARED, unchanged. The two new commits add no call site outside run_take; what moves is timing, after the cut has been applied. stinger_test.rs reports 12 passed ... finished in 20.29s in the same job, so the core is genuinely exercised.

Overlaps — none of the three kinds. No other open PR touches state/stinger.rs, gst/stinger.rs or mixer_ops.rs, and no open issue owns the stinger design.

Before merge — unchanged, and the only thing between this and Approve: the ci:macos label, the sole route that sets STROM_REQUIRE_GL. Not applying it myself — it fires a paid run. Merging on Linux-green and letting the post-merge macOS run cover the alpha half stays a legitimate call, but make it deliberately.

Confidence: HIGH

[wagenet]

From Claude, working with @wagenet. Reply to review 5301630969. The branch is rebased onto main and the head is now 3569e429.

Requested change (unreadable takes). Agreed: it was a visible artefact, not noise, and it is no longer absorbed. Tracing it showed that most such takes were not late clip frames. The take was giving up on the anchor at cut point + 500 ms of wall clock while the mixer's output trailed the clock, so the cut landed on whatever frame the mixer was on, often before the clip was on air. The take now waits for the mixer to reach the cut frame. The cases that remain are reported as StingerFailed with still_running: true, and the PR body now says what a viewer sees in each: a clip frame late at the cut, a clip that never reaches the mixer (cut made plainly, keyed input taken off air), or a stalled mixer. The cut test now fails any take that cuts with no clip on it without reporting it.

Alpha tests in CI. Still not run upstream. Locally at 3569e429, stinger_alpha_test passes 4/4 with STROM_REQUIRE_GL=1, so the GL case rendered. The ci:macos label stays your call.

Skipped wait with no framerate. Now logs at debug!.

Limits on the evidence: the late-clip-frame report has no deterministic test (it fired once in heavy-load runs, never at normal or CI-like load), and one compositor segfault in about 80 heavy-load runs is unattributed. Both are in the PR body.

[srperens]

Verdict: Comment — this supersedes my Comment of 2026-09-24 (head 09271e95), and the verdict stands. The branch was rebased. git range-diff shows the three reviewed commits unchanged (one-line rebase fix-up) plus one new commit, 3569e42. It is sound. What blocks approval is the unrun GL test and an overlap I missed last time.

Requested change

1. Coordinate with #789 on landing order. #789 adds three wildcard-free match self accessors on StromEvent (event_type(), flow_id(), is_high_frequency()), and this PR adds StingerStarted, StingerCompleted and StingerFailed. Whichever merges second fails to compile until it adds three arms to each accessor. Neither diff shows it; my earlier overlaps=none was wrong.

Claims

Claim	Verdict	Evidence
The new keyed-pad watch is safe on the per-buffer path	CONFIRMED	backend/src/state/stinger.rs:61 — ` seen.fetch_max(end, std::sync::atomic::Ordering::Relaxed);` is the probe's only work: no lock, no allocation. On every exit path, including the flow-restart return, Drop removes it: backend/src/state/stinger.rs:86 — ` self.pad.remove_probe(probe);`
The watch reads the same timeline as the cut frame	CONFIRMED	The bridge restamps clip buffers with the stream offset: backend/src/blocks/builtin/mediaplayer/bridge.rs:436 — ` let adjusted = (pts_val.nseconds() as i64 + offset_ns).max(0) as u64;`. cut_frame_pts is computed from that same offset
A take no longer cuts early on wall clock while the mixer lags	CONFIRMED	The wait now ends only when the mixer stalls: backend/src/state/stinger.rs:753 — ` if progress.stalled(Self::STINGER_CUT_STALL) {`. A cut with no clip is reported, not absorbed: backend/tests/stinger_test.rs:933 — ` !landed.contains(&Some(None)),`
The alpha half executed in this repo's CI	UNVERIFIED	Unchanged. Check (Linux) job 107872151230 reports stinger_alpha_test 4 passed ... finished in 0.07s, and alpha_survives_gl_videomixer still skips there. .github/workflows/ci.yml:685 — ` STROM_REQUIRE_GL: 1` sits in build-macos, and Build (macOS) is skipping

Diagnosis — The root cause is right: the old anchor gave up at cut point + 500 ms of wall clock, which is exactly the case where a lagging mixer cuts before the clip is on air. Following the mixer's position is the right layer. Coverage is BOUNDED in one respect: the wait for the clip's first buffer still gives up on wall clock (backend/src/state/stinger.rs:659 — ` return CutAnchor::NoClip;`). It is now reported.

The loop is CLOSED: it stops once the mixer reaches the cut frame, or after 5 s without progress. On the healthy path it costs one query_position every 2 ms, from the take to the cut. The two-frame repeat changes behaviour only on stinger-declared pads, and the teardown target adds the same window.

Radius — SHARED, unchanged. StingerFailed is new in this PR, so widening still_running breaks no client.

Overlaps — #789, which changes the same code path (the StromEvent accessors); see the requested change. #809 edits set_dsk_enabled next to the renamed function but not on it; it is also what gives the GL test a Linux run.

Tests & CI — Eight checks are green at 3569e429, and stinger_test reports 12 passed ... finished in 20.31s, so the rewritten cut test ran. The same job logs 647 gst_mini_object_unref criticals inside stinger_test. The previous head logged 619, so they predate this commit.

Before merge — Add the ci:macos label or land #809 first. I am not applying the label: it fires a paid run.

Confidence: HIGH