Put the repaired Cython fast paths behind an opt-in setting

[wbarnha]

Description

Two of the Cython fast paths never ran — each guarded by a condition that could not become true — so the extensions quietly did more work than the Python they were meant to accelerate. Repairing them (in #762 and #763) activates code that has, by definition, never executed in production. This gates it.

cython_optimizations defaults to False. With it off the extensions behave exactly as the released versions do, so upgrading changes nothing. Users opt in per app:

app = faust.App('myapp', cython_optimizations=True)

or CYTHON_OPTIMIZATIONS=1 in the environment (prefixed when env_prefix is set, like every other env-backed setting).

The flag is read once per StreamIterator and once per ConductorHandler — so once per stream and once per assigned TP, not per message — into a bint, leaving a predictable branch on the hot path rather than an attribute lookup into app.conf.

What it gates, and what it does not

Gated:

• StreamIterator._try_get_quick_value — taking values already in the channel queue instead of always awaiting.
• ConductorHandler event reuse — decoding once and reusing the event across channels with matching key/value types, instead of deserializing once per subscribed channel.

Not gated: the on_topic_buffer_full argument fix (#764). That one was wrong in both implementations, is not Cython-specific, and produced a metric that was simply incorrect — gating a wrong metric key behind a "Cython improvements" flag would be incoherent. It applies unconditionally.

One consequence worth stating plainly

While the setting is off, the Cython and pure-Python paths genuinely differ. That is not new — it is what has shipped for years — and the flag does not introduce the divergence, only makes it selectable.

The sharpest case is the conductor: a reused event is never decoded again, so a channel whose payload would fail to deserialize raises no error when the event is reused, and raises one when it is not. That changes which channels receive a message and how many acks it takes.

So the parity suites now run with the setting on, which is the configuration in which the two implementations are supposed to agree. Each suite also gains a test pinning the default-off behaviour, so the historical path — the one most users will actually run — stays covered: 5 awaits for 5 queued values in the iterator, one decode per channel in the conductor.

Verification

configuration	result
extensions built (FAUST_REQUIRE_CYTHON=1)	2274 passed
extensions absent	2208 passed
free-threaded 3.14t, PYTHON_GIL=0	2278 passed
mypy -p faust	clean
extra/tools/verify_doc_defaults.py	All OK

flake8 / black / isort clean. Docs build clean, with the setting rendered into the configuration reference and a new cython_optimizations section in the developer guide.

Note on docs/includes/settingref.txt: the entry was added by hand rather than by make configref. The committed file was generated by different tooling — regenerating it reformats every block in the file, which would bury this change in a tree-wide diff.

Note on the base branch

Based on claude/faust-topic-buffer-full-tp (#764), which is based on #763, which is based on #762. The flag gates changes introduced in #762 and #763, so it has to sit above them. Retarget down the stack as each lands.

---

Generated by Claude Code

[wbarnha]

Folded into #762 — the whole series is now one branch, so there is no stack to land in order. Both commits (cf10ab2, the cython_optimizations opt-in; e0ca537, making its eventual retirement a two-line change) are unchanged there.

---

Generated by Claude Code