bugfix: Fix cython _on_stream_event_in call to match python impl - #608
Merged
wbarnha merged 3 commits intoFeb 16, 2024
Merged
Conversation
wbarnha
self-requested a review
February 15, 2024 23:08
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #608 +/- ##
=======================================
Coverage 93.72% 93.72%
=======================================
Files 102 102
Lines 11122 11122
Branches 1545 1545
=======================================
Hits 10424 10424
Misses 607 607
Partials 91 91 ☔ View full report in Codecov by Sentry. |
Member
|
Thanks for the PR! Just run |
Contributor
Author
Running that script changes a large number of files so I opened another PR to match. If this is an issue with my local black config or flake8, please let me know and I'll take another look. I am assuming that there was a change in black as the linting failures that appear in the CI/CD job match the changes that I saw locally when running linting. PR here: #609 |
wbarnha
approved these changes
Feb 16, 2024
wbarnha
pushed a commit
that referenced
this pull request
Aug 7, 2026
The optional Cython accelerators were never executed by a single test. `pip install .` compiles them into site-packages, but pytest runs from the repository root, so `import faust` resolves to the source tree and every accelerated import sits behind `try: ... except ImportError`. With no .so next to the .pyx the fallback engaged silently, so the `use-cython: true` matrix legs differed from the `false` ones only in whether the build step succeeded. Build the extensions in place on those legs, and add FAUST_REQUIRE_CYTHON, which turns the silent fallback into a failure so the gap cannot quietly reopen. This matters beyond this branch: the parity tests proposed in #751 note they otherwise "just run the pure-Python one twice", which in CI was always. ## The bug this uncovered `StreamIterator._try_get_quick_value` carried two faults that concealed each other. `chan_queue_empty` holds the bound `queue.empty` method: # streams.py # streams.pyx if chan_queue_empty(): if self.chan_queue_empty: A bound method is always truthy, so the extension always reported "queue empty" and took the awaiting path. That made the `else` unreachable -- which hid the fact that it returned the bare value from `get_nowait()` instead of the `(need_slow_get, value)` pair the caller unpacks. Had the fast path ever run, `next()` would have raised TypeError, or silently mis-unpacked a two-element value into `need_slow_get, channel_value`. Both are fixed together; fixing only the condition would have activated the broken return. The pure-Python twin has always had this right, so this restores the fast path the extension was meant to provide and brings the two implementations back into agreement. Net effect: the compiled iterator has been doing strictly more work than the pure Python it was meant to accelerate, for as long as it has existed. ## Tests tests/unit/test_cython_parity.py covers the guard, window parity (HoppingWindow/SlidingWindow against their _Py twins across step boundaries), and both branches of the queue fast path. The stream tests drive `StreamIterator.next()` directly rather than `async for`, which would need a running worker, and count calls to `Channel.__anext__` -- the awaiting path -- because that is the only clean signal. The two obvious alternatives both fail: `get_nowait` is called by `Queue.get` on the slow path too, and `empty` is called from inside `get_nowait`, so both fire either way and only the counts differ. Verified in both directions: reintroducing the bug fails the test with 5 `__anext__` calls for 5 already-queued values, against 0 when fixed. Suite passes in every configuration: extensions built (2254 passed), absent (2207 passed, parity tests skipped), and free-threaded 3.14t with PYTHON_GIL=0 (2258 passed). ## Docs docs/developerguide/cython.rst records how to test the compiled code, the drift history that motivates parity tests (#608, the on_topic_buffer_full defect left unfixed because fixing one twin alone would desynchronise them, and the fast-path pair above), and the conventions for adding an accelerator -- including that the wins concentrate in per-call arithmetic, not in code whose body is mostly `await`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K8qT5E3rnSXvw7ibNLXrVr
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Changes the cython logic around invoking
_on_stream_event_into match the logic in the python implementation.Fixes #607