Skip to content

Async functions to await a full or empty queue#4

Merged
bikeshedder merged 12 commits into
deadpool-rs:masterfrom
overhacked:await_full_empty
Jan 10, 2024
Merged

Async functions to await a full or empty queue#4
bikeshedder merged 12 commits into
deadpool-rs:masterfrom
overhacked:await_full_empty

Conversation

@overhacked

Copy link
Copy Markdown
Contributor

Allow awaiting a full or empty queue.

  • Queue.empty() is useful for chaining futures
    in a multi-threaded program when some subsequent
    action, like a verification step, needs to be
    taken once the queue is emptied.
  • Queue.full() can help manage backpressure so that
    an autoscaling system can monitor when the queue fills
    before it starts increasing the number of workers.

@bikeshedder bikeshedder left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a nice addition to the crate.

I just think the "simple" implementations of those methods using just Semaphore::acquire_many are wrong. I'd like to see a test case that actually checks if calling the full and empty method don't block concurrent push and pop calls. The tests would look like that:

  • Full
    • Create a queue with capacity of 2
    • call full (tokio::spawn)
    • call push
    • call pop and make sure it doesn't block
    • call push twice and check that the full call returned
  • Empty
    • Create a queue with capacity of 2
    • call push twice
    • call empty (tokio::spawn)
    • call push
    • call pop and make sure it doesn't block
    • call pop twice and check that empty call returned

One more thing: I don't like the names empty and full. empty almost sounds like it is supposed to clear the queue. How about creating naming them wait_empty/wait_full or on_empty/on_full instead? That would make it very clear that the queues aren't modified but you're waiting for a specific event to happen.

Comment thread src/limited.rs Outdated
Comment thread src/limited.rs Outdated
Comment thread src/resizable.rs Outdated
@bikeshedder

bikeshedder commented Mar 18, 2022

Copy link
Copy Markdown
Collaborator

If you were to change the deadqueue::atomic::Available implementation so it returns the new (or old) value after add and sub you could use that to find out if the queue is full and/or empty. That way you could avoid calling available_permits and remove one possible race condition where the queue is empty for the fraction of a second but Semaphore::available_permits doesn't show that because of concurrent push/pop calls.

Even then you might miss an empty/full event, but it's less likely to happen than calling into Semaphore::available_permits outside of any atomic guarantees.

@overhacked

Copy link
Copy Markdown
Contributor Author

Hi, read all these comments. Thanks for the feedback! Trying to find time to rework the PR accordingly.

Allow awaiting a full or empty queue.

- `Queue.empty()` is useful for chaining futures
  in a multi-threaded program when some subsequent
  action, like a verification step, needs to be
  taken once the queue is emptied.
- `Queue.full()` can help manage backpressure so that
  an autoscaling system can monitor when the queue fills
  before it starts increasing the number of workers.
Same API as described in previous commit for limited::Queue,
but the implementation is different, because resizable::Queue's
capacity management is different. It doesn't use a pair of Semaphore
structs that can be awaited to detect full and empty conditions;
resizable::Queue only uses one Semaphore for push operations. This
can be awaited to detect an empty queue. Detecting a full queue
asynchronously is more complex: calls to `push()`, `try_push()`,
and `resize()` call a private function to check if the queue
has become full and notify any callers awaiting `full()`.
Also fix missing 0.2.3 and 0.2.4 version links.
Unlimited queues should be able to be awaited
for an empty status. Required a separate implementation
than limited and resizable queue due to different usage
of internal semaphore.
Barrier ensures that the main test thread doesn't
empty (or fill) the queue before the spawned task
has a chance to start. The test still passes
without the Barrier, but could be passing only due
to the short-circuiting `self.if_empty()` or
`self.capacity() == self.len()` logic in the body
of `async fn empty()`/`full()`.
Eliminates deadlocks highlighted in code review caused by
`Semaphore::acquire_many()`

Moved common `Notifier` type alias to `src/lib.rs`
If `limited` feature isn't enabled, doctest in
`src/lib.rs` fails to run, because it depends on
`crate::limited::Queue`. I've wrapped it in a
`cfg_attr` macro so that it is possible to run tests
like:
```shell
> cargo test --no-default-features --features unlimited
```
Discovered that `limited::Queue::available()` was returning incorrect
values, because `self.available.add()` had been omitted from `try_push()`.

Added unit tests for `available()` to all queue types. Also made a small
change in statement order to `unlimited::Queue` to make the code
identical to `limited` for easier comparison.
Per maintainer feedback, avoid using `Semaphore::available_permits` to
detect queue full/empty state, rather use return values of atomic
functions. That way notification is based on queue size at that moment
in time, not a subsequent call into `Semaphore` that might race with
other concurrent `push/pop` calls.
@overhacked

Copy link
Copy Markdown
Contributor Author

Finally circled back to this and updated the wait_full() and wait_empty() to use atomics instead of Semaphore::available_permits(). Thanks for your patience and not closing the PR.

@bikeshedder

Copy link
Copy Markdown
Collaborator

Sorry for taking so long. I completely forgot about that PR. 🙈

After reviewing it I only found two small details:

  1. Every time you use Available:add you add 1 to the previous value to see if the capacity has been reached. I'd rather change that code so the Available:add function already returns the new value so this comparison doesn't need to happen. The same can be done for Available::sub where comparing the new value against 0 is much clearer than comparing the previous value against 1.

  2. It would be nice if there was a way to get a hold of the watch channel of full and empty. That way an application that wants to perform some task every time the queue is full or empty doesn't need to continuously recreate the channel receiver.

If you happen to find time to look into that I'll wait a bit. I'm just as happy to merge this PR without those changes.

Comment thread src/resizable.rs Outdated
@bikeshedder

Copy link
Copy Markdown
Collaborator

Oh, and please explain this comment to me...

How is it borrowed mutably? 🤔

@overhacked

overhacked commented Nov 9, 2023 via email

Copy link
Copy Markdown
Contributor Author

@bikeshedder

Copy link
Copy Markdown
Collaborator

@overhacked Any updates on this PR?

Instead of having to write logic in consumers that adds or subtracts one
from the returned `add/sub` value to get the new value for comparisons,
have `Available` methods return the new value.
Expose the `subscribe()` method on the internal
`tokio::watch::Sender<()>` so that API consumers can subscribe to
repeated queue full/empty notifications without having to re-create and
await a receiver every time, as with the `wait_full()/wait_empty()`
methods.
Comment was regarding old `resize(&mut self, ...)` implementation, which
was changed to `&self` in 39e5013.
@overhacked

Copy link
Copy Markdown
Contributor Author

Requested changes pushed. Sorry I lost track of wrapping this up in the last few months of the year. Nothing going on in November or December... nothing else at all. 😉

Thanks for your patience and support with this!

@bikeshedder
bikeshedder merged commit b71cdce into deadpool-rs:master Jan 10, 2024
@bikeshedder

Copy link
Copy Markdown
Collaborator

Thanks a lot. Awesome work! 🎉

@overhacked
overhacked deleted the await_full_empty branch January 11, 2024 11:52
@bikeshedder

Copy link
Copy Markdown
Collaborator

I just saw that I never actually released a new version including your changes. 🤦

I just releated deadqueue 0.2.5 on crates.io:

📦 https://crates.io/crates/deadqueue/0.2.5

Thanks again for this PR! ❤️

@overhacked

Copy link
Copy Markdown
Contributor Author

Amazing, thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants