Skip to content

Serve repeated requests from the newest finished job, never a failed one (fkie-cad/mcritweb#47) - #160

Open
r0ny123 wants to merge 2 commits into
danielplohmann:mainfrom
r0ny123:fix/47-cached-job-prefers-finished
Open

Serve repeated requests from the newest finished job, never a failed one (fkie-cad/mcritweb#47)#160
r0ny123 wants to merge 2 commits into
danielplohmann:mainfrom
r0ny123:fix/47-cached-job-prefers-finished

Conversation

@r0ny123

@r0ny123 r0ny123 commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Closes fkie-cad/mcritweb#47 (labelled mcrit there). The cache selection was wrong: a force rematch that was still queued or running shadowed the finished job whose result a repeated plain request could actually use.

What changed

get_cached_job_id picked the newest job with the same descriptor that had attempts left and was not terminated, whatever its state. Both queues now prefer a finished job over an unfinished one and the newest within each group; jobs without attempts left (failed) or terminated ones are never reused, exactly the rule the issue proposes.

  • MongoQueue: one query, sorted finished_at descending then created_at descending. A descending sort places every date before null, so the finished jobs come first without a second query.
  • LocalQueue: the same rule over its jobs; the _descriptor_to_job map, which could only ever remember the last job, is gone.
  • test_job_cache_prio encoded the old rule (a queued forced job wins over the finished one) and is updated; new tests cover finished-over-running, newest-finished, and that failed/terminated jobs are skipped, on both queues.

Verified against a running instance

MongoDB 7 + this branch as server and worker, four samples, a finished 1vs1 job between two of them:

step result
force rematch, then cancel it, then plain request served by the earlier finished job
force rematch still running, plain request meanwhile served by the earlier finished job
forced job finished, plain request served by the newer finished job

ruff, ty and the full suite (mongo tests included) pass.

get_cached_job_id picked the newest job with the same descriptor that had
attempts left and was not terminated, whatever its state. A force rematch
that was still queued or running therefore shadowed the finished job whose
result the next plain request could actually use, and cancelling the forced
job was the only way to get the old result back (fkie-cad/mcritweb#47).

Both queues now prefer a finished job over an unfinished one and the newest
within each group; jobs without attempts left (failed) or terminated ones
are never reused. On MongoDB this is one sort - finished_at descending puts
every date before null - so no second query is needed. LocalQueue drops its
descriptor-to-job map, which could only ever remember the last job.

test_job_cache_prio encoded the old rule (a queued forced job wins over the
finished one) and is updated to the new one.
LocalQueue.get_job() indexed its defaultdict, so a lookup of an unknown or
already-cleaned id inserted a None entry that every later scan of the jobs
tripped over - the new cache scan included. The lookup uses .get() now, and
the cache test covers a lookup of an unknown id. Also narrows the queue
type in the mongo test helper for ty.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Queue result cache gets "destroyed" by force rematch.

1 participant