Skip to content

fix(iouring): fix the accept-direct SQE and gate fixed files behind an explicit opt-in (#541) - #553

Merged
FumingPower3925 merged 1 commit into
mainfrom
fix/iouring-fixed-files-explicit-gate-541
Sep 9, 2026
Merged

FumingPower3925 merged 1 commit into
mainfrom
fix/iouring-fixed-files-explicit-gate-541

Conversation

@FumingPower3925

Copy link
Copy Markdown
Contributor

Refs #541 — deliberately does not close it; that issue is now the readiness checklist.

The bug

prepMultishotAcceptDirect built on prepMultishotAccept, which sets
SOCK_NONBLOCK|SOCK_CLOEXEC, then wrote IORING_FILE_INDEX_ALLOC into file_index.
io_accept_prep rejects a fixed file slot combined with SOCK_CLOEXEC with -EINVAL: a direct
descriptor lives in the ring's file table, not the process fd table, so close-on-exec is meaningless
for it.

The probe therefore failed on every kernel, and the code blamed the kernel:

Res = -EINVAL (-22) means the kernel registered files but refuses ACCEPT_DIRECT (seen on
6.6.10-cix aarch64). Treat as unsupported.

It reproduces on 7.0.12 aarch64 as well. Dropping SOCK_CLOEXEC flips fixed_files=false to true
on the same kernel and container with nothing else changed.

Why the one-line fix alone would have been a trap

cs.fixedFile has never been true anywhere, so every branch gated on it is unexecuted code. A
four-surface audit, each finding attacked by an independent reviewer, found eleven defects and
refuted none
. The worst is the default receive path: prepRecv has no fixed-file variant and
never touches the flags byte, and the single-shot branch is the default (the buffer ring is only
built with CELERIS_IOURING_MULTISHOT_RECV=1). Every connection would arm a recv against a raw fd
equal to its slot index — low slots give -ENOTSOCK, higher ones collide with real sockets in the
process
and the ring reads their bytes. prepCloseDirect also omits the +1 its own comment
documents, and w.conns is indexed by two conflicting namespaces.

The decision

Fix the SQE, and hold the feature off with an explicit gate rather than an accident.

Leaving it to the -EINVAL was not safe: it depends on a kernel continuing to reject a malformed
SQE, and one that tolerated it would silently switch the whole broken path on. Finishing the feature
now means working through #541's checklist and deleting the gate — not finding the flags bug and
assuming that was all.

Also: a log that had become misleading

Both startup lines reported tier.SupportsFixedFiles(), the capability, so they printed
fixed_files=true while the feature was off. They now report the effective value through the same
helper the gate uses, so the two cannot disagree. This caught me during verification — I read
fixed_files=true and thought my gate had failed.

Verified

run result
default fixed_files=false, no warning
CELERIS_IOURING_FIXED_FILES=1 fixed_files=true + per-worker "this path is INCOMPLETE" warning

The test's load-bearing case is tier support without the opt-in — precisely what the SQE fix
newly makes reachable — and it fails without the gate:

fixed_files_gate_test.go:35: fixedFilesEnabled(tierSupports=true) with
CELERIS_IOURING_FIXED_FILES="" = true, want false

Full engine/iouring suite green (100.7s), lint clean.

…n explicit opt-in (#541)

prepMultishotAcceptDirect built on prepMultishotAccept, which sets
SOCK_NONBLOCK|SOCK_CLOEXEC, and then wrote IORING_FILE_INDEX_ALLOC into
file_index. io_accept_prep rejects a fixed file slot combined with
SOCK_CLOEXEC with -EINVAL — a direct descriptor lives in the ring's file
table, not the process fd table, so close-on-exec is meaningless for it.

So the runtime probe failed on every kernel, and the failure was attributed to
the kernel: "the kernel registered files but refuses ACCEPT_DIRECT (seen on
6.6.10-cix aarch64). Treat as unsupported." It reproduces on 7.0.12 aarch64
too, because it was our SQE. Dropping SOCK_CLOEXEC flips fixed_files=false to
true on the same kernel and container with nothing else changed.

That one-line fix would have been a trap. cs.fixedFile has never been true
anywhere, so every branch gated on it is unexecuted code, and an audit of
those branches — four surfaces, each finding attacked by an independent
reviewer — found eleven defects and refuted none. The worst is the DEFAULT
receive path: prepRecv has no fixed-file variant and never touches the flags
byte, and the single-shot branch is the default (the buffer ring is only built
with CELERIS_IOURING_MULTISHOT_RECV=1), so every connection would arm a recv
against a raw fd equal to its slot index. Low slots fail with -ENOTSOCK;
higher ones collide with real sockets in the process — another worker's listen
fd, a driver database connection — and the ring reads their bytes.
prepCloseDirect also omits the +1 its own comment documents, and w.conns is
indexed by two conflicting namespaces.

So the SQE is fixed and the feature is now held off by an explicit gate
instead. Leaving it to the -EINVAL was not safe: that depends on a kernel
continuing to reject a malformed SQE, and one that tolerated it would silently
switch the whole broken path on. celeris#541 carries the readiness checklist;
finishing the feature means working through it and deleting the gate, not
finding the flags bug and assuming that was all.

Also fixes a log that had become actively misleading: both startup lines
reported tier.SupportsFixedFiles(), the CAPABILITY, so they printed
fixed_files=true while the feature was off. They now report the effective
value through the same helper the gate uses, so the two cannot disagree.

Verified: default run logs fixed_files=false and no warning; with
CELERIS_IOURING_FIXED_FILES=1 it logs fixed_files=true plus a per-worker
warning that the path is incomplete. The test's load-bearing case is tier
support WITHOUT the opt-in — exactly what the SQE fix newly makes reachable —
and it fails without the gate.
@FumingPower3925
FumingPower3925 merged commit ba690ae into main Sep 9, 2026
10 checks passed
@FumingPower3925
FumingPower3925 deleted the fix/iouring-fixed-files-explicit-gate-541 branch September 9, 2026 21:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant