epoll: hijackConn mutates worker-thread-only liveConns and connCount from the dispatch goroutine (the epoll twin of celeris#539)

[FumingPower3925]

Found while fixing celeris#654 (PR #665). Not reproduced — this is a reading, with the exact lines at df1269c.

What runs off-thread

With AsyncHandlers: true the user handler runs inside ProcessH1 on the dispatch goroutine, so Context.Hijack → hijackConn (engine/epoll/loop.go:1466) performs the whole engine-side teardown on that goroutine. The lines it executes do not all stand on the same ground:

• EPOLL_CTL_DEL — a kernel call, no lock needed; after it the worker sees no further EPOLLIN for the fd.
• l.conns[fd] = nil — taken under driverMu, so it is serialized against growConns' copy + header swap, RegisterConn's occupancy check, and (since fix(epoll): re-check conn ownership after closeConn waits on detachMu, so a hijacked fd is not torn down twice (celeris#654) #665) closeConn's capture and ownership re-check.
• activeConns, closeCount — atomics.
• l.removeLiveConn(cs) (:1486) and l.connCount-- (:1490) — neither.

liveConns is documented Worker-thread-only — no synchronization. (#318) (loop.go:86) and connCount is a bare int (loop.go:80).

Two concrete consequences

1. Torn sweep. checkTimeouts (:2575, loop at :2577) and shutdown's phase 1 read len(l.liveConns) once and then index l.liveConns[i] each iteration. removeLiveConn swaps-with-last and shrinks the slice, so a hijack concurrent with a sweep can index out of range, or move an unvisited conn into an already-visited slot and skip it for that pass.
2. Lost decrement. l.connCount-- here races acceptAll's l.connCount++ (:914). A lost update leaves connCount permanently wrong, and l.connCount == 0 (:742) is the gate that lets a draining loop reach SUSPENDED — so a loop that loses one increment never suspends again.

Why this is separate from celeris#654

#654 / #665 removes the duplicate teardown: a loop-thread closeConn that parked on cs.detachMu and then redid work hijackConn had already done. It does not change where hijackConn's own teardown runs. #665's regression test observes correct counters only because its asyncClosed barrier orders the two goroutines — that ordering belongs to the test, not to production. The comment in hijackConn that used to assert these fields "are only read by the worker between epoll_wait returns, never while a dispatch goroutine is mid-ProcessH1" is exactly the premise #654 disproves; #665 replaces it with a per-line statement of what is and is not serialized, and points here.

io_uring does not carry this: its hijackConn refuses when w.async (celeris#539). The price of that refusal is celeris#558.

Fix directions

• (a) Refuse async hijack on epoll too, as io_uring does (io_uring: hijackConn mutates worker-owned state and submits SQEs, but runs on the dispatch goroutine in async mode #539). Small, but it removes Hijack under AsyncHandlers on the one engine where it currently works — io_uring: give Hijack a synchronous detach-queue round trip so it works under AsyncHandlers #558 is the io_uring counterpart of paying that bill.
• (b) Split the teardown. Keep the serialized/atomic half in hijackConn and defer removeLiveConn + connCount-- to drainDetachQueue's cs.hijacked branch, which already runs on the loop thread for exactly this conn. This delays connCount reaching 0 until the dispatch goroutine exits, which touches the DRAINING→SUSPENDED gate and the transplant accounting, so both need their own tests.

How to judge a fix

go test -race ./engine/epoll/ with a test that hijacks from a dispatch goroutine while a sweep walks liveConns: the race detector should report the liveConns / connCount accesses today and be silent after. Counter-level witness: accept and hijack concurrently in a loop, then assert connCount returns to 0 and the loop can still reach SUSPENDED.

[FumingPower3925]

PR #698 fixes this with option (b). Option (a), refusing async hijack, is not used: it would remove a working feature.

Confirmed on 9f4d89b. The unit arms run an off-thread hijackConn while checkTimeouts or the post-switch sweep walks liveConns. Under -race that is a data race on liveConns, on connCount, and on the sweep's dormancy fields that removeLiveConn writes. The end-to-end arm is this issue's counter witness: async hijacks under accept churn, then PauseAccept. On CI x86, run 36242060999, it left 0/2 loops SUSPENDED.

Option (b) as first written was not safe. hijackConn closes the descriptor at once. So while the live-set entry waits for the loop, the number can go to the next accept on the same loop. With liveConns keyed by fd number, the two entries alias:

• a swap-remove fixes up the wrong conn's liveIdx;
• the stale entry is never found again;
• shutdown would close that number twice.

TestHijackedEntryDoesNotAliasTheNextOwnerOfItsDescriptor pins this.

What the PR does:

• What stays off-thread. hijackConn, off-thread, now touches only what is serialized: EPOLL_CTL_DEL, the slot (under driverMu), the atomics, and hijacked, which is now atomic.
• The hand-off. It then enqueues the conn immediately. The loop removes it from the live set, decrements connCount and unlinks it from the dirty list at its next drain, exactly once. It does not wait for the dispatch goroutine to exit, because for a handler that serves the hijacked conn itself, the exit is the end of that session.
• Live-set entries. liveConns holds *connState. The reap, the sweep, shutdown and the dirty pass all skip a hijacked entry.
• Slot installs. Accept and adopt install their slot under driverMu, so the install is ordered after the hijack's clear of the same number.
• Pooling. A hijacked connState is never pooled.

The last item also fixes a pre-existing defect found on the way. A pipelined request whose predecessor's response was still in writeBuf could hijack; the goroutine then enqueued the conn twice. The first entry pooled the connState, and the second put the pooled struct on the dirty list. TestTwoQueueEntriesForAHijackedConnReachNoPooledConnState fails on 9f4d89b's code (arm64).

A second pre-existing defect also fails on 9f4d89b's code, in TestDirtyPassSkipsAHijackedConn: the dirty pass wrote a hijacked conn's queued bytes to its old descriptor number, which another file already owned.

Evidence (evidence/celeris-669-668/lane-20260926/), at the fixed head e038a40:

• CI x86, run 36245610547, every job log read: the epoll package is ok with 0 data-race reports, and the Adaptive job's results match main's name for name: 103 PASS lines over 101 test names, 0 FAIL, 0 SKIP on both. (Corrected in review round 2: an unanchored count had also taken the workflow's own echoed script lines, giving 106 PASS and 2 SKIP. Recount: round2/tools/recount_r1_adaptive.sh.)
• arm64, whole ./engine/epoll package under -race: 120 PASS, 0 FAIL, 6 SKIP. Main gives 95 / 0 / 6, and the 101 tests the two runs share have 0 outcome differences.
• arm64, the lane's tests x10: 340 PASS, 0 FAIL, 0 SKIP.
• The accept-churn arm: hijacked=64 ... suspended=2/2 in 10/10 runs, against 0/2 on 9f4d89b's code on both arches.
• Mutants: one per mechanism, all killed at e038a40. Among them:
  • hijackConn mutating off-thread again (31 data races);
  • no notice at hijack;
  • settling twice;
  • the reap reading a stale entry;
  • the dirty pass writing a hijacked conn's number;
  • an accept install without the lock.