Skip to content

cow: subtree-aware whiteouts with a durable deletion log and type-aware copy-up#162

Open
congwang-mk wants to merge 13 commits into
mainfrom
cow-deletion-subtree
Open

cow: subtree-aware whiteouts with a durable deletion log and type-aware copy-up#162
congwang-mk wants to merge 13 commits into
mainfrom
cow-deletion-subtree

Conversation

@congwang-mk

Copy link
Copy Markdown
Contributor

Redesign of the seccomp COW branch's deletion model. The flat exact-match HashSet<String> (inherited from the Python-era cowfs/_branch.py, carried verbatim into the Rust rewrite) is replaced by a subtree-aware whiteout set, and copy-up becomes file-type-aware.

Fixes #158
Fixes #159
Fixes #160
Fixes #161

The model

A new DeletionSet (cow/deletions.rs) owns whiteout semantics:

  • covers(rel) matches per path component: a whiteout on d hides d/x and d/x/y, never d2.
  • Entries are never removed. A path re-created after deletion is exposed by existing in the upper (upper presence shadows the whiteout), which makes rmdir d; mkdir d an opaque directory for free: the new d is visible, its old contents stay hidden.
  • Every insertion is appended to deleted.log beside upper/ and fdatasynced, so the deletion half of a change set is now self-describing on disk, like the upper already was. RAM stays the in-process source of truth; the log is the durability record a future recovery sweep (RFC RFC: Multi-sandbox transactions for pipelines #65 Phase 3, PR pipeline: transactional sequential pipelines over a shared COW workdir (RFC #65 Phase 1) #148's preserved branches) can replay.

All merged-view queries (is_deleted, open/stat/readlink, list_merged_dir, changes(), commit()) route through it, instead of each handler re-implementing its own slice of overlay semantics.

Per issue

The invariant

One test pins the property all four issues violated: the merged view the child observed during the run is exactly what commit() publishes. Plus each issue's own reproducer as a unit test, and three end-to-end regressions driving a real sandboxed child (rm -r, mv dir, rmdir non-empty).

Compatibility

No public API, CLI, or SDK signature changes. Child-visible behavior changes only toward kernel semantics (ENOTEMPTY, ENOENT under deleted dirs, dir renames preserving contents). The branch storage dir gains deleted.log; the layout is internal.

Suites: 555 lib / 308 integration / 386 Python locally; the two cli_test failures on my machine (test_learn_captures_openat2, test_write_collapse_skips_protected) reproduce identically on main and are unrelated.

@dzerik

dzerik commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Thanks for taking on the whole deletion model — the subtree-aware whiteouts and the merged-view rmdir/ENOTEMPTY are a real improvement. From the reporter side I can confirm #161 is fully closed (opaque dir after rmdir d; mkdir d), #159's core is closed (reads, listings and O_CREAT under a deleted dir all ENOENT/start-fresh — the O_CREAT-no-resurrect bonus holds), and #160's reported case is closed (a renamed dir keeps its children, whiteouted children do not reappear, a whiteouted source gives ENOENT). Type-aware copy-up does close the copy-up-time hang.

I re-ran each of the four as an in-cage test against this branch. A few things I would want resolved before it lands.

Blocker — #158 is only half-closed; the hang is relocated to commit().

prepare_copy (cow/seccomp.rs:288-306) virtualizes a FIFO/socket/device as an empty regular upper stub but never mark_deleteds the surviving lower special file. commit() applies deletions first (943-950 — the FIFO is not in the set), then walks the upper, sees the stub as a regular file, and at 989-995 does openat2_in_root(workdir, rel, O_WRONLY|O_CREAT|O_TRUNC|O_NOFOLLOW). That destination is the surviving lower FIFO: O_NOFOLLOW does not affect FIFOs and openat2_in_root adds no O_NONBLOCK, so an O_WRONLY open of a reader-less FIFO blocks forever. commit() runs in impl Drop with the error swallowed, so this is an unrecoverable supervisor hang, not a failed merge.

Repro: lower FIFO pipe, child does echo hi > pipe, BranchAction::Commit — the commit blocks (>25s watchdog, pipe still a FIFO on disk). The identical child with Abort passes, which isolates it to the commit publish path. The existing FIFO tests (copy_up_of_fifo, write_open_of_fifo, chmod_of_fifo) all stop before commit(), so CI stays green.

Fix options: mark_deleted the lower path when stubbing a non-regular entry (commit then unlinks-then-creates), unlink a type-mismatched destination before opening, or open the destination O_NONBLOCK and handle ENXIO.

Should-fix — a directory rename onto an existing lower-only destination silently merges the two trees (new, reachable via #160).

handle_rename (seccomp.rs:688-699) stages the source into the upper with copy_up_tree and then renameat_in_root(upper, old, new). When new exists only in the lower it is never whiteouted, so the merged view of new becomes the union of the renamed tree and the pre-existing lower tree, and the syscall returns success. With lower-only a/{a1} and b/{b1}, mv -T a b returns 0 and commits b = {a1, b1} — a state that never existed. POSIX wants ENOTEMPTY for a non-empty destination, or an atomic replace.

Should-fix — #159 gap: the O_DIRECTORY open path never consults is_deleted/covers.

handle_open returns Ok(None) for O_DIRECTORY unconditionally (450-452), and prepare_open's O_DIRECTORY branch resolves to the upper only when upper_dir.is_dir() && !lower_dir.is_dir() (519), otherwise SkipContinue → the kernel opens the lower directory. After a normal rm -rf d of a lower-only dir, stat("d") gives ENOENT but open("d", O_DIRECTORY) succeeds on the lower fd, and fstat on it leaks the deleted directory's inode/nlink/mtime. The blast radius is bounded (getdents is virtualized empty via list_merged_dir, and openat(dirfd, child) is re-gated through absolute-path reconstruction), but an open succeeding where the merged answer is ENOENT breaks the "present as absent" invariant this PR pins elsewhere. Short-circuiting is_deleted(rel) before the O_DIRECTORY split in both functions closes it.

Should-fix — copy_up_tree failure paths (seccomp.rs:402-426).

read_dir/statat errors are swallowed as Ok(()) (406, 416), so a mid-tree EACCES/EIO truncates the staged copy — and handle_rename then mark_deleteds the source anyway (697), so the untraversed children are silently lost at commit. It also recurses with no depth bound over child-controlled directory depth (a pre-seeded deep lower tree can overflow the supervisor's stack), and a mid-tree QuotaExceeded leaves disk_used charged and an orphan upper subtree behind with no rollback. Propagating those errors and making the staging all-or-nothing would match the guarantees the rest of the branch gives.

Nice-to-have — deleted.log format and durability (latent while load() is dead_code).

escape() (deletions.rs:27-37) escapes \\ and \n but not \r, while load() reads via BufRead::lines(), which strips a trailing \r. A file named "d/file\r" round-trips to "d/file", after which covers() hides the wrong sibling and the actually-deleted path resurrects. Separately, create() never fsyncs the parent branch_dir and insert() only sync_data()s the log file, so a power loss after the first deletion can lose the log's directory entry entirely — the exact crash class the log exists for. Escaping \r (or length-delimiting) and one fsync of branch_dir after create would close both.

Coupling with #148 — worth deciding the sequencing before either merges.

These two rework the same SeccompCowBranch commit/changes/deletion surface, and they collide semantically rather than textually:

Suggested order: land DeletionSet first, then rebuild #148's preserve/marker/remainder logic on a single deletion source of truth — either dropping the log's recovery role or having preserve() snapshot DeletionSet::iter() into the marker. Happy to drive that on the #148 side.

Only the #158 commit hang is a hard blocker for me; the rest is should-fix/nice-to-have plus the sequencing call.

@dzerik

dzerik commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Pushed the repros so you can run them directly: dzerik/sandlock branch review/pr-162-cow-repros, based on this PR's head (c712baf), test file only.

cargo test -p sandlock-core --test integration hunt_ -- --test-threads=1

Three fail here — hunt_158_fifo_commit_hang, hunt_159_opendir_whiteouted_dir_leaks_lower, hunt_160_rename_onto_existing_dir_merges — and the rest pass as the reporter-side confirmations that #161, #159's core and #160's reported case are closed.

Two notes on how they are written: the FIFO one drives run() + drop on its own OS thread and detects the hang by joining with a deadline, because commit() runs in impl Drop and a tokio timeout cannot interrupt a blocking drop; and the O_DIRECTORY one goes through the static rootfs helper, whose ls calls opendir() with no pre-stat, so it hits that open path exactly rather than being answered by the (correct) stat path. All children use only sh, coreutils and the helper — python3 does not start in every cage environment, which would otherwise mask a result.

@congwang-mk
congwang-mk force-pushed the cow-deletion-subtree branch from c712baf to 5f61e77 Compare July 26, 2026 18:35
The branch tracked deletions as a flat in-RAM set of exact paths, which
is the root of the #159/#160/#161 family: deleting a directory hid only
the directory node itself, children stayed visible through the lower
layer, and nothing about a deletion survived the process. Introduce a
DeletionSet whose covers() hides a path and everything beneath it,
matching per path component the way removing a directory hides its
contents, and mirror every insertion to an append-only escaped log
beside the upper so the deletion half of a change set is as
self-describing on disk as the upper directory already is. Entries are
never removed: a re-created path is exposed by upper presence shadowing
the whiteout, which keeps the set append-only and the log a pure
record. A log write failure degrades durability, not correctness, so it
is deliberately not fatal. This commit only adds the type; the branch
adopts it next.

Signed-off-by: Cong Wang <cwang@multikernel.io>
…visibility and resurrection (#159)

The flat deleted set answered only exact-path questions, so after the
child removed a lower directory its children still resolved through the
lower layer: reads and stats leaked pre-delete content, listings showed
ghost entries, and an O_CREAT over a deleted file copied the old bytes
back up as if the delete never happened (#159). Adopt DeletionSet
everywhere the branch consults deletions: is_deleted now means covered
and not shadowed, prepare_copy treats a covered path as having nothing
to copy so creates over a whiteout start fresh, list_merged_dir filters
covered children, changes() reports a re-created entry as a single Add
rather than Delete plus Modify, and commit applies deletions before
overlaying the upper, which is exactly the opaque-directory merge
order. Re-creating an entry in the upper shadows the whiteout instead
of erasing it, so a deleted-then-recreated directory stays opaque and
the set stays append-only.

Signed-off-by: Cong Wang <cwang@multikernel.io>
rmdir went straight to the path-based whiteout, so a directory that
still had children visible to the child was deleted while reporting
success, and the whole subtree the child never emptied vanished at
commit (#161). A real rmdir refuses with ENOTEMPTY, and the branch has
to give the same answer from the merged view because neither layer
alone can: entries may live only in the upper, only in the lower, or be
hidden by whiteouts. Answer emptiness from list_merged_dir and refuse
with ENOTEMPTY while any child is still visible, so rmdir only ever
deletes what the child actually drained.

Signed-off-by: Cong Wang <cwang@multikernel.io>
handle_rename staged only the single source entry, so renaming a lower
directory moved an empty upper husk: the merged view showed the new
name without its children, and commit published the rename as data
loss (#160). Stage the source tree into the upper first, entry by
entry through the same confined, quota-accounted single-entry copy-up
so symlinks are copied verbatim and never followed out of the tree
(issue #112), and skip children already hidden by whiteouts so deleted
entries do not reappear under the new name. Only then rename within the
upper and whiteout the surviving lower source.

Signed-off-by: Cong Wang <cwang@multikernel.io>
…158)

prepare_copy assumed every lower entry holds copyable bytes and opened
it to copy, but opening a FIFO blocks until the other end appears, so a
write to a lower FIFO under a COW workdir hung the supervisor at
copy-up time (#158). Classify the lower entry from the confined lstat
and dispatch on the type: symlinks are recreated verbatim, directories
are created empty, and non-regular entries (FIFOs, sockets, device
nodes) take a Passthrough plan that defers to the kernel instead of
reading content that is not filesystem data. execute_copy re-checks the
source type after opening so a swap between classification and copy
cannot make it read a special file either.

Signed-off-by: Cong Wang <cwang@multikernel.io>
…sions

The deletion model has two halves that must keep agreeing: what the
child sees mid-run (the merged view) and what lands on disk (commit
applying deletions before overlaying the upper). Every bug in the
#159/#160/#161 family was a disagreement between them, and each half
can regress independently of the other. Pin the agreement with branch
unit tests and with end-to-end regressions that drive rm, rmdir and mv
through a real sandboxed child, so a future change to either half
fails loudly instead of silently drifting apart.

Signed-off-by: Cong Wang <cwang@multikernel.io>
…d of passing through (#158)

The Passthrough plan sent writes to non-regular lower entries back to
the kernel, which re-runs the original syscall against the real file:
the child then needed real write permission on the lower entry and its
write really reached it. Both break the COW promise that writes under
the workdir neither require permission on nor cause side effects
through the real filesystem; learn mode COWs whole trees, so even
`> /dev/null` landed here needing a Landlock grant, and an O_WRONLY
open of a reader-less FIFO merely relocated the #158 hang from the
supervisor into the child. Drop Passthrough and virtualize the write
like any other COW write, onto an empty regular stub created in the
upper without ever opening the source.

Signed-off-by: Cong Wang <cwang@multikernel.io>
handle_rename staged the source into the upper and renamed it there
without ever looking at the destination. A directory renamed onto an
existing lower-only directory therefore committed the union of the two
trees while reporting success, a state no real filesystem transition
produces. The destination check has to come from the merged view, since
the kernel only ever sees one layer.

Classify both endpoints in the merged view before staging: a non-empty
directory destination refuses with ENOTEMPTY, type mismatches refuse
with EISDIR/ENOTDIR, and a destination that survives in the lower layer
is whiteouted after the upper rename so commit replaces it instead of
merging into it. handle_rename now answers the child directly with an
errno, the handle_unlink convention, which also stops a whiteouted
source from falling through to a real host rename in chroot mode.

Signed-off-by: Cong Wang <cwang@multikernel.io>
Both open handlers split on O_DIRECTORY before consulting the deletion
set, so opendir on a whiteouted directory skipped to the kernel and
succeeded on the surviving lower directory. The child then held a live
fd whose fstat leaked the deleted directory's inode metadata while
stat on the same path answered ENOENT, breaking the present-as-absent
invariant the merged view pins everywhere else. Check is_deleted before
the O_DIRECTORY early-return in handle_open and prepare_open so the
open answers ENOENT like every other path to a deleted entry.

Signed-off-by: Cong Wang <cwang@multikernel.io>
copy_up_tree swallowed stat and read_dir failures, so a mid-tree EACCES
or EIO truncated the staged copy silently; handle_rename then whiteouted
the source anyway and the untraversed children were gone at commit while
the child saw success. It also recursed over child-controlled directory
depth, letting a pre-seeded deep lower tree grow the supervisor's stack
without bound, and a mid-tree quota failure left the reservation charged
and an orphaned partial subtree in the upper.

Propagate every staging error to the child as its errno, walk the tree
with an explicit worklist instead of recursion, and on failure remove
the upper entries the staging created and re-derive the quota, so a
failed rename leaves the branch exactly as it was.

Signed-off-by: Cong Wang <cwang@multikernel.io>
…ntry

The log escaped backslash and newline but not carriage return, while
replay reads lines through BufRead::lines, which strips a trailing \r.
A path ending in \r therefore round-tripped to its \r-less sibling:
covers() would hide the wrong entry and the actually-deleted path would
resurrect. Escape \r the same way as \n.

Durability had a matching gap at the directory level: insert fdatasyncs
the log file, but nothing ever synced the branch directory, so a power
loss after the first deletion could drop the log's directory entry
entirely, the exact crash class the log exists for. Sync the parent
directory once when the log is created or reopened.

Signed-off-by: Cong Wang <cwang@multikernel.io>
handle_cow_stat is registered for the legacy x86_64 variants but read
every notification with the at-variant layout, dirfd in args[0] and
path in args[1]. Legacy stat carries the path in args[0], so the
handler read the statbuf pointer as the path, never matched the
workdir, and fell through to the kernel: a static-libc child emitting
legacy stat saw whiteouted lower entries that newfstatat already
reported as ENOENT, and legacy lstat picked its follow behavior from a
junk flags register. Same register-layout bug handle_cow_open already
fixed for legacy open.

Select path, statbuf and follow semantics by syscall number, and route
legacy access to the faccessat existence check instead of the stat
writeback, which would have written a stat struct through access's
mode argument.

Signed-off-by: Cong Wang <cwang@multikernel.io>
@congwang-mk
congwang-mk force-pushed the cow-deletion-subtree branch from 5f61e77 to 88b85d5 Compare July 26, 2026 18:44
@congwang-mk congwang-mk changed the title cow: subtree-aware whiteouts with a durable deletion log; type-aware copy-up cow: subtree-aware whiteouts with a durable deletion log and type-aware copy-up Jul 26, 2026
Mutation-testing the rename destination logic showed the whiteout of a
surviving lower destination was unprotected: with mark_deleted(new_rel)
deleted, every rename test still passed, because an empty-directory or
regular-file destination commits identical bytes whether the rename
replaces the destination or merges into it. The line only becomes
observable with a special-file destination, and the failure it prevents
there is severe: a FIFO destination hangs commit's publish open, the
same unrecoverable class as the #158 commit hang.

Renaming onto a lower symlink exercises the same code path but fails
cleanly instead of hanging: without the whiteout the symlink survives
into commit, whose O_NOFOLLOW publish refuses to write through it.
Verified the test fails with the whiteout removed and passes with it.

Signed-off-by: Cong Wang <cwang@multikernel.io>
@congwang-mk

Copy link
Copy Markdown
Contributor Author

Thanks for the reproducer and review, @dzerik . I have addressed them. Please take another look.

@dzerik

dzerik commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Thanks — I re-ran the repros against ae7df2b.

Closed on my side: the O_DIRECTORY gap, the rename-onto-existing-destination merge, the copy_up_tree failure paths, and the deleted.log \r / directory-fsync issues. hunt_159_opendir_whiteouted_dir_leaks_lower and hunt_160_rename_onto_existing_dir_merges now pass, and the earlier confirmations (#161, #159's core, #160's reported case) still do.

The #158 commit-time hang still reproduces, though: hunt_158_fifo_commit_hang blocks for >25s and the child's bytes are never published.

mark_deleted is still called only from handle_unlink (cow/seccomp.rs:713) and handle_rename (:801, :804), never from the non-regular stub path (~:308-320). So the lower FIFO survives unwhiteouted, commit()'s deletion pass does not remove it, and the upper walk then O_WRONLY-opens that surviving FIFO as its destination and blocks on a reader that never comes — in impl Drop, with the error swallowed.

Copy-up time is genuinely fixed; it is only the publish path that is left. Whiteouting the lower path when the stub is created (so commit unlinks-then-creates), unlinking a type-mismatched destination before opening it, or opening the destination O_NONBLOCK and handling ENXIO would each close it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment