vfs: add --vfs and --vfs-manifest startup flags - #3
Closed
pipobscure wants to merge 2 commits into
Closed
Conversation
pipobscure
force-pushed
the
ziparchives
branch
2 times, most recently
from
July 7, 2026 19:06
8eee064 to
0f2b879
Compare
pipobscure
force-pushed
the
ziparchives
branch
2 times, most recently
from
July 8, 2026 11:29
f157686 to
233f865
Compare
Add a node:vfs provider backed by a node:zlib ZIP archive - a ZipBuffer held in memory or a ZipFile on disk - that exposes the archive's members as a virtual filesystem tree. The provider is read-only unless the backing archive is writable, and offers both asynchronous and synchronous operations. Available as vfs.ZipProvider. Signed-off-by: Philipp Dunkel <pip@pipobscure.com>
Adds --vfs=<target>, which mounts a directory or ZIP archive and resolves the entry point (process.argv[1]) and all subsequent require()/import resolution against it instead of the real filesystem. A directory target is mounted with RealFSProvider at its own real path, gaining path containment it wouldn't otherwise have; a file target is opened as a read-only ZIP archive (zlib.ZipFile) and mounted with ZipProvider, turning that path into a virtual directory. Only paths under the mount are affected - the running program's own node:fs calls to other real paths are untouched, and module resolution never falls back to the real filesystem once it would step outside the mounted target. With --vfs active, process.argv[1] is unconditionally the mount root, exactly as if `node <mountRoot>` had been run - so the mount's own package.json "main"/index.js decides what runs, and any positional argument is the program's own (shifted to argv[2] onward), never an entry-point override. This makes a self-mounting shebang line (`#!/usr/bin/env node --vfs`) work: the kernel appends the script's own path as the argument that becomes --vfs's value, so the user's first real argument would otherwise land in argv[1] and be misread as an entry path. An active --vfs mount is also treated as an entry point by the C++ startup dispatch, so `node --vfs=<target>` with no positional argument runs the mount rather than falling through to the REPL or stdin. Workers are unaffected: they name their entry through `new Worker(filename)`, resolved against the inherited mount, not argv[1]. Getting there requires four of the CJS/ESM loader's module-resolution primitives - package.json reading, nearest-parent/scope lookup, legacy main resolution, and extensionless-file format sniffing - to stop bypassing the public fs module and calling straight into native bindings, since that bypass is exactly what let them ignore the mount. Each gets a VFS-aware replacement that defers to the real native binding unchanged for anything outside an active mount, so behavior for non-mounted paths is identical to before. Native addons are supported: a directory-backed mount dlopens the real underlying file directly; an archive-backed mount extracts the addon to a content-hashed temp file first, since there's no real file to point at. Worker threads inherit an active mount automatically in the common case, and explicitly when the caller supplies its own execArgv that omits it, so sandboxed code can't spawn an "escaped" worker. Also adds --vfs-manifest=<file>, used with a directory --vfs target: the path of every file actually read through the mount - by module resolution or by the program's own node:fs calls - is appended to <file> as it's read, via a small observer hook on the provider base class rather than patching any method. Workers append to the same file directly, since they share the real filesystem with the main thread. Native addons extracted from an archive-backed VFS mount were written under a single shared node-vfs-addons temp directory, keyed only by content hash. Sharing that directory across processes lets one process delete or overwrite a file another process still has dlopen'd/mapped. Scope the directory per pid (node-vfs-addons-<pid>) so each process owns its own extraction cache. Signed-off-by: Philipp Dunkel <pip@pipobscure.com>
pipobscure
added a commit
to pipobscure/bundles
that referenced
this pull request
Sep 2, 2026
The zip support in node:zlib is released (v26.8.0) and ZipProvider merged today as nodejs/node#64915, so the old framing — "three things that are not in any release", pointing at #64339 and pipobscure/node#3 — was wrong in both directions: it undersold what has landed and misnamed what has not. What is actually outstanding is nodejs/node#65748, the --vfs-mount / --vfs-load flags and the vfs.registerProvider() that ships with them, plus nodejs/node#65680 for loading native addons out of a mount. Reading those two turned up semantics this repo's prose had wrong: --vfs-load runs the *first* mount (or --vfs-load=<index>), not the last; --vfs-mount takes no target, since node assigns a reserved mount point; argv[1] is the mounted source's real path rather than the mount point; and registered providers are now offered directories as well as files, which retires the constraint that made tools/observe.ts a runner rather than a preload. The deck's status slide said "This is in Node" with the flags marked as landed. It now says "landing", carries the two open PRs and the addon work as their own rows, and the speaker notes say plainly that the keystone is still a pull request. Deck republished to the artifact link in the README.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds --vfs=, which mounts a directory or ZIP archive and resolves the entry point (process.argv[1]) and all subsequent require()/import resolution against it instead of the real filesystem. A directory target is mounted with RealFSProvider at its own real path, gaining path containment it wouldn't otherwise have; a file target is opened as a read-only ZIP archive (zlib.ZipFile) and mounted with ArchiveProvider, turning that path into a virtual directory. Only paths under the mount are affected - the running program's own node:fs calls to other real paths are untouched, and module resolution never falls back to the real filesystem once it would step outside the mounted target.
Getting there requires four of the CJS/ESM loader's module-resolution primitives - package.json reading, nearest-parent/scope lookup, legacy main resolution, and extensionless-file format sniffing - to stop bypassing the public fs module and calling straight into native bindings, since that bypass is exactly what let them ignore the mount. Each gets a VFS-aware replacement that defers to the real native binding unchanged for anything outside an active mount, so behavior for non-mounted paths is identical to before.
Native addons are supported: a directory-backed mount dlopens the real underlying file directly; an archive-backed mount extracts the addon to a content-hashed temp file first, since there's no real file to point at. Worker threads inherit an active mount automatically in the common case, and explicitly when the caller supplies its own execArgv that omits it, so sandboxed code can't spawn an "escaped" worker.
Also adds --vfs-manifest=, used with a directory --vfs target: the path of every file actually read through the mount - by module resolution or by the program's own node:fs calls - is appended to as it's read, via a small observer hook on the provider base class rather than patching any method. Workers append to the same file directly, since they share the real filesystem with the main thread.