vfs: integrate with CJS and ESM module loaders by mcollina · Pull Request #63653 · nodejs/node

mcollina · 2026-05-30T05:41:48Z

Makes `require()` and `import` resolve files served by `node:vfs`. Before this PR, mounted VFS files were only visible through `fs.*`; the loaders went straight to the real filesystem.

Mechanism: toggleable wrappers in the loaders. Null fast-path when no VFS is mounted; otherwise the VFS answers stat / read / realpath / package.json.

User-visible side effects:

Each `VirtualFileSystem` instance has a stable `layerId` (per-process monotonic counter).
ESM URLs resolved through a VFS gain `?vfs-layer=N` — visible in `import.meta.url`, matches the HMR cache-busting pattern, lets unmount scope-purge the cascaded loader's cache instead of flushing it globally.
CJS `__filename` is unchanged.

Review guide

Out of scope

SEA + VFS, migrating the C++ `package_configs_` cache, broader permission-model integration.

nodejs-github-bot · 2026-05-30T05:41:54Z

Review requested:

@nodejs/loaders

codecov · 2026-05-30T06:50:02Z

Codecov Report

❌ Patch coverage is 90.71207% with 60 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.32%. Comparing base (39b481b) to head (51b033a).
⚠️ Report is 26 commits behind head on main.

Files with missing lines	Patch %	Lines
lib/internal/vfs/setup.js	85.54%	59 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #63653      +/-   ##
==========================================
- Coverage   91.96%   90.32%   -1.64%     
==========================================
  Files         379      732     +353     
  Lines      166638   237271   +70633     
  Branches    25497    44727   +19230     
==========================================
+ Hits       153242   214325   +61083     
- Misses      13099    14656    +1557     
- Partials      297     8290    +7993

Files with missing lines	Coverage Δ
lib/internal/modules/cjs/loader.js	`98.15% <100.00%> (+18.48%)`	⬆️
lib/internal/modules/esm/get_format.js	`94.83% <100.00%> (+21.03%)`	⬆️
lib/internal/modules/esm/load.js	`91.47% <100.00%> (+8.22%)`	⬆️
lib/internal/modules/esm/resolve.js	`99.04% <100.00%> (+11.14%)`	⬆️
lib/internal/modules/helpers.js	`98.75% <100.00%> (+7.70%)`	⬆️
lib/internal/modules/package_json_reader.js	`99.47% <100.00%> (+11.86%)`	⬆️
lib/internal/vfs/setup.js	`86.14% <85.54%> (-0.55%)`	⬇️

... and 472 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

mcollina · 2026-05-30T10:27:16Z

@joyeecheung take a look, should be easier to review.

@jsumners-nr

Restore the "DO NOT depend on the patchability" warnings in esm/load.js and esm/resolve.js that were dropped along with the fs imports. The warning still applies; it now also points at node:vfs as one of the formal hook mechanisms callers should reach for instead. Addresses review feedback from @jsumners-nr in nodejs#63653

nodejs-github-bot · 2026-06-03T08:04:09Z

CI: https://ci.nodejs.org/job/node-test-pull-request/73901/

nodejs-github-bot · 2026-06-03T11:52:41Z

CI: https://ci.nodejs.org/job/node-test-pull-request/73902/

joyeecheung

A design question recently occurred to me: have we explored the versioning of the mounting?

mcollina · 2026-06-03T12:40:18Z

A design question recently occurred to me: have we explored the versioning of the mounting?

what do you mean? You mean multiple vfs layers on top of each other?

joyeecheung · 2026-06-03T13:02:48Z

what do you mean? You mean multiple vfs layers on top of each other?

For the stacks to have some kind of version number/ID to identify the current status?

BTW I just noticed that there's no mention of unmount() and mount() in the VFS docs..

mcollina · 2026-06-03T20:10:47Z

BTW I just noticed that there's no mention of unmount() and mount() in the VFS docs..

I did purge them when doing the splitting; I forgot to bring them back. I'll add them to this PR.

mcollina · 2026-06-03T20:11:45Z

For the stacks to have some kind of version number/ID to identify the current status?

No but we totally should.

Each VirtualFileSystem now exposes a per-process monotonically increasing `layerId`, assigned at construction. The id is stable across mount/unmount cycles for the lifetime of the instance and surfaces in: - debug() output for register / deregister so the layer stack is visible when NODE_DEBUG=vfs is enabled; - the overlap ERR_INVALID_STATE message, which now names the layer ids of the conflicting mounts. The id is the building block for tagging cache entries with the owning VFS, which a follow-up will use to replace the global loader-cache flush in deregisterVFS with a scoped purge. Refs: nodejs#63653 Signed-off-by: Matteo Collina <hello@matteocollina.com>

Replace the global loader-cache flush in deregisterVFS with a scope-purge that only drops entries owned by the unmounting VFS. Per-layer ownership is determined two ways: - For CJS-style filename-keyed caches (Module._cache, Module._pathCache, the CJS stat cache, the helpers.js realpath cache, and the package.json caches) entries are filtered with `vfs.shouldHandle(filename)`. __filename stays a clean absolute path so user code that does `path.dirname(__filename)` or similar is unaffected. - For the ESM cascaded loader's loadCache, entries are tagged at resolve time: when finalizeResolution() detects the resolved path is VFS-owned (via the new loaderGetLayerForPath hook), it appends `?vfs-layer=<id>` to the URL. The tag surfaces in `import.meta.url`, matching the cache-busting pattern used by HMR tooling. On deregister, entries whose URL carries the tag for the unmounting layer are deleted. Multi-mount setups no longer pay the cross-VFS cache-warmup penalty when a single VFS unmounts, and ESM modules loaded from a VFS become reachable for purge instead of leaking forever in the cascaded loader. New helpers exposed for VFS: - cjs/loader.js: clearStatCacheForVFS - helpers.js: purgeRealpathCacheForVFS, loaderGetLayerForPath - package_json_reader.js: purgePackageJSONCacheForVFS Adds test-vfs-scoped-cache-purge covering both the multi-mount isolation and the import.meta.url tag visibility. Refs: nodejs#63653 Signed-off-by: Matteo Collina <hello@matteocollina.com>

mcollina · 2026-06-05T06:47:09Z

@joyeecheung PTAL

nodejs-github-bot · 2026-06-16T10:10:10Z

CI: https://ci.nodejs.org/job/node-test-pull-request/74176/

ShogunPanda

LGTM!

mcollina · 2026-06-19T16:08:18Z

@joyeecheung PTAL

nodejs-github-bot · 2026-06-19T16:20:03Z

CI: https://ci.nodejs.org/job/node-test-pull-request/74271/

nodejs-github-bot · 2026-06-24T00:00:25Z

CI: https://ci.nodejs.org/job/node-test-pull-request/74396/

nodejs-github-bot · 2026-06-26T13:19:52Z

CI: https://ci.nodejs.org/job/node-test-pull-request/74420/

mcollina · 2026-06-26T17:54:24Z

@jasnell @joyeecheung updated with all your feedbacks.

Co-authored-by: James M Snell <jasnell@gmail.com> Signed-off-by: Matteo Collina <hello@matteocollina.com>

Each VirtualFileSystem now tracks the absolute filenames it has handled (ownedFilenames) and the Module._pathCache keys that resolve to a VFS-owned filename (ownedPathCacheKeys). Recording happens at routing time inside the loader overrides (findVFSForStat / findVFSForRead / findVFSWith, the inline package.json override loops, and a pathCache write recorder installed on the cjs loader). On deregister, purgeLoaderCachesForVFS walks the per-VFS sets and removes entries from Module._cache, Module._pathCache, the stat / realpath / package.json caches in O(owned) instead of scanning every cache and calling vfs.shouldHandle() on each entry. The pathCache recorder uses a Set.has() lookup over activeVFSList so the per-write overhead is M cheap Set checks rather than M path normalizations. Also encapsulates the previous direct access to Module._cache / Module._pathCache from internal/vfs/setup.js behind the cjs loader's new purgeModuleCacheForVFS / setPathCacheWriteRecorder / cachePathResolution helpers, so the loader's private data structures stay owned by the loader module. Signed-off-by: Matteo Collina <hello@matteocollina.com>

jasnell · 2026-06-27T13:16:12Z

+function clearStatCache() {
+  if (statCache !== null) {
+    statCache = new SafeMap();
+  }


Nit: could this skip the conditional check and just always set to a new Map?

nodejs-github-bot added lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. labels May 30, 2026

mcollina requested review from ShogunPanda, jasnell and joyeecheung May 30, 2026 10:26

ShogunPanda approved these changes May 30, 2026

View reviewed changes

jsumners-nr reviewed Jun 1, 2026

View reviewed changes

Comment thread lib/internal/modules/esm/load.js

Comment thread lib/internal/modules/esm/resolve.js

mcollina force-pushed the vfs-module-loader-integration branch from 6321e08 to 51b033a Compare June 1, 2026 15:54

ShogunPanda approved these changes Jun 2, 2026

View reviewed changes

mcollina added the request-ci Add this label to start a Jenkins CI on a PR. label Jun 3, 2026

github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jun 3, 2026

joyeecheung reviewed Jun 3, 2026

View reviewed changes

Comment thread lib/internal/vfs/setup.js Outdated

mcollina force-pushed the vfs-module-loader-integration branch from 51b033a to 294a19c Compare June 4, 2026 07:23

coderabbitai Bot mentioned this pull request Jun 12, 2026

build: migrate from Workers to Docker jsdelivr/globalping-dash#204

Merged

arcanis mentioned this pull request Jun 15, 2026

module: add --experimental-strip-private-modules #63869

Open

ShogunPanda approved these changes Jun 15, 2026

View reviewed changes

mcollina force-pushed the vfs-module-loader-integration branch from 75bb5c7 to 99a5a5c Compare June 15, 2026 09:00

github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jun 16, 2026

mcollina force-pushed the vfs-module-loader-integration branch 2 times, most recently from 4dc22f7 to 4bb502d Compare June 17, 2026 09:31

ShogunPanda approved these changes Jun 19, 2026

View reviewed changes

mcollina force-pushed the vfs-module-loader-integration branch from 4bb502d to 10412dc Compare June 19, 2026 16:08

ShogunPanda approved these changes Jun 19, 2026

View reviewed changes

mcollina added the request-ci Add this label to start a Jenkins CI on a PR. label Jun 19, 2026

github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jun 19, 2026

mcollina force-pushed the vfs-module-loader-integration branch 2 times, most recently from 572c4bc to efc7db4 Compare June 21, 2026 06:34

ShogunPanda approved these changes Jun 21, 2026

View reviewed changes

mcollina added the request-ci Add this label to start a Jenkins CI on a PR. label Jun 21, 2026

github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jun 21, 2026

This comment was marked as outdated.

Sign in to view

trivikr added the vfs Issues and PRs related to the virtual filesystem subsystem. label Jun 24, 2026

ShogunPanda approved these changes Jun 26, 2026

View reviewed changes

mcollina added the request-ci Add this label to start a Jenkins CI on a PR. label Jun 26, 2026

github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jun 26, 2026

mcollina and others added 3 commits June 27, 2026 12:57

vfs: integrate with CJS and ESM module loaders

82e8cb5

Co-authored-by: James M Snell <jasnell@gmail.com> Signed-off-by: Matteo Collina <hello@matteocollina.com>

vfs: fix module loader paths on Windows

8b1b4ea

mcollina force-pushed the vfs-module-loader-integration branch from 61ada45 to c65d084 Compare June 27, 2026 10:57

jasnell approved these changes Jun 27, 2026

View reviewed changes

#	File	Role
1	`lib/internal/modules/helpers.js`	The hook surface: 9 `loader*` wrappers + 2 setters.
2	`lib/internal/modules/cjs/loader.js`	`stat()` + TS read routed through wrappers.
3	`lib/internal/modules/esm/resolve.js`	Routed; appends `?vfs-layer=N` after `finalizeResolution`.
4	`lib/internal/modules/esm/load.js`	`getSourceSync` reads via the wrapper.
5	`lib/internal/modules/esm/get_format.js`	Extensionless format detection routed.
6	`lib/internal/modules/package_json_reader.js`	4 C++ binding calls routed; adds scoped purge.
7	`lib/internal/vfs/file_system.js`	Exposes `vfs.layerId`.
8	`lib/internal/vfs/setup.js`	Heart of the PR. Installs overrides, reimplements `legacyMainResolve` + package.json ops with parity to `src/node_modules.cc` / `src/node_file.cc`, scope-purges caches on unmount.
9	`doc/api/vfs.md`	Mounting section + Module loader integration section.

Uh oh!

Uh oh!

Conversation

mcollina commented May 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review guide

Out of scope

Uh oh!

nodejs-github-bot commented May 30, 2026

Uh oh!

codecov Bot commented May 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

mcollina commented May 30, 2026

Uh oh!

Uh oh!

Uh oh!

nodejs-github-bot commented Jun 3, 2026

Uh oh!

nodejs-github-bot commented Jun 3, 2026

Uh oh!

joyeecheung left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mcollina commented Jun 3, 2026

Uh oh!

joyeecheung commented Jun 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mcollina commented Jun 3, 2026

Uh oh!

mcollina commented Jun 3, 2026

Uh oh!

mcollina commented Jun 5, 2026

Uh oh!

nodejs-github-bot commented Jun 16, 2026

Uh oh!

ShogunPanda left a comment

Choose a reason for hiding this comment

Uh oh!

mcollina commented Jun 19, 2026

Uh oh!

nodejs-github-bot commented Jun 19, 2026

Uh oh!

This comment was marked as outdated.

This comment was marked as outdated.

nodejs-github-bot commented Jun 24, 2026

Uh oh!

nodejs-github-bot commented Jun 26, 2026

Uh oh!

mcollina commented Jun 26, 2026

Uh oh!

jasnell Jun 27, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

mcollina commented May 30, 2026 •

edited

Loading

codecov Bot commented May 30, 2026 •

edited

Loading

joyeecheung commented Jun 3, 2026 •

edited

Loading