Skip to content

[R4] Slim the npm tarball: drop test files and source map weight, fix pack-smoke #191

Description

@LinuxDevil

Goal

The published SDK is 16.5 MB unpacked and 4.4 MB packed (1.0.0-alpha.8, 720 files). Most of that is source maps that embed the source a third time, plus one 2.25 MB map of the Agent Forge server, plus 15 test-only files. This ticket cuts the tarball to about 10.5 MB unpacked and 3 MB packed without breaking any feature, and makes pack-smoke measure the tarball that is really published (today it does not build Agent Forge, so it never saw the real size and its limits are stale).

Current state

Measured on the published tarball (npm pack @lousho/build-ai-agent@1.0.0-alpha.8, unpacked here):

Part Unpacked Packed (gzip) Notes
dist/**/*.map (138 files) 5.53 MB 1.37 MB Each map has sourcesContent, so src/ is embedded again (3.70 MB of the 5.53). Without sourcesContent the maps are 1.83 MB.
dist (code and .d.ts, .d.mts) 3.06 MB 0.74 MB
apps/agent-forge/dist-server/index.cjs.map 2.27 MB 0.50 MB The map of the bundled Forge server (express, cors, ws inside). index.cjs itself is 1.38 MB.
apps/agent-forge/dist-server/index.cjs + dist/ (client) 2.58 MB 0.73 MB Needed by lousho studio.
src/ 1.91 MB 0.54 MB See below: needed at runtime.
docs, llms*.txt, other ~1.1 MB 0.34 MB
  • package.json files (lines ~70-90): dist, src, !src/**/*.test.ts, !src/**/*.eval.ts, !src/**/__fixtures__, bin, apps/agent-forge/package.json, apps/agent-forge/dist, apps/agent-forge/dist-server, README.md, llms.txt, llms-full.txt, docs/*.md, !docs/eslint-baseline-followup.md, LICENSE, CHANGELOG.md, CONTRIBUTING.md.
  • The 15 test-only files that still ship (!src/**/*.test.ts does not match them): 11 *.test-d.ts files (src/dynamicConfig.test-d.ts, src/execution/agentEvents.test-d.ts, hooks.test-d.ts, structuredOutput.test-d.ts, src/flows/flowTypes.test-d.ts, src/react/react.test-d.ts, src/session/outputTyping.test-d.ts, src/storage/StorageService.test-d.ts, src/svelte/svelte.test-d.ts, src/tools/defineTool.test-d.ts, src/vue/vue.test-d.ts), 3 *.testkit.ts files (src/deploy/buildLock.testkit.ts, src/providers/aiMajor.testkit.ts, src/providers/aiShapes.testkit.ts) and src/cli/init/__snapshots__/templates.test.ts.snap. Not to be touched: src/testing/* is the public ./testing subpath (mockModel), not test-only.
  • src/ cannot be dropped. src/deploy/bundle.ts:53,67,73,86,111 resolve src/deploy/runtime.ts, runtime.worker.ts, src/index.ts and the shims under src/deploy/shims/ from the installed package root (findSdkRoot()) and bundle them with esbuild for lousho build --target cloudflare-worker and the node-server target. Without src/ those builds fail.
  • Source maps: tsup.config.ts sets sourcemap: true; apps/agent-forge/server/tsup.config.ts sets sourcemap: true. dist/**/*.map sources point at ../src/..., which exists in the package, so maps still resolve to real files without sourcesContent.
  • scripts/pack-smoke.ts: thresholds MAX_ENTRIES = 900, MAX_UNPACKED_BYTES = 14 MB, MAX_PACKED_BYTES = 4 MB (comment: "alpha.8 tarball is ~705 entries / ~11.4 MB"). Its main() builds only the SDK and create-lousho-agent (npm run build), not npm run build:studio, but prepublishOnly runs both (package.json: "prepublishOnly": "npm run build && npm run build:studio"). So the script packs a tarball without the Forge build (about 11.7 MB) and the real publish is 16.5 MB. ALLOWED_TEST_LIKE = [/\.test-d\.ts$/] allows the type tests on purpose; FORBIDDEN_PATHS does not list .testkit.ts or snapshots.
  • The pack-smoke CI job (.github/workflows/ci.yml, last job) runs npm ci then npm run pack-smoke.

Scope

Owner decision first. Do not start until the issue records the owner's choice. The question:

Publishing configuration changes (the files list and the source map settings). Which tarball do you want for the next release?
A (recommended): exclude the 15 test-only files; no source map for the Agent Forge server; SDK source maps without embedded source (sourcesContent off), src/ stays. About 10.5 MB unpacked, 3 MB packed. Stack traces keep working with --enable-source-maps because src/ ships.
B: A, and also ship no SDK source maps at all (!dist/**/*.map). About 8.7 MB unpacked, 2.5 MB packed; no source-mapped stack traces for users.
C: change nothing in the package; only fix pack-smoke and raise its limits to 18 MB unpacked and 5 MB packed.

If the owner picks A, implement as below. For B add "!dist/**/*.map" to files and skip the tsup change. For C implement only item 3.

In (option A):

  1. package.json files: add "!src/**/*.test-d.ts", "!src/**/*.testkit.ts", "!src/**/__snapshots__", "!apps/agent-forge/dist-server/*.map". Keep src.
  2. tsup.config.ts: add esbuildOptions(options) { options.sourcesContent = false; } so the SDK maps do not embed the source. Check dist/index.js.map has no sourcesContent and sources entries still resolve from dist/ to ../src/... files that exist.
  3. scripts/pack-smoke.ts:
    • main() also runs npm run build:studio (unless --skip-build), so the tarball is the published one. In CI the pack-smoke job needs no new step (the script builds).
    • Set the thresholds from a real run after the change, with about 20% headroom: MAX_UNPACKED_BYTES, MAX_PACKED_BYTES, MAX_ENTRIES; update the comment with the measured numbers.
    • Remove ALLOWED_TEST_LIKE and add /\.test-d\.ts$/, /\.testkit\.ts$/ and /__snapshots__\// to FORBIDDEN_PATHS.
    • Add a check that src/deploy/runtime.worker.ts, src/deploy/shims/node.worker.ts and src/index.ts are in the tarball (the Worker build needs them), and one that apps/agent-forge/dist-server/index.cjs and apps/agent-forge/dist/index.html are in it. A missing file fails the script.
    • Add a check that no apps/agent-forge/dist-server/*.map is in the tarball.
  4. A functional proof that nothing needed was dropped, in the pull request: after npm pack, install the tarball in a temp project and run npx lousho build --target cloudflare-worker on a scaffolded spec agent (the lousho init --template yaml output; see docs/deployment.md) and npx lousho studio (needs R1a if it is not merged yet; if not, only the build). Paste the output.
  5. CHANGELOG under ## [Unreleased], "Changed": the package is smaller (give the new sizes); source maps no longer embed source; Agent Forge's server map is not published.

Out:

  • Dropping src/ or changing how lousho build finds the SDK (a larger design change; open a separate issue if wanted).
  • Deleting the .d.mts duplicates of the type files (1.1 MB unpacked); not touched.
  • The release itself: publishing and version bumps stay with the owner.

Acceptance criteria

  • The issue records the owner's answer (A, B or C) before work starts; the pull request quotes it.
  • npm pack --dry-run --json output after the change is pasted: entry count, unpacked size and packed size against the 720 files / 16.5 MB / 4.4 MB baseline.
  • None of the 15 files listed above is in the tarball; src/testing/* still is.
  • npm run build:studio then npm run pack-smoke passes with the new thresholds and the new file-presence checks; it fails if one of them is removed (show a run with a deliberately forbidden file, then revert).
  • The functional proof (item 4) is pasted.
  • npm run test:coverage, npx tsc --noEmit, npm run lint, npm run fallow pass.
  • CHANGELOG entry added; README and docs unchanged unless a doc states the package size (grep docs/ for "MB"; if one does, update it).

Live test

None: this ticket spends nothing.

Dependencies

None to start, but the owner's answer is required. R1a is helpful for the lousho studio proof. Conflicts: package.json (R2 adds an exports entry and A1 to A3 change exports; this ticket edits only files), tsup.config.ts (R2 adds an entry; this ticket adds one option), scripts/pack-smoke.ts (R1b moves helpers out of it; whoever merges second rebases).

Notes for the implementer

  • Decision: keep src/ in the package. The Worker and node-server bundlers read it (src/deploy/bundle.ts), and it is also what the shipped source maps point at.
  • The files negations use the same style as the existing !src/**/*.test.ts line. After editing, confirm with npm pack --dry-run --json rather than by reading the patterns: negated globs are order sensitive and !apps/agent-forge/dist-server/*.map must come after the positive apps/agent-forge/dist-server.
  • The 2.25 MB map is also written by apps/agent-forge/server/tsup.config.ts (sourcemap: true); leave that setting alone, so a source checkout keeps its map for debugging, and exclude the file from the package with files.
  • tsup's esbuildOptions runs per build format; the sourcesContent option is valid on esbuild 0.25 (build.sourcesContent). If tsup passes the maps through its own post-processing that re-adds content, fall back to stripping sourcesContent in the existing onSuccess hook (it already runs copyDevUi()): read each dist/**/*.map, delete the field, write it back.
  • Do not run npm publish, not even --dry-run against the real registry beyond what pack-smoke already does.

Round 2 ticket R4. Before starting, read the agent brief (worktree rules, verification list, live-test budget) and the plan. One ticket is one pull request; put Closes #<this issue> in it.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    model:sonnetWell specified; a Sonnet agent can take itowner-decisionNeeds the owner's answer before work startsround-2Round 2 plan ticketwave-0Round 2, wave 0

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions