Skip to content

[NO-JIRA] Build releases with bun 1.4.0 to stop the 1.5 GB/day /tmp libopentui.so leak - #54

Merged
johnnymo87 merged 3 commits into
mainfrom
opentui-so-leak
Sep 21, 2026
Merged

johnnymo87 merged 3 commits into
mainfrom
opentui-so-leak

Conversation

@johnnymo87

Copy link
Copy Markdown
Owner

Description

The attached TUI leaks a 13.35 MB copy of libopentui.so into /tmp on every launch and never removes it. Measured on cloudbox 2026-09-21: 1626 orphaned copies, 19.84 GB, accruing 1.1–2.1 GB/day across ~15 concurrent TUIs, on a disk at 81%. A second, smaller family of leaked files is the fff-core file-picker native library, same mechanism.

The leak is not opencode's code and not @opentui's. @opentui/core-linux-*/index.bun.js resolves the native library with import("./libopentui.so", { with: { type: "file" } }), which makes it a bun embedded file that bun must materialize on disk before the TUI can dlopen it. bun < 1.4.0 wrote a fresh .<16 hex>-<8 digit>.so per launch and never unlinked it — upstream oven-sh/bun#40076 (closed; its report names opencode's libopentui.so as the real-world case), fixed by #29587 in bun 1.4.0, which extracts once to a reused, content-hashed $TMPDIR/.bun-{uid}-{hash}.{ext}.

bun build --compile bakes the bun runtime — and therefore that extractor — into the shipped binary, so the bun version this workflow compiles with is the only lever. A host upgrading its own bun changes nothing.

This PR:

  • bun-version: 1.3.14 → 1.4.0 in build-release.yml.
  • New patches/sse-cancel-rejection.patch — a backport of upstream #44944, which lands after v1.18.18 and so is not on the line the cron hold keeps us on. Under bun 1.4, reader.cancel() on a just-aborted fetch body rejects; both SSE chunk-timeout wrappers discard it with void, making it an unhandled rejection, which bun answers by killing the process. chunkTimeout is armed on four providers in the deployed config, so one stalled stream would take a serve and every session it owns down with it. This is latent under 1.3.14 and fatal at 1.4.x — the patch and the pin ship together.
  • New named CI step running the upstream test that goes red without that patch under bun 1.4.

Why 1.4.0 and not 1.4.2, which is current: bun 1.4.1 rewrote --splitting, and script/build.ts compiles with splitting: true. A 1.4.1/1.4.2-compiled opencode binary crashes on the first agent turn (TypeError: undefined is not an object (evaluating 'node.name') in layer-node.ts) — this is why upstream's own bump PR #44946 is still open; tracked in oven-sh/bun#42837 and opencode#48397. 1.4.0 has the leak fix and predates the splitting rewrite.

Verification

All on aarch64, against artifacts built by this workflow:

What Result
bun#40076 repro, 3 launches 1.3.3 → 3 files in /tmp, one per launch. 1.4.0 and 1.4.2 → one reused $TMPDIR/.bun-1000-<hash>.so
dry_run: true run of this workflow on this branch (35632784070) green: patches apply, all named test steps pass under bun 1.4.0, 4 targets build, macOS sign + smoke
That run's linux-arm64 artifact, 3 TUI launches exactly one .bun-1000-e6e54fcdf671fcbb.so of 13,995,736 bytes (libopentui.so to the byte), mtime unchanged. The deployed 1.3.14 binary, same invocation: one fresh file per launch
Same artifact, real agent turn (isolated OPENCODE_DB/XDG, real credentials) completes. The 1.4.2-built artifact fails it with exactly the node.name crash above — the splitting bug reproduced on our own build, which is why the pin is 1.4.0
New CI step non-vacuous with the patch: 1 pass. With the patch removed on a throwaway branch (35631423987): (fail) chunkTimeout raises a response stream error when SSE body stalls, same bun 1.4.0

Deliberate deviation: upstream v1.18.18 and current dev both declare packageManager: bun@1.3.14, so this pin no longer matches upstream's. The old rule ("match upstream's packageManager") is replaced in-comment by a stronger one: a dry-run green is necessary and not sufficient — it never executes a linux binary and never takes an agent turn, which is exactly the shape of the 1.4.1 regression.

Follow-ups (not in this PR)

  • Merging changes nothing by itself. A release (revision=4) must be dispatched, and the workstation Nix derivation repointed with new asset hashes. The cron hold pins the opencode line at 1.18.18 and is unaffected — v1.18.18-patched.4 stays inside it.
  • Before dispatching that release, take one real agent turn with the darwin-arm64 artifact on the Mac. The macOS job smoke-tests --version only, and --version returns success on a binary with this crash class.
  • Nothing reclaims the ~19.6 GB already on disk; that is tracked separately as workstation-o5s1.24.
  • Commit 1 proposes 1.4.2 and commit 2 corrects it to 1.4.0. If squashing with the default "commit messages" body, use commit 2's.

References

`bun build --compile` bakes the bun runtime, and therefore its
embedded-file extractor, into the shipped binary, so the bun version this
workflow pins is what every launch of the released binary runs.

@opentui/core's platform package resolves its native library through
`import("./libopentui.so", { with: { type: "file" } })`, which makes the
13.35 MB shared object an embedded file that bun must materialize on disk
before the TUI can dlopen it. bun < 1.4.0 wrote a fresh
`$TMPDIR/.<16 hex>-<8 digit>.so` on every launch and never unlinked it,
not even on clean exit. Measured on a shared box on 2026-09-21: 1626
orphaned copies totalling 19.84 GB, accruing 1.1-2.1 GB/day across ~15
concurrent attach TUIs.

That is oven-sh/bun#40076 (closed 2026-08-22; its report names opencode's
libopentui.so as the real-world case), a duplicate of #29585, fixed by
#29587 in bun 1.4.0: an embedded file now extracts once to a
content-hashed, reused `$TMPDIR/.bun-{uid}-{hash}.{ext}`.

Nothing above bun can fix this. The extraction is not opencode's code and
not @OpenTui's; @opentui/core 0.4.5 is what upstream opencode dev still
pins as of 2026-09-21, and upstream opencode still declares
`packageManager: bun@1.3.14`, so this pin deliberately no longer matches
upstream's. The prior rule ("match upstream's packageManager") is
replaced by the same gate it always implied: a bun bump ships only after
a `dry_run: true` run of this workflow on the bumped branch is green.

Verified locally on aarch64 with the #40076 repro (a 69 KB lib.so
imported `with { type: "file" }` and dlopen'd, compiled and run 3x):
bun 1.3.3 left three files, one per launch, in /tmp and ignored $TMPDIR;
bun 1.4.2 left one reused `$TMPDIR/.bun-1000-3a39a40e08be468e.so`.

Refs: workstation-o5s1.24, workstation-o5s1.27
…kport

Two corrections to the previous commit, both from evidence that the bump
alone would have shipped broken binaries.

1. 1.4.2 -> 1.4.0. bun 1.4.1 rewrote `--splitting`, and
   packages/opencode/script/build.ts compiles with
   `minify: true, splitting: true`. A 1.4.1- or 1.4.2-compiled opencode
   binary crashes on the first agent turn with `TypeError: undefined is
   not an object (evaluating 'node.name')` in
   packages/core/src/effect/layer-node.ts -- reported against stock
   upstream `dev` on darwin-arm64 in anomalyco/opencode#44946 (the
   upstream bun-bump PR, still open for this reason) and tracked as
   oven-sh/bun#42837, where a 1.4.0-compiled binary passes and a 1.4.1
   one fails. 1.4.0 carries the embedded-file fix (#29587) and predates
   the splitting rewrite. Verified locally that a 1.4.0-compiled binary
   reuses one `$TMPDIR/.bun-{uid}-{hash}.so` across launches, same as
   1.4.2.

   That regression is invisible to a `--version` smoke test and to the
   named test steps here, so the comment's "dry_run green is enough"
   rule went with it: a bun bump now also requires executing the dry
   run's linux artifact for one real agent turn.

2. New patches/sse-cancel-rejection.patch, a backport of upstream
   #44944 (merged 2026-09-02, after v1.18.18, so not on the line the
   cron hold keeps us on). Under bun 1.4, `reader.cancel()` on a fetch
   body that was just aborted rejects; both SSE chunk-timeout wrappers
   discard that promise with `void`, which is an unhandled rejection,
   which bun answers by killing the process. chunkTimeout is armed on
   four providers in the deployed config, so one stalled stream would
   take a serve and every session it owns down with it. The two hunks
   are byte-identical to upstream's; upstream's third hunk is a
   test-only flake fix and is deliberately not carried.

   build-release now names the upstream test that goes red without the
   patch under bun 1.4 (test/provider/header-timeout.test.ts, per
   #44943), so neither half of this change can regress unnoticed.

Refs: workstation-o5s1.24, workstation-o5s1.27
- Cite the local reproduction of the 1.4.1+ splitting crash rather than
  oven-sh/bun#42837's own matrix, whose PASS criterion is an /api/agent
  200 -- the same smoke test this comment says returns 200 on a broken
  binary. Add anomalyco/opencode#48397, which traces the crash to a
  filesystem import cycle and is where the fix may actually land.
- Say what staying on 1.4.0 costs (the 1.4.1 fetch/dns fixes forgone),
  and why patching build.ts to `splitting: false` to reach 1.4.2 is
  worse than waiting: it puts us on a bundle shape nobody upstream runs.
- Scope the re-bump rule to every platform a deployed host consumes, not
  just linux, and record that darwin has no agent-turn coverage today.
- Narrow the new tripwire to the one test that exercises wrapSSE (the
  five headerTimeout tests return early before that path, and one races
  a 500 ms margin on a cold runner), record that it was proved
  non-vacuous on CI, and say plainly that the aisdk.ts wrapper is
  patched but not tripwired.
- apply.sh: the sunset is 'when upstream already contains #44944', at
  which point dropping the patch is mandatory. The previous wording read
  as an absolute ban while the pin is >= 1.4.0.
@johnnymo87
johnnymo87 merged commit 32b64e5 into main Sep 21, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant