Skip to content

Let a scroll expand an uncompressed tarball - #14

Merged
suffro merged 3 commits into
mainfrom
plain-tar-assets
Sep 12, 2026
Merged

suffro merged 3 commits into
mainfrom
plain-tar-assets

Conversation

@suffro

@suffro suffro commented Sep 11, 2026

Copy link
Copy Markdown
Owner

What

assetArchives accepted zip or tar.gz only. A publisher who ships a plain .tar could not be consumed at all — and that is the usual choice when every member inside is already compressed, where a second gzip pass costs minutes to save nothing.

tar now joins the enum and goes through the same listing, validation and extraction as tar.gz. Nothing about the safety rules changes: entries are listed and checked before anything is written, and links and special entries are still refused outright.

One thing that turned out not to be true

The read paths passed gzip: true to node-tar, which reads like a guarantee that the archive really is gzipped. It is not one — node-tar reads the header and decompresses or not on its own, and its gzip option applies to writing. Probed directly on all four combinations before deciding:

plain.tar     gzip=false -> OK      packed.tar.gz gzip=false -> OK
plain.tar     gzip=true  -> OK      packed.tar.gz gzip=true  -> OK

So the option is dropped rather than threaded through, and the white paper now says what actually holds the author to their claim: the scroll's tar / tar.gz distinction records what they pinned, and the asset's SHA-256 is what makes it binding.

Tests

Two added to archive-security.test.mjs: a plain tarball expands with stripComponents, and the entry rules apply to it exactly as to a compressed one.

Full suite on Linux: 566 passed, 36 files. On a Windows host 43 tests fail with EPERM: operation not permitted, symlink — pre-existing, unrelated, and true on main too.

Where it came from

A project whose model bundles 1.86 GB of pickled molecule definitions as a plain mols.tar. Third gap this line of work has found by real use, after PyPI licences (1.1.0) and download-then-expand (1.1.1).

🤖 Generated with Claude Code

`assetArchives` accepted `zip` or `tar.gz`, so a publisher who ships a plain
`.tar` could not be consumed at all — and that is the usual choice when every
member inside is already compressed, where a second pass costs minutes to save
nothing. The archive goes through the same listing, validation and extraction as
`tar.gz`; the only thing missing was permission to say so.

Compression is now left to the library on read, which is what it was doing
anyway: node-tar reads the header and decompresses or not, and its `gzip` option
applies to writing. Passing `gzip: true` on a read looked like a guarantee and
was not one. The `tar` / `tar.gz` distinction records what the author pinned, and
the asset's SHA-256 is what holds them to it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Sep 11, 2026 •

Copy link
Copy Markdown

Deploying scrollcase with  Cloudflare Pages  Cloudflare Pages

Latest commit: 64cb078
Status: ✅  Deploy successful!
Preview URL: https://e40a0391.scrollcase.pages.dev
Branch Preview URL: https://plain-tar-assets.scrollcase.pages.dev

View logs

suffro added a commit to Liatir/liatir-app that referenced this pull request Sep 11, 2026
Boltz-2 builds, signs, self-tests and now predicts. Given ubiquitin's 76
residues and nothing else, it reproduces the experimental structure of PDB 1UBQ
to a backbone RMSD of 1.99 A against a 3 A limit, at pLDDT 0.925. The largest
single deviation, 11.5 A, falls on the C-terminal tail, which is the part of
ubiquitin known to be mobile: physics, not a wrong fold. Two runs at the same
seed differ by exactly zero, and a separate validator process reproduced the
RMSD to the last decimal.

Every prediction here runs single-sequence, because an offline box has no
alignment server. That is the accuracy this product can actually promise, so it
is the accuracy that gets measured.

The evidence is bound to the signed box rather than to a directory: the
validator verifies the release signature and walks the payload digest before it
runs anything. Archive 609a3cc5, 12.69 GB, 20.3 GB installed. Development key,
dirty tree; a production build makes a different archive and must be measured
there.

Three things this turned up that only running it could:

  - Boltz's CUDA path imports cuequivariance_torch unconditionally, even though
    upstream calls those kernels an optional extra and the reviewed decision was
    to leave them out. The first prediction died on ModuleNotFoundError. Boltz
    keeps a pure-PyTorch implementation of the same triangular multiplication
    behind --no_kernels, which the runner now passes: same mathematics, slower,
    and no closed-source NVIDIA kernel shipped for an unmeasured speed-up.

  - pandas 3.0.5 works with Boltz 2.2.1. That was the one loose upstream bound,
    left for a real import to settle rather than pinned pre-emptively.

  - mols.tar is an uncompressed tar, which Scrollcase did not accept at all.
    Fixed upstream in suffro/scrollcase#14, which also drops a gzip: true that
    read like a guarantee and was not one. Until that release lands, this scroll
    cannot be built from a clean checkout.

The legal record is approved. mols.tar is a derivative of the wwPDB Chemical
Component Dictionary, which the PDB archive places under CC0 - traced through
upstream's generation script, the archive's own contents and the wwPDB policy,
and re-checked at build time: all 45,227 members are named by a PDB chemical
component identifier and nothing else.

One claim in that record was wrong and is corrected. frozendict was not this
project's first copyleft dependency. Every published box carries between 8 and
26 GPL-family conda packages, and native-tools, which ships inside the app, has
18 including bcftools and bwa under plain GPL-3.0. What was new is that it was
the first one anyone could see.

Protenix splits in two. v2 stays blocked on a checkpoint that is not ours to
unblock: HTTP 403 again, five days on, while every neighbouring object serves
200, with no official mirror anywhere. Mini now resolves, after the reviewed
manifest turned out to have no conda-first solution at all - 24 conda entries,
116 PyPI, none without a SHA-256, every upstream pin honoured. All 116 of its
licences are reviewed, and two of them should not be there: cuequivariance-ops
ships closed-source NVIDIA kernels that Protenix never imports.

test:verify 6/6. syngraphe check clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
suffro and others added 2 commits September 12, 2026 12:55
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@suffro
suffro merged commit 946ab6e into main Sep 12, 2026
16 checks passed
@suffro
suffro deleted the plain-tar-assets branch September 12, 2026 17:11

This branch had an error being deployed

1 failed deployment
npm — 946ab6e3 Deployed Sep 12, 2026 by suffro via Publish to npm #3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant