Run the test suites in CI, and make the fmt and clippy gates pass - #139
Closed
ParallelEntrepreneur wants to merge 9 commits into
Closed
ParallelEntrepreneur wants to merge 9 commits into
ParallelEntrepreneur wants to merge 9 commits into
Conversation
…settlers into a crew (#71) Implements the Settler Showcase design (claude.ai/design): the ant, the card and parallel settlers. The ant (AntAvatar.tsx, index.css) is now a pixel-art worker in side profile, 26 by 20 pixels with crisp edges and a joint for every moving part. The ground, thought pixels, scribble and lens move a pixel at a time. - working: a tripod gait, carrying what the tool is (a lens to search or read, a leaf to edit, a block to build or install, a flag to test). - thinking: looks up while a thought fills in, dot by dot. - writing: nods over a line it writes and rewrites. - done: hops once and rests with a check. - paused: sits with its antennae drooped. - A failed tool result makes it stumble over a pebble, then carry on. - Each role wears its own mark: a Scout's pack, a Builder's hard hat, an Inspector's monocle, and so on. - Reduced motion freezes every state in a readable pose. The card (SettlerCard.tsx) keeps the settler's name, task and one status line. It adds a step count and a trail along its foot that moves while the settler works, and the frame turns green when the settler is done. Collapsed, a done card shows the first sentence of its report. Opened, it shows the report as markdown, then the steps: in plain words with the command or path each acted on, or the raw transcript in the technical view. Grouping: settlers the orchestrator sends out together are one crew. Their events interleave while they work, and each settler now keeps one card for the whole burst instead of splitting into "continued" cards every time another settler speaks. A crew's cards hang off one rail under a strip of its ants on a shared trail ("3 settlers · 2 at work · 1 done"), and their loops are staggered so they never move in lockstep. Departures from the design, where it showed data colonies don't send: - A done card doesn't show a duration, because the stream doesn't record when a settler finished. - A failed step says "didn't work", not "didn't work · retried", because nothing says it was retried. The mock's colony now sends a crew of three (Scout, Tester, Inspector) with a failing test run, so ?mock=1 shows every part of this.
…e top of the README (#72) Installing was explained only in the README, about 200 lines down, and the site's homepage showed `scripts/install.sh` with no word that it has to run inside a clone. docs/install.md is now the one place for it, mirrored to colonizer.dev/docs/install alongside vision, architecture and protocol. It covers: - what a machine needs, including Claude Code on Linux; - the build-and-run commands, and the two installer options; - the first run, and what the installer does on a Mac; - where settings, credentials, colonies and the app live; - how to update. The README's "Run it" moves up to right after the introduction: the four commands and a pointer to the guide. The introduction no longer says Linux only, since an Apple Silicon Mac has run a colony end to end (#36). colonizer.dev/docs/install only exists once the website re-syncs its docs after this merges.
A version tag now publishes a GitHub release with a prebuilt app for Linux x86_64
and one for Apple Silicon, plus an installer:
curl -fsSL https://colonizer.dev/install.sh | sh
Nothing on the machine needs Rust or Node.js to install it.
.github/workflows/release.yml
- linux-binaries: builds colonizer-agentd and rtk for both colony architectures,
and the Linux harness, as static musl binaries. They are built inside
rust:1-alpine with `docker run`, because GitHub's JavaScript actions don't run
in Alpine containers. A static harness starts on any glibc; one built on the
Ubuntu 24.04 runner would not start on Debian 12 or Ubuntu 22.04.
- bundle: runs scripts/install.sh --bundle on ubuntu-24.04 and macos-15, then a
smoke test. The app starts from the unpacked archive, finds its assets there,
reports agentd, the web UI and the claude-code module, and serves the UI.
- release: runs for tags only. The tag must match crates/colonizer's version. It
writes SHA256SUMS and publishes the archives with install.sh.
A release contains no Anthropic code. The Claude Agent SDK is "all rights reserved",
and its optional platform packages are Claude Code itself (224 MB).
- The bundle installs the module with --omit=optional. The runner already passes
pathToClaudeCodeExecutable, so those packages were never used.
- scripts/record-fetch-at-install.mjs takes the SDK out and writes its lockfile
tarball URL and sha256, after checking the tarball against package-lock.json's
integrity.
- scripts/install-release.sh fetches the SDK from the npm registry against that
record. On a Mac it also fetches the linux-arm64 Claude Code build from
Anthropic's stable channel, checked against the manifest. macOS's own plutil
reads the manifest, so no Node.js is needed. The Workflow's Package step fails
if the SDK is found in a bundle.
scripts/install-release.sh (published as install.sh)
- Refuses platforms it can't serve, including Linux without read-write /dev/kvm.
- Checks the archive against SHA256SUMS.
- Swaps ~/.local/share/colonizer/app in whole, and links ~/.local/bin/colonizer.
- Running it again updates in place. It reuses a Claude Code binary that still
matches the manifest, and never touches settings or colonies.
- Accepts COLONIZER_VERSION, COLONIZER_APP and --pull-image.
- Everything runs inside main(), so a truncated download runs nothing.
scripts/install.sh --bundle skips the KVM check and the Claude Code fetch, and
uses binaries from $COLONIZER_PREBUILT instead of building them.
build-agentd.sh and build-rtk.sh accept COLONIZER_BUILD_HERE=1, to build inside
an existing rust:1-alpine instead of a microVM. rtk's source is then unpacked
outside the repository, because cargo refused to build it as a stray member of
the harness workspace (the local test below caught that).
Tested on an Apple Silicon Mac:
- In a rust:1-alpine microVM with COLONIZER_BUILD_HERE=1: agentd, rtk and the
harness built as static aarch64 musl binaries. The static harness started in a
Debian (node:24-bookworm) microVM and served /api/status.
- `install.sh --bundle` with those guest binaries: a 129 MB app, a 44 MB archive,
and no Agent SDK inside.
- `curl … | sh` against the archive served locally, into an empty HOME, took 41 s.
It fetched the SDK (sha256 checked) and Claude Code 2.1.267 (manifest checked).
- The installed app reported its assets in the new location: agentd, web,
claude-code, the guest Claude binary, and msb 0.6.18 from the relocated vendor
directory. It served the UI.
- Running the installer again reused Claude Code, left no app.new or app.old,
and kept the data directories.
- The installed colonizer-agentd, rtk and claude-guest each ran `--version`
inside node:24-bookworm.
- The runner's 58 tests passed against an SDK restored from the npm tarball
without optional packages.
On GitHub (this pull request's run): both linux-binaries jobs built in rust:1-alpine, and both
bundles passed the smoke test from their unpacked archives (97 MB for linux-x86_64, with headscale and
tailscale; 43 MB for darwin-arm64).
Not tested yet:
- Installing the linux-x86_64 release on a Linux machine.
- Launching a colony from an installed release.
- colonizer.dev/install.sh, which the website adds once a release exists. Until
then, install from the release URL directly.
docs/install.md opens with "Install a release": - `curl -fsSL https://colonizer.dev/install.sh | sh`; - what the installer checks and where it installs; - the two things it fetches from Anthropic's own channels, since a release carries no Anthropic code; - updating by running it again; - how to install a particular version, or pull the colony image up front. The build steps are now "Build from source". "What you need" separates what a release needs (git, gh, curl, tar) from what a source build adds (Node.js 20+, Rust 1.88+). "Where things live" and "Updating" cover both ways. The README's "Run it" leads with the one command, with the source build under it. v0.1.0 was installed from the published release into an empty HOME on an Apple Silicon Mac (52 s): it fetched the SDK and Claude Code 2.1.267, started, and reported agentd, the web UI, the claude-code module, the guest Claude binary and msb 0.6.18. The Linux release is smoke-tested in CI but hasn't been installed on a Linux machine yet. colonizer.dev/install.sh is added by the website.
…, and publish them on release (#76) Reserves the names on crates.io and puts the source there. The crates are not an install: `cargo install` gives the mothership binary without microsandbox, the in-VM daemon, the agent module or the web UI. Each crate's README says so and points to `curl -fsSL https://colonizer.dev/install.sh | sh`. The site and docs don't mention cargo. - The harness package is renamed `colonizer-harness`, because `colonizer` on crates.io belongs to an unrelated project. Its binary is still `colonizer`, so the command, target/release/colonizer and the release archives don't change. scripts/install.sh and release.yml build it with `-p colonizer-harness`. - The Headroom bundle pins move from vendor/vendor.lock into crates/colonizer/headroom.lock, because `include_str!` reached outside the crate and `cargo publish` packages only the crate. fetch-vendor.sh loses its skip for kind `bundle`, which no longer occurs. docs/protocol.md and headroom-bundle.yml name the new file. - Both crates are at 0.1.1 and have a crates.io README, a homepage and a readme field. - release.yml: the release job checks the tag against both crate versions. A new crates job, tags only and after the release, publishes whichever of colonizer-agentd and colonizer-harness crates.io doesn't have yet. It uses Trusted Publishing (rust-lang/crates-io-auth-action v1.0.5, pinned, with id-token: write), so no token is stored. crates.io won't create a crate that way ("Trusted Publishing tokens do not support creating new crates. Publish the crate manually, first"), so the first version is published by hand, and later tags publish on their own. Checked: - `cargo test --workspace` passes. - `cargo publish --dry-run --locked` packages and verifies both crates. The harness package includes headroom.lock and README.md. - actionlint passes on release.yml.
…m to 0.1.2 (#77) The crates.io pages now open like the repository: a banner, a badge row, then what the crate is. - assets/banners/banner.html draws both banners in colonizer.dev's colours and type: a mono eyebrow, a headline, a lead, a foot line and a diagram leading to the command. - colonizer-harness: "The mothership.", with the colony graph from the site, ending in `colonizer`. - colonizer-agentd: "Inside every colony.", showing a colony microVM with agentd between Claude Code, the event log and the terminal, and the private mesh up to the mothership. - scripts/render-crate-banners.sh renders them at 2560×800 with headless Chrome into assets/banners/<crate>.png. The READMEs link them by their github.com/ghraw URL on main, because crates.io renders a README outside the repository. - crates/colonizer/README.md and crates/colonizer-agentd/README.md: - banner and badges: crates.io version, latest release, docs, MIT; - what each crate is, with the install command, and why the crate alone isn't an install; - for the harness, what Colonizer does and the two crates; - for agentd, what it does and its four endpoints. The claims come from the README and docs/protocol.md §3. - Both crates are at 0.1.2: crates.io shows a README only with a new version. Tagging v0.1.2 publishes them through Trusted Publishing. `cargo publish --dry-run` packages and verifies both crates.
…laiming no runtime downloads (#78) docs/vision.md defined a settler as "the agent module working inside a colony". Since #70 and #71, the colony chat also calls the subagents that module sends out settlers, named for their role (Scout, Builder, Inspector…), and colonizer.dev is getting a page about colonies and settlers. The vocabulary now covers both: the agent module is a colony's first settler, and the subagents it sends out are settlers too, each with a role. Principle 4 said "No runtime downloads". That hasn't been true for a while: microsandbox pulls a colony image the first time a colony needs it, and switching Headroom on downloads its bundle. docs/install.md already says so. The principle now keeps what holds (your hardware, its own dependencies, no cloud account) and names the two downloads after install.
…itches it on (#79) colonizer.dev/live will show a dot for every area, about 25 km across, where a mothership is online, lit while its colonies work. This adds both ends. The receiver, services/telemetry, is a Cloudflare Worker with a D1 table, deployed at telemetry.colonizer.dev: - POST /v1/heartbeat takes {install_id, version, platform, colonies} and ignores any other field. The location is Cloudflare's own estimate for the request, snapped to a 25 km grid cell; only the cell is stored. The install id is kept as a SHA-256 hash. The IP address is used, in memory, for rate limiting only. {install_id, online: false} deletes the row. - GET /v1/presence is the public view: totals and counts per cell, never per mothership. Online means a heartbeat in the last 12 minutes. - Rows older than an hour are pruned during heartbeats. A cron trigger would need a workers.dev subdomain on the account, and this needs nothing. - The privacy-relevant logic is in src/presence.js with node tests. The mothership, crates/colonizer/src/telemetry.rs: - Off until the user answers. The answer and a random v4 install id live in <config>/telemetry.json. The id is created on switching on and forgotten on switching off, so two periods on the map can't be linked. - While on, a heartbeat every 5 minutes (the service may name 60 s to 1 h), and within a minute of the colony count changing. Switching off and shutting down send online: false, so the dot goes at once. - DO_NOT_TRACK or COLONIZER_TELEMETRY=off keep it off whatever Settings says; COLONIZER_TELEMETRY_URL points it elsewhere. - GET/PUT /api/telemetry return the status and the exact next heartbeat. The web UI asks once, after GitHub and Claude are connected, with "Show on the map" / "No thanks" / "What is sent". Settings has a Live map section with the switch and the heartbeat as JSON. docs/telemetry.md says what is sent, what is kept and for how long, what is public, and how to keep it off; the README's trust model, configuration and development sections and docs/protocol.md point to it. The crates are bumped to 0.1.3, the first release with the live map.
Colonizer session 0ee041a4 Co-Authored-By: Colonizer <noreply@colonizer.dev>
Collaborator
Author
|
Superseded by #142. This branch's only new commit was a whole-tree One deliberate difference: #142 adds a |
ParallelEntrepreneur
added a commit
that referenced
this pull request
Sep 23, 2026
#138 left the fmt gate out on purpose: reformatting the whole tree is a dedicated change, not a drive-by. This is that change, redone against main rather than rebasing #139's version, whose base predated eight merged commits. `rustfmt.toml` sets width 130 instead of taking rustfmt's 100-column default. The tree was written wider — 99% of its lines already fit in 129 columns — and at 100 the one-time reflow is nearly twice the churn, for lines nobody asked to be broken. Everything else is rustfmt's default. Nothing but whitespace changed: `cargo test --workspace` is 240 passed before and after, and clippy stays clean. CI runs `cargo fmt --all --check` after the tests and the lint, so a formatting nit never hides a real failure, and the rust job's toolchain now installs rustfmt and clippy rather than relying on what the runner image happens to ship.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
README.mdlisted "CI running the Rust, runner and UI test suites" asPLANNED, and said"No CI yet". Neither was quite true and neither was the whole story:
release.ymlalreadybuilds, bundles and publishes to crates.io, but nothing ever ran a test. This adds the
workflow that does, and makes the lint gates it enforces actually pass.
What changed
.github/workflows/ci.yml(new). Runs on every pull request, on pushes tomain, and onworkflow_dispatch, as three parallel jobs onubuntu-24.04:rust—cargo fmt --all -- --check, thencargo clippy --workspace --all-targets --locked -- -D warnings, thencargo test --workspace --locked.node— the Claude Code runner's suite (npm ci+npm test) and the telemetry receiver's(
node --test; it has no dependencies, so it needs no install step).web—npm ci+npm run build(tsc --noEmit && vite build). There is no UI test suite,so this is a build check and the README says so.
It follows
release.yml's conventions: actions pinned by commit SHA with a# vX.Y.Zcomment(the same pins already in use),
rustup toolchain install stable --profile minimalrather thana setup action, top-level
permissions: contents: read, aconcurrencygroup that cancelssuperseded PR runs, and
timeout-minuteson every job.A genuinely broken test command, fixed.
modules/agents/claude-code'stestscript wasnode --test test/, which fails on Node 24 — the positional argument is treated as a literalentry file, so it exits 1 in 25ms with
Cannot find module '.../claude-code/test'withoutrunning anything.
npm testin that package was broken, as was the command documented inREADME.md. Barenode --testdiscoverstest/*.test.mjscorrectly and passes all 58 tests.Fixed in the script and the README.
cargo fmt --allacross the workspace (26 files, formatting only) andcargo clippy --fix,so the two new gates go green. The clippy pass touched 10 files: 13
collapsible_ifcollapses,one
while_let_on_iterator, onemanual_contains, oneitems_after_test_modulecode motion,and one hand-written fix for
unnecessary_sort_byinmemory.rs(sort_by→sort_by_key(|p| Reverse(...))). No#[allow]was used to silence anything.rust-version = "1.88"in both crate manifests. Thecollapsible_iffixes producelet-chains, which need Rust 1.88 — edition 2024 alone only requires 1.85. Since both crates are
published to crates.io, this is now declared rather than left as an undocumented effective
minimum, so cargo gives consumers a clear error instead of a syntax failure.
README.md — the roadmap row flipped to
SHIPPINGand reworded to match what CI really does(three test suites plus a UI build, not a "UI test suite"); the "No CI yet, and nothing is
published to crates.io or npm" bullet rewritten, since only the npm half was still true; and the
Development section now notes CI and shows the
npm cia fresh clone actually needs.How this was verified
Everything below was run in the sandbox, and then re-run from scratch by a separate agent that
read
ci.ymland executed each step's command verbatim:cargo fmt --all -- --checkcargo clippy --workspace --all-targets --locked -- -D warningscargo test --workspace --lockednpm ci+npm testnode --testnpm ci+npm run buildactionlintv1.7.12 reports the workflow clean. The action SHAs were compared literally againstrelease.yml. Therustup ... --component rustfmt,clippyline was verified empirically ratherthan by eye — clippy was removed from the toolchain and the exact workflow line re-run, which
reinstalled it. The old
node --test test/was run under Node 24 to confirm the failure is realrather than inferred.
Worth a close look
This branch is based on a stale
origin/mainand will conflict on merge. It could not berebased here (git metadata is read-only in the colony). Simulating the merge in a throwaway
clone:
HEADalready conflicts withorigin/mainon 5 files before any of this work —Cargo.lock, both crateCargo.tomls,web/src/components/ChatPanel.tsxandweb/src/sessionStream.ts. The workspace-wide reformat adds 5 more, all Rust:headroom.rs,main.rs,memory.rs,sessions.rsandtelemetry.rs(~28 marker blocks). Most arestyle-versus-semantic and resolve by taking main's side and re-running
cargo fmt, butmemory.rsneeds real care — #97 rewrote 253 lines of it inside a region this reformat alsotouched. Note that
headroom.rsandtelemetry.rsare byte-identical on both sides today; thereformat alone is what breaks that identity. Rebasing onto
origin/mainand re-runningcargo fmt --allis likely to be much less work than resolving the reformat by hand.The two
Cargo.tomlfiles that gainedrust-versionare themselves already in that baselineconflict set.
The roadmap row is marked
SHIPPING, whichREADME.md:78-81defines as "merged, in thisrepository, and exercised on a real machine". That is true by the time this merges — GitHub runs
pull_requestworkflows from the PR's merge ref, so this PR exercises its own CI — but theclaim is slightly ahead of the evidence until that first run is green. Worth confirming the run
before merging.
The clippy autofixes are the only semantic changes in a very large diff, and each was reviewed
individually: the
sort_by_key(Reverse(...))rewrite preserves ordering and tie-breaking (bothare stable sorts on a
Copykey),chars.by_ref()keeps the iterator usable after the loop asthe surrounding code requires, and every collapsed condition with an
||was parenthesised topreserve precedence. The remaining ~3900 lines of Rust diff are pure formatting churn and are
best reviewed with whitespace ignored.
🤖 Generated by Colonizer in a microVM