diff --git a/CLAUDE.md b/CLAUDE.md index be5bc539..20cd2478 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,20 +1,44 @@ # Working in this fork `hotdata-dev/liquid-cache` is a fork of `datafusion-contrib/liquid-cache`. -`main` is upstream's `0033b15` with our patches on top — upstream's history is -fully contained, so `git merge-base main ` resolves. +`main` contains upstream's history in full plus our patches, so +`git merge-base main ` resolves. + +Nothing here states where upstream currently is, or where we are: both move. +Ask git instead. + +``` +git fetch https://github.com/datafusion-contrib/liquid-cache main +git log --oneline FETCH_HEAD..main # ours that upstream does not have +git log --oneline main..FETCH_HEAD # upstream's that we do not have +``` `AGENTS.md` and `README.md` are upstream's and describe the project itself. This file is ours and describes only what differs here. Nothing in this repo should edit an upstream-owned file to record a fork convention: that conflicts on every sync. Add a file upstream does not have instead. +## Remotes + +**Check before using a remote name. They vary by clone and they are not what +you would guess** — in at least one working copy `origin` is *upstream* and the +fork is a second remote named `fork`, which is the reverse of the usual +arrangement. + +``` +git remote -v +``` + +Commands below name repositories by URL rather than by remote, so they are +correct in any clone. Do the same when writing instructions for anyone else: a +bare `origin/main` is ambiguous here and has already been misread as the +opposite of what it meant. + ## Branches -- Work off `main`. Fetch first — a local `main` goes stale with no signal, and - branching off a stale one silently drops everything merged since. The remote - is `origin` in a fresh clone; if you cloned upstream and added this fork as a - second remote, use that name instead. +- Work off `main`, and fetch before branching — a local `main` goes stale with + no signal, and branching off a stale one silently drops everything merged + since. - **Upstream PRs branch from upstream, not from `main`**, and are named `upstream/`. A branch cut from `main` carries our whole patch stack into the PR diff. @@ -26,8 +50,9 @@ on every sync. Add a file upstream does not have instead. Before pushing, this must list only the commits you wrote — anything else is a fork patch that would land in the upstream diff. Re-fetch upstream on the - line above it: `FETCH_HEAD` holds whatever the last fetch wrote, so after a - `git fetch origin` it is *our* `main` and the check hides every fork patch. + line above it: `FETCH_HEAD` holds whatever the last fetch wrote, so after + fetching any other remote it is no longer upstream and the check hides every + fork patch. ``` git fetch https://github.com/datafusion-contrib/liquid-cache main @@ -46,16 +71,64 @@ on every sync. Add a file upstream does not have instead. have (`DiskResidue`, `reclaim_orphaned_disk`, `settle`) and are not upstreamable at all. +## Syncing from upstream + +Merge upstream into a branch off `main` and raise a PR; do not use GitHub's +"Sync fork" button, which offers to discard our commits when the merge is not +a fast-forward. + +Fetch *this fork* first and branch from what came back, not from local `main` +— the upstream fetch below never refreshes `main`, so a stale one would make +the sync PR revert fork commits merged since. Use each `FETCH_HEAD` +immediately: it holds only the last fetch. + +``` +git fetch https://github.com/hotdata-dev/liquid-cache main +git checkout -b sync/upstream- FETCH_HEAD + +git fetch https://github.com/datafusion-contrib/liquid-cache main +git merge FETCH_HEAD +``` + +**Merge this PR, do not squash it.** A squash gives the result a single parent, +so upstream's history never enters `main`'s ancestry: the merge-base does not +move, the same upstream commits stay missing, and nothing reports it. + +Verify afterwards. The merge happens on GitHub, so local `main` does not have +it and must not be what you check; and `FETCH_HEAD` holds only the last fetch, +so capture upstream before fetching the fork over it: + +``` +git fetch https://github.com/datafusion-contrib/liquid-cache main +upstream=$(git rev-parse FETCH_HEAD) +git fetch https://github.com/hotdata-dev/liquid-cache main +git merge-base --is-ancestor "$upstream" FETCH_HEAD && echo ok +``` + +A change we contributed upstream comes back as their squash of it. The content +matches but the commit does not, so the merge conflicts where both sides +touched the same lines — typically a module list that each side appended to. +Keep ours *for the returned change*, and keep any other upstream edit in the +same hunk: upstream may have appended something of its own next to it, and +taking the whole hunk from our side drops that silently. Read the hunk rather +than resolving by rule. + +Sync promptly rather than letting such a conflict wait: alone it is obvious, +bundled with real upstream work later it is not. + ## Building and testing -- **`cargo +1.96.0`.** The dependency tree needs 1.95+ (`vortex-*`, `sysinfo`) - and DataFusion 55 needs 1.94. A bare `cargo` on an older default fails - resolution with a wall of `requires rustc 1.9x` lines. +- **Pin the toolchain: `cargo +1.96.0 ...`.** There is no + `rust-toolchain.toml`, so a bare `cargo` uses whatever default is installed, + and an older one fails resolution with a wall of `requires rustc 1.9x` lines + naming `vortex-*` and `sysinfo`. Those lines state the minimum each crate + wants; use a toolchain at least that new. - **Shuttle tests need a filter**: `cargo +1.96.0 test -p liquid-cache --features shuttle --lib shuttle_`. The feature swaps `crate::sync` to shuttle primitives for the whole test build, so running it unfiltered fails - ~49 unrelated tests with "Are you accessing a Shuttle primitive outside of a - Shuttle test?". That is by design, not a regression. + every test that touches a lock outside a shuttle runner, with "Are you + accessing a Shuttle primitive outside of a Shuttle test?". That is by design, + not a regression. - **`dev-tools` needs `dev/dev-tools/assets/tailwind.css`**, which CI generates and the repo does not carry. To run its tests locally, create a placeholder and delete it before committing. Do not habitually pass