chore: merge new changes from ipfs/kubo master - #2
Open
alvin-reyes wants to merge 986 commits into
Open
Conversation
* fix(routing): update kad-dht with peerstore address clone fix closes #11116 See #11116 for context of this fix * fix(routing): update kad-dht with CPL exploration fix fixes an infinite loop when all peers share the same CPL during provider exploration See libp2p/go-libp2p-kad-dht#1216 * fix(routing): update kad-dht with shutdown loop check libp2p/go-libp2p-kad-dht#1217 * depend on latest kad-dht fix * bump kad-dht to v0.37.0 --------- Co-authored-by: guillaumemichel <guillaume@michel.id> Co-authored-by: Guillaume Michel <guillaumemichel@users.noreply.github.com>
- upgrade go-ds-pebble to [v0.5.9](https://github.com/ipfs/go-ds-pebble/releases/tag/v0.5.9) - git ignore ipfswatch binary
pebble, leveldb, and badger should be updated via go-ds-* wrappers to ensure compatibility
* Upgrade to Boxo v0.36.0 * sharness: add missing metrics
) Bumps [github.com/tidwall/gjson](https://github.com/tidwall/gjson) from 1.16.0 to 1.18.0. - [Commits](tidwall/gjson@v1.16.0...v1.18.0) --- updated-dependencies: - dependency-name: github.com/tidwall/gjson dependency-version: 1.18.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* ci: add caching to ipfs-webui interop tests cache node_modules, Playwright browsers, and test build output to speed up repeated CI runs. also use node version from ipfs-webui/.tool-versions instead of hardcoding, and upload test artifacts on failure. * docs(ci): add header comment to interop workflow explain what helia-interop and ipfs-webui jobs do
* feat(dns): resolve libp2p.direct addresses locally without network I/O p2p-forge hostnames encode IP addresses directly (e.g., 1-2-3-4.peerID.libp2p.direct -> 1.2.3.4), so DNS queries are wasteful. kubo now parses these IPs in-memory. - applies to both default libp2p.direct and custom AutoTLS.DomainSuffix - TXT queries still delegate to network for ACME DNS-01 compatibility - #11140 (comment) use fallback to network DNS instead of returning errors when local parsing fails, ensuring forward compatibility with future DNS records - #11140 (comment) add peerID validation using peer.Decode(), matching libp2p.direct server behavior, with fallback on invalid peerID - #11140 (comment) document interaction with DNS.Resolvers in config.md - #11140 (comment) add AutoTLS.SkipDNSLookup config flag to disable local resolution (useful for debugging or custom DNS override scenarios) - #11140 (comment) add E2E test verifying libp2p.direct resolves locally even when DNS.Resolvers points to a broken server additional improvements: - use madns.BasicResolver interface instead of custom basicResolver - add compile-time interface checks for p2pForgeResolver and madns.Resolver - refactor tests: merge IPv4/IPv6, add helpers, use config.DefaultDomainSuffix - improve changelog to explain public good benefit (reducing DNS load) Fixes #11136
- docs/README.md: restructure to surface 20+ previously undiscoverable docs - docs/README.md: fix broken github-issue-guide.md link (file was removed) - docs/add-code-flow.md: rewrite with current code flow and mermaid diagrams - docs/customizing.md, docs/gateway.md: use specs.ipfs.tech URLs - README.md: fix orphan #nix anchor, use go.dev links, link to contributors graph - remove stale docs/AUTHORS and docs/generate-authors.sh (last updated 2016)
* feat(key): add 'ipfs key ls' as alias for 'ipfs key list' Add 'ls' as an alias for the 'list' subcommand in 'ipfs key' to be consistent with other ipfs commands like 'ipfs repo ls' and 'ipfs pin ls' which use 'ls' instead of 'list'. Fixes #10976 Signed-off-by: Vedant Madane <6527493+VedantMadane@users.noreply.github.com> * feat(key): make 'ipfs key ls' canonical, deprecate 'list' aligns with other commands like 'ipfs pin ls' and 'ipfs files ls'. 'ipfs key list' still works but shows deprecation warning. * fix(key): correct --key option description in verify command was copy-pasted from sign command and said "signing" instead of "verifying" --------- Signed-off-by: Vedant Madane <6527493+VedantMadane@users.noreply.github.com> Co-authored-by: Marcin Rataj <lidel@lidel.org>
* fix http header when compress enabled for get command Closes #2376 * fix(rpc): set Content-Type for ipfs get based on output format - set application/x-tar when outputting tar (default and --archive) - set application/gzip when compression is enabled (--compress) - update go-ipfs-cmds with Tar encoding type and RFC 6713 compliant MIME types (application/gzip instead of application/x-gzip) * test(rpc): add Content-Type header tests for ipfs get * feat(rpc): add Content-Type headers for binary responses set proper Content-Type headers for RPC endpoints that return binary data: - `dag export`: application/vnd.ipld.car - `block get`: application/vnd.ipld.raw - `diag profile`: application/zip - `get`: application/x-tar or application/gzip (already worked, migrated to new API) uses the new OctetStream encoding type and SetContentType() method from go-ipfs-cmds to specify custom MIME types for binary responses. refs: #2376 * feat(rpc): add `ipfs name get` command for IPNS record retrieval add dedicated command to retrieve raw signed IPNS records from the routing system. returns protobuf-encoded IPNS record with Content-Type `application/vnd.ipfs.ipns-record`. this provides a more convenient alternative to `ipfs routing get /ipns/<name>` which returns JSON with base64-encoded data. the raw output can be piped directly to `ipfs name inspect`: ipfs name get <name> | ipfs name inspect spec: https://specs.ipfs.tech/ipns/ipns-record/ * feat(rpc): add `ipfs name put` command for IPNS record storage adds `ipfs name put` to complement `ipfs name get`, allowing users to store IPNS records obtained from external sources without needing the private key. useful for backup, restore, and debugging workflows. the command validates records by default (signature, sequence number). use `--force` to bypass validation for testing how routing handles malformed or outdated records. also reorganizes test/cli files: - rename http_rpc_* -> rpc_* to match existing convention - merge name_get_put_test.go into name_test.go - add file header comments documenting test purposes * chore(deps): update go-ipfs-cmds to latest master includes SetContentType() for dynamic Content-Type headers --------- Co-authored-by: Marcin Rataj <lidel@lidel.org>
…7.0 (#11178) Bumps [github.com/coreos/go-systemd/v22](https://github.com/coreos/go-systemd) from 22.5.0 to 22.7.0. - [Release notes](https://github.com/coreos/go-systemd/releases) - [Commits](coreos/go-systemd@v22.5.0...v22.7.0) --- updated-dependencies: - dependency-name: github.com/coreos/go-systemd/v22 dependency-version: 22.7.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
… to 0.5.0 in the ipfs-ecosystem group (#11177) * chore(deps): bump github.com/multiformats/go-multiaddr-dns Bumps the ipfs-ecosystem group with 1 update: [github.com/multiformats/go-multiaddr-dns](https://github.com/multiformats/go-multiaddr-dns). Updates `github.com/multiformats/go-multiaddr-dns` from 0.4.1 to 0.5.0 - [Release notes](https://github.com/multiformats/go-multiaddr-dns/releases) - [Commits](multiformats/go-multiaddr-dns@v0.4.1...v0.5.0) --- updated-dependencies: - dependency-name: github.com/multiformats/go-multiaddr-dns dependency-version: 0.5.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: ipfs-ecosystem ... Signed-off-by: dependabot[bot] <support@github.com> * chore: run make mod_tidy --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com>
…#11160) * chore(deps): bump github.com/hashicorp/go-version from 1.7.0 to 1.8.0 Bumps [github.com/hashicorp/go-version](https://github.com/hashicorp/go-version) from 1.7.0 to 1.8.0. - [Release notes](https://github.com/hashicorp/go-version/releases) - [Changelog](https://github.com/hashicorp/go-version/blob/main/CHANGELOG.md) - [Commits](hashicorp/go-version@v1.7.0...v1.8.0) --- updated-dependencies: - dependency-name: github.com/hashicorp/go-version dependency-version: 1.8.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * chore: run make mod_tidy --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com>
* fix: recognize content-type application/x-tar For context, see: ipfs/go-ipfs-cmds#320 Closes: #11179
add missing fixes from v0.47.0 release
* test: IPIP-523 format query precedence over Accept header update boxo to ipfs/boxo#1074 and gateway-conformance tests to ipfs/gateway-conformance#252 for testing IPIP-523 changes where ?format= URL query parameter takes precedence over Accept HTTP header * chore: update boxo and gateway-conformance for IPIP-523 testing - boxo@9aeb0c835899fedb4f886040a4ccf3ba09fd97d4 - gateway-conformance@c82a2a9bc79303e0f07216a80cf454ef2a1e042c * chore(ci): switch to gateway-conformance@v0.9 * chore: update boxo with IPIP-523 changes * chore: update boxo to main after ipfs/boxo#1074 merge * docs: add gateway-conformance v0.9 to changelog
swap to last working snapshots until libp2p project fixes their websites Right now its either broken due to Fleek shutting down: https://web.archive.org/web/20260201232813/https://docs.libp2p.io/concepts/pubsub/overview/ or URLs changed by being moved to new place at: https://web.archive.org/web/20260204171903/https://libp2p.io/guides/circuit-relay/ we can undo/switch to real URLs once things stabilize for a few months
* feat(config): Import.* and unixfs-v1-2025 profile implements IPIP-499: add config options for controlling UnixFS DAG determinism and introduces `unixfs-v1-2025` and `unixfs-v0-2015` profiles for cross-implementation CID reproducibility. changes: - add Import.* fields: HAMTDirectorySizeEstimation, SymlinkMode, DAGLayout, IncludeEmptyDirectories, IncludeHidden - add validation for all Import.* config values - add unixfs-v1-2025 profile (recommended for new data) - add unixfs-v0-2015 profile (alias: legacy-cid-v0) - remove deprecated test-cid-v1 and test-cid-v1-wide profiles - wire Import.HAMTSizeEstimationMode() to boxo globals - update go.mod to use boxo with SizeEstimationMode support ref: https://specs.ipfs.tech/ipips/ipip-0499/ * feat(add): add --dereference-symlinks, --empty-dirs, --hidden CLI flags add CLI flags for controlling file collection behavior during ipfs add: - `--dereference-symlinks`: recursively resolve symlinks to their target content (replaces deprecated --dereference-args which only worked on CLI arguments). wired through go-ipfs-cmds to boxo's SerialFileOptions. - `--empty-dirs` / `-E`: include empty directories (default: true) - `--hidden` / `-H`: include hidden files (default: false) these flags are CLI-only and not wired to Import.* config options because go-ipfs-cmds library handles input file filtering before the directory tree is passed to kubo. removed unused Import.UnixFSSymlinkMode config option that was defined but never actually read by the CLI. also: - wire --trickle to Import.UnixFSDAGLayout config default - update go-ipfs-cmds to v0.15.1-0.20260117043932-17687e216294 - add SYMLINK HANDLING section to ipfs add help text - add CLI tests for all three flags ref: ipfs/specs#499 * test(add): add CID profile tests and wire SizeEstimationMode add comprehensive test suite for UnixFS CID determinism per IPIP-499: - verify exact HAMT threshold boundary for both estimation modes: - v0-2015 (links): sum(name_len + cid_len) == 262144 - v1-2025 (block): serialized block size == 262144 - verify HAMT triggers at threshold + 1 byte for both profiles - add all deterministic CIDs for cross-implementation testing also wires SizeEstimationMode through CLI/API, allowing Import.UnixFSHAMTSizeEstimation config to take effect. bumps boxo to ipfs/boxo@6707376 which aligns HAMT threshold with JS implementation (uses > instead of >=), fixing CID determinism at the exact 256 KiB boundary. * feat(add): --dereference-symlinks now resolves all symlinks Previously, resolving symlinks required two flags: - --dereference-args: resolved symlinks passed as CLI arguments - --dereference-symlinks: resolved symlinks inside directories Now --dereference-symlinks handles both cases. Users only need one flag to fully dereference symlinks when adding files to IPFS. The deprecated --dereference-args still works for backwards compatibility but is no longer necessary. * chore: update boxo and improve changelog - update boxo to ebdaf07c (nil filter fix, thread-safety docs) - simplify changelog for IPIP-499 section - shorten test names, move context to comments * chore: update boxo to 5cf22196 * chore: apply suggestions from code review Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com> * test(add): verify balanced DAG layout produces uniform leaf depth add test that confirms kubo uses balanced layout (all leaves at same depth) rather than balanced-packed (varying depths). creates 45MiB file to trigger multi-level DAG and walks it to verify leaf depth uniformity. includes trickle subtest to validate test logic can detect varying depths. supports CAR export via DAG_LAYOUT_CAR_OUTPUT env var for test vectors. * chore(deps): update boxo to 6141039ad8ef switches to ipfs/boxo@6141039 changes since 5cf22196ad0b: - refactor(unixfs): use arithmetic for exact block size calculation - refactor(unixfs): unify size tracking and make SizeEstimationMode immutable - feat(unixfs): optimize SizeEstimationBlock and add mode/mtime tests also clarifies that directory sharding globals affect both `ipfs add` and MFS. * test(cli): improve HAMT threshold tests with exact +1 byte verification - add UnixFSDataType() helper to directly check UnixFS type via protobuf - refactor threshold tests to use exact +1 byte calculations instead of +1 file - verify directory type directly (ft.TDirectory vs ft.THAMTShard) instead of inferring from link count - clean up helper function signatures by removing unused cidLength parameter * test(cli): consolidate profile tests into cid_profiles_test.go remove duplicate profile threshold tests from add_test.go since they are fully covered by the data-driven tests in cid_profiles_test.go. changes: - improve test names to describe what threshold is being tested - add inline documentation explaining each test's purpose - add byte-precise helper IPFSAddDeterministicBytes for threshold tests - remove ~200 lines of duplicated test code from add_test.go - keep non-profile tests (pinning, symlinks, hidden files) in add_test.go * chore: update to rebased boxo and go-ipfs-cmds PRs * docs: add HAMT threshold fix details to changelog * feat(mfs): use Import config for CID version and hash function make MFS commands (files cp, files write, files mkdir, files chcid) respect Import.CidVersion and Import.HashFunction config settings when CLI options are not explicitly provided. also add tests for: - files write respects Import.UnixFSRawLeaves=true - single-block file: files write produces same CID as ipfs add - updated comments clarifying CID parity with ipfs add * feat(files): wire Import.UnixFSChunker and UnixFSDirectoryMaxLinks to MFS `ipfs files` commands now respect these Import.* config options: - UnixFSChunker: configures chunk size for `files write` - UnixFSDirectoryMaxLinks: triggers HAMT sharding in `files mkdir` - UnixFSHAMTDirectorySizeEstimation: controls size estimation mode previously, MFS used hardcoded defaults ignoring user config. changes: - config/import.go: add UnixFSSplitterFunc() returning chunk.SplitterGen - core/node/core.go: pass chunker, maxLinks, sizeEstimationMode to mfs.NewRoot() via new boxo RootOption API - core/commands/files.go: pass maxLinks and sizeEstimationMode to mfs.Mkdir() and ensureContainingDirectoryExists(); document that UnixFSFileMaxLinks doesn't apply to files write (trickle DAG limitation) - test/cli/files_test.go: add tests for UnixFSDirectoryMaxLinks and UnixFSChunker, including CID parity test with `ipfs add --trickle` related: boxo@54e044f1b265 * feat(files): wire Import.UnixFSHAMTDirectoryMaxFanout and UnixFSHAMTDirectorySizeThreshold wire remaining HAMT config options to MFS root: - Import.UnixFSHAMTDirectoryMaxFanout via mfs.WithMaxHAMTFanout - Import.UnixFSHAMTDirectorySizeThreshold via mfs.WithHAMTShardingSize add CLI tests: - files mkdir respects Import.UnixFSHAMTDirectoryMaxFanout - files mkdir respects Import.UnixFSHAMTDirectorySizeThreshold - config change takes effect after daemon restart add UnixFSHAMTFanout() helper to test harness update boxo to ac97424d99ab90e097fc7c36f285988b596b6f05 * fix(mfs): single-block files in CIDv1 dirs now produce raw CIDs problem: `ipfs files write` in CIDv1 directories wrapped single-block files in dag-pb even when raw-leaves was enabled, producing different CIDs than `ipfs add --raw-leaves` for the same content. fix: boxo now collapses single-block ProtoNode wrappers (with no metadata) to RawNode in DagModifier.GetNode(). files with mtime/mode stay as dag-pb since raw blocks cannot store UnixFS metadata. also fixes sparse file writes where writing past EOF would lose data because expandSparse didn't update the internal node pointer. updates boxo to v0.36.1-0.20260203003133-7884ae23aaff updates t0250-files-api.sh test hashes to match new behavior * chore(test): use Go 1.22+ range-over-int syntax * chore: update boxo to c6829fe26860 - fix typo in files write help text - update boxo with CI fixes (gofumpt, race condition in test) * chore: update go-ipfs-cmds to 192ec9d15c1f includes binary content types fix: gzip, zip, vnd.ipld.car, vnd.ipld.raw, vnd.ipfs.ipns-record * chore: update boxo to 0a22cde9225c includes refactor of maxLinks check in addLinkChild (review feedback). * ci: fix helia-interop and improve caching skip '@helia/mfs - should have the same CID after creating a file' test until helia implements IPIP-499 (tracking: ipfs/helia#941) the test fails because kubo now collapses single-block files to raw CIDs while helia explicitly uses reduceSingleLeafToSelf: false changes: - run aegir directly instead of helia-interop binary (binary ignores --grep flags) - cache node_modules keyed by @helia/interop version from npm registry - skip npm install on cache hit (matches ipfs-webui caching pattern) * chore: update boxo to 1e30b954 includes latest upstream changes from boxo main * chore: update go-ipfs-cmds to 1b2a641ed6f6 * chore: update boxo to f188f79fd412 switches to boxo@main after merging ipfs/boxo#1088 * chore: update go-ipfs-cmds to af9bcbaf5709 switches to go-ipfs-cmds@master after merging ipfs/go-ipfs-cmds#315 --------- Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com>
…11090) * feat(gateway): IPIP-0524 Gateway.AllowCodecConversion config option Wire up boxo's AllowCodecConversion config to control codec conversion behavior per IPIP-0524. When false (default), the gateway returns 406 Not Acceptable if the requested format doesn't match the block's codec. Clients should fetch raw blocks (`?format=raw`) and convert client-side. Ref: ipfs/specs#524 Ref: ipfs/boxo#1077 Ref: ipfs/gateway-conformance#254 * chore: update boxo for improved 406 codec conversion error boxo now returns an actionable hint when codec conversion is rejected: suggests fetching raw block with ?format=raw and converting client-side. * chore: bump boxo and gateway-conformance to v0.10 * docs: add IPLD Logical Format note to AllowCodecConversion
* feat: add swarm addrs autonat command fixes #11171 by adding a self service way to debug public reachability with autonat * test: add test for ipfs swarm addr autonat command * docs: add ipfs swarm addrs autonat to changelog * test: update failing test * fix: swarm addrs autonat bugfixes and cleanup - fix help text to show capitalized reachability values (Public, Private, Unknown) matching actual output from network.Reachability.String() - default Reachability to "Unknown" instead of empty string when the host interface assertion fails - extract multiaddrsToStrings and writeAddrSection helpers to deduplicate repeated conversion loops and text formatting blocks --------- Co-authored-by: Marcin Rataj <lidel@lidel.org>
IPIP-499's unixfs-v1-2025 profile uses 1MiB chunks. with --raw-leaves=false, protobuf wrapping pushes blocks slightly over 1MiB. the previous 1MiB SoftBlockLimit rejected these blocks on dag import. raise SoftBlockLimit to 2MiB to match the bitswap spec, which requires implementations to support blocks up to 2MiB. - raise SoftBlockLimit to 2MiB per the bitswap spec - update error messages and help text - bump boxo to main with ipfs/boxo#1101 (raised ChunkSizeLimit/BlockSizeLimit, 256-byte overhead budget) - update sharness tests for 2MiB boundary - add test/cli boundary tests for block put, dag put, dag import, ipfs add (raw and wrapped leaves), and bitswap exchange including regression tests for the libp2p message size hard limit
…11183) go-libp2p v0.47.0 [1] returns addresses from all interfaces when listening on 0.0.0.0, improving VPN/WireGuard/Tailscale connectivity. note impact for users without the server profile. [1] https://github.com/libp2p/go-libp2p/releases/tag/v0.47.0
* docs: loud deprecation of badger v1 datastore badger v1 (go-ds-badger) has not been maintained by its upstream maintainers for years and has known bugs (startup timeouts, shutdown hangs, fd exhaustion). make the deprecation loud and unavoidable: - print ERROR log line and bordered stderr message on every daemon start when a badger-based datastore is detected - mark badgerds and badgerds-measure init profiles as DEPRECATED - update docs/datastores.md and docs/config.md with migration guidance - add changelog highlight for v0.40 badger v1 support will be removed later in 2026. part of #11186 * docs(changelog): link badger deprecation to #11186
* feat: update to Go 1.26 replace deprecated httputil.NewSingleHostReverseProxy (Director) with ReverseProxy.Rewrite, switch math/rand to math/rand/v2 in production code, update Dockerfile base image. * fix test to accept response with HTTP status of 307 and 308 where 302 and 301 are expected --------- Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com>
* chore: apply go fix modernizers from Go 1.26
automated refactoring: interface{} to any, slices.Contains,
and other idiomatic updates.
* feat(ci): add `go fix` check to Go analysis workflow
ensures Go 1.26 modernizers are applied, fails CI if `go fix ./...`
produces any changes (similar to existing `go fmt` enforcement)
* feat(cli): add --human and --sort-size to ipfs ls
- --human (-H): SI human-readable sizes in text output (humanize.Bytes)
- --sort-size (-S): sort directory entries by size, largest first
- Validation: --sort-size + --stream and --sort-size + --size=false errors
- Unit tests for formatSize and sort helpers
- CLI integration tests for both flags
- Changelog highlight in v0.44
* test(ls): make sort tests fail when sorting breaks
The tests covering --sort-size could not detect a broken feature. The
unit tests copied the comparator into the test body and sorted with their
own copy, so they passed regardless of what ls.go did. The CLI tests named
each fixture after its size, which left alphabetical order and size order
in agreement, so every ordering subtest passed even with --sort-size
disabled outright.
- extract lsLinkByName and lsLinkBySize so the tests exercise shipped code
- point the unit tests at those two functions
- name fixtures so their alphabetical order disagrees with their sizes
- pin the real directory behaviour: UnixFS directories carry no Filesize,
so they sort as 0, tie with empty files and break by name rather than
landing strictly last
* fix(ls): return 400 not 500 for bad flag combos
Passing --sort-size together with --stream or --size=false made
/api/v0/ls answer 500 Internal Server Error, telling API clients the
server had broken when the caller had simply combined flags that cannot
work together. Clients that retry on 5xx would retry a request that can
never succeed.
cmds.ErrClient maps to 400, matching how the rest of core/commands
reports caller mistakes. CLI output is unchanged.
* docs: fix --human size examples and JSON claims
The help for --human advertised "1K 234M 2G", which no kubo command has
ever printed. All three use humanize.Bytes, so the real output is SI with
a space: 1.2 kB, 234 MB, 2.0 GB. The stale example was copied into
'ipfs ls' from the two commands that already carried it, so correct all
three together.
- ls, repo stat, bitswap stat: examples now match real output
- drop the claim that --enc=json reports bytes. On the CLI, 'ipfs ls'
has a PostRun that prints the text table whatever --enc says, so no
JSON is produced there at all. Only the /api/v0/ls response is JSON,
and that part is true
- directories have no UnixFS Filesize, so they sort as 0 and tie with
empty files. They do not land strictly last, so stop saying they do
- changelog: a #### highlight with a TOC entry, kept short and pointing
at 'ipfs ls --help' for the details
- format sizes with strconv.FormatUint rather than fmt.Sprintf("%d")
---------
Co-authored-by: Marcin Rataj <lidel@lidel.org>
* chore: bump go-libp2p for sorted confirmed addrs Pin the head commit of libp2p/go-libp2p#3526: AutoNAT V2's ConfirmedAddrs returned unsorted buckets, and removeNotInSource silently dropped webrtc-direct from the confirmed set. Switch to a master pseudo-version once the PR merges. * fix: keep browser transports in provider records Provider records sent to HTTP routers were narrowed to the addresses AutoNAT V2 confirmed reachable, which silently dropped the only two transports a browser can dial: the AutoTLS /tls/ws address and webrtc-direct. A publicly reachable node was invisible to browser and Helia clients that found it through a delegated router, even though ipfs id and the DHT both advertised those addresses. AutoNAT only ever sees listen addresses, so the AutoTLS address, which the AddrsFactory synthesizes afterwards, can never reach the confirmed set. webrtc-direct does get confirmed, but go-libp2p loses it again in getConfirmedAddrs, which feeds an unsorted slice to a scan that assumes sorted input; that one is fixed upstream in libp2p/go-libp2p#3526. Announce host.Addrs() instead, the same set identify sends to peers and the DHT already publishes, narrowed to globally routable addresses so loopback and LAN entries stay out of a public index. Nodes with no public address keep announcing what they have, so LAN-only setups pointing at a local router are unaffected. - core/node/libp2p/routingopt.go: drop the ConfirmedAddrs branch from httpRouterAddrFunc, filter host.Addrs() with manet.IsPublicAddr; AppendAnnounce is emitted exactly once and does not count toward the public-addr check - core/commands/swarm_addrs_autonat.go: take over the BasicHost compile-time assertion, now the only ConfirmedAddrs consumer Fixes #11369 * docs: move highlight to v0.43 and scope it The fix ships in v0.43, so the entry moves out of v0.44.md and in next to the other browser-retrieval highlights. - names the config it applies to: Routing.Type=custom with a provide method on an HTTP router. Default auto provides over the DHT alone and is unaffected, since constructDefaultHTTPRouters leaves ProvideRouter as a noop. - cites bitsocial.net, which runs libp2p in the browser and uses delegated routers to find peers, as the app the gap broke
Telemetry reports to https://telemetry.ipshipyard.dev by default again, as it did through v0.42. This is a stopgap: it holds while the devgrant support window is active. Every way to turn telemetry off now lives in one place, so ending it later is a config change or a one-line diff rather than a rewrite. - endpoint is a linker-settable var: building with -ldflags "-X ...telemetry.defaultEndpoint=" yields a binary with no destination, which collects nothing and writes no identifier - DO_NOT_TRACK is honored, ranking between IPFS_TELEMETRY and the config Mode, so one variable opts a machine out of every tool - a collector answering 410 Gone retires itself: the node drops its identifier and never sends there again, on this run or a later one, which stops reporting across deployed nodes without a release - first-run notice names DO_NOT_TRACK next to the Kubo switches - docs/telemetry.md leads with how to disable, including at build time - AGENTS.md: telemetry opt-outs are a rule, not a courtesy - changelog: drop the opt-in highlight, v0.43 ships no telemetry change
* fix(routing): keep peers found before the timeout The DHT returns the closest peers it reached together with the context error when a lookup runs past its deadline. We dropped both, so any lookup slower than the routing server's per-request timeout came back as HTTP 500 with nothing in it, indistinguishable from a lookup that found no peers at all. Return what we have, and only error when the set is empty. * test: use local dht swarm for routing v1 test GetClosestPeers joined the public Amino DHT with real bootstrap peers, so the assertions depended on a CI runner reaching bootstrap.libp2p.io from a cold repo. When it could not, the test retried for five minutes and failed; ten such failures since v0.42.0, every one green on re-run. Bootstrap from the harness's in-process DHT peers instead, which the provider tests already use and this one predates. The window drops from five minutes to sixty seconds because there is no longer anything slow to wait for, and passing runs go from tens of seconds to under one. * test: stop handing out ports the kernel reuses NewRandPort binds port zero, notes the number, closes the socket and hands the number to the caller, which leaves a window for anything else on the machine to take it. The number also came from the ephemeral range, the same pool every outgoing connection draws from, and the CLI suite opens a lot of those. Both TestP2PForeground tunnel subtests died on "bind: address already in use" for a server the test binds itself. - NewTCPListener hands back the bound listener, closing that window for callers that listen in-process - ports for daemons we spawn now come from below the ephemeral range, so an outgoing connection cannot land on one * test: sync gc tests to the adder, not the clock TestAddGCLive asserted that gc had not started yet, but the only thing it waited for was the first file's output event. Between that event and the adder reaching the next file there is a gap, and the adder hands the pin lock to a waiting gc at exactly that boundary, so on a loaded runner gc really had started and the assertion was right to fail. Wrap the pipe so the test learns when the adder is inside the hanging file, and poll GCRequested instead of sleeping 100ms to know gc is queued. TestAddMultipleGCLive gets the same treatment for its two sleeps: too short there means gc never gets the lock and the test waits out its five second timeout instead. * test: move watched file in atomically os.WriteFile creates the file and fills it in two steps, and ipfswatch adds whatever is on disk when the create event wakes it. Catch it between the two and it adds an empty file, so the CID the test pulls out of the log reads back as nothing. Stage the file outside the watched directory and rename it in, which the watcher sees as one event for a file that is already complete. * test(sharness): poll the daemon request log The test backgrounded "ipfs log tail", slept 100ms and expected the daemon to be listing the request. The daemon only sees it once the client has started up and connected, which on a loaded runner takes longer than that, and then both the active and the inactive assertion fail together because the entry never appears at all. Poll for each state instead. The extra requests that polling makes push the daemon closer to the point where it drops finished entries from the log, so keep them with "diag cmds set-time" first. * test(sharness): drop stale peer count check The connect case opened by re-asserting that the previous case had left zero peers connected. Disconnecting is not permanent: the DHT keeps the other node in its routing table and re-dials it on any refresh, so that count is only true for as long as nothing else runs. What this case is named for, connecting with a bare /p2p/ address, is still covered by the connect itself and the peer count after it. * test(fuse): mount one node at a time Every parallel subtest does identical setup before mounting, so they all reach the mount together and around twenty setuid fusermount helpers open /dev/fuse inside the same instant. One occasionally comes back with a bare exit status 1. Take a lock for the mount call itself, which the subtests only hold for tens of milliseconds. Also report the failure instead of panicking: a panic failed all 37 tests in the package and left daemons behind, and the daemon's stderr, where fusermount says what actually went wrong, was captured and then thrown away. * test: compare cat output byte for byte The payload is 100 random bytes and the comparison ran through Trimmed(), which strips one trailing newline. Roughly one run in 256 ends in 0x0a and loses it. * test: wait for the fast-provide log line The daemon writes the line before it answers the RPC, but the test reads a buffer that a goroutine fills by copying the daemon's stderr, and that copy can still be behind when the command returns. Wait for the line rather than assuming it has landed. * test: allow for ipns republish mid-test A minute after the daemon starts, the republisher re-signs every key and publishes it again, giving the same value a new signature and expiry. The test captured one PUT body and compared it byte for byte with what routing returned, so a run slow enough to straddle that minute compared the first record against the second. Keep every record the mock is sent and require that routing's answer is one of them, which is what the assertion was reaching for. * fix(examples): turn off mdns in library example The example connects its two nodes by address, but left mDNS on, so local discovery could connect them first. A connection opened while a node is still being built is invisible to that node's bitswap, which only learns about connections made after it registers its notifier, and with no routing configured there is nothing to fall back on. The final fetch then waited forever and the test died on its two minute timeout with no clue why. Turning mDNS off makes the explicit dial the only way the two can meet, and keeps the example off the reader's LAN. Alongside that: - connectToPeers returns dial errors instead of logging and continuing into a fetch that cannot succeed - the example's own deadline now fits inside the test budget, so a stall names the step that hung - CommandContext so a hung child does not outlive the test * ci: make helia-interop job resilient Seven failures since v0.42.0 came from this job's setup rather than from any incompatibility. It installs whatever @helia/interop published last, and upstream shipped three packages in a row whose test config does not work from inside node_modules; a GitHub blip took out the rest. - find the compiled specs and pass them to aegir, instead of patching the config upstream ships into node_modules and grepping its text - pin node to a major: setup-node resolves an lts/ alias through a GitHub manifest with no retry and no fallback, and newer node rejects a flag aegir sets unconditionally - retry the registry lookup and fail loudly, since the old one-liner could not fail and left an empty cache key behind - install the exact version the cache key names, and only save the cache once the install is known good - drop the playwright apt packages, unused since this job stopped running browser targets
(cherry picked from commit c8ed996)
* chore: bump go-libp2p for sorted confirmed addrs Pin the head commit of libp2p/go-libp2p#3526: AutoNAT V2's ConfirmedAddrs returned unsorted buckets, and removeNotInSource silently dropped webrtc-direct from the confirmed set. Switch to a master pseudo-version once the PR merges. * fix: keep browser transports in provider records Provider records sent to HTTP routers were narrowed to the addresses AutoNAT V2 confirmed reachable, which silently dropped the only two transports a browser can dial: the AutoTLS /tls/ws address and webrtc-direct. A publicly reachable node was invisible to browser and Helia clients that found it through a delegated router, even though ipfs id and the DHT both advertised those addresses. AutoNAT only ever sees listen addresses, so the AutoTLS address, which the AddrsFactory synthesizes afterwards, can never reach the confirmed set. webrtc-direct does get confirmed, but go-libp2p loses it again in getConfirmedAddrs, which feeds an unsorted slice to a scan that assumes sorted input; that one is fixed upstream in libp2p/go-libp2p#3526. Announce host.Addrs() instead, the same set identify sends to peers and the DHT already publishes, narrowed to globally routable addresses so loopback and LAN entries stay out of a public index. Nodes with no public address keep announcing what they have, so LAN-only setups pointing at a local router are unaffected. - core/node/libp2p/routingopt.go: drop the ConfirmedAddrs branch from httpRouterAddrFunc, filter host.Addrs() with manet.IsPublicAddr; AppendAnnounce is emitted exactly once and does not count toward the public-addr check - core/commands/swarm_addrs_autonat.go: take over the BasicHost compile-time assertion, now the only ConfirmedAddrs consumer Fixes #11369 * docs: move highlight to v0.43 and scope it The fix ships in v0.43, so the entry moves out of v0.44.md and in next to the other browser-retrieval highlights. - names the config it applies to: Routing.Type=custom with a provide method on an HTTP router. Default auto provides over the DHT alone and is unaffected, since constructDefaultHTTPRouters leaves ProvideRouter as a noop. - cites bitsocial.net, which runs libp2p in the browser and uses delegated routers to find peers, as the app the gap broke (cherry picked from commit 4b7a94f)
Telemetry reports to https://telemetry.ipshipyard.dev by default again, as it did through v0.42. This is a stopgap: it holds while the devgrant support window is active. Every way to turn telemetry off now lives in one place, so ending it later is a config change or a one-line diff rather than a rewrite. - endpoint is a linker-settable var: building with -ldflags "-X ...telemetry.defaultEndpoint=" yields a binary with no destination, which collects nothing and writes no identifier - DO_NOT_TRACK is honored, ranking between IPFS_TELEMETRY and the config Mode, so one variable opts a machine out of every tool - a collector answering 410 Gone retires itself: the node drops its identifier and never sends there again, on this run or a later one, which stops reporting across deployed nodes without a release - first-run notice names DO_NOT_TRACK next to the Kubo switches - docs/telemetry.md leads with how to disable, including at build time - AGENTS.md: telemetry opt-outs are a rule, not a courtesy - changelog: drop the opt-in highlight, v0.43 ships no telemetry change (cherry picked from commit f64c770)
* fix(routing): keep peers found before the timeout The DHT returns the closest peers it reached together with the context error when a lookup runs past its deadline. We dropped both, so any lookup slower than the routing server's per-request timeout came back as HTTP 500 with nothing in it, indistinguishable from a lookup that found no peers at all. Return what we have, and only error when the set is empty. * test: use local dht swarm for routing v1 test GetClosestPeers joined the public Amino DHT with real bootstrap peers, so the assertions depended on a CI runner reaching bootstrap.libp2p.io from a cold repo. When it could not, the test retried for five minutes and failed; ten such failures since v0.42.0, every one green on re-run. Bootstrap from the harness's in-process DHT peers instead, which the provider tests already use and this one predates. The window drops from five minutes to sixty seconds because there is no longer anything slow to wait for, and passing runs go from tens of seconds to under one. * test: stop handing out ports the kernel reuses NewRandPort binds port zero, notes the number, closes the socket and hands the number to the caller, which leaves a window for anything else on the machine to take it. The number also came from the ephemeral range, the same pool every outgoing connection draws from, and the CLI suite opens a lot of those. Both TestP2PForeground tunnel subtests died on "bind: address already in use" for a server the test binds itself. - NewTCPListener hands back the bound listener, closing that window for callers that listen in-process - ports for daemons we spawn now come from below the ephemeral range, so an outgoing connection cannot land on one * test: sync gc tests to the adder, not the clock TestAddGCLive asserted that gc had not started yet, but the only thing it waited for was the first file's output event. Between that event and the adder reaching the next file there is a gap, and the adder hands the pin lock to a waiting gc at exactly that boundary, so on a loaded runner gc really had started and the assertion was right to fail. Wrap the pipe so the test learns when the adder is inside the hanging file, and poll GCRequested instead of sleeping 100ms to know gc is queued. TestAddMultipleGCLive gets the same treatment for its two sleeps: too short there means gc never gets the lock and the test waits out its five second timeout instead. * test: move watched file in atomically os.WriteFile creates the file and fills it in two steps, and ipfswatch adds whatever is on disk when the create event wakes it. Catch it between the two and it adds an empty file, so the CID the test pulls out of the log reads back as nothing. Stage the file outside the watched directory and rename it in, which the watcher sees as one event for a file that is already complete. * test(sharness): poll the daemon request log The test backgrounded "ipfs log tail", slept 100ms and expected the daemon to be listing the request. The daemon only sees it once the client has started up and connected, which on a loaded runner takes longer than that, and then both the active and the inactive assertion fail together because the entry never appears at all. Poll for each state instead. The extra requests that polling makes push the daemon closer to the point where it drops finished entries from the log, so keep them with "diag cmds set-time" first. * test(sharness): drop stale peer count check The connect case opened by re-asserting that the previous case had left zero peers connected. Disconnecting is not permanent: the DHT keeps the other node in its routing table and re-dials it on any refresh, so that count is only true for as long as nothing else runs. What this case is named for, connecting with a bare /p2p/ address, is still covered by the connect itself and the peer count after it. * test(fuse): mount one node at a time Every parallel subtest does identical setup before mounting, so they all reach the mount together and around twenty setuid fusermount helpers open /dev/fuse inside the same instant. One occasionally comes back with a bare exit status 1. Take a lock for the mount call itself, which the subtests only hold for tens of milliseconds. Also report the failure instead of panicking: a panic failed all 37 tests in the package and left daemons behind, and the daemon's stderr, where fusermount says what actually went wrong, was captured and then thrown away. * test: compare cat output byte for byte The payload is 100 random bytes and the comparison ran through Trimmed(), which strips one trailing newline. Roughly one run in 256 ends in 0x0a and loses it. * test: wait for the fast-provide log line The daemon writes the line before it answers the RPC, but the test reads a buffer that a goroutine fills by copying the daemon's stderr, and that copy can still be behind when the command returns. Wait for the line rather than assuming it has landed. * test: allow for ipns republish mid-test A minute after the daemon starts, the republisher re-signs every key and publishes it again, giving the same value a new signature and expiry. The test captured one PUT body and compared it byte for byte with what routing returned, so a run slow enough to straddle that minute compared the first record against the second. Keep every record the mock is sent and require that routing's answer is one of them, which is what the assertion was reaching for. * fix(examples): turn off mdns in library example The example connects its two nodes by address, but left mDNS on, so local discovery could connect them first. A connection opened while a node is still being built is invisible to that node's bitswap, which only learns about connections made after it registers its notifier, and with no routing configured there is nothing to fall back on. The final fetch then waited forever and the test died on its two minute timeout with no clue why. Turning mDNS off makes the explicit dial the only way the two can meet, and keeps the example off the reader's LAN. Alongside that: - connectToPeers returns dial errors instead of logging and continuing into a fetch that cannot succeed - the example's own deadline now fits inside the test budget, so a stall names the step that hung - CommandContext so a hung child does not outlive the test * ci: make helia-interop job resilient Seven failures since v0.42.0 came from this job's setup rather than from any incompatibility. It installs whatever @helia/interop published last, and upstream shipped three packages in a row whose test config does not work from inside node_modules; a GitHub blip took out the rest. - find the compiled specs and pass them to aegir, instead of patching the config upstream ships into node_modules and grepping its text - pin node to a major: setup-node resolves an lts/ alias through a GitHub manifest with no retry and no fallback, and newer node rejects a flag aegir sets unconditionally - retry the registry lookup and fail loudly, since the old one-liner could not fail and left an empty cache key behind - install the exact version the cache key names, and only save the cache once the install is known good - drop the playwright apt packages, unused since this job stopped running browser targets (cherry picked from commit 8ee3dcb)
Requested by @sanderpick in #11298 (comment)
Release v0.43.0
# Conflicts: # version.go
Merge release v0.43.0
Signed-off-by: weifanglab <weifanglab@outlook.com> Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com>
Rewrites the security policy around a single reporting route and links it from the issue template chooser. Reports go to security@ipfs.io by email. - SECURITY.md: drops the stale go-ipfs issue link and the "heavy development" framing, adds what to do if nobody replies - ISSUE_TEMPLATE: security contact link, listed first - ipfs.tech/help returned 404, replaced with docs.ipfs.tech/community
The key expired in 2018 and the pgp.mit.edu lookup it pointed at returns 503, so anyone following it hits a dead end twice. No successor key is published for security@ipfs.io on any keyserver, so the notes point at the repository security policy instead. Seeding the init docs changes their directory CID, so the constants the tests assert against move with it.
* docs: guide for running on low-memory devices Add docs/production/low-memory.md with tuning for 8 GiB devices like Raspberry Pi: GOMEMLIMIT, systemd MemoryHigh/MemoryMax, and Provide.DHT.MaxWorkers, plus how to verify with cgroup memory pressure. - README.md, docs/README.md: link the guide - docs/config.md: link from lowpower profile and ResourceMgr notes, caution against setting Swarm.ResourceMgr.MaxMemory too low - misc/systemd: commented example limits pointing at the guide * docs: announcement sizing and dht client advice Extend the low-memory guide with measured announcement rates, the 48h record expiry floor, and a worked 10M-CID example (MaxWorkers=6, Interval=32h). Advise client-only routing (autoclient), disabling the AutoNAT and relay services, and warn against enabling the accelerated DHT client on constrained hardware. - docs/environment-variables.md: document GOMEMLIMIT - docs/config.md: broaden lowpower profile pointer to the guide
* ci: remove self-hosted runners * test: fix zsh completion test on hosted runners The test runs compinit in a non-interactive zsh. When a directory in fpath is writable by group or others, compinit asks a question. There is no terminal to answer it, so compinit aborts and the test fails. GitHub-hosted runner images have such a directory. compinit -i skips insecure directories without asking. The test checks kubo's generated completion script, not the permissions of the host's zsh directories. --------- Co-authored-by: Marcin Rataj <lidel@lidel.org>
…11428) * fix(key): restrict overwritten key exports to owner-only permissions Signed-off-by: questfever <questfever@outlook.com> * fix(atomicfile): temp file leak and name limit Both problems surface through `ipfs key export`, which now writes through this helper, but they affect every caller: config writes, repo migrations and `ipfs update`. - remove the temporary file when the rename fails, so one holding private key material is not left next to the target - keep the ".tmp-" prefix and the random suffix within the 255 byte file name limit, so a target with a long name can still be written * fix(key): route key export by target type Choosing the write path with os.Lstat treated /dev/stdout, /dev/stderr and /dev/fd/N as plain paths, because they are symlinks into /proc/self/fd, so the atomic write failed on targets that worked before. Decide by what the path resolves to, and state the whole contract in the command help. - regular file or nothing yet: written to a temporary file and renamed over the target, following symlinks, including one whose target does not exist yet - character device or pipe: streamed in place, confirmed on the open descriptor and without O_TRUNC, so a path swapped for a regular file can neither receive the key nor be emptied - anything else: refused, naming the path - errors name the file the user asked for, and the temporary file is flushed before the rename * fix(key): stop export landing on the wrong file An export could replace a file the target symlink does not point at. resolveSymlink joined a relative link target onto the path as typed, so ".." collapsed lexically. Where a parent component was itself a symlink, the join named a file outside the directory the link resolves to: the key was renamed over that file, and the intended target was never written. The link's parent is now resolved with filepath.EvalSymlinks before the join. * test(key): drop umask dependency in export test os.WriteFile applies the umask, so under umask 077 the fixture was created 0600 and the check that a failed export leaves the file at 0644 failed for reasons unrelated to the code under test. The CLI test already chmods for the same reason. --------- Signed-off-by: questfever <questfever@outlook.com> Co-authored-by: Marcin Rataj <lidel@lidel.org>
* fix(fuse): report a link count of 1 st_nlink was left at 0, which POSIX gives an inode with no remaining names, so tools can read a live file as one on its way out. Neither IPFS nor MFS has hard links. Directories report 1 as well, which keeps GNU find from trusting a subdirectory count and skipping entries. * fix(ipns): fill attrs in key directory lookups The /ipns root answers lookups for its key directories and alias symlinks itself, and the reply carried zeroed attributes. Every later lookup refreshed the kernel's cache with the same zeroes, so the Getattr that would have corrected them never ran and a key directory showed up as d--------- with no link count. * fix(fuse): give mounts stable inode numbers go-fuse numbers any node left with a zero StableAttr.Ino itself, and picks a new number every time. The kernel drops a mount's dentries once EntryTimeout expires, so a file nobody touched came back from the next lookup under a different st_ino, and programs that compare file identity over time read that as the file being replaced. vim abandons a save with "E949: File changed while writing", which is what made the FUSE CI job flaky once it moved to slower runners and the save started crossing the one second cache boundary. - /ipfs takes the number from the multihash digest inside the CID, so the same content is one object whichever path reaches it; inline CIDs hash the whole CID instead, their digest being the content itself - /ipns and /mfs allocate per mount from a counter keyed by parent and name, retired on unlink, rmdir and rename so a name that is created again is never handed a removed entry's number - writable nodes carry a generation of their own, so go-fuse builds a fresh node per lookup instead of reusing one bound to an *mfs.File that boxo has since replaced - mount points report inode 1 instead of 0, and readdir reports the same numbers as stat * fix(fuse): tell two CIDs apart on /ipfs go-fuse matches a lookup against the nodes it already holds by the whole of StableAttr, so two entries that agree on it are served as one object. The inode number alone is 63 bits, and two CIDs can end up sharing one, by chance or by choice. Reading the second one then returned the first one's bytes. A dag-pb CID and the raw CID of the same block hit this every time, because the number ignored the codec. The number and the generation now both come from a hash of the codec and the multihash, so it takes a match on 128 bits to confuse two entries. The mount also sets FirstAutomaticIno instead of relying on go-fuse's default, so the range it keeps for itself stays where the code says it is. * fix(fuse): stat /ipfs entries we cannot read A stat of a child whose block is missing, or of one in a codec this mount does not decode, read UnixFS metadata that was never loaded and panicked. go-fuse does not recover a panic in its serve loop, so this took the whole daemon down. Neither case is exotic: a dag-cbor object linked from a UnixFS directory needs no missing blocks at all. A lookup that cannot read the block now fails instead of building an entry from it, and a block that is not UnixFS is reported as a file of its own size. * fix(fuse): keep a file a rename cannot move `mv /ipns/<key>/f /ipns/f` unlinked the source before finding out that the /ipns root holds no files of its own, then failed with EINVAL and left the file nowhere. The destination is now checked before anything is written. * fix(fuse): store a moved file where it landed A rename across directories only wrote the source directory back, so the file's new name lived in memory until something else flushed it, and a daemon that stopped first lost the file. The destination is written back first, so an interrupted rename leaves the file under both names rather than under neither. * fix(fuse): keep the inode number over a rename Both names gave up their inode numbers on rename, so a moved file came back about a second later as a different file, which is the problem this branch fixes everywhere else. A moved directory was worse: its entries are keyed by its number, so the whole subtree was renumbered and the old keys were left behind until unmount. The number now moves with the entry. Nothing else has to change to make that safe: go-fuse cannot hand back the old node anyway, because every node gets a generation of its own. The comments that credited the renumbering for it were wrong. * test(fuse): check listings and stat agree Each mount fills in the inode number of a directory entry separately from the one it reports to stat, and nothing compared the two. Tools read whichever is cheaper for them. * docs(config): warn about writing to a mounted mfs `ipfs files` writes to the same tree as the /mfs and /ipns mounts without the mount knowing, so the two can lose each other's writes.
* fix(fuse): keep a rename's writes go-fuse hands the kernel's existing node to the new name once Dir.Rename returns, and that node still held the MFS handle the rename had unlinked. MFS treats such a handle as gone: a write through it was accepted and then dropped, so `mv a b` followed by a write to `b` read back the old contents a second later, once the entry cache expired. A directory was worse. Creating a file in one that had just been renamed flushed through the dead handle, which carried the name the rename had moved away from, so the new file was lost and the old directory came back for good. Each node now reaches its MFS handle through an atomic, and a rename points the moved node at the entry that exists afterwards. Entries the kernel had already looked up underneath a renamed directory hang off the handle it was reached through, so the walk follows them down; it covers what the kernel is holding, not the whole tree. Invalidating the entry instead was tried and does not work: the kernel processes FUSE_NOTIFY_INVAL_ENTRY while holding the parent inode lock, so notifying from inside Rename deadlocks, and notifying asynchronously still loses most of the writes it races. Left unfixed: a write through a file descriptor held open across the rename still goes to the descriptor opened from the old handle. * fix(fuse): refuse to replace a non-empty directory A rename may only overwrite a directory that is empty. MFS removes a directory and everything under it without complaint, so `mv -T src dst` took dst's contents with it and reported success. Rmdir already had the check; Rename now makes it too, before it unlinks anything. The check for an absent destination also goes through errors.Is now. It compares against a sentinel that boxo returns bare today, and the cost of that changing is the source file, which by then has been unlinked. * fix(fuse): read /ipfs blocks we cannot decode A UnixFS directory can link to a block of any codec. stat reports one the mount cannot decode as a file the size of the block, but every read of it failed, because the read path went looking for a UnixFS DAG that is not there. A size stat promises has to be a size reads deliver, so serve the block itself. * fix(fuse): list a directory with a missing block One child whose block is not held locally failed the whole listing, and with an errno the caller could make nothing of: ipld.ErrNotFound has no mapping, so it arrived as ENOSYS. The readable entries are worth having, so report the one that is missing with no type and let a stat of it say what is wrong. * fix(fuse): report the CID the path used The ipfs.cid xattr answered with a CID the caller had never seen. A lookup rebuilds the node by decoding the block, which drops the version and codec of the path it came from, so a v1 dag-pb path reported its v0 form. Keep the CID the entry resolved to and report that. The changelog entry also covers the rename check from the commit before it, which landed without one. * test(fuse): make the rename tests catch their bugs TestRenameOntoNamespaceRoot read the file back through the mount, which answers from the entry the kernel still has cached and so succeeds whether or not the rename took the file away. It passed against the bug it was written for. Ask MFS instead. The dirent helper also loops on a record length it never checks, which would spin rather than fail if the kernel ever sent zero. * test(ipns): settle the repo path before mounting TestStatfs assigned Root.RepoPath once the server was already serving, and Statfs reads it from a FUSE handler goroutine, so `go test -race ./fuse/...` reported a data race on every run. The mfs and readonly tests already settle it before their mount; do the same here.
Modeled on libp2p/go-libp2p-kad-dht#1292. Lead with refusal mechanics that outrank the task prompt, add hard rules for wire compatibility, tracking, and dependency-introduced behavior, and a User Agency section: no resource appropriation, no silent CID injection, mandatory opt-outs, and a policy-change bar (standalone AGENTS.md PR naming a DRI) for monetization or cryptocurrency integration. Cite the tests that pin each rule where they exist.
Both failure modes showed up while running the FUSE suite locally, and both outlive the test process that caused them. A mount left behind by a killed or deadlocked test blocks every later stat() of it in uninterruptible D state, which hangs find /tmp, the tmpwatch cron, and backups. Releasing those processes needs a connection abort; the unmount alone does not do it. A symlink loop left in /tmp sends fuser into an unbounded readlink() walk, so the daily tmpwatch --fuser cron pins a core and /tmp stops being cleaned.
Shipyard's IPFS maintenance work ends on 2026-09-30: https://ipshipyard.com/blog/2026-the-end-of-ipfs-at-shipyard/
* feat: Ipfs-Uri gateway header (IPIP-548) Bump boxo to the IPIP-548 implementation (ipfs/boxo#1209): gateway responses carry a canonical percent-encoded Ipfs-Uri header and stop sending the deprecated X-Ipfs-Path, which cannot represent every UnixFS file name. - sharness: CORS expects Ipfs-Uri exposed, X-Ipfs-Path gone - gateway-conformance CI pinned to the IPIP-548 test suite (ipfs/gateway-conformance#301) until a release ships - reverse-proxy doc and v0.44 changelog updated Refs ipfs/specs#548 * feat: opt-in Gateway.DeprecatedXIpfsPath Expose boxo's opt-in for the legacy X-Ipfs-Path response header as a kubo config flag, default off. Unsafe: the legacy value cannot represent every UnixFS file name, so it must only be used to facilitate migration to Ipfs-Uri, and even when enabled the header is still skipped when the value would include non-ASCII byte sequences. Refs ipfs/specs#548 * ci: bump gateway-conformance pin * ci: gateway-conformance v0.14 * chore: boxo with IPIP-548 from boxo/main * docs: Ipfs-Uri changelog in v0.43.1 * docs: assemble v0.43.1 changelog Move the v0.44 highlights and dependency lines into a new v0.43.1 section, and add the missing entry for owner-only key exports (#11428). v0.44.md returns to an empty skeleton. * docs: note boxo v0.42.2 fixes in v0.43.1
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.