Owner reports: ipctool output, backups and photos in the board catalogue, kept apart from every import - #341
Merged
Merged
Conversation
…ery import A new catalogue entity for what camera owners and AI agents send about a board: ipctool's YAML, optionally a full-flash backup, photos, a boot log and the U-Boot environment. POST /api/v1/reports stores it at once and answers a receipt; nothing is public until `openipc reports publish`. The public copy has the MAC, die ID and cloud ID replaced by keyed hashes, and a backup is served only when its owner sent consent=public. POST /api/v1/boards/identify matches ipctool's output to catalogue boards and stores nothing. A report exists once, on this host, so nothing the board importers do can reach it: - its own tables (migration 016), which no package but internal/reports may name (deploytest); - triggers that refuse UPDATE, DELETE and TRUNCATE unless takedown or unlink stood them down for their own transaction; - links to board models ON DELETE RESTRICT, so a model delete fails instead of cascading; - files content-addressed and written once under /srv/www/shared/owner-reports, not the analytics' shared/reports, which dev serves with autoindex; - survival_test.go, which runs the archive import, every donor snapshot, every vendor-firmware push, the purge and the migrations twice over stored reports and requires them byte-identical. A planted DELETE in the snapshot importer fails it, and so does one that stands the guard down. Each file goes to S3 the night after it arrives, under the boards/ prefix the backup's IAM policy already allows. `openipc reports verify` re-hashes every file nightly. nginx answers the upload on port 80 too, because ipctool on stock firmware has no TLS.
…e file ipctool upload --note on dev stored the note as a 25-byte file: the part name is both a field and a file kind. A part with a filename is a file, one without is a field.
…HTTP for uget
A camera on stock firmware has no curl and no TLS, and uget, the
downloader an owner pastes in over telnet, speaks HTTP on port 80 and
follows no redirects. It cannot fetch GitHub's release links. OpenIPC/
ipctool's release job pushes each build to PUT /api/v1/tools/{name} over
the builds' OIDC check. The service checks it is an ELF for the machine
its name says and installs it atomically under /srv/www/shared/tools.
nginx serves http://openipc.org/ipctool (-mips32, -arm64) on port 80;
over HTTPS /ipctool still redirects to the project on GitHub.
internal/tools/PUSH.md has the contract.
…unt but not fetch `openipc serve --role nfs` answers what busybox's `mount -o nolock openipc.org:/ipctool /tmp/o` asks, with no other option: the portmapper on 111, then MOUNT v1/v3 and NFS v2/v3, over UDP and TCP. Everything that would write is refused as a read-only file system. No dependency. go-nfs is TCP only, with no portmapper, and a stock busybox asks both over UDP. NFS READ over UDP is a reflection amplifier, so a file handle is keyed to the address it was issued to. A spoofer cannot hold one that works from a victim's address, and UDP answers are budgeted per address as well. Tried on the lab Hi3516EV300 (busybox 1.36, kernel 4.9): mount -o nolock mounted over TCP, ipctool ran from the mount, and the md5 matched. Forced proto=udp,mountproto=udp,vers=3 mounted too, with rsize 8192. That kernel has no NFSv2, so v2 is covered by the unit test only. It runs as go-nfs beside production (one portmapper per host), started by deploy.sh prod.
… a board - /cameras/report (en, ru, zh): how a new board reaches the catalogue, from a shell on the camera or from an AI coding agent. The shell route never says curl, because stock firmware has none. uget is pasted in as text (its six release scripts are vendored under /uget/ and picked by the C library ls /lib/ld-* shows), then fetches http://openipc.org/ipctool; an NFS mount is the alternative. - With ?id= the page is the receipt ipctool prints: state, what arrived, who may see each file, and the catalogue board the report most likely is. - A board's panel lists its published reports (GET /api/v1/reports?model=), with identifiers already hashed and a private backup shown only as existing. - The featured-hardware strip's "send us the report" goes here, not to /community.
…ra and report it Served from the bundle at /agents.md, as text/markdown (nginx has no .md type, and octet-stream reads as a download to a fetch tool). Rules first: the owner's camera only, no password guessing, the owner's say-so before a flash read. Then ipctool by uget or NFS, identify, what to collect, the upload and its refusals, the receipt. The upload now refuses a tool field over 200 characters with a 400. The column holds 200, and a longer one was a 500.
…anslated On a 375px screen the page ran off the right edge. The agent's one-line prompt could not wrap, and the grid columns had no min-w-0, so they grew to it. The prompt now wraps. Copy uget and the receipt lookup use the site's button styles. The panel's "app" and "board" labels come from the dictionary (they read "stock app", in English, on every locale).
PR Summary by QodoAdd protected owner reports and stock-firmware ipctool access
AI Description
Diagram
High-Level Assessment
Files changed (79)
|
Code Review by Qodo
1.
|
From the review of #341: - A published note was served unredacted. Migration 018 adds note_public, redacted like the YAML, and only it is served. - Redaction finds a MAC with dots too (02.8f.5c..., Cisco's 028f.5c94.d7e7). - A board with both cloudId and chip-id has both redacted, not the first. - Takedown could delete a file a simultaneous upload of the same bytes had just named. Uploads now put their files in the store inside the insert transaction, under a shared advisory lock per file. Takedown removes each candidate under the exclusive lock, after checking that no row names it. - Uploads racing from one address could pass the daily limit together. The count is taken again under the address's lock in the insert transaction. The test fails with the recount disabled: 20 in against a limit of 10. - The NFS TCP listener served any number of idle connections. It now holds at most 128, closes the rest at once, and allows 15 s to a first call and a minute between calls. - The tools push refuses a dynamically linked build (PT_INTERP): a stock camera has no loader for it. - The receipt's board links keep the reader's language.
openipc-ai
added a commit
that referenced
this pull request
Sep 29, 2026
…cks the upstream - TestInheritedHeaders read only server-level add_header lines indented four spaces, and dev's X-Robots-Tag is indented three. So every dev location that set a header of its own dropped the noindex policy: the ten added by #341 and this PR, and the wizard, explorer, build push and vendor-firmware locations before them. The test now reads both indents, and all fourteen repeat the header. - TestEveryAPIRouteReachesTheService now requires the upstream of the route's role in the vhost's own environment (3002/3003 prod, 3012/3013 dev, or that environment's openipc-route variable). With dev's tools location pointed at 3002 it fails.
openipc-ai
added a commit
that referenced
this pull request
Sep 29, 2026
…ute is routed (#342) * nginx: route /api/v1/tools to the service, and test that every API route is routed The tools push (PUT /api/v1/tools/<name>) and its listing had no location in either vhost. They fell through to @fallback and answered a 302 to the home page, so ipctool's first release push after #341 reached nothing and http://openipc.org/ipctool stayed empty. They are now proxied to the web role, and the three push addresses are exempt from the datacentre block, because GitHub's runners are Azure. TestEveryAPIRouteReachesTheService picks, for every /api/ route in routes.json, the location nginx would choose (exact, longest prefix, then regex) and requires that it proxies to the service. On master's vhosts it fails for exactly these two routes. * Review: dev locations keep the noindex header; the API route test checks the upstream - TestInheritedHeaders read only server-level add_header lines indented four spaces, and dev's X-Robots-Tag is indented three. So every dev location that set a header of its own dropped the noindex policy: the ten added by #341 and this PR, and the wizard, explorer, build push and vendor-firmware locations before them. The test now reads both indents, and all fourteen repeat the header. - TestEveryAPIRouteReachesTheService now requires the upstream of the route's role in the vhost's own environment (3002/3003 prod, 3012/3013 dev, or that environment's openipc-route variable). With dev's tools location pointed at 3002 it fails.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A new catalogue entity, the owner report: what a camera owner, or an AI coding agent on a bench, sends about a board. That is ipctool's YAML, optionally a full-flash backup, photos, a boot log and the U-Boot environment. Along with it come two ways for stock firmware, which has no curl and no TLS, to get ipctool at all. The ipctool side is OpenIPC/ipctool#225.
Reports (
service/internal/reports, migration 016)POST /api/v1/reportsstores a report at once and answers a receipt. The body is multipart (or ipctool's output as the whole body). Parts are streamed to disk, so a backup can be up to 256 MB. The limit is 10 a day per address.openipc reports publish <id> --model <board>. The public copy has the MAC, die ID and cloud ID replaced by keyed hashes, in the YAML and in any log. A backup is served only if its owner sentconsent=public.GET /api/v1/reports/{id}is the receipt: state, what arrived, and the catalogue board it most likely is.GET /api/v1/reports?model=lists a board's published reports.POST /api/v1/boards/identifymatches ipctool's output to catalogue boards and stores nothing.openipc reports list|show|publish|reject|link|unlink|takedown|verify.Why a Xiongmai or Anjoy import cannot destroy them
internal/reportsmay name in SQL (deploytest/reports_test.go).takedown/unlinkstood them down for their own transaction.ON DELETE RESTRICT: a model delete like migration 008's fails instead of cascading./srv/www/shared/owner-reports, besideBOARDS_ROOT. That directory is notshared/reports, which is the analytics' own and which dev serves with autoindex. The test caught that clash.internal/boards/survival_test.goruns the archive import, every donor snapshot (cctvsp, xiongmai, tehno32, jftech, anjoy), every vendor-firmware push, the purge and the migrations twice over stored reports, and requires them byte-identical. A planted DELETE in the snapshot importer fails it, and so does one that stands the guard down first.boards/owner-reports/…, inside the IAM-allowed prefix).openipc reports verifyre-hashes every file nightly, and RESTORE.md has the step.ipctool for stock firmware (
internal/tools,internal/nfsro, migration 017)PUT /api/v1/tools/{name}over the builds' OIDC check (tools/PUSH.md). The build must be an ELF for the machine its name says, and it is installed atomically.http://openipc.org/ipctool(-mips32,-arm64) on port 80 with no redirect, for uget. Over HTTPS,/ipctoolstill redirects to GitHub.serve --role nfs(thego-nfscontainer, next to production) exports the same files read-only. It answers whatmount -o nolock openipc.org:/ipctool /tmp/oasks: the portmapper on 111, MOUNT v1/v3 and NFS v2/v3 on 2049, over UDP and TCP. There is no dependency; go-nfs is TCP-only and has no portmapper. File handles are keyed to the client's address, so the UDP READ path cannot be used for reflection, and UDP answers are budgeted per address.Site
/cameras/report(en, ru, zh) covers two routes, from a shell (uget pasted as text, whose six release scripts are vendored under/uget/, or NFS) and from an AI agent. With?id=it shows the receipt./agents.mdis the protocol for coding agents, served astext/markdown.Validated on dev (df877ab … 4d79127)
ipctool uploadagainst dev.openipc.org over the internet. The report and a private 16 MB backup were stored, and publish, the redacted public YAML, and a 404 for the private backup (by its API path and by the internal path) all behaved as intended.uget dev.openipc.org/ipctoolon the camera fetched the released binary, md5-identical, and it ran.service/run.sh test, frontend lint, typecheck and test, the bundle check, andcheck-config.sh --seamall pass.After merge
openipc-deploy prod <sha>runs migrations 016 and 017, createsowner-reportsandtools, and startsgo-nfs. Thenpush-nginx.sh --apply, and setOPENIPC_ORG_TOOLS_PUSH=truein OpenIPC/ipctool.