feat(cmcd): CTA-5004 v1 spec completion - #3
Open
bbetter173 wants to merge 31 commits into
Open
Conversation
Replace AampCMCDCollector's dependency on the CMCDHeaders class family in support/aampmetrics (libmetrics.so) with per-media-type state owned by the collector and a serialization layer (AampCMCDSerializer) inside libaamp. Header output is byte-identical, pinned by characterization tests that assert exact header strings through the public API.
Complete the CMCD v1 key set and make serialization CTA-5004 conformant:
- v - CMCD version. Constant 1.
- sf - Streaming format: "d" (DASH) / "h" (HLS) / "s" (Smooth), mapped
from the session MediaFormat at tune; omitted for progressive and
other formats.
- st - Stream type: "l" (live) or "v" (VOD). Fed by the HLS/DASH fragment
collectors on each manifest parse; omitted until first parse.
- cid - Manifest URL, stripped of query/fragment params so auth tokens are
not leaked.
- pr - Playback rate; supports trickplay and normal playback rates.
Refreshed on every request (NotifySpeedChanged alone is unreliable
at tune); emitted whenever not 1, including 0 ("not playing").
- d - Duration of object in ms. Parsed from the manifest: the segment
duration the fragment collector supplies with each download. Media
segments only; init instances never carry d.
- dl - Deadline in ms: buffered duration / |playback rate|, so trick and
slow rates scale the drain. Omitted when not playing (pr=0) or the
buffer level is unknown.
- mtp - Measured throughput in kbps, from the ABR manager's bandwidth
estimator (fed by real download samples, so valid for
single-profile streams too); omitted until samples exist.
- su - Startup urgency: set on initial tune or active rebuffer, recomputed
per request; also set on init-segment requests using the parent
track's buffer state.
- rtp - Requested max throughput: 2x the encoded bitrate (br), per the
spec's client-discretion clause (matches ExoPlayer's default).
Omitted when br is unknown.
Breaking Changes:
- Subtitle object type corrected from the undefined "s" to "c".
- String keys (sid/cid/nor/nrr) are quoted with backslash escaping
- keys sort alphabetically within each header
- unavailable standard keys are omitted instead of being reported as 0.
- Per-key rounding per spec: bl/dl round to the nearest 100 ms, mtp/rtp to
the nearest 100 kbps
- bs is latched: a starvation seen since the prior request is reported once
on the resumption request, then cleared.
Breaks from CTA-5004:
- The com.comcast-* vendor keys keep their deployed presence rules
unchanged (fb/lb on every media segment request, dns when available,
values unrounded) so downstream consumers that assume they always exist
are unaffected; only nor/nrr changed in that block (quoted, and nor is
omitted when the next URL is unknown).
bbetter173
force-pushed
the
feat/cmcd-cta5004-v2
branch
from
July 20, 2026 04:27
908d8cb to
d76f070
Compare
Close the two remaining deliberate deviations from CTA-5004: - v is no longer emitted. The spec says the version SHOULD only be sent when not equal to 1, and this implementation is CMCD v1. - nor is now expressed relative to the request it rides on, per the spec, instead of carrying the absolute next-object URL. CMCDGetHeaders takes the current request URL (defaulted, so the manifest path is unchanged) and GetFile supplies it. A next object in the same directory yields the bare segment name; the same origin yields an absolute-path reference (a valid RFC 3986 relative reference); a different origin or unknown current URL cannot be expressed relatively, so the key is omitted per the optional-key rule. Values originate from playlist URLs and are already URL-encoded, so no re-encoding is applied. nrr is unaffected. The only remaining intentional deviation is the always-present vendor custom keys, kept for downstream compatibility. L1: new collector case covers all four relativization outcomes; existing expectations updated for the dropped v token. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Port the XiOne (armv7 NEON hard-float) cross-compile tooling onto the cmcd-cta5004 line: cmake/xione-armhf.cmake, cmake/FindEthanLog.cmake, and the CMAKE_MODULE_PATH append that lets find_package() see them. CMakeLists.txt already gates find_package(EthanLog REQUIRED) behind CMAKE_USE_ETHAN_LOG, but this branch's lineage never carried the module itself. Ported from 23a462e on feat/xione-build-tooling rather than merged: that branch descends from xione-release (the Sky 8.4 line) and this one from the sprint line, so their merge-base is ancient and a merge would drag in unrelated history. Only the two cmake modules and the module-path append are wanted here. Kept this branch's CMAKE_CXX_STANDARD 17 (the xione-release line was on 14). cmake/xione-armhf.cmake carries one addition over the version on feat/xione-build-tooling: a force-append of the multiarch -L onto the linker flag cache vars. The _INIT seeding it replaces is a no-op when those vars arrive pre-set on the cmake command line, which is what the Bazel build in the following commit does. See the comment at that block. The device-harvest sysroot scripts from that branch are deliberately not ported; the Bazel build that follows assembles a sysroot from pinned Debian packages instead, with no device in the loop.
The 12 RDK downstream patches for bitmovin libdash stable_3_0, byte-for-byte as scripts/install_libdash.sh fetches them from meta-rdk-ext. Vendoring them lets a hermetic build apply the patch set without cloning meta-rdk-ext at build time, which the Bazel libdash cross-build in the following commit relies on. Ported from 0e09706 on bazel/libdash-patches, which descends from the xione-release line rather than this one.
Add a Bazel module that drives this repo's own CMake build (via rules_foreign_cc and cmake/xione-armhf.cmake) to cross-compile AAMP for the Sky XiOne — armv7 NEON hard-float, glibc 2.35, new C++11 string ABI. CMake stays the interface RDK maintains and the only supported host build; Bazel exists here solely so CI can produce release artifacts reproducibly, with no device and no developer machine in the loop. Nothing in a normal CMake build is affected: //:aamp is target_compatible_with //bazel/constraints:xione-stb, so it is skipped in wildcards and only builds under --platforms=//bazel/platforms:xione. No device in the build. scripts/build-xione-sysroot.sh assembles its sysroot partly by harvesting ABI-exact .so files from a live XiOne over SSH, which cannot run in CI and is not reproducible. //third_party/xione_sysroot replaces that with 36 sha256-pinned Debian armhf .debs for headers/.pc/link stand-ins, ethanlog and libdash built from pinned source, and JSC reduced to a generated link stub. Ships all nine .so files the build emits, not just libaamp.so and libaampjsbindings.so. With CMAKE_INBUILT_AAMP_DEPENDENCIES=ON this branch builds middleware/ and support/ as separate shared libraries and libaamp.so carries a DT_NEEDED on five of them, so the two headline libraries alone cannot load on a device — its stock AAMP predates that split. The older xione-release line absorbed the middleware into a monolithic libaamp.so, which is why the earlier prototype only collected two. Two cross-build quirks are handled in //BUILD.bazel with comments: JSC_INCDIR is pre-seeded because find_path() cannot resolve it under CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY once pkg-config has already returned a sysroot-absolute hint, and the host pkg-config is preferred because bootstrapping it from source compiles a vendored glib that is invalid C23. Verified: builds nine ELF32 ARM hard-float libraries with a self-contained internal DT_NEEDED closure. Not yet verified on a device — see XIONE-BUILD.md.
Add CI for the cross-build and a release pipeline that publishes the libraries
as a GitHub Release asset, so downstream repos pin a tarball by URL + sha256
rather than reproducing the cross-build themselves.
tools/release/stage_artifacts.sh does the packaging, and two steps in it are
load-bearing rather than cosmetic:
- patchelf --set-rpath '$ORIGIN' on every .so. rules_foreign_cc bakes the build
sandbox path into DT_RPATH; left unpatched a bundled libaampjsbindings.so
silently loads the device's stock /usr/lib/libaamp.so and defeats the swap
with no error, just the wrong code running. Doing it in the producer makes
every published artifact correct by construction.
- the cross strip from the Bootlin toolchain, not the host's. Host binutils
cannot read ELF32 ARM at all ("Unable to recognise the architecture").
The tarball unpacks to usr/lib/*.so + manifest.json, laid out to drop straight
into a .wgt tree. manifest.json records the AAMP commit, the Bootlin toolchain
sha256 and the deb manifest hash, because the artifact is a function of all of
them and no upstream version identifies it.
xione-ci.yml asserts what actually breaks on a device rather than just that the
build exits zero: every library is ELF32 / EM_ARM / hard-float, the library count
matches, and the AAMP-internal DT_NEEDED closure is satisfied. It also runs the
staging script so an RPATH or packaging regression surfaces in CI rather than at
release time. Paths-scoped, since this is a full cross-compile of the player.
Release tags are prefixed xione-v* so they cannot collide with upstream
rdkcentral tags, and the version is packaging semver rather than an AAMP
version. Notes are passed via body_path: a `body:` YAML block scalar is not
shell-interpolated, so a $(...) placed there publishes literally — shipping
release notes whose stated checksums are absent while telling consumers to copy
hashes from them.
Runs on ubuntu-latest with no remote cache or cloud credentials: the Bootlin
cross-compiler is x86_64-host-only so the runner must be x86_64, and a warm
build is fast enough that actions/cache over ~/.cache/bazel is sufficient.
Verified locally end to end: produces a 2.1M tarball of nine stripped ELF32 ARM
libraries, all with RPATH=$ORIGIN, all assertions passing.
Add a `xione-build-<shortsha>` tag mode alongside `xione-v<semver>`, so a specific commit can be published and pinned downstream before anything is blessed as a version. Commit builds go out as GitHub prereleases and their tags are disposable — delete both when the commit is superseded. Promoting one is just tagging the same commit `xione-v<semver>` and repointing the consumer; the build, staging and notes are identical either way, and manifest.json records the inputs regardless. Driven by tag push rather than workflow_dispatch because dispatch only works once the workflow is on the default branch, and this needs to run from a feature branch. The dispatch path stays for later.
Removes the host `ar` and `nm` dependencies from the repository rules, and adopts
webrtc-sim's `.bazelrc` hermeticity block. Aligning with webrtc-sim was the driver:
its own repo rules contain zero `ctx.which`/`ctx.execute` calls, so these escapes
were a divergence from house style rather than an instance of it.
`nm` was the one that mattered. jsc_deb ran the HOST nm over an ELF32 ARM `.so` to
derive the JSC link-stub symbol list, which only works if the distro built binutils
with the ARM target in BFD — host `strip` already fails outright on these same files
("Unable to recognise the architecture of the input file"). On a leaner CI image the
*fetch* would break, which is a confusing place to fail. The symbol list is now
derived in a build action (//third_party/jsc:wpe_webkit_syms) using the cross nm
from the Bootlin toolchain we already fetch: hermetic and target-correct. It also
now fails the build on an empty symbol list, which previously would have produced a
stub exporting nothing — links fine, dies on the device.
`ar` turned out to be unnecessary rather than replaceable: Bazel's own extractor
understands the `ar` container a `.deb` is, so `ctx.extract` unwraps it and a second
extract unpacks the payload. Confirmed by probe before relying on it. This also
retires the last argument in the xione_sysroot README for preferring
rules_distroless, which had already been prototyped and rejected because its
transitive per-package closures drag Debian's glibc into a sysroot the cross
toolchain owns.
Verified: with shims that make host `ar` and `nm` exit 127 placed first on PATH, a
`clean --expunge` cold build succeeds, and both outputs are byte-identical to the
previous implementation — the 4257-entry sysroot tree (paths, file hashes and
symlink targets) and wpe-webkit.syms (sha256 5ef685e9…, 168 symbols). `bazel build
//...` on a host still skips the cross targets.
Still host-dependent, and deliberately left: `ln` for the multiarch fixups
(ctx.symlink produces absolute links, which do not survive rules_foreign_cc copying
the tree — webrtc-sim documents the same constraint), `mkdir`/`cp` for header
staging, host pkg-config, and the ~14 coreutils rules_foreign_cc's generated script
calls. That last one is the ceiling: this build will always assume a POSIX userland.
Completes the removal of host-tool escapes from the fetch phase: `deb_sysroot` and `jsc_deb` now use only ctx.download/extract/read/file, so `grep -rn 'ctx.which\| ctx.execute' bazel/` is empty. That matches webrtc-sim, whose own repo rules have never contained either call. The multiarch fixups move into assemble_sysroot rather than being rewritten. A repository rule cannot create a *relative* symlink without shelling out to `ln` (ctx.symlink produces absolute links, which do not survive rules_foreign_cc copying the tree — webrtc-sim documents the same constraint at bazel/extensions/zig.bzl), but assemble_sysroot is already a build action that creates relative symlinks with `ln -sfn`, so the fixups just belong there. No new dependency, and the mechanism was already in the codebase. Their order inside that action is load-bearing and commented as such: the include mirror must run against the deb tree alone, before the glibc overlay, because it only links names that do not already exist and glibc contributes colliding usr/include entries (sys/, bits/, gnu/). Running it after glibc would silently create fewer links. Header staging uses ctx.read/ctx.file instead of `cp`. The eight classic JSC C-API headers are pure US-ASCII, so the round-trip is byte-exact. The assembled sysroot tree legitimately CHANGES, in a strictly better direction: 11 relative directory symlinks (usr/include/curl -> arm-linux-gnueabihf/curl) replace 22 absolute per-file symlinks that previously pointed into the local Bazel cache. Those absolute links were an artifact of Bazel expanding a repo-side directory symlink into individual files, and are exactly the cross-runner fragility webrtc-sim warns about. Verified. Fetch phase: with shims making host ar/nm/ln/cp/mkdir all exit 127 first on PATH, a `clean --expunge` cold fetch succeeds. Compiled output: every library's size, dynamic-symbol count and DT_NEEDED list is unchanged, and all nine dynamic symbol-name sets hash identically to the CI-published artifact built before this change. Release staging and its assertions still pass; `bazel build //...` on a host still skips the cross targets. Remaining host dependencies are all build-phase and deliberately out of scope: the coreutils assemble_sysroot itself calls, host pkg-config, and the ~14 coreutils rules_foreign_cc's generated script uses. That is the ceiling while CMake drives the compile.
Completes the host-tool removal for everything this repo writes. `assemble_sysroot` was the last holdout: it shelled out to `ln -sfn` for the multiarch include mirror, the `libz.so` alias and the unversioned `.so` aliases, on the reasoning that only `ln` can make a *relative* symlink. That is true of repository rules, but not of build actions — `ctx.actions.declare_symlink` produces exactly that, and Bazel stages such an artifact as the symlink itself, so `cp -a` (already used throughout the action) places it with its link text intact. Which names get mirrored now comes from the deb filegroup at analysis time rather than from a shell loop over the assembled tree. Same 11 links, but the set is visible in the analysis graph, and the load-bearing ordering becomes a property of the inputs: the mirror must consider the deb tree alone, since it only links names that tree lacks (`openssl` is the live case) and the glibc overlay contributes colliding usr/include entries. Verified with a positive control, because the obvious way to test this is wrong: `--action_env=PATH` does not reach this action (a `run_shell` action gets an empty env, so PATH is bash's built-in default), and a shim under /tmp is invisible inside the sandbox, which mounts a fresh tmpfs there. With the shim outside /tmp and a PATH pinned into the action, the previous revision fails on `ln` (exit 127) and this one succeeds. Both sysroot trees come out with the same 13 relative links as before, and all nine libraries are unchanged in size, dynamic-symbol count, symbol-set hash and DT_NEEDED. One `ln` remains reachable in a full build and is not ours to remove: rules_foreign_cc's generated script stages its tool binaries with `ln -sf`, and the symlink is load-bearing rather than incidental — CMake locates CMAKE_ROOT by resolving argv[0] back to the real binary, so a *copy* of `cmake` dies with "Could not find CMAKE_ROOT" (confirmed directly). Dropping `ln` from PATH entirely therefore needs CMake out of the picture, not a patch to rules_foreign_cc. XIONE-BUILD.md now says so explicitly instead of listing `ln` among our own actions' coreutils. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The scan cannot run here and never could. It calls an rdkcentral reusable workflow that takes four FOSSID container/host credentials as secrets; those exist on rdkcentral/aamp and cannot be provisioned on a fork. With them absent the secrets resolve to empty strings and the called workflow fails template validation — "Unexpected value ''" at its lines 22-23 — before a single step runs. So the check is red on every PR on this fork, from the first commit onwards, and nothing a PR does can turn it green. A permanently red check that carries no signal trains people to ignore red checks, which is worse than having no check. Guarded on the repository rather than deleted or detriggered: the file, its `on: pull_request` trigger and the secret plumbing stay byte-identical to upstream, so the job still runs there and merges from upstream do not have to re-resolve a deletion. Here it simply reports as skipped. Reverting is deleting one line, which is what should happen if the credentials ever reach the fork. This PR is its own test — Fossid runs on `pull_request`, from the head ref, so the check on it should come back skipped rather than failed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ci(fossid): skip the stateless diff scan on forks
…xione-bazel-build
The toolchain file set CMAKE_{C,CXX}_FLAGS_INIT to
"-I${XIONE_SYSROOT}/include/libdash" and promised, in a comment, that the libdash
include dir was added "to every compile". Neither held. The ABI block further down
re-sets the same two variables as CACHE entries, and creating a cache entry removes
the normal variable of the same name from scope, so the value CMake actually used
was only the -march/-mfpu/-mfloat-abi set. The path was wrong too:
assemble_sysroot stages those headers at usr/include/libdash, not include/libdash.
Dropped rather than corrected. It has never been in effect, and the build has
always resolved libdash headers through the sysroot's default include path plus
CPATH — so making it real now would add an unexercised compile flag to a
device-verified build in order to satisfy a comment. The comment now says that,
and says where to put the -I (with the usr/ prefix) if a consumer ever needs it.
All nine libraries are unchanged by this: same size, dynamic symbol set and
DT_NEEDED, as expected for a flag that was never applied.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…nonical repo name Both cmake() targets spelled the Bootlin toolchain's path as `external/+http_archive+xione_bootlin_toolchain` — Bazel's internal canonical repo-name form, which has already changed shape once (`~http_archive~name`), and which moving the archive behind a module extension would change again. Either way the variable silently points at a directory that does not exist, and the symptom is CMake failing with "CMAKE_C_COMPILER not found", which names nothing near the cause. Derive it from a file inside the toolchain instead: `:nm` is a single-file filegroup at bin/<triple>-nm, so two dirnames give the root. That also retires the two `# buildifier: disable=external-path` suppressions, which existed only to silence this. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
postfix_script collected each of the nine libraries with `find $BUILD_TMPDIR -name $so -type f | head -1`. Two problems with `head -1`: on no match the command substitution is empty and cp fails with "missing destination file operand", which says nothing about which library is missing; and on more than one match — AAMP's CMake does copy libraries around its build tree — which one gets staged is filesystem-order dependent, so one source revision could produce different bytes on different machines with nothing in the log to say why. Require exactly one match per name, and fail with the name and the candidates. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two silent-wrong-result paths in assemble_sysroot, both of which only misbehave once something upstream moves — which is when a clear error is worth most. overlay_files staged the file whose basename matched the destination and otherwise fell back to `tfiles[0]`. A soname bump (libdash.so -> libdash.so.3) or a cmake() layout change makes the match fail, and the fallback then stages an arbitrary sibling — plausibly an include *directory* — at usr/lib/…/libdash.so. Nothing notices until AAMP's link reports "file format not recognized" about a path that looks right. It now requires exactly one match and names the label, destination, expected basename and what the target actually offered. _root_of indexed files[0] with no empty check, so a filegroup resolving to nothing crashed analysis with a bare Starlark "index out of range" naming neither the attribute nor the target. Callers now pass the label. Verified the new fail() fires rather than assuming it: pointing an overlay_files destination at a basename no file carries reports "expected exactly one file named ethanlog-bogus.h, found 0 among [\"ethanlog.h\"]" instead of staging the wrong file. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`sed -i -E` is GNU-only: BSD/macOS sed reads the argument after -i as the backup suffix, so -E becomes the suffix and the expression becomes a filename. Repository rules always run on the local host — even under --config=rbe, which is how the macOS path would otherwise be argued — so a macOS client fetching this toolchain fails inside patch_cmds. Rewrite through a temp file, the same form assemble_sysroot already uses for the identical ld-script fixup. Behaviour is unchanged on Linux: the sysroot's ld scripts get the same `=`-prefixed paths, and all nine libraries build byte-for-byte the same size, symbol set and DT_NEEDED. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The README said the flag is applied "on compile + link". It is compile-only, and deliberately so — the rationale is entirely about how the compile step reports builtin header dirs, and putting it on the link action changed zig link output for no benefit, which is why it was reverted. Someone debugging a zig link would have trusted the README over the code. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`bazel build //...` fetched @xione_bootlin_toolchain — a ~500 MB Linux-x86_64 archive, plus its patch_cmds — on every host, even though all six cross targets are skipped as incompatible. Measured with a fresh output base: the archive appears in external/ after a plain wildcard build, and it is the only cross repo that does (the deb sysroot, JSC, libdash and ethanlog stay lazy). Neither target in this package was gated, and both name a label inside the fetched repo, so wildcard expansion analyses them and the fetch follows. `target_compatible_with` on the toolchain() rule does not help: those constraints describe what the toolchain is *for*, they do not stop the target itself from being analysed. Verified the registration is not the trigger by removing it and re-running — the fetch still happened. On a macOS host this is the one thing that drags a wildcard build into the fetch phase's shell, for an archive nothing on that host can execute. Tagging both `manual` makes the cross build opt-in: `//...` no longer fetches it (verified, fresh output base), while `bazel build //:aamp --platforms=//bazel/platforms:xione` still does and `register_toolchains` resolves the label directly, which `manual` does not affect. All nine libraries remain identical to the device-verified build. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Replaces the `http_archive` + `patch_cmds` with a repository rule that does the same download and the same one ld-script rewrite, in Starlark. The fetch phase now uses no host tools at all, which is what XIONE-BUILD.md has been claiming while patch_cmds ran find, grep and sed. The shell form was also not portable. `sed -i` with no argument is GNU-only — BSD and macOS sed read the following argument as a backup suffix, so `-E` became the suffix and the expression became a filename — and repository rules always run on the local host, even under --config=rbe, so a macOS client failed inside the fetch rather than anywhere informative. Mechanically: `path.readdir()` walks the sysroot breadth-first (Starlark has neither recursion nor `while`, so the walk is a bounded loop that fails rather than half-scanning a deeper tree), and only `*.so` files are read, since an ld script always stands in for a shared library and reading all ~2300 files including multi-megabyte `.a` archives to find a marker would be gratuitous. Exactly one script needs the rewrite here, `usr/lib/libc.so`; `lib/libgcc_s.so` is an ld script too but names its members relatively. The rule fails if it rewrites nothing. An unrelocated script does not error at link time — it silently resolves against the host x86_64 libc — so a toolchain bump that moves or renames the scripts has to be loud at fetch. Verified: both ld scripts come out byte-identical to what sed produced, with the same 0644 mode; the fail() path was exercised by pointing the scan at a directory with no scripts; and all nine libraries are unchanged in size, symbol set and DT_NEEDED. Note this changes the canonical repo name to `+bootlin_toolchain+xione_bootlin_toolchain`, which is exactly the breakage the preceding XIONE_TOOLCHAIN_DIR commit removed — with the old hardcoded `external/+http_archive+…` this commit would have failed with "CMAKE_C_COMPILER not found". Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Removes the last `sed` from the build. assemble_sysroot ended every run with a `find | grep | sed` pass to `=`-prefix absolute GROUP/INPUT paths in the merged tree; with the Bootlin fetch now doing that in Starlark, the only script it still had to fix was our own -lpthreads alias, which deb_sysroot generates. So the rewrite moves into deb_sysroot too, and the shared implementation lands in //bazel/repo_rules:ld_scripts.bzl. Both sysroot sources now guarantee the property at fetch time and the merged tree inherits it — which is strictly better than the old catch-all, because a `.deb` that ships its own ld script (libc6-dev would) is covered without anything downstream having to notice, and neither the fetch nor the assemble action needs a GNU coreutils userland for it. The `*.so`-only filter needs one non-obvious guard: a deb-derived sysroot is full of dangling dev symlinks whose versioned target lives in a runtime package the manifest omits (`libpcre2-32.so`), and `ctx.read` on a dangling symlink raises FileNotFoundException rather than returning empty. Found by the build, not by reading — the Bootlin tree has no such links, so the deb side was the first to hit it. Verified: both trees carry the same two `=`-prefixed scripts as before (Bootlin's usr/lib/libc.so, our usr/lib/arm-linux-gnueabihf/libpthreads.so), all nine libraries are unchanged in size, symbol set and DT_NEEDED, and `sed`, `find` and `grep` now appear in bazel/ only inside comments. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
stage_artifacts.sh built the path to the Bootlin `strip` out of
`external/+http_archive+xione_bootlin_toolchain`, so moving the toolchain to a repo
rule renamed the directory out from under it. The script fails hard when the strip
is missing, which is right — but it never fired, because every runner and every
developer machine that had built before still had the old directory in its Bazel
cache. CI's "Verify release staging" step passed on the very commit that broke it.
A clean build, or a cache miss, is the first thing that would have failed, and the
error ("cross strip not found at …") points at a path rather than at the rename.
Match the directory by suffix instead. The leading `+<rule>+` is Bazel's internal
canonical-repo form and changes whenever the fetch mechanism does — `~http_archive~`
before bzlmod's current scheme, `+http_archive+` until this week — while the
trailing apparent name is ours and stable.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Switches both workflows off `ubuntu-latest`, which was only ever the fallback. Self-hosted ARC runners were registered for this repo specifically — not org-wide, because their docker-in-docker boundary is a concern for public/forked repos — and the instruction that came with that was to select them as `gke-arc-aamp` rather than webrtc-sim's `gke-arc-std` (rillanetwork/core-infra#133). Until now the enablement sat unused. `sudo` is no longer assumed: the ARC image runs as root without it, GitHub-hosted requires it, so the apt step picks whichever exists and the workflows still run on either. That keeps a revert to `ubuntu-latest` a one-line change if the label turns out not to be picked up — the failure mode there is a job that queues forever, which is exactly what happened on rdk-packages before the runners existed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The switch to the self-hosted label was tried and does not work: both XiOne jobs sat `queued` for over 8 minutes with no runner assigned, then were cancelled. That is the same failure rdk-packages hit before its runners existed — a label nobody listens on does not error, it hangs, and it would have left this PR's checks pending forever. rillanetwork/core-infra#133, the PR referenced when the runners were set up, is titled "add rdk-packages ARC runner scale set" — it provisions the scale set for rdk-packages, the repo the build moved *away* from. Whether anything was ever registered for this repo cannot be checked without runner-read permission on it. Keeps the sudo-optional apt step from the previous commit, so adopting the runners later is a one-line change per workflow rather than a re-litigation of the image assumptions. GitHub-hosted is also what was signed off for this in review ("since this is probably a one off thing, I'm ok to use github hosted runners too") and the cross-build completes there in about three minutes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adds //tools/verify:xione_libs_ok, which checks the built libraries are actually
loadable on the device: every one ELF32 / EM_ARM / hard-float, all nine present, and
the AAMP-internal DT_NEEDED closure self-contained. These are the failure modes a
zero exit status misses — a build that silently targeted the wrong architecture, or
lost a middleware library that still links and then fails at dlopen on the box with
nothing reporting why.
The checks already existed, as ~40 lines of bash in xione-ci.yml. In Bazel they are
cached and only re-run when the libraries change, and — the reason I wanted them
here — they can be run locally with one command instead of reading a workflow file to
find out what CI enforces.
A genrule rather than a test, and not by preference: `bazel test
--platforms=//bazel/platforms:xione` fails toolchain resolution, because tests want
an execution platform matching their target platform and this one is an armv7
set-top box ("No matching toolchains found for
@bazel_tools//tools/test:default_test_toolchain_type"). The ways around that are
registering a fake exec platform, or transitioning the libraries into a host-side
test — and the transition has a real cost: it would make the check host-compatible,
so `bazel build //...` would start fetching the cross toolchain and running a
multi-minute build, which the preceding commit deliberately stopped. As a genrule the
check runs in the same configuration as the thing it checks.
Uses the cross readelf for the same reason //third_party/jsc uses the cross nm: the
host's may or may not have the ARM target in BFD.
Verified both directions rather than just the happy path: it reports "ok: 9
libraries…", and with the expected count set to 10 it fails the build with "expected
10 shared libraries, got 9". A host `bazel build //...` still skips it and still
fetches no toolchain.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two changes to the workflow, both about letting Bazel decide what can be reused. **The cache split.** `actions/cache` covered `~/.cache/bazel` wholesale, which also captured the *output_base* — so every run started from a previous run's `external/` tree. That is not a theoretical problem: it is how a renamed repository directory stayed alive across runs and let CI's staging step pass on the very commit that broke it, until a clean build failed locally. Now a user bazelrc points `--repository_cache` and `--disk_cache` at separate directories, cached separately: downloads (36 debs off snapshot.debian.org, ~500 MB toolchain — the genuinely slow part) stay shared, action outputs are keyed on action hashes rather than on a key I wrote by hand, and the output_base is never restored, so analysis always starts clean. The bazelrc lives at $HOME/.bazelrc so nested invocations pick it up too, including the ones inside stage_artifacts.sh. Saves are restricted to pushes. PR runs read the shared entry instead of each writing one and evicting it, which is the same split webrtc-sim's setup uses. **The assertions move to Bazel.** The inline bash is replaced by building //tools/verify:xione_libs_ok, which subsumes it and builds //:aamp on the way. Failures now come from a script that can be run locally. Verified with a fresh output_base and the two caches configured: the build succeeds and the disk cache populates (187 MB, ~3.5k entries), so the bazelrc is in effect for every invocation. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Public fork of rdkcentral/aamp carrying the Bazel build tooling; call that out at the top of README and XIONE-BUILD so readers do not mistake it for upstream or for something production ready.
build(bazel): hermetic XiOne cross-build alongside the CMake build
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Brings AAMP's CMCD output into compliance with CTA-5004 (v1) in three commits: a behaviour-preserving refactor that makes CMCD collection self-contained in libaamp, the spec-compliance and key-set completion on top, and a follow-up closing the last two spec deviations.
support/aampmetricsis untouched — the change is contained entirely within libaamp.Commit 1 — refactor(cmcd): make CMCD collection self-contained in libaamp
AampCMCDCollectornow owns per-media-type state directly and serializes via a newAampCMCDSerializer(namespaceAampCMCD) inside libaamp, replacing its use of theCMCDHeadersclass family from libmetrics.Commit 2 — feat(cmcd): CTA-5004 serialization compliance and full v1 key set
New keys:
sf,st,cid(manifest URL, query/fragment stripped),pr(per-request, emitted when ≠ 1),d(manifest segment duration),dl(= bl / |rate|),mtp(ABR bandwidth estimate),su(tune/seek/rebuffer, incl. init segments),rtp(2× br).Breaking changes:
stocsid/cid/nor/nrr) quoted with backslash escapingbl/dlto nearest 100 ms,mtp/rtpto nearest 100 kbps;br/tb/dstay plain integersbslatched: a starvation since the prior request is reported once, then clearedDeliberate break from CTA-5004: the pre-existing vendor-specific custom keys keep their presence rules unchanged (emitted on media segment requests exactly as before, values unrounded) so downstream consumers that assume they always exist are unaffected.
Commit 3 — fix(cmcd): omit v at its default and emit nor as a relative reference
Closes the two remaining spec deviations from commit 2:
vis no longer emitted (spec: SHOULD only be sent when ≠ 1; this is a v1 implementation).noris now expressed relative to the request it rides on instead of carrying the absolute next-object URL: same directory → bare segment name, same origin → absolute-path reference, otherwise (different origin / current URL unknown) the key is omitted.CMCDGetHeadersgains a defaultedcurrentUrlparameter fed fromGetFile.Tests
L1 under
test/utests/tests/:AampCMCDSerializerTests(quoting, rounding, ordering) andAampCMCDCollectorTestsasserting exact header strings for the full key set — including thebslatch,cidstripping,dlrate scaling, the vendor-key presence contract, and all fournorrelativization outcomes.🤖 Generated with Claude Code