Repository navigation
fix: redact exec root before = in build script env output - #84
Open
pawelchcki wants to merge 1 commit into
Open
pawelchcki wants to merge 1 commit into
pawelchcki wants to merge 1 commit into
Conversation
Build scripts receive CARGO_ENCODED_RUSTFLAGS containing
`--remap-path-prefix=${pwd}=.`, and some (rav1e, av-scenechange) echo it
back as `cargo:rustc-env`. redact_exec_root only rewrote the exec root when
followed by `/` or `\`, so the absolute per-action sandbox path survived
into _bs.env, giving the crate's Rustc action a different key on every
execution and defeating remote/disk caching on fresh output bases.
Also redact when the exec root is followed by `=`, which keeps the existing
protection against matching sibling paths that share the prefix.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
redact_exec_rootnow also rewrites the exec root when=follows it, as well as/and\.--remap-path-prefix=<exec_root>=.. The test also checks that a sibling path sharing the prefix (/abs/exec_root2/...) is left untouched.Problem
cargo_build_scriptgives build scriptsand the runner expands
${pwd}to the absolute exec root before it runs the script. Some build scripts echo that variable back ascargo:rustc-env, for example rav1e 0.8 and av-scenechange 0.14. On the way back,redact_exec_rootturns--sysroot=<exec_root>/...into${pwd}/.... It leaves--remap-path-prefix=<exec_root>=.alone, because only<exec_root>/and<exec_root>\are matched. As a result,_bs.envends up containing something like:The sandbox directory number changes on every execution. So the crate's
Rustcaction gets a new key every time and can't be served from a remote or disk cache on any fresh output base, on CI or on another machine. Upstream bazelbuild/rules_rust replaces every occurrence of the exec root and doesn't have this problem. It appeared when the replacement was narrowed to avoid matching sibling paths that share the prefix. Matching on a trailing=keeps that protection.How it was found
I built the same tree from clean twice, in two checkouts with different output bases, then diffed the two
--execution_log_json_filelogs. The only actions with the same key but different outputs were theCargoBuildScriptRunactions for rav1e and av-scenechange, and the only difference was the line above. With this change applied as a patch through rules_rs'srules_rust.patch(...), every spawn's action key and outputs matched between the two builds.Testing
exec_root_followed_by_equals_is_redactedtest fails without the fix (--remap-path-prefix=/abs/exec_root=.stays in the output) and passes with it.cargo_build_script_runnerpass, withTEST_TMPDIRset forcargo_manifest_dir::tests::replace_symlinks_in_out_dir.rustfmt --checkis clean.