fix(contract): reject control characters in the scan target remote - #233
Open
rohanpoudel2 wants to merge 4 commits into
Open
fix(contract): reject control characters in the scan target remote#233rohanpoudel2 wants to merge 4 commits into
rohanpoudel2 wants to merge 4 commits into
Conversation
`validateCanonicalContract` extracted the URL authority with a regex whose negated class `[^/?#]` matches ASCII tab, LF and CR, and the follow-up `new URL(remote)` check did not catch them either: the WHATWG URL parser strips exactly those three characters from its input before parsing. The value that survived validation was therefore not the value that was validated, and the unsanitized original was handed back to callers as `manifest.scan.target.remote`. A manifest carrying "https://example.com\nhttps://evil.example.net" loaded successfully while `new URL()` saw host "example.comhttps". Reject C0/C1 control characters and Unicode line separators in `remote` before parsing it, mirroring the class `requireModelSafeOutputDir` already applies in runtime.ts for the same reason. Nothing under src/ consumes `remote` today, so this closes a broken guarantee rather than a live exploit: SDK consumers that render or re-emit the field were inheriting a multi-line value that had passed validation. Fixes openai#231
Collaborator
|
Codex Review: Didn't find any major issues. Can't wait for the next one! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
Security review completed. No security issues were found in this pull request. Reviewed commit: Only the user who started this review can view the report in Codex. ℹ️ About Codex security reviews in GitHubThis is an experimental Codex feature. Security reviews are triggered when:
Once complete, Codex will leave suggestions, or a comment if no findings are found. |
The loader-side fix left the producer behind. `_validate_remote` only inspected what `urlsplit` returned, and `urlsplit` strips ASCII tab, LF and CR before parsing exactly like the WHATWG URL parser does, so "https://example.com\nhttps://evil.example.net" split to netloc "example.comhttps:" and sealed cleanly. With the loader now rejecting that value, a scan could complete, seal, and produce a manifest the SDK refuses to load. Reject the same class the loader rejects -- C0 (U+0000-U+001F), DEL and C1 (U+007F-U+009F), U+2028 and U+2029 -- before the `urlsplit` checks, matching the order `validateCanonicalContract` applies in src/contract.ts. The new constant names its counterpart there so the two rules stay linked. A probe over both layers now agrees on all 19 inputs: the LF, TAB, CR, U+2028, U+2029, NUL, DEL and U+0085 smuggling forms are rejected on both sides, and plain https, trailing .git, punycode and Unicode IDN hosts, explicit :443, a non-default port, a mixed-case host, git+ssh:// and ssh://, an IPv6 literal and a percent-encoded path are still accepted on both sides. The regression test in tests-ts/contract.test.ts drives the bundled `_validate_remote` and `loadContract` over the same list. Deliberately leaving the manifest schema pattern alone. Its "^(?![^:/?#]+://[^/?#]*@)[^?#]+$" does match control characters, but the schema is a published contract frozen at schemaVersion 1.0: tightening the pattern would reject manifests that older validators accepted, and the exact pattern string is additionally pinned in two safe-pattern allowlists (one in the finalizer, one in src/contract.ts). The validator runs on every seal, so nothing can be sealed through the looser schema.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #231
Problem
validateCanonicalContractvalidatedscan.target.remoteagainst a string that was not the string it stored.The authority is extracted with:
[^/?#]matches ASCII tab, LF and CR. The follow-upnew URL(remote)check does not catch them either, because the WHATWG URL parser strips exactly those three characters from its input before parsing. So a manifest carrying:loaded successfully —
new URL()saw hostexample.comhttps, whileloadContracthanded the original two-line string straight back to callers.Neither surrounding guard closes it: the schema pattern
^(?![^:/?#]+://[^/?#]*@)[^?#]+$also matches newlines, andvalidateParsedJsonchecks only for well-formed Unicode.The producer had the same hole from the other side.
_validate_remoteinfinalize_scan_contract.pyinspected only whaturlsplitreturned, andurlsplitstrips ASCII tab, LF and CR before parsing exactly like the WHATWG parser does, so that same value split to netlocexample.comhttps:and sealed cleanly.Change
One rule, applied on both halves of the contract.
Loader —
sdk/typescript/src/contract.tsrejects C0/C1 control characters and Unicode line separators inremotebefore the authority regex and thenew URL()check run, mirroring the classrequireModelSafeOutputDiralready applies inruntime.tsfor the same reason. That constant is not exported, so it is mirrored locally with a comment naming the source of truth rather than wideningruntime.ts's public surface for one caller.Producer —
_bundled_plugin/scripts/finalize_scan_contract.pyapplies the identical class in_validate_remote, ahead of theurlsplitchecks, in the same order the loader uses. Without this half the loader-side fix would turn a smuggled remote into a scan that completes, seals, and then cannot be loaded at all.The broad class earns its width. Measured against the current code, position matters:
new URL("https://exa<c>mple.com")already threw on 64 of the 67 code points in the class. Only TAB, LF and CR slipped through.new URL()percent-encodes rather than throwing, so all 67 slipped through before this change, and the raw code point stayed in the stored string. The class is load-bearing there, not merely defence in depth.Why not require
href === remoteI tested this and rejected it. The WHATWG serializer legitimately rewrites valid URLs — it strips a default
:443, lowercases mixed-case hostnames, and punycode-encodes internationalized ones — so exact canonical equality would reject real remotes. The bundled producer validatesremoteonly viaurlsplitscheme and netloc checks and imposes no canonical-equality requirement, so it can legitimately emit any of those forms.Why not widen the class further to cover the remaining smuggling characters
Because a character class is the wrong instrument for what is left, and widening it would reject valid remotes.
This change narrows the smuggling alphabet without emptying it. Driving the real
loadContractand the real_validate_remoteover the Problem section's own payload shape, seven non-control characters still seal and load:new URL().hostthe validator sawgithub.laiyagushi.comhttpsgithub.laiyagushi.comhttpsgithub.laiyagushi.com.httpsgithub.laiyagushi.com.httpsgithub.laiyagushi.com.httpsgithub.xn--comhttps-g03dgithub.xn--comhttps-0f7dAdding them to the class is not available: U+3002 is a legitimate IDN label separator.
https://例え。テスト/repo.gitandhttps://例え.テスト/repo.gitboth resolve to hostxn--r8jz45g.xn--zckzah, and the producer accepts both today, so rejecting the ideographic form would reject a valid remote.Closing the remainder needs a structural rule instead — require the authority's port segment, outside any
[...]IPv6 literal, to be digits — which changes the published validation contract on both halves and deserves its own issue and its own argument. Keeping this change to the control-character class keeps it revert-proof and free of false positives.Impact, stated plainly
remoteis not inert once it leaves the contract layer.finalize_scan_contract.pyemits it as SARIFversionControlProvenance[].repositoryUriforgit_revisiontargets, and_bundled_plugin/skills/track-findings/SKILL.mdinstructs the agent to readscan.targetfrom the manifest and "prefer its canonical remote" when resolving a tracking repository. A two-line value that passed validation therefore reached GitHub code scanning and the agent's repository resolution.Nothing under
src/reads the field, so this closes a broken guarantee rather than a live CLI exploit. Credential smuggling remains correctly blocked — any@in the authority is still rejected before URL parsing, and that check is untouched.Upgrade cost is real but narrow. At load, the only verdicts that flip from accept to reject are remotes carrying a C0/C1 control character, U+2028 or U+2029; across a 311-input probe corpus, 73 inputs change verdict and nothing else does. An already-sealed scan carrying such a remote also stops projecting to SARIF, because
_load_sealed_contractre-runs_validate_manifest. No formgit remote get-urlproduces is affected.Verification
Both halves are revert-proof. Run from
sdk/typescript:bun test --timeout 30000 ./tests-ts/contract.test.tscontract.tsfinalize_scan_contract.pyDeleting the loader guard fails
seals and loads the same set of remote URLsonexpect(await loaderAcceptsRemote(remote)).toBe(false)—Expected: false, Received: true— and also fails the table-drivenrejects schema-valid but canonically invalid contract data. Deleting the producer guard fails the parity test onexpect(producer[remote]).toContain("canonical absolute URL")withReceived: "". The parity test drives the bundled_validate_remotein a subprocess andloadContractin-process over the same 19 remotes, so neither side can drift without failing it, and the 11 accepted remotes (plain HTTPS, trailing.git, punycode and Unicode IDN hosts, explicit:443, a non-default port, a mixed-case host,git+ssh://,ssh://, an IPv6 literal, a percent-encoded path) prove the rule cannot pass by rejecting everything.Full suite: 740 pass / 5 skip / 0 fail, 5204 expect() calls, 745 tests across 34 files.
node scripts/generate-models.cjs --checkandtsc --noEmitare clean.prettier --checkreports "All matched files use Prettier code style!".python3 -m py_compileis clean on the changed script;node-ci.ymlruns no Python linter, so that is the available check.Every figure above was measured by running the command, and the residual table in "Why not widen the class further" was measured by driving the real
loadContractand the real_validate_remoteover the payloads rather than by reading the code.