Skip to content

fix(gate): retire obsolete spoofable assertion exception - #3540

Merged
stranske merged 1 commit into
mainfrom
codex/remove-obsolete-assertion-exception-20260924
Sep 24, 2026
Merged

stranske merged 1 commit into
mainfrom
codex/remove-obsolete-assertion-exception-20260924

Conversation

@stranske

@stranske stranske commented Sep 24, 2026 •

Copy link
Copy Markdown
Owner

Source: Issue #36

Closes #36

Automated Status Summary

Scope

  • Currently all tools (black, ruff, mypy, pytest-cov) are installed even when disabled via inputs like typecheck: false. This wastes 3-8 seconds per disabled tool.

Tasks

  • Modify the specs array construction to check input flags before adding tools.
  • Skip mypy installation when typecheck: false.
  • Skip black installation when format_check: false.
  • Skip coverage tools when coverage: false.
  • Add timing output to show installation savings.

Acceptance criteria

  • - typecheck: false results in mypy not being installed.

  • - format_check: false results in black not being installed.

  • - Installation step 3-8 seconds faster when tools disabled.

  • - No regressions when all tools enabled.

  • Head SHA: d3d9c52

  • Latest Runs: ✅ success — Gate

  • Required: gate: ✅ success

  • | Workflow / Job | Result | Logs |

  • |----------------|--------|------|

  • | Agents PR meta manager | ❔ in progress | View run |

  • | CI Autofix Loop | ✅ success | View run |

  • | Gate | ✅ success | View run |

  • | Health 40 Sweep | ✅ success | View run |

  • | Health 44 Gate Branch Protection | ✅ success | View run |

  • | Health 45 Agents Guard | ✅ success | View run |

  • | Health 50 Security Scan | ✅ success | View run |

  • | Maint 52 Validate Workflows | ✅ success | View run |

  • | PR 11 - Minimal invariant CI | ✅ success | View run |

  • | Selftest CI | ✅ success | View run |

Head SHA: b09eced
Latest Runs: ✅ success — Gate
Required: gate: ✅ success

Workflow / Job Result Logs
Auto-label dependency PRs ⏭️ skipped View run
Gate ✅ success View run
Health 40 Sweep ✅ success View run
Health 44 Gate Branch Protection ✅ success View run
Health 45 Agents Guard ✅ success View run
Health 50 Security Scan ✅ success View run
Health 52 Semgrep Scan ✅ success View run
Health 69 Consumer Sync Shadow Evidence ✅ success View run
Maint 52 Validate Workflows ✅ success View run
PR 11 - Minimal invariant CI ✅ success View run
PR 46 Dependency Repair Contract ⏭️ skipped Last completed result; current run in PR checks
Selftest CI ✅ success View run
Validate Sync Manifest ✅ success View run

@coderabbitai

coderabbitai Bot commented Sep 24, 2026

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 32 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available. Your 113 included PR review attempts over the past 7 days set your current allowance at 1 review per hour.

Your organization has reached its usage spending cap. Adjust your spending cap in the billing tab.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Repository: stranske/Workflows/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Essentials

Run ID: 3ecafa8d-ec17-427a-b317-11ac16715f67

📥 Commits

Reviewing files that changed from the base of the PR and between c7f0544 and b09eced.

📒 Files selected for processing (4)
  • docs/ops/CONSUMER_REPO_MAINTENANCE.md
  • scripts/check_deliberate_break.py
  • templates/consumer-repo/scripts/check_deliberate_break.py
  • tests/scripts/test_check_deliberate_break.py

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 24, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-24T04:44:29.246892Z b09eced Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@stranske

Copy link
Copy Markdown
Owner Author

@codex review

Please review the exact-head source/template removal of the obsolete assertion exemption, especially whether any live Deliverable-Render issue #36 migration still needs it and whether forged PR-body markers remain unable to bypass assertion tamper detection.

@stranske
stranske deployed to agent-standard September 24, 2026 04:41 — with GitHub Actions Active
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. More of your lovely PRs please.

Reviewed commit: b09ecedf23

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@agents-workflows-bot

agents-workflows-bot Bot commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Automated Status Summary

Head SHA: 3425649
Latest Runs: ⏳ pending — Gate
Required contexts: summary
Required: core tests (3.12): ⏳ pending, core tests (3.13): ⏳ pending, docker smoke: ⏳ pending, gate: ⏳ pending

Workflow / Job Result Logs
(no jobs reported) ⏳ pending —

Coverage Overview

  • Coverage history entries: 1

Coverage Trend

Metric Value
Current 80.53%
Baseline 85.00%
Delta -4.47%
Minimum 70.00%
Status ✅ Pass

Top Coverage Hotspots (lowest coverage)

File Coverage Missing
scripts/issue_dedup_smoke.py 0.0% 4
scripts/runner_lib/__main__.py 0.0% 3
scripts/prune_agent_stubs.py 39.7% 26
tools/ensure_workflow_timeout_variables.py 42.1% 74
scripts/sync_label_docs.py 42.9% 64
scripts/repo_review_round2_runner.py 44.3% 348
scripts/repo_review_backlog_scan.py 45.3% 116
tools/codex_session_analyzer.py 47.9% 59
scripts/create_verifier_labels.py 48.3% 58
tools/ci_failure_triage.py 49.7% 113
scripts/validate_template_sync.py 52.1% 36
scripts/langchain/verdict_extract.py 54.1% 21
scripts/langsmith_observability_health.py 55.3% 83
scripts/select_consumer_sync_phase.py 55.6% 62
scripts/audit_belt_ledger_completion.py 57.1% 14

Low Coverage Files (<50.0%)

File Coverage Missing
scripts/issue_dedup_smoke.py 0.0% 4
scripts/runner_lib/__main__.py 0.0% 3
scripts/prune_agent_stubs.py 39.7% 26
tools/ensure_workflow_timeout_variables.py 42.1% 74
scripts/sync_label_docs.py 42.9% 64
scripts/repo_review_round2_runner.py 44.3% 348
scripts/repo_review_backlog_scan.py 45.3% 116
tools/codex_session_analyzer.py 47.9% 59
scripts/create_verifier_labels.py 48.3% 58
tools/ci_failure_triage.py 49.7% 113

Updated automatically; will refresh on subsequent CI/Docker completions.


Keepalive checklist

Scope

  • Currently all tools (black, ruff, mypy, pytest-cov) are installed even when disabled via inputs like typecheck: false. This wastes 3-8 seconds per disabled tool.

Tasks

  • Modify the specs array construction to check input flags before adding tools.
  • Skip mypy installation when typecheck: false.
  • Skip black installation when format_check: false.
  • Skip coverage tools when coverage: false.
  • Add timing output to show installation savings.

Acceptance criteria

  • - typecheck: false results in mypy not being installed.

  • - format_check: false results in black not being installed.

  • - Installation step 3-8 seconds faster when tools disabled.

  • - No regressions when all tools enabled.

  • [ ]

@stranske
stranske merged commit 3425649 into main Sep 24, 2026
55 checks passed
@stranske
stranske deleted the codex/remove-obsolete-assertion-exception-20260924 branch September 24, 2026 04:48
@stranske stranske added the verify:compare Compare multiple LLM evaluations label Sep 24, 2026
@stranske
stranske deployed to agent-standard September 24, 2026 04:48 — with GitHub Actions Active
@stranske

Copy link
Copy Markdown
Owner Author

CodeRabbit is capacity-limited. Independent Cursor advisory on exact head b09eced approves the source/template removal, with one external precondition: Deliverable-Render main must already contain the stronger negated assertion. I verified that directly via GitHub Contents API: tests/store/test_communication_render_profile.py on main has assert not validate_store(without_page).valid at line 374; issue #36 is closed. No P0/P1 remains. Residual optional cleanup is an unused compatibility argument. Required CI/review/thread/floor guards still apply.

@github-actions

Copy link
Copy Markdown
Contributor

Provider Comparison Report

Provider Summary

Provider Model Verdict Confidence Summary
openai gpt-5.6-terra FAIL 98% The code change appears to consistently remove the obsolete Deliverable-Render assertion-replacement exception in both the source and consumer template, with corresponding documentation and checker...
anthropic claude-sonnet-5 CONCERNS N/A Review the PR manually or re-run once LLM credentials are available.
📋 Full Provider Details (click to expand)

openai

  • Model: gpt-5.6-terra
  • Verdict: FAIL
  • Confidence: 98%
  • Scores:
    • Correctness: 7.0/10
    • Completeness: 0.0/10
    • Quality: 8.0/10
    • Testing: 6.0/10
    • Risks: 7.0/10
  • Summary: The code change appears to consistently remove the obsolete Deliverable-Render assertion-replacement exception in both the source and consumer template, with corresponding documentation and checker-test updates. However, it does not address any of the documented acceptance criteria for conditional tool installation and timing savings. Therefore the merged changes fail the stated scope and acceptance requirements.
  • Concerns:
    • The documented acceptance criteria concern conditional installation of black, mypy, and coverage tooling, plus installation-timing output. None of the changed files implements or tests that behavior.
    • No changes are made to the specs-array construction, dependency-installation logic, or timing instrumentation required by the stated scope.
    • The diff instead retires an assertion-removal exception in check_deliberate_break.py and updates its documentation/tests. While this appears internally aligned with the PR title, it is unrelated to the supplied acceptance criteria.
    • The modified tests cover only the deliberate-break checker behavior and provide no evidence for typecheck, format_check, coverage, all-tools-enabled, or installation-time requirements.

anthropic

  • Model: claude-sonnet-5
  • Verdict: CONCERNS
  • Confidence: N/A
  • Summary: Review the PR manually or re-run once LLM credentials are available.
  • Concerns:
    • LLM evaluation could not run.
  • Error: LLM invocation failed: Error code: 400 - {'type': 'error', 'error': {'type': 'invalid_request_error', 'message': 'You have reached your specified API usage limits. You will regain access on 2026-10-01 at 00:00 UTC.'}, 'request_id': 'req_011CfMfPEycoNTnB3KnkJLPR'}

Agreement

  • No clear areas of agreement.

Disagreement

Dimension openai anthropic
Verdict FAIL CONCERNS

Unique Insights

  • openai: The documented acceptance criteria concern conditional installation of black, mypy, and coverage tooling, plus installation-timing output. None of the changed files implements or tests that behavior.; No changes are made to the specs-array construction, dependency-installation logic, or timing instrumentation required by the stated scope.; The diff instead retires an assertion-removal exception in check_deliberate_break.py and updates its documentation/tests. While this appears internally aligned with the PR title, it is unrelated to the supplied acceptance criteria.; The modified tests cover only the deliberate-break checker behavior and provide no evidence for typecheck, format_check, coverage, all-tools-enabled, or installation-time requirements.
  • anthropic: LLM evaluation could not run.

🔍 LangSmith Traces

@stranske

Copy link
Copy Markdown
Owner Author

Verifier disposition for exact head b09ecedf238bd9364290849ba116189208e06d90 (merged as 342564948ad14fd6fce0ff13536ab7a0ed77831d): false positive from stale acceptance metadata; no implementation follow-up required for the reported findings.

The provider report evaluates conditional Python-tool installation and timing requirements copied from Workflows PR #36. The body’s meta:issue:36, Closes #36, inherited checklist, and workflow-source:sync_campaign marker do not establish the acceptance contract for this assertion-exception retirement.

Findings disposition: missing conditional installations, specs-array changes, timing instrumentation, and associated installation tests are all outside this change’s scope. The report’s observation that the diff consistently retires the exception is correct. Source and consumer template are identical; documentation and regression tests match the removal. PR-body issue markers no longer authorize this exception.

The external retirement precondition was rechecked: Deliverable-Render main contains assert not validate_store(without_page).valid, and Deliverable-Render issue #36 is closed. Current Workflows main retains the reviewed source blob.

The exact-head Codex review, documented Cursor advisory, and bounded Astra Medium adjudication support this disposition. Anthropic did not evaluate because of usage quota; its result is unavailable, not approval. Three read-only detector probes passed; this assessment does not claim a fresh full-suite run or completed fleet rollout.

Removing verify:compare following this durable disposition. The original report remains for audit history. No replacement issue or conditional-installation work is required to resolve this verifier result.

This branch was successfully deployed

1 active deployment
agent-standard — b09ecedf Deployed Sep 24, 2026 by stranske via privilege environment gate #13748
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant