From b8f030e0c13b6f3c52ff6c91a33d6aafee4c2d7b Mon Sep 17 00:00:00 2001 From: jinjunnn Date: Thu, 30 Jul 2026 22:10:44 -0400 Subject: [PATCH 1/5] fix(verification): correct E7 billing probe gates --- .../2026-07-27-e7-packaged-live/README.md | 138 ++--- .../2026-07-27-e7-packaged-live/probe.ts | 481 ++++++++++++++---- 2 files changed, 471 insertions(+), 148 deletions(-) diff --git a/docs/verification/2026-07-27-e7-packaged-live/README.md b/docs/verification/2026-07-27-e7-packaged-live/README.md index 4ef80ac47dfe..73ebf01f3f47 100644 --- a/docs/verification/2026-07-27-e7-packaged-live/README.md +++ b/docs/verification/2026-07-27-e7-packaged-live/README.md @@ -4,7 +4,7 @@ kind: verification status: active owners: - alpha-code maintainers -last_reviewed: 2026-07-27 +last_reviewed: 2026-07-30 review_after: 2026-10-27 --- @@ -21,15 +21,15 @@ review_after: 2026-10-27 ## 1. 被测件 -| 项 | 值 | -| --- | --- | -| 应用 | `/Applications/alpha-code.app`(`ship:mac` 装机版,非 `dist/` 直跑) | -| 构建时间 | 2026-07-27T21:31:42 -0400(= 2026-07-28T01:31:42Z) | -| 基线 commit | `e578e00ae`(`alpha`,工作树干净) | +| 项 | 值 | +| ------------------------------------- | ------------------------------------------------------------------ | +| 应用 | `/Applications/alpha-code.app`(`ship:mac` 装机版,非 `dist/` 直跑) | +| 构建时间 | 2026-07-27T21:31:42 -0400(= 2026-07-28T01:31:42Z) | +| 基线 commit | `e578e00ae`(`alpha`,工作树干净) | | `sha256(Contents/Resources/app.asar)` | `60589c59c58e44ac0daede93fc7397a8a04365f5345eac4312e205a0d8f48e44` | -| CFBundleShortVersionString | `0.1.2` | -| 引擎版本 | `1.17.13` | -| userData | `~/Library/Application Support/ai.opencode.desktop.dev` | +| CFBundleShortVersionString | `0.1.2` | +| 引擎版本 | `1.17.13` | +| userData | `~/Library/Application Support/ai.opencode.desktop.dev` | `app.asar` 的 sha256 被钉进 `probe.ts`(`PINNED_ASAR_SHA256`)。探针第一件事就是重算它并比对 —— **在错的构建上跑出来的绿是假绿**,这条判据把它挡住。重新打包后必须同时更新此处与 `probe.ts` 的常量。 @@ -58,6 +58,8 @@ eager ajv `new Function` 撞 renderer CSP —— 本次均未复现:`renderer.lo - 默认(无参数)= **登录态**相位:未登录即 `blocked` + exit 2。 - `--keyless` = **登出态**相位:仍在登录态即 `blocked` + exit 2。 +- `--self-test` = **判据反向闸**:不连应用、不发网络请求,逐条证明 #651 修正后的 + P1.2/P1.3/P1.5/P3.8 与 Ledger V1 计费判据在对应绕过下会变红。 每一项都写死了**运行前就定好的判据**(结果 JSON 里的 `criterion` 字段),输出机器可读 JSON 到 [`results/`](results),带采集时刻、被验 commit、asar 指纹、应用版本。 @@ -98,43 +100,43 @@ cd ~/app/alpha-code && bun docs/verification/2026-07-27-e7-packaged-live/probe.t ### 登录态相位(默认) -| 项 | AC | 判据 | 结果 | -| --- | --- | --- | --- | -| P0.1 | 前置 | `sha256(app.asar)` 等于本文件钉的值 | ☑ pass(2026-07-28T01:37Z) | -| P0.2 | 前置 | CDP 端口列出 renderer page target | ☑ pass | -| P0.3 | 前置 | `GET /global/health` → `{healthy:true}` | ☑ pass | -| P0.4 | 前置 | `auth.getState()` 为 `{status:"logged-in",mode:"platform"}` **且** `alpha-secrets/ALPHA_CLOUD_TOKEN` 在位 | ☐ 待 owner(未登录时已实测 `blocked`+exit 2) | -| P1.1 | AC1 | `GET /mcp` 里 `cloud` 的 `status === "connected"` | ☐ | -| P1.2 | AC1 | `config.mcp.cloud.url` 等于应用解析出的 mcp 端点,且 `headers.Authorization` 形如 `Bearer {file:…ALPHA_CLOUD_TOKEN}`(token 不进配置) | ☐ | -| P1.3 | AC1 | **LIVE-PATH GATE ①** 用应用自己的 cloud token 对已部署端点做 `tools/list`,存在匹配 `/web[_-]?search/` 的工具 | ☐ | -| P1.4 | AC1 | 记录引擎侧真实工具 id(`sanitize("cloud")+"_"+sanitize(<远端名>)`),**不假定**是 `cloud_web_search` | ☐ | -| P1.5 | AC1 | `config.permission.websearch === "deny"`,且每个注入 agent 的 `permission.websearch === "deny"` | ☐ | -| P1.6 | AC1 | `/config/providers` 里存在网关 provider 且有 `capabilities.toolcall` 模型 | ☐ | -| P1.7 | AC1 | `GET /experimental/tool?provider&model` **不含** `websearch`(本地 keyless 被抑制) | ☐ | -| P2.1 | AC1 | **打包真调**:一次真实模型轮次产出该云工具的 tool part,`status==="completed"`,输出解析出 `{query,results}` | ☐ | -| P2.2 | AC1 | **LIVE-PATH GATE ②** 用应用自己的 token 直接 `tools/call`,返回 `{query,results}` 且 `isError !== true` | ☐ | -| P2.3 | AC3 | **计费**:两次真调前后 `account.summary()` 的 `walletUsedFen` 上升 / `balanceFen` 下降,或出现新的 usage 流水 | ☐ | -| P3.1 | AC3 | **401**:无 Authorization 打 `POST {platform}/v1/tools/web_search` → 401 | ☐ | -| P3.2 | AC3 | **400**:带真 bearer、body `{}` → 400 | ☐ | -| P3.3 | AC3 | **400**:带真 bearer、坏 JSON → 400 | ☐ | -| P3.4 | AC3 | **403** —— `not-producible`(桌面端只持有 `model.invoke` / `cloud.dispatch` 两个 route-purpose 绑定令牌,造不出 scope 不足的令牌);映射由 L1 `alpha-websearch-failure.test.ts` 覆盖 | ☐ | -| P3.5 | AC3 | **502** —— `not-producible`(需已部署 gateway 同时缺 `TAVILY_API_KEY`/`BRAVE_API_KEY`;两把钥匙都在位,拆生产配置不在探针权限内) | ☐ | -| P3.6 | AC3 | **意外状态 LOUD** —— `not-producible`(桌面端够得着的请求形态都落在 {400,401,402,403,502} 内);映射由 L1 覆盖 | ☐ | -| P3.7 | AC3 | **402 / 余额**:带真 bearer 打 `/v1/tools/web_search` → 402(账户额度+余额双空时);200 ⇒ 账户被预授权通过 = 今天产生不了,记 `not-producible`;其它状态 LOUD。见 §6 | ☐ | -| P3.8 | AC3 | **defect 消失**:代付态下调用被 deny 的本地 `websearch`,模型拿到的是可辨 tool error(含「denied by alpha sovereignty」或以 `Web search failed:` 开头),不是崩溃 | ☐ | -| P3.9 | AC3 | 云侧失败 loud 但**不可分类**(平台薄壳丢弃 `r.status`)—— 已登记缺口 `alpha-platform#105` | ☐ | +| 项 | AC | 判据 | 结果 | +| ---- | ---- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------- | +| P0.1 | 前置 | `sha256(app.asar)` 等于本文件钉的值 | ☑ pass(2026-07-28T01:37Z) | +| P0.2 | 前置 | CDP 端口列出 renderer page target | ☑ pass | +| P0.3 | 前置 | `GET /global/health` → `{healthy:true}` | ☑ pass | +| P0.4 | 前置 | `auth.getState()` 为 `{status:"logged-in",mode:"platform"}` **且** `alpha-secrets/ALPHA_CLOUD_TOKEN` 在位 | ☐ 待 owner(未登录时已实测 `blocked`+exit 2) | +| P1.1 | AC1 | `GET /mcp` 里 `cloud` 的 `status === "connected"` | ☐ | +| P1.2 | AC1 | 从打包 sidecar 的原始 `OPENCODE_CONFIG_CONTENT` 断言 URL + `Bearer {file:…ALPHA_CLOUD_TOKEN}`;`GET /config` 只另证替换后 URL/值与密钥文件一致 | ☐ | +| P1.3 | AC1 | **LIVE-PATH catalog gate**:已部署端点的匿名 `tools/list` 存在匹配 `/web[_-]?search/` 的工具;本项不再冒充账户授权证据,账户绑定由 P2.2 证明 | ☐ | +| P1.4 | AC1 | 记录引擎侧真实工具 id(`sanitize("cloud")+"_"+sanitize(<远端名>)`),**不假定**是 `cloud_web_search` | ☐ | +| P1.5 | AC1 | 对 `GET /agent` 每个运行时 agent 按引擎 `Wildcard.match + findLast` 语义计算 `websearch` 有效判决,要求全部为 `deny`;后置用户 agent allow 会使本项变红 | ☐ | +| P1.6 | AC1 | `/config/providers` 里存在网关 provider 且有 `capabilities.toolcall` 模型 | ☐ | +| P1.7 | AC1 | `GET /experimental/tool?provider&model` **不含** `websearch`(本地 keyless 被抑制) | ☐ | +| P2.1 | AC1 | **打包真调**:一次真实模型轮次产出该云工具的 tool part,`status==="completed"`,输出解析出 `{query,results}` | ☐ | +| P2.2 | AC1 | **LIVE-PATH GATE ②** 用应用自己的 token 直接 `tools/call`,返回 `{query,results}` 且 `isError !== true` | ☐ | +| P2.3 | AC3 | **计费**:有界轮询 `waitUntil` 后台结算,要求 Ledger V1 新增 `kind=usage_settled`、`actionId=tool.web_search` 且 `seq` 不在调用前页面中的事实 | ☐ | +| P3.1 | AC3 | **401**:无 Authorization 打 `POST {platform}/v1/tools/web_search` → 401 | ☐ | +| P3.2 | AC3 | **400**:带真 bearer、body `{}` → 400 | ☐ | +| P3.3 | AC3 | **400**:带真 bearer、坏 JSON → 400 | ☐ | +| P3.4 | AC3 | **403** —— `not-producible`(桌面端只持有 `model.invoke` / `cloud.dispatch` 两个 route-purpose 绑定令牌,造不出 scope 不足的令牌);映射由 L1 `alpha-websearch-failure.test.ts` 覆盖 | ☐ | +| P3.5 | AC3 | **502** —— `not-producible`(需已部署 gateway 同时缺 `TAVILY_API_KEY`/`BRAVE_API_KEY`;两把钥匙都在位,拆生产配置不在探针权限内) | ☐ | +| P3.6 | AC3 | **意外状态 LOUD** —— `not-producible`(桌面端够得着的请求形态都落在 {400,401,402,403,502} 内);映射由 L1 覆盖 | ☐ | +| P3.7 | AC3 | **402 / 余额**:带真 bearer 打 `/v1/tools/web_search` → 402(账户额度+余额双空时);200 ⇒ 账户被预授权通过 = 今天产生不了,记 `not-producible`;其它状态 LOUD。见 §6 | ☐ | +| P3.8 | AC3 | **defect 消失**:平台模型工具表不含本地 `websearch`,所以模型不可能尝试它;直接拒绝的可辨文案留给既有 L1 `alpha-websearch-failure.test.ts` | ☐ | +| P3.9 | AC3 | 云侧失败 loud 但**不可分类**(平台薄壳丢弃 `r.status`)—— 已登记缺口 `alpha-platform#105` | ☐ | ### 登出态相位(`--keyless`) -| 项 | AC | 判据 | 结果 | -| --- | --- | --- | --- | -| K0.4 | 前置 | `auth.getState().status === "logged-out"` 且无 `ALPHA_CLOUD_TOKEN` 文件 | ☑ pass(2026-07-28T01:37Z) | -| K1.1 | AC2 | `GET /mcp` 无 `cloud` 键(登出态云暗) | ☑ pass | -| K1.2 | AC2 | `config.mcp.cloud` 不存在,且 `config.permission.websearch !== "deny"`(keyless 还原) | ☑ pass | -| K1.3 | AC2 | 存在非网关 provider 的 `toolcall` 模型 | ☑ pass(`deepseek-byok` / `deepseek-v4-flash`) | -| K1.4 | AC2 | `GET /experimental/tool?provider&model` **含** `websearch` | ☑ pass | -| K1.5 | AC2 | **keyless 真调**:真实模型轮次产出 `websearch` tool part,`status==="completed"` 且输出非空 | ☑ **pass** —— 见 §5 | -| K1.6 | AC3 | keyless 失败也必须是可辨错误(不是匿名 defect) | ☑ pass(本次 `completed`,无失败可辨) | +| 项 | AC | 判据 | 结果 | +| ---- | ---- | ----------------------------------------------------------------------------------------- | ---------------------------------------------- | +| K0.4 | 前置 | `auth.getState().status === "logged-out"` 且无 `ALPHA_CLOUD_TOKEN` 文件 | ☑ pass(2026-07-28T01:37Z) | +| K1.1 | AC2 | `GET /mcp` 无 `cloud` 键(登出态云暗) | ☑ pass | +| K1.2 | AC2 | `config.mcp.cloud` 不存在,且 `config.permission.websearch !== "deny"`(keyless 还原) | ☑ pass | +| K1.3 | AC2 | 存在非网关 provider 的 `toolcall` 模型 | ☑ pass(`deepseek-byok` / `deepseek-v4-flash`) | +| K1.4 | AC2 | `GET /experimental/tool?provider&model` **含** `websearch` | ☑ pass | +| K1.5 | AC2 | **keyless 真调**:真实模型轮次产出 `websearch` tool part,`status==="completed"` 且输出非空 | ☑ **pass** —— 见 §5 | +| K1.6 | AC3 | keyless 失败也必须是可辨错误(不是匿名 defect) | ☑ pass(本次 `completed`,无失败可辨) | 登出态相位整体 **exit 0,10/10 必需项通过**,且在新产物上**独立跑了两遍**都是 10/10 (K1.5 不是一次侥幸): @@ -150,6 +152,22 @@ cd ~/app/alpha-code && bun docs/verification/2026-07-27-e7-packaged-live/probe.t [`results/keyless-20260727T095253Z.json`](results/keyless-20260727T095253Z.json)、 [`results/logged-in-20260727T095330Z.json`](results/logged-in-20260727T095330Z.json)。 +### 2026-07-30 判据校正(#651) + +2026-07-28 的登录态结果暴露出三条假红与一条空绿。此次只修探针和判读文档,不改产品代码, +也不改任何旧 `results/` 原始证据: + +1. P1.2 不再在已经完成 `{file:}` 替换的 `GET /config` 上找引用,改看打包 sidecar 的源侧 + `OPENCODE_CONFIG_CONTENT`;运行时视图只证明替换后的 URL 与密钥文件一致。 +2. P1.5 不再要求每份配置对象都显式带 deny,改按 `/agent` 的最终 ruleset 算有效判决。 +3. P1.3 明确是匿名 catalog 可用性,不再写“for this account”;P2.2 的真 `tools/call` + 才是账户绑定的 LIVE-PATH gate。 +4. P3.8 不再要求模型调用一项已被 P1.7 从工具表移除的工具;它与 P1.7 共用可观察事实, + 可辨拒绝文案由 L1 负责。 +5. 计费差分同步适配 Ledger V1:`id` 已不存在,以 `seq` 判断新事实,并轮询等待 Workers + `waitUntil` 结算出现精确的 `usage_settled/tool.web_search` 事实,而不是固定睡 4 秒后 + 接受任意新流水。 + ## 5. 曾经的阻断项 —— keyless 真调在 `94a76b669` 上是坏的,`e578e00ae` 上已修复 **状态:已解除。** 下面是缺陷本身的记录;修复(PR #648,commit `e578e00ae`)已在新产物上 @@ -162,11 +180,11 @@ K1.5 曾在旧打包应用上稳定复现:真实 `websearch` 调用返回 用引擎自身的传输写法脱机复现(同一请求,三种读法): -| 读法 | chunk 数 | 收到字节 | -| --- | --- | --- | -| `Stream.runForEachWhile`(旧 `readBoundedBody` 的写法) | 1 | 4,090 | -| `Stream.runForEach` | 3 | 18,063 | -| `response.text` | — | 18,034 字符 | +| 读法 | chunk 数 | 收到字节 | +| ----------------------------------------------------- | -------- | ----------- | +| `Stream.runForEachWhile`(旧 `readBoundedBody` 的写法) | 1 | 4,090 | +| `Stream.runForEach` | 3 | 18,063 | +| `response.text` | — | 18,034 字符 | `Stream.runForEachWhile` 在谓词恒为 `true` 时仍在第一个 chunk 后停止 —— `packages/opencode/src/tool/mcp-websearch.ts` 的 `readBoundedBody` 因此把**任何超过一个 chunk 的 @@ -203,18 +221,18 @@ commit `e578e00ae`:`readBoundedBody` 换成 `Stream.runForEach` 读全 + `BodyCa 依据 alpha-platform `packages/gateway/src/worker.ts` 的 `webSearchHandler`: -| 臂 | 触发条件 | 桌面端可达? | -| --- | --- | --- | +| 臂 | 触发条件 | 桌面端可达? | +| -------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **A. per-job 预算耗尽**(`perJobPrecall` → `kind:"over"`) | `auth.via === "job"`,即一枚 `JOB_TOKEN_SECRET` 签发、claims 带 `job_id` 的 job token(`lib/tenant-auth.ts:114-123`) | **不可达**。桌面端登录拿到的是 route-purpose 绑定的 JWT(`via:"jwt"`),`auth.jobId` 恒空 ⇒ `perJobPrecall` 直接 `{kind:"pass",enforced:false}`。与 P3.4 同源:桌面端铸不出那个形状的凭证 | -| **B. `accountPreauth` 拒绝** | 账户服务回 `{ok:false}`(「超出会员额度且钱包余额不足」,`worker.ts:243`) | **仅当账户余额与会员额度双空**。预估价是路由常量(`BILLABLE_ROUTES[…].estimatedCostUsd`),请求体只有 `{query,max_results}`,客户端**没有任何调价/调额杠杆** | +| **B. `accountPreauth` 拒绝** | 账户服务回 `{ok:false}`(「超出会员额度且钱包余额不足」,`worker.ts:243`) | **仅当账户余额与会员额度双空**。预估价是路由常量(`BILLABLE_ROUTES[…].estimatedCostUsd`),请求体只有 `{query,max_results}`,客户端**没有任何调价/调额杠杆** | ### 判读规则(写在探针里,运行前定好) -| 观测 | 记法 | 含义 | -| --- | --- | --- | -| HTTP **402** | ☑ `pass`(required) | 真拿到了 402 证据 | +| 观测 | 记法 | 含义 | +| ------------ | ----------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | +| HTTP **402** | ☑ `pass`(required) | 真拿到了 402 证据 | | HTTP **200** | `not-producible`(非 required) | 账户被预授权通过 —— **这本身就是「今天产生不了 402」的可观测证据**,不是绿、也不是静默跳过。JSON 里带实测 `balanceFen`/`walletUsedFen`/`plan` 作为理由 | -| 其它状态 | ☒ `fail`(required) | 意外状态必须 LOUD,先查清再接受这一轮 | +| 其它状态 | ☒ `fail`(required) | 意外状态必须 LOUD,先查清再接受这一轮 | **为什么不主动把账户打空去凑 402**:那是对 owner 计费状态的破坏性变更,而且会连带让同一轮的 P2.1/P2.2(真调)与 P2.3(计费)一起失败 —— 402 证据与 AC1/计费证据**不可能在同一个账户形态下同时取到**。 @@ -244,11 +262,11 @@ P1.4 的 `observed` 里有三个字段:`remoteName`(worker 实测返回的远端 它们盯的是一个引擎里并不存在的工具名 —— 也就是说这些闸**当前是空闸门**(闸在,但永不命中)。 必须按 `derivedEngineToolId` 逐个重核,**另开一张 CODE 票**,不要在本 VERIFY 票里改源码: - | 要重核的闸 | 位置 | 下错了会怎样 | - | --- | --- | --- | - | permission deny(云优先时抑制本地 keyless) | `packages/ui-mac/src/main/cloud-web-search.ts` 注入的 `permission.websearch` / 云工具 id | 抑制不生效或误伤,P1.5/P1.7 的绿变成假绿 | - | ext 的 `tool.execute.before` 钩子 | `packages/ext/src/`(云 websearch kill-switch 与主权闸) | kill-switch(#223 AC4)拦不住真实工具名 = 关不掉 | - | 引擎侧工具枚举/展示 | `packages/opencode/src/mcp/catalog.ts:117-119` 的拼名 | 名字对不上,模型看到的工具与闸盯的工具不是同一个 | + | 要重核的闸 | 位置 | 下错了会怎样 | + | ----------------------------------------- | ---------------------------------------------------------------------------------------- | ----------------------------------------------- | + | permission deny(云优先时抑制本地 keyless) | `packages/ui-mac/src/main/cloud-web-search.ts` 注入的 `permission.websearch` / 云工具 id | 抑制不生效或误伤,P1.5/P1.7 的绿变成假绿 | + | ext 的 `tool.execute.before` 钩子 | `packages/ext/src/`(云 websearch kill-switch 与主权闸) | kill-switch(#223 AC4)拦不住真实工具名 = 关不掉 | + | 引擎侧工具枚举/展示 | `packages/opencode/src/mcp/catalog.ts:117-119` 的拼名 | 名字对不上,模型看到的工具与闸盯的工具不是同一个 | 重核判据不是「grep 到字面量」,而是**用真实 id 跑一遍这三个闸并确认它们真的命中** —— 本仓的教训是「闸门是假的」比逻辑错更常见(见 `AGENTS.md` / 闸门四形态)。 diff --git a/docs/verification/2026-07-27-e7-packaged-live/probe.ts b/docs/verification/2026-07-27-e7-packaged-live/probe.ts index 39c91a2eb5fb..10b7656e6978 100644 --- a/docs/verification/2026-07-27-e7-packaged-live/probe.ts +++ b/docs/verification/2026-07-27-e7-packaged-live/probe.ts @@ -108,8 +108,7 @@ async function connectCdp(): Promise { url: string webSocketDebuggerUrl: string }> - const target = - list.find((t) => t.type === "page" && t.url.startsWith("oc://")) ?? list.find((t) => t.type === "page") + const target = list.find((t) => t.type === "page" && t.url.startsWith("oc://")) ?? list.find((t) => t.type === "page") if (!target) throw new Error("no renderer page target on the CDP port") const ws = new WebSocket(target.webSocketDebuggerUrl) await new Promise((resolve, reject) => { @@ -145,11 +144,19 @@ async function connectCdp(): Promise { // ── engine HTTP (through the packaged app's own sidecar credential) ────────── -type Engine = { base: string; auth: string; get: (p: string) => Promise; post: (p: string, b: unknown) => Promise } +type Engine = { + base: string + auth: string + get: (p: string) => Promise + post: (p: string, b: unknown) => Promise +} /** A model turn can run for minutes; anything longer than this is a hang, and a hung probe is * worse than a failed one — the owner would sit and wait instead of reading a red line. */ const TURN_TIMEOUT_MS = Number(process.env.ALPHA_E7_TURN_TIMEOUT_MS ?? 240_000) +/** Gateway settlement deliberately runs under Workers `waitUntil`; poll the account truth instead + * of assuming a fixed four-second delay is enough. */ +const BILLING_SETTLE_TIMEOUT_MS = Number(process.env.ALPHA_E7_BILLING_SETTLE_TIMEOUT_MS ?? 45_000) function makeEngine(base: string, auth: string): Engine { const call = async (method: string, p: string, body?: unknown) => { @@ -192,6 +199,240 @@ function readSecret(name: string): string | undefined { return registerSecret(value) && value ? value : undefined } +type CloudDefinition = { + type?: string + url?: string + enabled?: boolean + headers?: Record +} + +type PermissionRule = { + permission: string + pattern: string + action: string +} + +type LedgerFact = { + seq: number + kind: string + domain: string + amount: number + actionId?: string + reservationId?: string + createdAt: number +} + +function cloudDefinitionUsesFileReference( + config: { mcp?: Record } | undefined, + expectedUrl: string, + expectedSecretFile: string, +) { + const definition = config?.mcp?.[CLOUD_MCP_SERVER_NAME] + return definition?.url === expectedUrl && definition.headers?.Authorization === `Bearer {file:${expectedSecretFile}}` +} + +function wildcardMatch(value: string, pattern: string) { + const normalizedValue = value.replaceAll("\\", "/") + const normalizedPattern = pattern.replaceAll("\\", "/") + let escaped = normalizedPattern + .replace(/[.+^${}()|[\]\\]/g, "\\$&") + .replace(/\*/g, ".*") + .replace(/\?/g, ".") + if (escaped.endsWith(" .*")) escaped = escaped.slice(0, -3) + "( .*)?" + return new RegExp(`^${escaped}$`, process.platform === "win32" ? "si" : "s").test(normalizedValue) +} + +function effectivePermission(rules: PermissionRule[], permission: string, pattern: string) { + const index = rules.findLastIndex( + (rule) => wildcardMatch(permission, rule.permission) && wildcardMatch(pattern, rule.pattern), + ) + return index < 0 + ? { action: "ask", index, rule: { permission, pattern: "*", action: "ask" } } + : { action: rules[index]!.action, index, rule: rules[index]! } +} + +function hasRemoteWebSearch(tools: Array<{ name: string }>) { + return tools.find((tool) => /web[_-]?search/i.test(tool.name)) +} + +function localWebSearchIsHidden(toolIDs: string[] | undefined) { + return toolIDs !== undefined && !toolIDs.includes(LOCAL_WEB_SEARCH_TOOL_ID) +} + +function newSettledWebSearchFacts(facts: LedgerFact[], before: Set) { + return facts.filter( + (fact) => !before.has(fact.seq) && fact.kind === "usage_settled" && fact.actionId === "tool.web_search", + ) +} + +/** `ps eww` is a source-side observation: unlike GET /config, it sees the sidecar's raw + * OPENCODE_CONFIG_CONTENT before ConfigVariable substitutes `{file:...}`. Parse only the balanced + * JSON value and never retain or print the rest of the process environment. */ +function extractJsonEnvironment(line: string, name: string) { + const marker = `${name}=` + const start = line.indexOf(marker) + if (start < 0) return undefined + const valueStart = start + marker.length + if (line[valueStart] !== "{") return undefined + let depth = 0 + let quoted = false + let escaped = false + for (let index = valueStart; index < line.length; index++) { + const char = line[index]! + if (escaped) { + escaped = false + continue + } + if (quoted && char === "\\") { + escaped = true + continue + } + if (char === '"') { + quoted = !quoted + continue + } + if (quoted) continue + if (char === "{") depth++ + if (char !== "}") continue + depth-- + if (depth !== 0) continue + try { + return JSON.parse(line.slice(valueStart, index + 1)) as { mcp?: Record } + } catch { + return undefined + } + } + return undefined +} + +function readPackagedSidecarConfigSource(expectedUrl: string) { + // Resolve only the packaged NodeService PID without environments first. Asking `ps eww -ax` + // would unnecessarily copy unrelated processes' environments into this probe. + const processList = Bun.spawnSync(["ps", "-axo", "pid=,command="]) + if (processList.exitCode !== 0) + return { + config: undefined, + packagedCandidates: 0, + error: processList.stderr.toString().trim().slice(0, 300) || `ps exited ${processList.exitCode}`, + } + const sidecarPids = processList.stdout + .toString() + .split("\n") + .filter( + (line) => + line.includes(APP) && + line.includes("--type=utility") && + line.includes("--utility-sub-type=node.mojom.NodeService"), + ) + .flatMap((line) => { + const pid = line.trim().match(/^(\d+)\s/)?.[1] + return pid ? [pid] : [] + }) + const configs = sidecarPids + .map((pid) => Bun.spawnSync(["ps", "eww", "-p", pid, "-o", "command="])) + .filter((result) => result.exitCode === 0) + .map((result) => result.stdout.toString()) + .flatMap((line) => { + const config = extractJsonEnvironment(line, "OPENCODE_CONFIG_CONTENT") + return config ? [config] : [] + }) + return { + config: configs.find((config) => config.mcp?.[CLOUD_MCP_SERVER_NAME]?.url === expectedUrl), + packagedCandidates: configs.length, + error: + sidecarPids.length === 0 + ? "no packaged NodeService sidecar process found" + : configs.length === 0 + ? `OPENCODE_CONFIG_CONTENT was not observable on ${sidecarPids.length} packaged sidecar process(es)` + : undefined, + } +} + +function runCriterionSelfTests() { + const expectedUrl = "https://cloud.example/mcp" + const expectedSecret = "/tmp/ALPHA_CLOUD_TOKEN" + const goodSource = { + mcp: { + cloud: { + url: expectedUrl, + headers: { Authorization: `Bearer {file:${expectedSecret}}` }, + }, + }, + } + const extractedSource = extractJsonEnvironment( + `sidecar OPENCODE_CONFIG_CONTENT=${JSON.stringify(goodSource)} OTHER_ENV=1`, + "OPENCODE_CONFIG_CONTENT", + ) + const denied = [{ permission: "websearch", pattern: "*", action: "deny" }] + const tests = [ + { + name: "P1.2 accepts the source-side file reference", + ok: + cloudDefinitionUsesFileReference(goodSource, expectedUrl, expectedSecret) && + cloudDefinitionUsesFileReference(extractedSource, expectedUrl, expectedSecret), + }, + { + name: "P1.2 rejects an inlined bearer bypass", + ok: !cloudDefinitionUsesFileReference( + { mcp: { cloud: { url: expectedUrl, headers: { Authorization: "Bearer inline-token" } } } }, + expectedUrl, + expectedSecret, + ), + }, + { + name: "P1.5 accepts an effective final deny", + ok: effectivePermission(denied, "websearch", "*").action === "deny", + }, + { + name: "P1.5 rejects a later user-agent allow bypass", + ok: + effectivePermission([...denied, { permission: "websearch", pattern: "*", action: "allow" }], "websearch", "*") + .action !== "deny", + }, + { + name: "P1.3 fails when the catalog drops web search", + ok: + Boolean(hasRemoteWebSearch([{ name: "cloud_web_search" }])) && !hasRemoteWebSearch([{ name: "cloud_status" }]), + }, + { + name: "P3.8 fails when local websearch returns to the model tool set", + ok: localWebSearchIsHidden(["read"]) && !localWebSearchIsHidden(["read", "websearch"]), + }, + { + name: "P2.3 requires a new settled web-search ledger fact", + ok: + newSettledWebSearchFacts( + [ + { + seq: 2, + kind: "usage_settled", + domain: "wallet", + amount: -1, + actionId: "tool.web_search", + createdAt: 1, + }, + ], + new Set([1]), + ).length === 1 && + newSettledWebSearchFacts( + [ + { + seq: 2, + kind: "reservation_created", + domain: "wallet", + amount: -1, + actionId: "tool.web_search", + createdAt: 1, + }, + ], + new Set([1]), + ).length === 0, + }, + ] + for (const test of tests) console.log(`[${test.ok ? "PASS" : "FAIL"}] ${test.name}`) + process.exit(tests.every((test) => test.ok) ? 0 : 1) +} + /** MCP Streamable-HTTP one-shot. Returns the parsed JSON-RPC envelope plus the raw HTTP status. */ async function mcpCall(url: string, bearer: string, method: string, params: unknown, timeoutMs = 45_000) { const controller = new AbortController() @@ -209,7 +450,12 @@ async function mcpCall(url: string, bearer: string, method: string, params: unkn }) const body = await res.text() let envelope: any - const line = body.startsWith("{") ? body : body.split("\n").find((l) => l.startsWith("data: "))?.slice(6) + const line = body.startsWith("{") + ? body + : body + .split("\n") + .find((l) => l.startsWith("data: ")) + ?.slice(6) try { envelope = line ? JSON.parse(line) : undefined } catch { @@ -267,7 +513,8 @@ async function pickModel(engine: Engine, platformBase: string, want: "platform" for (const provider of candidates) { const model = Object.values(provider.models ?? {}).find((m: any) => m?.capabilities?.toolcall) // Never let a provider record (it carries plaintext BYOK keys) escape this function. - if (model?.id) return { providerID: provider.id, modelID: model.id as string, providerIDs: raw.providers.map((p) => p.id) } + if (model?.id) + return { providerID: provider.id, modelID: model.id as string, providerIDs: raw.providers.map((p) => p.id) } } return { providerID: undefined, modelID: undefined, providerIDs: raw.providers.map((p) => p.id) } } @@ -278,6 +525,7 @@ async function pickModel(engine: Engine, platformBase: string, want: "platform" * auto-detect: a probe that silently switches phases can never report "未登录,无法取证", * and a gate you cannot see fail is not a gate. */ const MODE: "logged-in" | "keyless" = process.argv.includes("--keyless") ? "keyless" : "logged-in" +if (process.argv.includes("--self-test")) runCriterionSelfTests() const runAt = new Date().toISOString() let cdp: Cdp | undefined @@ -471,9 +719,11 @@ if (MODE === "keyless") { }) if (byok.providerID && byok.modelID) { - const toolIDs = ((await engine.get( - `/experimental/tool?provider=${encodeURIComponent(byok.providerID)}&model=${encodeURIComponent(byok.modelID)}`, - )) as Array<{ id: string }>).map((t) => t.id) + const toolIDs = ( + (await engine.get( + `/experimental/tool?provider=${encodeURIComponent(byok.providerID)}&model=${encodeURIComponent(byok.modelID)}`, + )) as Array<{ id: string }> + ).map((t) => t.id) assertCheck( { id: "K1.4", @@ -564,33 +814,59 @@ const engineConfig = (await engine.get("/config")) as { } const cloudDef = engineConfig.mcp?.[CLOUD_MCP_SERVER_NAME] const expectedMcpUrl = endpoints.mcp ?? `${endpoints.cloud}/mcp` +const sourceConfig = readPackagedSidecarConfigSource(expectedMcpUrl) +const sourceUsesFileReference = cloudDefinitionUsesFileReference( + sourceConfig.config, + expectedMcpUrl, + secretFile("ALPHA_CLOUD_TOKEN"), +) +const resolvedAuthorizationMatchesSecret = cloudDef?.headers?.Authorization === `Bearer ${cloudToken}` assertCheck( { id: "P1.2", ac: "AC1 packaged 登录态 listTools", - title: "cloud MCP definition points at the app-resolved URL and carries the token by {file:} reference only", - criterion: `config.mcp.${CLOUD_MCP_SERVER_NAME}.url === "${expectedMcpUrl}" AND headers.Authorization matches /^Bearer \\{file:.*ALPHA_CLOUD_TOKEN\\}$/`, + title: "the source-side cloud MCP definition uses a {file:} token reference and resolves to the expected endpoint", + criterion: + `the packaged sidecar's raw OPENCODE_CONFIG_CONTENT has mcp.${CLOUD_MCP_SERVER_NAME}.url === "${expectedMcpUrl}" ` + + `AND Authorization === "Bearer {file:/alpha-secrets/ALPHA_CLOUD_TOKEN}"; ` + + "GET /config separately resolves that definition to the same URL and the secret-file value", required: true, }, - cloudDef?.url === expectedMcpUrl && - /^Bearer \{file:.*ALPHA_CLOUD_TOKEN\}$/.test(String(cloudDef?.headers?.Authorization ?? "")), - { expectedMcpUrl, cloudDef }, + sourceUsesFileReference && cloudDef?.url === expectedMcpUrl && resolvedAuthorizationMatchesSecret, + { + expectedMcpUrl, + sourceProcessCandidates: sourceConfig.packagedCandidates, + sourceObservationError: sourceConfig.error, + sourceUsesFileReference, + resolvedUrl: cloudDef?.url, + resolvedAuthorizationMatchesSecret, + secretFileMode: (statSync(secretFile("ALPHA_CLOUD_TOKEN")).mode & 0o777).toString(8), + secretDirectoryMode: (statSync(path.dirname(secretFile("ALPHA_CLOUD_TOKEN"))).mode & 0o777).toString(8), + }, ) -// LIVE-PATH GATE ①: listTools against the deployed worker, with the app's own cloud token. +// LIVE-PATH catalog gate: tools/list is intentionally anonymous on alpha-cloud. It proves transport +// catalog availability only; the account-bound authorization claim belongs to P2.2 tools/call. const listed = await mcpCall(expectedMcpUrl, cloudToken!, "tools/list", {}) const remoteTools: Array<{ name: string }> = listed.envelope?.result?.tools ?? [] -const remoteWebSearch = remoteTools.find((t) => /web[_-]?search/i.test(t.name)) +const remoteWebSearch = hasRemoteWebSearch(remoteTools) assertCheck( { id: "P1.3", ac: "AC1 packaged 登录态 listTools", - title: "LIVE-PATH GATE ① — deployed cloud worker lists a web-search tool for this account", - criterion: "MCP tools/list on the app-resolved endpoint (app's own bearer) contains a tool matching /web[_-]?search/", + title: "LIVE-PATH catalog gate — the deployed cloud worker advertises a web-search tool", + criterion: + "MCP tools/list on the app-resolved endpoint contains a tool matching /web[_-]?search/; this anonymous catalog check makes no account-authorization claim (P2.2 does)", required: true, }, Boolean(remoteWebSearch), - { httpStatus: listed.status, remoteToolNames: remoteTools.map((t) => t.name), matched: remoteWebSearch?.name }, + { + httpStatus: listed.status, + remoteToolNames: remoteTools.map((t) => t.name), + matched: remoteWebSearch?.name, + accountBound: false, + accountBoundGate: "P2.2 tools/call with the app-issued cloud bearer", + }, ) const derivedEngineId = remoteWebSearch ? engineToolId(remoteWebSearch.name) : undefined @@ -609,20 +885,34 @@ record({ : undefined, }) -const permWebsearch = engineConfig.permission?.[LOCAL_WEB_SEARCH_TOOL_ID] -const agentsMissingDeny = Object.entries(engineConfig.agent ?? {}) - .filter(([, a]) => a && a.permission?.[LOCAL_WEB_SEARCH_TOOL_ID] !== "deny") - .map(([name]) => name) +const agents = (await engine.get("/agent")) as Array<{ name: string; permission: PermissionRule[] }> +const agentDecisions = agents.map((agent) => ({ + name: agent.name, + ...effectivePermission(agent.permission, LOCAL_WEB_SEARCH_TOOL_ID, "*"), +})) +const agentsWithoutEffectiveDeny = agentDecisions + .filter((decision) => decision.action !== "deny") + .map((decision) => decision.name) assertCheck( { id: "P1.5", ac: "AC1 云优先(本地 keyless 被抑制)", - title: "platform-pays denies the local keyless websearch globally and on every injected agent", - criterion: `config.permission.${LOCAL_WEB_SEARCH_TOOL_ID} === "deny" AND every config.agent[*].permission.${LOCAL_WEB_SEARCH_TOOL_ID} === "deny"`, + title: "every runtime agent's effective permission decision denies local keyless websearch", + criterion: + `for every GET /agent entry, Permission.evaluate("${LOCAL_WEB_SEARCH_TOOL_ID}", "*", ruleset) ` + + "using engine-equivalent Wildcard.match + findLast semantics returns action=deny; a later user-agent allow must make this gate fail", required: true, }, - permWebsearch === "deny" && agentsMissingDeny.length === 0, - { globalPermission: permWebsearch, agentsMissingDeny, agents: Object.keys(engineConfig.agent ?? {}) }, + agentDecisions.length > 0 && agentsWithoutEffectiveDeny.length === 0, + { + agentsWithoutEffectiveDeny, + decisions: agentDecisions.map((decision) => ({ + name: decision.name, + action: decision.action, + matchedRuleIndex: decision.index, + matchedRule: decision.rule, + })), + }, ) // Pick a platform-gateway model with tool-calling, so the real turn below runs on the paid path. @@ -641,11 +931,13 @@ record({ required: true, }) +let platformModelToolIDs: string[] | undefined if (platform.providerID && platform.modelID) { - const modelTools = (await engine.get( - `/experimental/tool?provider=${encodeURIComponent(platform.providerID)}&model=${encodeURIComponent(platform.modelID)}`, - )) as Array<{ id: string }> - const ids = modelTools.map((t) => t.id) + platformModelToolIDs = ( + (await engine.get( + `/experimental/tool?provider=${encodeURIComponent(platform.providerID)}&model=${encodeURIComponent(platform.modelID)}`, + )) as Array<{ id: string }> + ).map((tool) => tool.id) assertCheck( { id: "P1.7", @@ -654,8 +946,8 @@ if (platform.providerID && platform.modelID) { criterion: `GET /experimental/tool?provider&model does NOT contain "${LOCAL_WEB_SEARCH_TOOL_ID}"`, required: true, }, - !ids.includes(LOCAL_WEB_SEARCH_TOOL_ID), - { builtinToolIDs: ids }, + localWebSearchIsHidden(platformModelToolIDs), + { builtinToolIDs: platformModelToolIDs }, ) } @@ -663,7 +955,7 @@ if (platform.providerID && platform.modelID) { const summaryBefore = await cdp!.eval("window.api.account.summary()") const txBefore = await cdp!.eval("window.api.account.transactions(50)") -const txIdsBefore = new Set((txBefore?.transactions ?? []).map((t: any) => t.id)) +const txSeqBefore = new Set((txBefore?.transactions ?? []).map((fact: LedgerFact) => fact.seq)) let realCall: { sessionID: string; parts: ToolPart[] } | undefined let cloudToolPart: ToolPart | undefined @@ -760,37 +1052,59 @@ assertCheck( }, ) -// 计费 (ledger/settle): the same two real calls above must move the account ledger. -await sleep(4000) -const summaryAfter = await cdp!.eval("window.api.account.summary()") -const txAfter = await cdp!.eval("window.api.account.transactions(50)") -const newTx = ((txAfter?.transactions ?? []) as any[]).filter((t) => !txIdsBefore.has(t.id)) +// 计费 (ledger/settle): settlement is intentionally scheduled through Cloudflare `waitUntil`. +// Poll the account ledger's append-only truth until the exact web-search settlement appears; a +// fixed sleep is not a valid assertion about background-task completion. +const settlementPollStartedAt = Date.now() +let summaryAfter = await cdp!.eval("window.api.account.summary()") +let txAfter = await cdp!.eval("window.api.account.transactions(50)") +let settledWebSearchFacts = newSettledWebSearchFacts(txAfter?.transactions ?? [], txSeqBefore) +while (settledWebSearchFacts.length === 0 && Date.now() - settlementPollStartedAt < BILLING_SETTLE_TIMEOUT_MS) { + await sleep(1000) + summaryAfter = await cdp!.eval("window.api.account.summary()") + txAfter = await cdp!.eval("window.api.account.transactions(50)") + settledWebSearchFacts = newSettledWebSearchFacts(txAfter?.transactions ?? [], txSeqBefore) +} +const newFacts = ((txAfter?.transactions ?? []) as LedgerFact[]).filter((fact) => !txSeqBefore.has(fact.seq)) const walletDelta = Number(summaryAfter?.walletUsedFen ?? 0) - Number(summaryBefore?.walletUsedFen ?? 0) const balanceDelta = Number(summaryBefore?.balanceFen ?? 0) - Number(summaryAfter?.balanceFen ?? 0) assertCheck( { id: "P2.3", ac: "AC3 计费(ledger/settle)证据", - title: "the real web-search calls are billed to this tenant's ledger", + title: "waitUntil-backed web-search settlement reaches this tenant's append-only ledger", criterion: - "account summary walletUsedFen increased OR balanceFen decreased OR a new usage transaction appeared between the pre-call and post-call snapshots", + `within ${BILLING_SETTLE_TIMEOUT_MS}ms after the real calls, the account ledger appends a fact with ` + + 'kind="usage_settled" AND actionId="tool.web_search" and a seq absent from the pre-call page', required: true, }, - walletDelta > 0 || balanceDelta > 0 || newTx.length > 0, + settledWebSearchFacts.length > 0, { + settlementObservedAfterMs: Date.now() - settlementPollStartedAt, + settlementTimeoutMs: BILLING_SETTLE_TIMEOUT_MS, walletUsedFenBefore: summaryBefore?.walletUsedFen, walletUsedFenAfter: summaryAfter?.walletUsedFen, balanceFenBefore: summaryBefore?.balanceFen, balanceFenAfter: summaryAfter?.balanceFen, walletDelta, balanceDelta, - newTransactions: newTx.map((t) => ({ - id: t.id, - type: t.type, - title: t.title, - amountFen: t.amountFen, - status: t.status, - createdAt: t.createdAt, + settledWebSearchFacts: settledWebSearchFacts.map((fact) => ({ + seq: fact.seq, + kind: fact.kind, + domain: fact.domain, + amount: fact.amount, + actionId: fact.actionId, + reservationId: fact.reservationId, + createdAt: fact.createdAt, + })), + newLedgerFacts: newFacts.map((fact) => ({ + seq: fact.seq, + kind: fact.kind, + domain: fact.domain, + amount: fact.amount, + actionId: fact.actionId, + reservationId: fact.reservationId, + createdAt: fact.createdAt, })), }, ) @@ -799,7 +1113,11 @@ assertCheck( const gateway = `${endpoints.platform}/v1/tools/web_search` async function gatewayProbe(headers: Record, body: string) { - const res = await fetch(gateway, { method: "POST", headers: { "content-type": "application/json", ...headers }, body }) + const res = await fetch(gateway, { + method: "POST", + headers: { "content-type": "application/json", ...headers }, + body, + }) return { status: res.status, body: (await res.text()).slice(0, 600) } } @@ -848,7 +1166,8 @@ record({ ac: "AC3 真实失败集 403", title: "403 (action_forbidden / job_not_enforceable)", status: "not-producible", - criterion: "would require a token whose scope excludes model.invoke, or a non-enforceable job — neither is mintable from a normal desktop login", + criterion: + "would require a token whose scope excludes model.invoke, or a non-enforceable job — neither is mintable from a normal desktop login", observed: { reason: "the desktop only ever holds route-purpose-bound tokens for model.invoke and cloud.dispatch", coveredAtL1: "packages/opencode/test/tool/alpha-websearch-failure.test.ts (403 → forbidden, error.code preserved)", @@ -863,7 +1182,8 @@ record({ status: "not-producible", criterion: "would require the deployed gateway to have neither TAVILY_API_KEY nor BRAVE_API_KEY", observed: { - reason: "both secrets are configured on the deployed gateway (docs/verification/2026-07-22-e7-deploy-probe.md §4); removing them is a production mutation, out of bounds for a probe", + reason: + "both secrets are configured on the deployed gateway (docs/verification/2026-07-22-e7-deploy-probe.md §4); removing them is a production mutation, out of bounds for a probe", coveredAtL1: "packages/opencode/test/tool/alpha-websearch-failure.test.ts (502 → upstream)", }, required: false, @@ -914,7 +1234,7 @@ const p402Observed = { walletUsedFen: summaryAfter?.walletUsedFen, plan: summaryAfter?.plan?.id ? { id: summaryAfter.plan.id, status: summaryAfter.plan.status } : undefined, }, - armA: "unreachable by construction — the desktop holds a route-purpose JWT (via:\"jwt\"), never a job token, so perJobPrecall returns pass/enforced:false", + armA: 'unreachable by construction — the desktop holds a route-purpose JWT (via:"jwt"), never a job token, so perJobPrecall returns pass/enforced:false', armB: p402.status === 402 ? "fired — accountPreauth rejected this call" @@ -939,42 +1259,27 @@ record({ : "unexpected status on the 402 probe — investigate before accepting this run", }) -// defect 消失: the local keyless tool is denied under platform-pays. Invoking it must yield a -// model-visible, discernible tool error — not an unhandled defect. -let denialPart: ToolPart | undefined -if (platform.providerID && platform.modelID) { - const denial = await realTurn( - engine, - { providerID: platform.providerID, modelID: platform.modelID }, - `Call the \`${LOCAL_WEB_SEARCH_TOOL_ID}\` tool exactly once with the query "alpha-code e7 denial probe". Do not call any other tool.`, - { [LOCAL_WEB_SEARCH_TOOL_ID]: true }, - "You are an evidence probe. When told to call a tool, call exactly that tool once, then stop.", - ) - denialPart = denial.parts.find((p) => p.tool === LOCAL_WEB_SEARCH_TOOL_ID) - const message = String(denialPart?.state?.error ?? "") - record({ - id: "P3.8", - ac: "AC3 defect 消失 / 主权拒绝 LOUD", - title: "the denied local websearch surfaces a discernible tool error, not a crash", - status: denialPart - ? message.includes(SOVEREIGNTY_DENIED_MARK) || message.startsWith(FAILURE_PREFIX) - ? "pass" - : "fail" - : "fail", - criterion: `the tool part exists with state.status === "error" and a model-visible message containing "${SOVEREIGNTY_DENIED_MARK}" (or starting with "${FAILURE_PREFIX}") — no unhandled defect`, - observed: redact({ - sessionID: denial.sessionID, - toolPartsSeen: denial.parts.map((p) => ({ tool: p.tool, status: p.state?.status })), - status: denialPart?.state?.status, - message: message.slice(0, 600), - metadata: denialPart?.state?.metadata, - }), - required: true, - note: denialPart - ? undefined - : "the model declined to call the denied tool — re-run; the check needs the tool to actually be attempted", - }) -} +// defect 消失: P1.7 proves the permission filter removes local websearch before model execution. +// A user `tools: {websearch:true}` map can only turn offered tools off; it cannot resurrect one +// filtered out by Permission.disabled. Requiring an impossible tool part made the old P3.8 +// permanently red. The model-visible typed-failure wording remains covered at L1. +record({ + id: "P3.8", + ac: "AC3 defect 消失 / 主权拒绝", + title: "platform-pays makes the denied local websearch impossible for the model to attempt", + status: localWebSearchIsHidden(platformModelToolIDs) ? "pass" : "fail", + criterion: + `GET /experimental/tool for the selected platform model omits "${LOCAL_WEB_SEARCH_TOOL_ID}"; ` + + "the user tools map cannot add an omitted tool, while discernible direct-denial wording remains an L1 responsibility", + observed: { + platformProviderID: platform.providerID, + platformModelID: platform.modelID, + builtinToolIDs: platformModelToolIDs, + modelCanAttemptLocalWebSearch: platformModelToolIDs?.includes(LOCAL_WEB_SEARCH_TOOL_ID), + coveredAtL1: "packages/opencode/test/tool/alpha-websearch-failure.test.ts", + }, + required: true, +}) record({ id: "P3.9", From 9dcbbeb6f7d388980d3b0aca2541bb2ca6b5ce49 Mon Sep 17 00:00:00 2001 From: jinjunnn Date: Thu, 30 Jul 2026 22:13:20 -0400 Subject: [PATCH 2/5] docs(verification): repin #643 packaged build --- .../2026-07-27-e7-packaged-live/README.md | 17 +++++++++++++---- .../2026-07-27-e7-packaged-live/probe.ts | 10 +++++----- 2 files changed, 18 insertions(+), 9 deletions(-) diff --git a/docs/verification/2026-07-27-e7-packaged-live/README.md b/docs/verification/2026-07-27-e7-packaged-live/README.md index 73ebf01f3f47..849d1fddec4b 100644 --- a/docs/verification/2026-07-27-e7-packaged-live/README.md +++ b/docs/verification/2026-07-27-e7-packaged-live/README.md @@ -24,9 +24,9 @@ review_after: 2026-10-27 | 项 | 值 | | ------------------------------------- | ------------------------------------------------------------------ | | 应用 | `/Applications/alpha-code.app`(`ship:mac` 装机版,非 `dist/` 直跑) | -| 构建时间 | 2026-07-27T21:31:42 -0400(= 2026-07-28T01:31:42Z) | -| 基线 commit | `e578e00ae`(`alpha`,工作树干净) | -| `sha256(Contents/Resources/app.asar)` | `60589c59c58e44ac0daede93fc7397a8a04365f5345eac4312e205a0d8f48e44` | +| 构建时间 | 2026-07-30T22:12:02 -0400(= 2026-07-31T02:12:02Z) | +| 基线 commit | `b8f030e0c`(`e7-probe-refresh`,工作树干净) | +| `sha256(Contents/Resources/app.asar)` | `dded6b38f023e2bbaba3c152032a857f80221d280949c6838374741aea6f42b9` | | CFBundleShortVersionString | `0.1.2` | | 引擎版本 | `1.17.13` | | userData | `~/Library/Application Support/ai.opencode.desktop.dev` | @@ -34,7 +34,16 @@ review_after: 2026-10-27 `app.asar` 的 sha256 被钉进 `probe.ts`(`PINNED_ASAR_SHA256`)。探针第一件事就是重算它并比对 —— **在错的构建上跑出来的绿是假绿**,这条判据把它挡住。重新打包后必须同时更新此处与 `probe.ts` 的常量。 -### 为什么重打了一次 +### 2026-07-30 为什么再次重钉 + +`alpha-code#651` 修正了登录态探针的四个错误观测点,本次又发现旧 P2.3 仍按 Ledger V1 +硬切前的 `transaction.id` 做差分,因此永远看不到新流水。先在干净提交 `b8f030e0c` +完成判据修正、自测、类型检查与文档校验,再从该提交执行标准 `ship:mac`; +`/Applications/alpha-code.app` 的新指纹为 `dded6b38…` 且通过 `codesign --verify --deep --strict`。 +产品源码相对 `origin/alpha` 没有额外修改,但被测包与判据现在共同指向一个可追踪提交。 +旧结果及其旧指纹均保留不改。 + +### 2026-07-27 为什么重打了一次 上一份装机产物是 `94a76b669` / `8706d0c4…`(2026-07-27T05:29:35 -0400),**早于** `e578e00ae`(PR #648,`readBoundedBody` 读全响应体)。在那份产物上登出态 keyless 真调 diff --git a/docs/verification/2026-07-27-e7-packaged-live/probe.ts b/docs/verification/2026-07-27-e7-packaged-live/probe.ts index 10b7656e6978..85e765fb3020 100644 --- a/docs/verification/2026-07-27-e7-packaged-live/probe.ts +++ b/docs/verification/2026-07-27-e7-packaged-live/probe.ts @@ -29,11 +29,11 @@ const USER_DATA = const CDP_PORT = Number(process.env.ALPHA_E7_CDP_PORT ?? 9222) const OUT_DIR = path.join(import.meta.dir, "results") -/** The build this evidence directory was cut for: alpha @ e578e00ae, ship:mac 2026-07-28T01:31:42Z. - * Re-pinned from 94a76b669 / 8706d0c0… — that build predated the `readBoundedBody` fix (#648), - * on which K1.5 (keyless 真调) could not pass. See README §1 and §5. */ -const PINNED_ASAR_SHA256 = "60589c59c58e44ac0daede93fc7397a8a04365f5345eac4312e205a0d8f48e44" -const PINNED_COMMIT = "e578e00ae" +/** The build this evidence directory is cut for: alpha @ b8f030e0c, ship:mac + * 2026-07-31T02:12:02Z. Re-pinned after correcting the #651 criteria and Ledger V1 settlement + * observation; earlier evidence remains immutable under results/. See README §1. */ +const PINNED_ASAR_SHA256 = "dded6b38f023e2bbaba3c152032a857f80221d280949c6838374741aea6f42b9" +const PINNED_COMMIT = "b8f030e0c" /** `packages/ui-mac/src/main/cloud-web-search.ts` — the two ids alpha pins. */ const CLOUD_MCP_SERVER_NAME = "cloud" From 1fb1d5cbd2afe5ac3977ce6e4a1a6dd47f4246f3 Mon Sep 17 00:00:00 2001 From: jinjunnn Date: Thu, 30 Jul 2026 22:23:10 -0400 Subject: [PATCH 3/5] fix(verification): correlate live billing settlement --- .../2026-07-27-e7-packaged-live/README.md | 90 +++--- .../2026-07-27-e7-packaged-live/probe.ts | 298 ++++++++++-------- 2 files changed, 216 insertions(+), 172 deletions(-) diff --git a/docs/verification/2026-07-27-e7-packaged-live/README.md b/docs/verification/2026-07-27-e7-packaged-live/README.md index 849d1fddec4b..1ab32c8e70f3 100644 --- a/docs/verification/2026-07-27-e7-packaged-live/README.md +++ b/docs/verification/2026-07-27-e7-packaged-live/README.md @@ -16,8 +16,9 @@ review_after: 2026-10-27 验的是**前提**(worker 已部署、匿名 `tools/list` 有 `cloud_web_search`、gateway 规范路径 fail-closed), 且比 #639 旧,不能支撑本票任何一条 AC。 -**本目录目前完成了「取证准备 + 登出态那一半」。** 登录态那一半需要 owner 本人在真机上登录后执行 -[`probe.ts`](probe.ts);未登录时探针**拒绝产出证据**并以非零退出(见 §4)。 +**本目录的登录态与登出态取证均已完成。** 2026-07-31 在重钉产物上用真实登录态执行 +[`probe.ts`](probe.ts),23 项、0 个必需失败;此前同一套探针的登出态相位为 10/10。 +探针仍对错误登录态 fail-closed:前置不满足时拒绝产出证据并以非零退出(见 §4)。 ## 1. 被测件 @@ -77,9 +78,8 @@ eager ajv `new Function` 撞 renderer CSP —— 本次均未复现:`renderer.lo ## 3. 复现步骤(owner 照敲) -**登出态那一半(⑤)已经跑完并通过**,机器现在停在:新产物已装、`ALPHA_CDP=1` 已开、应用**处于登出态** -(为跑 keyless 相位而登出;走的是应用自己的「退出登录」路径,BYOK 钥匙没动)。 -所以 owner 只需要 ②③ 两步;①④⑤ 留作完整复现记录。 +**两相位均已有通过证据。** 机器现在停在:重钉产物已装、`ALPHA_CDP=1` 已开、应用处于 +真实登录态。①–⑤ 留作完整复现记录;再次执行默认相位会产生真实 web-search 调用与计费。 ```bash # ① 用本仓标准 CDP 口子重启打包应用(这是拿到 sidecar 凭证的唯一通道) @@ -87,14 +87,14 @@ eager ajv `new Function` 撞 renderer CSP —— 本次均未复现:`renderer.lo pkill -f "/Applications/alpha-code.app" ; sleep 2 ALPHA_CDP=1 open -a /Applications/alpha-code.app -# ② 在应用里登录(平台代付模式),等模型目录出来 ← owner 从这里开始 +# ② 在应用里登录(平台代付模式),等模型目录出来 # ③ 登录态取证 —— 一条命令跑完 cd ~/app/alpha-code && bun docs/verification/2026-07-27-e7-packaged-live/probe.ts # ④ 在应用里登出(设置 → 退出登录) -# ⑤ 登出态 keyless 兜底取证 —— 已在 2026-07-28T01:37Z 跑过,exit 0 +# ⑤ 登出态 keyless 兜底取证 —— 已在 2026-07-28 跑过,exit 0 cd ~/app/alpha-code && bun docs/verification/2026-07-27-e7-packaged-live/probe.ts --keyless ``` @@ -105,35 +105,36 @@ cd ~/app/alpha-code && bun docs/verification/2026-07-27-e7-packaged-live/probe.t ## 4. 逐项判据与结果 `AC` 列对应 #643 正文三条。`结果` 列由探针填(`results/latest-.json` 是真源); -下表中已填的行来自新产物(`e578e00ae` / `60589c59c…`)上的真实运行,登录态那一半待 owner 执行。 +登录态结果来自重钉产物 `b8f030e0c` / `dded6b38…` 的 +[`logged-in-20260731T022038Z.json`](results/logged-in-20260731T022038Z.json)。 ### 登录态相位(默认) -| 项 | AC | 判据 | 结果 | -| ---- | ---- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------- | -| P0.1 | 前置 | `sha256(app.asar)` 等于本文件钉的值 | ☑ pass(2026-07-28T01:37Z) | -| P0.2 | 前置 | CDP 端口列出 renderer page target | ☑ pass | -| P0.3 | 前置 | `GET /global/health` → `{healthy:true}` | ☑ pass | -| P0.4 | 前置 | `auth.getState()` 为 `{status:"logged-in",mode:"platform"}` **且** `alpha-secrets/ALPHA_CLOUD_TOKEN` 在位 | ☐ 待 owner(未登录时已实测 `blocked`+exit 2) | -| P1.1 | AC1 | `GET /mcp` 里 `cloud` 的 `status === "connected"` | ☐ | -| P1.2 | AC1 | 从打包 sidecar 的原始 `OPENCODE_CONFIG_CONTENT` 断言 URL + `Bearer {file:…ALPHA_CLOUD_TOKEN}`;`GET /config` 只另证替换后 URL/值与密钥文件一致 | ☐ | -| P1.3 | AC1 | **LIVE-PATH catalog gate**:已部署端点的匿名 `tools/list` 存在匹配 `/web[_-]?search/` 的工具;本项不再冒充账户授权证据,账户绑定由 P2.2 证明 | ☐ | -| P1.4 | AC1 | 记录引擎侧真实工具 id(`sanitize("cloud")+"_"+sanitize(<远端名>)`),**不假定**是 `cloud_web_search` | ☐ | -| P1.5 | AC1 | 对 `GET /agent` 每个运行时 agent 按引擎 `Wildcard.match + findLast` 语义计算 `websearch` 有效判决,要求全部为 `deny`;后置用户 agent allow 会使本项变红 | ☐ | -| P1.6 | AC1 | `/config/providers` 里存在网关 provider 且有 `capabilities.toolcall` 模型 | ☐ | -| P1.7 | AC1 | `GET /experimental/tool?provider&model` **不含** `websearch`(本地 keyless 被抑制) | ☐ | -| P2.1 | AC1 | **打包真调**:一次真实模型轮次产出该云工具的 tool part,`status==="completed"`,输出解析出 `{query,results}` | ☐ | -| P2.2 | AC1 | **LIVE-PATH GATE ②** 用应用自己的 token 直接 `tools/call`,返回 `{query,results}` 且 `isError !== true` | ☐ | -| P2.3 | AC3 | **计费**:有界轮询 `waitUntil` 后台结算,要求 Ledger V1 新增 `kind=usage_settled`、`actionId=tool.web_search` 且 `seq` 不在调用前页面中的事实 | ☐ | -| P3.1 | AC3 | **401**:无 Authorization 打 `POST {platform}/v1/tools/web_search` → 401 | ☐ | -| P3.2 | AC3 | **400**:带真 bearer、body `{}` → 400 | ☐ | -| P3.3 | AC3 | **400**:带真 bearer、坏 JSON → 400 | ☐ | -| P3.4 | AC3 | **403** —— `not-producible`(桌面端只持有 `model.invoke` / `cloud.dispatch` 两个 route-purpose 绑定令牌,造不出 scope 不足的令牌);映射由 L1 `alpha-websearch-failure.test.ts` 覆盖 | ☐ | -| P3.5 | AC3 | **502** —— `not-producible`(需已部署 gateway 同时缺 `TAVILY_API_KEY`/`BRAVE_API_KEY`;两把钥匙都在位,拆生产配置不在探针权限内) | ☐ | -| P3.6 | AC3 | **意外状态 LOUD** —— `not-producible`(桌面端够得着的请求形态都落在 {400,401,402,403,502} 内);映射由 L1 覆盖 | ☐ | -| P3.7 | AC3 | **402 / 余额**:带真 bearer 打 `/v1/tools/web_search` → 402(账户额度+余额双空时);200 ⇒ 账户被预授权通过 = 今天产生不了,记 `not-producible`;其它状态 LOUD。见 §6 | ☐ | -| P3.8 | AC3 | **defect 消失**:平台模型工具表不含本地 `websearch`,所以模型不可能尝试它;直接拒绝的可辨文案留给既有 L1 `alpha-websearch-failure.test.ts` | ☐ | -| P3.9 | AC3 | 云侧失败 loud 但**不可分类**(平台薄壳丢弃 `r.status`)—— 已登记缺口 `alpha-platform#105` | ☐ | +| 项 | AC | 判据 | 结果 | +| ---- | ---- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------- | +| P0.1 | 前置 | `sha256(app.asar)` 等于本文件钉的值 | ☑ pass(2026-07-31T02:20Z) | +| P0.2 | 前置 | CDP 端口列出 renderer page target | ☑ pass | +| P0.3 | 前置 | `GET /global/health` → `{healthy:true}` | ☑ pass | +| P0.4 | 前置 | `auth.getState()` 为 `{status:"logged-in",mode:"platform"}` **且** `alpha-secrets/ALPHA_CLOUD_TOKEN` 在位 | ☑ pass | +| P1.1 | AC1 | `GET /mcp` 里 `cloud` 的 `status === "connected"` | ☑ pass | +| P1.2 | AC1 | 由打包引擎创建短命 PTY 子进程,从继承的原始 `OPENCODE_CONFIG_CONTENT` 只输出布尔断言 URL + `Bearer {file:…ALPHA_CLOUD_TOKEN}`;`GET /config` 另证替换后 URL/值与密钥文件一致 | ☑ pass | +| P1.3 | AC1 | **LIVE-PATH catalog gate**:已部署端点的匿名 `tools/list` 存在匹配 `/web[_-]?search/` 的工具;本项不再冒充账户授权证据,账户绑定由 P2.2 证明 | ☑ pass | +| P1.4 | AC1 | 记录引擎侧真实工具 id(`sanitize("cloud")+"_"+sanitize(<远端名>)`),**不假定**是 `cloud_web_search` | ☑ pass:`cloud_cloud_web_search`(`#650`) | +| P1.5 | AC1 | 对 `GET /agent` 每个运行时 agent 按引擎 `Wildcard.match + findLast` 语义计算 `websearch` 有效判决,要求全部为 `deny`;后置用户 agent allow 会使本项变红 | ☑ pass | +| P1.6 | AC1 | `/config/providers` 里存在网关 provider 且有 `capabilities.toolcall` 模型 | ☑ pass | +| P1.7 | AC1 | `GET /experimental/tool?provider&model` **不含** `websearch`(本地 keyless 被抑制) | ☑ pass | +| P2.1 | AC1 | **打包真调**:一次真实模型轮次产出该云工具的 tool part,`status==="completed"`,输出解析出 `{query,results}` | ☑ pass | +| P2.2 | AC1 | **LIVE-PATH GATE ②** 用应用自己的 token 直接 `tools/call`,返回 `{query,results}` 且 `isError !== true` | ☑ pass:HTTP 200 | +| P2.3 | AC3 | **计费**:有界轮询 `waitUntil` 后台结算,要求 Ledger V1 新增 `reservation_created(actionId=tool.web_search) → usage_settled → reservation_settled`,三者共享新 `reservationId` | ☑ pass:2 笔各 15 分,总差分 30 分 | +| P3.1 | AC3 | **401**:无 Authorization 打 `POST {platform}/v1/tools/web_search` → 401 | ☑ pass | +| P3.2 | AC3 | **400**:带真 bearer、body `{}` → 400 | ☑ pass | +| P3.3 | AC3 | **400**:带真 bearer、坏 JSON → 400 | ☑ pass | +| P3.4 | AC3 | **403** —— `not-producible`(桌面端只持有 `model.invoke` / `cloud.dispatch` 两个 route-purpose 绑定令牌,造不出 scope 不足的令牌);映射由 L1 `alpha-websearch-failure.test.ts` 覆盖 | ◇ not-producible | +| P3.5 | AC3 | **502** —— `not-producible`(需已部署 gateway 同时缺 `TAVILY_API_KEY`/`BRAVE_API_KEY`;两把钥匙都在位,拆生产配置不在探针权限内) | ◇ not-producible | +| P3.6 | AC3 | **意外状态 LOUD** —— `not-producible`(桌面端够得着的请求形态都落在 {400,401,402,403,502} 内);映射由 L1 覆盖 | ◇ not-producible | +| P3.7 | AC3 | **402 / 余额**:带真 bearer 打 `/v1/tools/web_search` → 402(账户额度+余额双空时);200 ⇒ 账户被预授权通过 = 今天产生不了,记 `not-producible`;其它状态 LOUD。见 §6 | ◇ not-producible:当前账户有余额,HTTP 200 | +| P3.8 | AC3 | **defect 消失**:平台模型工具表不含本地 `websearch`,所以模型不可能尝试它;直接拒绝的可辨文案留给既有 L1 `alpha-websearch-failure.test.ts` | ☑ pass | +| P3.9 | AC3 | 云侧失败 loud 但**不可分类**(平台薄壳丢弃 `r.status`)—— 已登记缺口 `alpha-platform#105` | ◇ not-producible / 已登记 `#105` | ### 登出态相位(`--keyless`) @@ -166,16 +167,31 @@ cd ~/app/alpha-code && bun docs/verification/2026-07-27-e7-packaged-live/probe.t 2026-07-28 的登录态结果暴露出三条假红与一条空绿。此次只修探针和判读文档,不改产品代码, 也不改任何旧 `results/` 原始证据: -1. P1.2 不再在已经完成 `{file:}` 替换的 `GET /config` 上找引用,改看打包 sidecar 的源侧 - `OPENCODE_CONFIG_CONTENT`;运行时视图只证明替换后的 URL 与密钥文件一致。 +1. P1.2 不再在已经完成 `{file:}` 替换的 `GET /config` 上找引用。打包引擎经自己的 PTY API + 创建短命子进程;该进程继承引擎运行时的原始 `OPENCODE_CONFIG_CONTENT`,但只输出 URL/文件引用 + 是否匹配的布尔值,不输出配置或令牌。`GET /config` 只另证替换后的 URL 与密钥文件一致。 2. P1.5 不再要求每份配置对象都显式带 deny,改按 `/agent` 的最终 ruleset 算有效判决。 3. P1.3 明确是匿名 catalog 可用性,不再写“for this account”;P2.2 的真 `tools/call` 才是账户绑定的 LIVE-PATH gate。 4. P3.8 不再要求模型调用一项已被 P1.7 从工具表移除的工具;它与 P1.7 共用可观察事实, 可辨拒绝文案由 L1 负责。 5. 计费差分同步适配 Ledger V1:`id` 已不存在,以 `seq` 判断新事实,并轮询等待 Workers - `waitUntil` 结算出现精确的 `usage_settled/tool.web_search` 事实,而不是固定睡 4 秒后 - 接受任意新流水。 + `waitUntil` 结算出现 `reservation_created(actionId=tool.web_search)` 及同一 + `reservationId` 下的 `usage_settled`、`reservation_settled`。`actionId` 属于预留事实, + 结算事实不重复携带它;探针不再错误要求该字段。 + +第一次真实登录态运行 +[`logged-in-20260731T021405Z.json`](results/logged-in-20260731T021405Z.json) +被原探针自身打出 P1.2/P2.3 两条假红,但同一份记录已经显示: +替换后配置命中密钥文件,且两笔 web-search 都有完整结算链、余额合计扣 30 分。 +该失败记录保留不改,用于证明判据为何必须修正。修正后的 +[`logged-in-20260731T022038Z.json`](results/logged-in-20260731T022038Z.json) +在同一 `app.asar` 上 23 项、0 个必需失败;两次调用继续产生两笔各 15 分的精确结算。 + +本次生产被测端点对应部署版本: + +- `alpha-gateway`: `8c42c99b-c8d7-4414-b63f-f197e042c36f` +- `alpha-cloud`: `06b278d4-a7e3-4d32-bca4-4520010dbdd8` ## 5. 曾经的阻断项 —— keyless 真调在 `94a76b669` 上是坏的,`e578e00ae` 上已修复 diff --git a/docs/verification/2026-07-27-e7-packaged-live/probe.ts b/docs/verification/2026-07-27-e7-packaged-live/probe.ts index 85e765fb3020..b6e9c43486be 100644 --- a/docs/verification/2026-07-27-e7-packaged-live/probe.ts +++ b/docs/verification/2026-07-27-e7-packaged-live/probe.ts @@ -149,6 +149,7 @@ type Engine = { auth: string get: (p: string) => Promise post: (p: string, b: unknown) => Promise + delete: (p: string) => Promise } /** A model turn can run for minutes; anything longer than this is a hang, and a hung probe is @@ -182,6 +183,7 @@ function makeEngine(base: string, auth: string): Engine { auth, get: (p) => call("GET", p), post: (p, b) => call("POST", p, b), + delete: (p) => call("DELETE", p), } } @@ -259,92 +261,111 @@ function localWebSearchIsHidden(toolIDs: string[] | undefined) { return toolIDs !== undefined && !toolIDs.includes(LOCAL_WEB_SEARCH_TOOL_ID) } -function newSettledWebSearchFacts(facts: LedgerFact[], before: Set) { - return facts.filter( - (fact) => !before.has(fact.seq) && fact.kind === "usage_settled" && fact.actionId === "tool.web_search", +function newWebSearchSettlements(facts: LedgerFact[], before: Set) { + const fresh = facts.filter((fact) => !before.has(fact.seq)) + const reservations = fresh.filter( + (fact) => + fact.kind === "reservation_created" && + fact.actionId === "tool.web_search" && + typeof fact.reservationId === "string", ) + return Array.from(new Set(reservations.map((fact) => fact.reservationId!))).map((reservationId) => ({ + reservationId, + reservations: reservations.filter((fact) => fact.reservationId === reservationId), + usage: fresh.filter((fact) => fact.reservationId === reservationId && fact.kind === "usage_settled"), + terminal: fresh.filter((fact) => fact.reservationId === reservationId && fact.kind === "reservation_settled"), + })) } -/** `ps eww` is a source-side observation: unlike GET /config, it sees the sidecar's raw - * OPENCODE_CONFIG_CONTENT before ConfigVariable substitutes `{file:...}`. Parse only the balanced - * JSON value and never retain or print the rest of the process environment. */ -function extractJsonEnvironment(line: string, name: string) { - const marker = `${name}=` - const start = line.indexOf(marker) - if (start < 0) return undefined - const valueStart = start + marker.length - if (line[valueStart] !== "{") return undefined - let depth = 0 - let quoted = false - let escaped = false - for (let index = valueStart; index < line.length; index++) { - const char = line[index]! - if (escaped) { - escaped = false - continue - } - if (quoted && char === "\\") { - escaped = true - continue - } - if (char === '"') { - quoted = !quoted - continue - } - if (quoted) continue - if (char === "{") depth++ - if (char !== "}") continue - depth-- - if (depth !== 0) continue - try { - return JSON.parse(line.slice(valueStart, index + 1)) as { mcp?: Record } - } catch { - return undefined - } - } - return undefined +type ConfigSourceObservation = { + present: boolean + parsed: boolean + urlMatches: boolean + authorizationMatchesFileReference: boolean } -function readPackagedSidecarConfigSource(expectedUrl: string) { - // Resolve only the packaged NodeService PID without environments first. Asking `ps eww -ax` - // would unnecessarily copy unrelated processes' environments into this probe. - const processList = Bun.spawnSync(["ps", "-axo", "pid=,command="]) - if (processList.exitCode !== 0) - return { - config: undefined, - packagedCandidates: 0, - error: processList.stderr.toString().trim().slice(0, 300) || `ps exited ${processList.exitCode}`, - } - const sidecarPids = processList.stdout - .toString() - .split("\n") - .filter( - (line) => - line.includes(APP) && - line.includes("--type=utility") && - line.includes("--utility-sub-type=node.mojom.NodeService"), - ) - .flatMap((line) => { - const pid = line.trim().match(/^(\d+)\s/)?.[1] - return pid ? [pid] : [] +/** Spawn a short-lived child through the packaged engine's own PTY API. Pty.Service merges the + * engine's current `process.env` into every child, so this observes raw OPENCODE_CONFIG_CONTENT + * before ConfigVariable resolves `{file:...}`. The child prints only booleans; neither the config + * nor either credential can reach stdout/evidence. */ +async function observePackagedConfigSource( + engine: Engine, + expectedUrl: string, + expectedSecretFile: string, +): Promise<{ observation?: ConfigSourceObservation; error?: string }> { + const marker = "ALPHA_E7_P1_2=" + const child = [ + "const result = { present: false, parsed: false, urlMatches: false, authorizationMatchesFileReference: false };", + "const raw = process.env.OPENCODE_CONFIG_CONTENT;", + 'result.present = typeof raw === "string" && raw.length > 0;', + "try {", + ' const config = JSON.parse(raw ?? "{}");', + " result.parsed = true;", + ` const definition = config?.mcp?.[${JSON.stringify(CLOUD_MCP_SERVER_NAME)}];`, + " result.urlMatches = definition?.url === process.argv[1];", + ' result.authorizationMatchesFileReference = definition?.headers?.Authorization === "Bearer {file:" + process.argv[2] + "}";', + "} catch {}", + `console.log(${JSON.stringify(marker)} + JSON.stringify(result));`, + "await Bun.sleep(5000);", + ].join("\n") + + let ptyID: string | undefined + let socket: WebSocket | undefined + try { + const created = await engine.post("/pty", { + command: process.execPath, + args: ["-e", child, expectedUrl, expectedSecretFile], + cwd: import.meta.dir, + title: "Alpha E7 config-source probe", }) - const configs = sidecarPids - .map((pid) => Bun.spawnSync(["ps", "eww", "-p", pid, "-o", "command="])) - .filter((result) => result.exitCode === 0) - .map((result) => result.stdout.toString()) - .flatMap((line) => { - const config = extractJsonEnvironment(line, "OPENCODE_CONFIG_CONTENT") - return config ? [config] : [] + ptyID = created?.id + if (!ptyID) return { error: "engine PTY create returned no id" } + + const next = new URL(`${engine.base}/pty/${encodeURIComponent(ptyID)}/connect`) + next.protocol = next.protocol === "https:" ? "wss:" : "ws:" + next.searchParams.set("cursor", "0") + next.searchParams.set("auth_token", engine.auth.replace(/^Basic\s+/i, "")) + const result = await new Promise<{ observation?: ConfigSourceObservation; error?: string }>((resolve) => { + let output = "" + let settled = false + const finish = (value: { observation?: ConfigSourceObservation; error?: string }) => { + if (settled) return + settled = true + clearTimeout(timer) + resolve(value) + } + const timer = setTimeout(() => finish({ error: "timed out waiting for sanitized PTY observation" }), 10_000) + socket = new WebSocket(next) + socket.onmessage = (event) => { + const data = + typeof event.data === "string" + ? event.data + : event.data instanceof ArrayBuffer + ? new TextDecoder().decode(event.data) + : "" + if (data.charCodeAt(0) === 0) return + output += data + const markerIndex = output.indexOf(marker) + if (markerIndex < 0) return + const tail = output.slice(markerIndex + marker.length) + const lineEnd = tail.search(/\r?\n/) + if (lineEnd < 0) return + const line = tail.slice(0, lineEnd) + try { + finish({ observation: JSON.parse(line) as ConfigSourceObservation }) + } catch { + finish({ error: "sanitized PTY observation was not valid JSON" }) + } + } + socket.onerror = () => finish({ error: "PTY websocket failed" }) + socket.onclose = () => finish({ error: "PTY closed before emitting the sanitized observation" }) }) - return { - config: configs.find((config) => config.mcp?.[CLOUD_MCP_SERVER_NAME]?.url === expectedUrl), - packagedCandidates: configs.length, - error: - sidecarPids.length === 0 - ? "no packaged NodeService sidecar process found" - : configs.length === 0 - ? `OPENCODE_CONFIG_CONTENT was not observable on ${sidecarPids.length} packaged sidecar process(es)` - : undefined, + return result + } catch (error) { + return { error: String(error).slice(0, 400) } + } finally { + socket?.close() + if (ptyID) await engine.delete(`/pty/${encodeURIComponent(ptyID)}`).catch(() => undefined) } } @@ -359,17 +380,38 @@ function runCriterionSelfTests() { }, }, } - const extractedSource = extractJsonEnvironment( - `sidecar OPENCODE_CONFIG_CONTENT=${JSON.stringify(goodSource)} OTHER_ENV=1`, - "OPENCODE_CONFIG_CONTENT", - ) const denied = [{ permission: "websearch", pattern: "*", action: "deny" }] + const goodLedger: LedgerFact[] = [ + { + seq: 2, + kind: "reservation_created", + domain: "wallet", + amount: 15, + actionId: "tool.web_search", + reservationId: "reservation-good", + createdAt: 1, + }, + { + seq: 3, + kind: "usage_settled", + domain: "wallet", + amount: -15, + reservationId: "reservation-good", + createdAt: 2, + }, + { + seq: 4, + kind: "reservation_settled", + domain: "wallet", + amount: 0, + reservationId: "reservation-good", + createdAt: 2, + }, + ] const tests = [ { name: "P1.2 accepts the source-side file reference", - ok: - cloudDefinitionUsesFileReference(goodSource, expectedUrl, expectedSecret) && - cloudDefinitionUsesFileReference(extractedSource, expectedUrl, expectedSecret), + ok: cloudDefinitionUsesFileReference(goodSource, expectedUrl, expectedSecret), }, { name: "P1.2 rejects an inlined bearer bypass", @@ -399,34 +441,20 @@ function runCriterionSelfTests() { ok: localWebSearchIsHidden(["read"]) && !localWebSearchIsHidden(["read", "websearch"]), }, { - name: "P2.3 requires a new settled web-search ledger fact", - ok: - newSettledWebSearchFacts( - [ - { - seq: 2, - kind: "usage_settled", - domain: "wallet", - amount: -1, - actionId: "tool.web_search", - createdAt: 1, - }, - ], - new Set([1]), - ).length === 1 && - newSettledWebSearchFacts( - [ - { - seq: 2, - kind: "reservation_created", - domain: "wallet", - amount: -1, - actionId: "tool.web_search", - createdAt: 1, - }, - ], + name: "P2.3 correlates reservation and settlement facts by reservationId", + ok: (() => { + const [complete] = newWebSearchSettlements(goodLedger, new Set([1])) + const [mismatched] = newWebSearchSettlements( + goodLedger.map((fact) => (fact.kind === "usage_settled" ? { ...fact, reservationId: "other" } : fact)), new Set([1]), - ).length === 0, + ) + return ( + complete?.usage.length === 1 && + complete.terminal.length === 1 && + mismatched?.usage.length === 0 && + mismatched.terminal.length === 1 + ) + })(), }, ] for (const test of tests) console.log(`[${test.ok ? "PASS" : "FAIL"}] ${test.name}`) @@ -814,12 +842,12 @@ const engineConfig = (await engine.get("/config")) as { } const cloudDef = engineConfig.mcp?.[CLOUD_MCP_SERVER_NAME] const expectedMcpUrl = endpoints.mcp ?? `${endpoints.cloud}/mcp` -const sourceConfig = readPackagedSidecarConfigSource(expectedMcpUrl) -const sourceUsesFileReference = cloudDefinitionUsesFileReference( - sourceConfig.config, - expectedMcpUrl, - secretFile("ALPHA_CLOUD_TOKEN"), -) +const sourceConfig = await observePackagedConfigSource(engine, expectedMcpUrl, secretFile("ALPHA_CLOUD_TOKEN")) +const sourceUsesFileReference = + sourceConfig.observation?.present === true && + sourceConfig.observation.parsed === true && + sourceConfig.observation.urlMatches === true && + sourceConfig.observation.authorizationMatchesFileReference === true const resolvedAuthorizationMatchesSecret = cloudDef?.headers?.Authorization === `Bearer ${cloudToken}` assertCheck( { @@ -827,7 +855,7 @@ assertCheck( ac: "AC1 packaged 登录态 listTools", title: "the source-side cloud MCP definition uses a {file:} token reference and resolves to the expected endpoint", criterion: - `the packaged sidecar's raw OPENCODE_CONFIG_CONTENT has mcp.${CLOUD_MCP_SERVER_NAME}.url === "${expectedMcpUrl}" ` + + `a short-lived child spawned by the packaged engine inherits raw OPENCODE_CONFIG_CONTENT with mcp.${CLOUD_MCP_SERVER_NAME}.url === "${expectedMcpUrl}" ` + `AND Authorization === "Bearer {file:/alpha-secrets/ALPHA_CLOUD_TOKEN}"; ` + "GET /config separately resolves that definition to the same URL and the secret-file value", required: true, @@ -835,8 +863,8 @@ assertCheck( sourceUsesFileReference && cloudDef?.url === expectedMcpUrl && resolvedAuthorizationMatchesSecret, { expectedMcpUrl, - sourceProcessCandidates: sourceConfig.packagedCandidates, sourceObservationError: sourceConfig.error, + sourceObservation: sourceConfig.observation, sourceUsesFileReference, resolvedUrl: cloudDef?.url, resolvedAuthorizationMatchesSecret, @@ -1058,12 +1086,14 @@ assertCheck( const settlementPollStartedAt = Date.now() let summaryAfter = await cdp!.eval("window.api.account.summary()") let txAfter = await cdp!.eval("window.api.account.transactions(50)") -let settledWebSearchFacts = newSettledWebSearchFacts(txAfter?.transactions ?? [], txSeqBefore) -while (settledWebSearchFacts.length === 0 && Date.now() - settlementPollStartedAt < BILLING_SETTLE_TIMEOUT_MS) { +let webSearchSettlements = newWebSearchSettlements(txAfter?.transactions ?? [], txSeqBefore) +const settlementComplete = () => + webSearchSettlements.some((settlement) => settlement.usage.length > 0 && settlement.terminal.length > 0) +while (!settlementComplete() && Date.now() - settlementPollStartedAt < BILLING_SETTLE_TIMEOUT_MS) { await sleep(1000) summaryAfter = await cdp!.eval("window.api.account.summary()") txAfter = await cdp!.eval("window.api.account.transactions(50)") - settledWebSearchFacts = newSettledWebSearchFacts(txAfter?.transactions ?? [], txSeqBefore) + webSearchSettlements = newWebSearchSettlements(txAfter?.transactions ?? [], txSeqBefore) } const newFacts = ((txAfter?.transactions ?? []) as LedgerFact[]).filter((fact) => !txSeqBefore.has(fact.seq)) const walletDelta = Number(summaryAfter?.walletUsedFen ?? 0) - Number(summaryBefore?.walletUsedFen ?? 0) @@ -1074,11 +1104,11 @@ assertCheck( ac: "AC3 计费(ledger/settle)证据", title: "waitUntil-backed web-search settlement reaches this tenant's append-only ledger", criterion: - `within ${BILLING_SETTLE_TIMEOUT_MS}ms after the real calls, the account ledger appends a fact with ` + - 'kind="usage_settled" AND actionId="tool.web_search" and a seq absent from the pre-call page', + `within ${BILLING_SETTLE_TIMEOUT_MS}ms after the real calls, the account ledger appends ` + + 'reservation_created(actionId="tool.web_search"), usage_settled, and reservation_settled facts sharing one new reservationId', required: true, }, - settledWebSearchFacts.length > 0, + settlementComplete(), { settlementObservedAfterMs: Date.now() - settlementPollStartedAt, settlementTimeoutMs: BILLING_SETTLE_TIMEOUT_MS, @@ -1088,14 +1118,12 @@ assertCheck( balanceFenAfter: summaryAfter?.balanceFen, walletDelta, balanceDelta, - settledWebSearchFacts: settledWebSearchFacts.map((fact) => ({ - seq: fact.seq, - kind: fact.kind, - domain: fact.domain, - amount: fact.amount, - actionId: fact.actionId, - reservationId: fact.reservationId, - createdAt: fact.createdAt, + webSearchSettlements: webSearchSettlements.map((settlement) => ({ + reservationId: settlement.reservationId, + reservationSeqs: settlement.reservations.map((fact) => fact.seq), + usageSeqs: settlement.usage.map((fact) => fact.seq), + terminalSeqs: settlement.terminal.map((fact) => fact.seq), + amountSettled: settlement.usage.reduce((sum, fact) => sum + fact.amount, 0), })), newLedgerFacts: newFacts.map((fact) => ({ seq: fact.seq, From eb7ff05fb7494d17f4ecdfdb176011d724e8e82a Mon Sep 17 00:00:00 2001 From: jinjunnn Date: Thu, 30 Jul 2026 22:23:31 -0400 Subject: [PATCH 4/5] docs(verification): record #643 live billing evidence --- .../results/latest-logged-in.json | 579 ++++++++++-- .../results/logged-in-20260731T021405Z.json | 805 +++++++++++++++++ .../results/logged-in-20260731T022038Z.json | 833 ++++++++++++++++++ 3 files changed, 2123 insertions(+), 94 deletions(-) create mode 100644 docs/verification/2026-07-27-e7-packaged-live/results/logged-in-20260731T021405Z.json create mode 100644 docs/verification/2026-07-27-e7-packaged-live/results/logged-in-20260731T022038Z.json diff --git a/docs/verification/2026-07-27-e7-packaged-live/results/latest-logged-in.json b/docs/verification/2026-07-27-e7-packaged-live/results/latest-logged-in.json index bc2d1c726b58..f607a04d8a6c 100644 --- a/docs/verification/2026-07-27-e7-packaged-live/results/latest-logged-in.json +++ b/docs/verification/2026-07-27-e7-packaged-live/results/latest-logged-in.json @@ -2,24 +2,24 @@ "schema": "alpha-code/e7-packaged-live/v1", "issue": "alpha-code#643", "phase": "logged-in", - "capturedAt": "2026-07-28T02:05:45.876Z", - "finishedAt": "2026-07-28T02:06:07.248Z", + "capturedAt": "2026-07-31T02:20:38.799Z", + "finishedAt": "2026-07-31T02:20:56.635Z", "build": { "app": "/Applications/alpha-code.app", - "asarSha256": "60589c59c58e44ac0daede93fc7397a8a04365f5345eac4312e205a0d8f48e44", - "appMtime": "2026-07-28T01:31:42.882Z", - "pinnedCommit": "e578e00ae", + "asarSha256": "dded6b38f023e2bbaba3c152032a857f80221d280949c6838374741aea6f42b9", + "appMtime": "2026-07-31T02:12:02.218Z", + "pinnedCommit": "b8f030e0c", "appVersion": "1.17.13" }, "engine": { - "url": "http://127.0.0.1:50360", + "url": "http://127.0.0.1:50152", "version": "1.17.13" }, "auth": { "status": "logged-in", "mode": "platform", "account": {}, - "expiresAt": 1785205182135, + "expiresAt": 1785464921725, "platformStatus": "ready" }, "endpoints": { @@ -38,15 +38,15 @@ "ac": "preflight", "title": "app under test is the build this evidence directory was cut for", "status": "pass", - "criterion": "sha256(app.asar) === 60589c59c58e44ac0daede93fc7397a8a04365f5345eac4312e205a0d8f48e44 (alpha @ e578e00ae)", + "criterion": "sha256(app.asar) === dded6b38f023e2bbaba3c152032a857f80221d280949c6838374741aea6f42b9 (alpha @ b8f030e0c)", "observed": { "asarPath": "/Applications/alpha-code.app/Contents/Resources/app.asar", - "asarSha": "60589c59c58e44ac0daede93fc7397a8a04365f5345eac4312e205a0d8f48e44", - "appMtime": "2026-07-28T01:31:42.882Z", - "pinnedCommit": "e578e00ae" + "asarSha": "dded6b38f023e2bbaba3c152032a857f80221d280949c6838374741aea6f42b9", + "appMtime": "2026-07-31T02:12:02.218Z", + "pinnedCommit": "b8f030e0c" }, "required": true, - "at": "2026-07-28T02:05:45.967Z" + "at": "2026-07-31T02:20:38.908Z" }, { "id": "P0.2", @@ -58,7 +58,7 @@ "port": 9222 }, "required": true, - "at": "2026-07-28T02:05:45.987Z" + "at": "2026-07-31T02:20:38.923Z" }, { "id": "P0.3", @@ -67,14 +67,14 @@ "status": "pass", "criterion": "GET /global/health → { healthy: true }", "observed": { - "url": "http://127.0.0.1:50360", + "url": "http://127.0.0.1:50152", "health": { "healthy": true, "version": "1.17.13" } }, "required": true, - "at": "2026-07-28T02:05:45.995Z" + "at": "2026-07-31T02:20:38.943Z" }, { "id": "P0.4", @@ -87,7 +87,7 @@ "status": "logged-in", "mode": "platform", "account": {}, - "expiresAt": 1785205182135, + "expiresAt": 1785464921725, "platformStatus": "ready" }, "cloudTokenFile": { @@ -107,7 +107,7 @@ } }, "required": true, - "at": "2026-07-28T02:05:45.997Z" + "at": "2026-07-31T02:20:38.948Z" }, { "id": "P1.1", @@ -121,34 +121,36 @@ "status": "connected" } }, - "at": "2026-07-28T02:05:48.103Z" + "at": "2026-07-31T02:20:38.960Z" }, { "id": "P1.2", "ac": "AC1 packaged 登录态 listTools", - "title": "cloud MCP definition points at the app-resolved URL and carries the token by {file:} reference only", - "criterion": "config.mcp.cloud.url === \"https://alpha-cloud.tidelabs.click/mcp\" AND headers.Authorization matches /^Bearer \\{file:.*ALPHA_CLOUD_TOKEN\\}$/", + "title": "the source-side cloud MCP definition uses a {file:} token reference and resolves to the expected endpoint", + "criterion": "a short-lived child spawned by the packaged engine inherits raw OPENCODE_CONFIG_CONTENT with mcp.cloud.url === \"https://alpha-cloud.tidelabs.click/mcp\" AND Authorization === \"Bearer {file:/alpha-secrets/ALPHA_CLOUD_TOKEN}\"; GET /config separately resolves that definition to the same URL and the secret-file value", "required": true, - "status": "fail", + "status": "pass", "observed": { "expectedMcpUrl": "https://alpha-cloud.tidelabs.click/mcp", - "cloudDef": { - "type": "remote", - "url": "https://alpha-cloud.tidelabs.click/mcp", - "enabled": true, - "headers": { - "Authorization": "Bearer " - }, - "oauth": false - } + "sourceObservation": { + "present": true, + "parsed": true, + "urlMatches": true, + "authorizationMatchesFileReference": true + }, + "sourceUsesFileReference": true, + "resolvedUrl": "https://alpha-cloud.tidelabs.click/mcp", + "resolvedAuthorizationMatchesSecret": true, + "secretFileMode": "600", + "secretDirectoryMode": "700" }, - "at": "2026-07-28T02:05:48.125Z" + "at": "2026-07-31T02:20:39.045Z" }, { "id": "P1.3", "ac": "AC1 packaged 登录态 listTools", - "title": "LIVE-PATH GATE ① — deployed cloud worker lists a web-search tool for this account", - "criterion": "MCP tools/list on the app-resolved endpoint (app's own bearer) contains a tool matching /web[_-]?search/", + "title": "LIVE-PATH catalog gate — the deployed cloud worker advertises a web-search tool", + "criterion": "MCP tools/list on the app-resolved endpoint contains a tool matching /web[_-]?search/; this anonymous catalog check makes no account-authorization claim (P2.2 does)", "required": true, "status": "pass", "observed": { @@ -163,9 +165,11 @@ "cloud_web_search", "cloud_schedule_delete" ], - "matched": "cloud_web_search" + "matched": "cloud_web_search", + "accountBound": false, + "accountBoundGate": "P2.2 tools/call with the app-issued cloud bearer" }, - "at": "2026-07-28T02:05:48.768Z" + "at": "2026-07-31T02:20:40.100Z" }, { "id": "P1.4", @@ -180,30 +184,121 @@ }, "required": true, "note": "MISMATCH: alpha pins \"cloud_web_search\" but the engine will register \"cloud_cloud_web_search\" — every alpha gate keyed on the pinned id is currently an EMPTY gate (it watches a tool id the engine never registers). Re-check each one against the real id and open a CODE ticket; README §7 lists them and the judgement method.", - "at": "2026-07-28T02:05:48.768Z" + "at": "2026-07-31T02:20:40.100Z" }, { "id": "P1.5", "ac": "AC1 云优先(本地 keyless 被抑制)", - "title": "platform-pays denies the local keyless websearch globally and on every injected agent", - "criterion": "config.permission.websearch === \"deny\" AND every config.agent[*].permission.websearch === \"deny\"", + "title": "every runtime agent's effective permission decision denies local keyless websearch", + "criterion": "for every GET /agent entry, Permission.evaluate(\"websearch\", \"*\", ruleset) using engine-equivalent Wildcard.match + findLast semantics returns action=deny; a later user-agent allow must make this gate fail", "required": true, - "status": "fail", + "status": "pass", "observed": { - "globalPermission": "deny", - "agentsMissingDeny": [ - "general", - "explore" - ], - "agents": [ - "general", - "explore", - "alpha-automation", - "alpha-readonly", - "alpha-automation-standard" + "agentsWithoutEffectiveDeny": [], + "decisions": [ + { + "name": "build", + "action": "deny", + "matchedRuleIndex": 33, + "matchedRule": { + "permission": "websearch", + "pattern": "*", + "action": "deny" + } + }, + { + "name": "alpha-automation", + "action": "deny", + "matchedRuleIndex": 46, + "matchedRule": { + "permission": "websearch", + "pattern": "*", + "action": "deny" + } + }, + { + "name": "alpha-automation-standard", + "action": "deny", + "matchedRuleIndex": 73, + "matchedRule": { + "permission": "websearch", + "pattern": "*", + "action": "deny" + } + }, + { + "name": "alpha-readonly", + "action": "deny", + "matchedRuleIndex": 43, + "matchedRule": { + "permission": "websearch", + "pattern": "*", + "action": "deny" + } + }, + { + "name": "compaction", + "action": "deny", + "matchedRuleIndex": 32, + "matchedRule": { + "permission": "websearch", + "pattern": "*", + "action": "deny" + } + }, + { + "name": "explore", + "action": "deny", + "matchedRuleIndex": 61, + "matchedRule": { + "permission": "websearch", + "pattern": "*", + "action": "deny" + } + }, + { + "name": "general", + "action": "deny", + "matchedRuleIndex": 32, + "matchedRule": { + "permission": "websearch", + "pattern": "*", + "action": "deny" + } + }, + { + "name": "plan", + "action": "deny", + "matchedRuleIndex": 38, + "matchedRule": { + "permission": "websearch", + "pattern": "*", + "action": "deny" + } + }, + { + "name": "summary", + "action": "deny", + "matchedRuleIndex": 32, + "matchedRule": { + "permission": "websearch", + "pattern": "*", + "action": "deny" + } + }, + { + "name": "title", + "action": "deny", + "matchedRuleIndex": 32, + "matchedRule": { + "permission": "websearch", + "pattern": "*", + "action": "deny" + } + } ] }, - "at": "2026-07-28T02:05:48.768Z" + "at": "2026-07-31T02:20:40.108Z" }, { "id": "P1.6", @@ -221,7 +316,7 @@ ] }, "required": true, - "at": "2026-07-28T02:05:48.773Z" + "at": "2026-07-31T02:20:40.111Z" }, { "id": "P1.7", @@ -250,7 +345,7 @@ "alpha_ping" ] }, - "at": "2026-07-28T02:05:48.788Z" + "at": "2026-07-31T02:20:40.115Z" }, { "id": "P2.1", @@ -258,20 +353,23 @@ "title": "the packaged app really invoked the cloud web-search tool through its own engine", "criterion": "a real model turn produced a tool part whose id is the cloud web-search tool, state.status === \"completed\", and whose output parses to an object with `query` and `results`", "required": true, - "status": "fail", + "status": "pass", "observed": { - "sessionID": "ses_05987684effeq6UCQ4kjJ5WxQg", + "sessionID": "ses_04a06ba39ffezzNQSdJIUR6r1K", "toolPartsSeen": [ { "tool": "cloud_cloud_web_search", - "status": "error" + "status": "completed" } ], "calledTool": "cloud_cloud_web_search", - "state": "error", - "error": "{\"error\":\"forbidden\"}" + "state": "completed", + "outputKeys": [ + "query", + "results" + ] }, - "at": "2026-07-28T02:05:55.594Z" + "at": "2026-07-31T02:20:49.163Z" }, { "id": "P2.2", @@ -279,33 +377,308 @@ "title": "LIVE-PATH GATE ② — one real call on the app-resolved endpoint returns {query, results}", "criterion": "MCP tools/call with the app's own cloud bearer returns isError !== true and a text payload parsing to an object with `query` and `results`", "required": true, - "status": "fail", + "status": "pass", "observed": { "httpStatus": 200, - "isError": true, + "isError": false, "payloadKeys": [ - "error" + "query", + "results" ] }, - "at": "2026-07-28T02:05:55.850Z" + "at": "2026-07-31T02:20:51.851Z" }, { "id": "P2.3", "ac": "AC3 计费(ledger/settle)证据", - "title": "the real web-search calls are billed to this tenant's ledger", - "criterion": "account summary walletUsedFen increased OR balanceFen decreased OR a new usage transaction appeared between the pre-call and post-call snapshots", + "title": "waitUntil-backed web-search settlement reaches this tenant's append-only ledger", + "criterion": "within 45000ms after the real calls, the account ledger appends reservation_created(actionId=\"tool.web_search\"), usage_settled, and reservation_settled facts sharing one new reservationId", "required": true, - "status": "fail", + "status": "pass", "observed": { - "walletUsedFenBefore": 0, - "walletUsedFenAfter": 0, - "balanceFenBefore": 100000, - "balanceFenAfter": 100000, - "walletDelta": 0, - "balanceDelta": 0, - "newTransactions": [] + "settlementObservedAfterMs": 75, + "settlementTimeoutMs": 45000, + "walletUsedFenBefore": 45, + "walletUsedFenAfter": 75, + "balanceFenBefore": 99955, + "balanceFenAfter": 99925, + "walletDelta": 30, + "balanceDelta": 30, + "webSearchSettlements": [ + { + "reservationId": "6b5b9c7a-a5bb-4d54-ba24-ad9d32ff3cdc", + "reservationSeqs": [ + 155, + 154, + 153 + ], + "usageSeqs": [ + 156 + ], + "terminalSeqs": [ + 157 + ], + "amountSettled": -15 + }, + { + "reservationId": "cb5ae08a-fe3e-482c-a13d-e8212cf93e8d", + "reservationSeqs": [ + 141, + 140, + 139 + ], + "usageSeqs": [ + 142 + ], + "terminalSeqs": [ + 143 + ], + "amountSettled": -15 + } + ], + "newLedgerFacts": [ + { + "seq": 157, + "kind": "reservation_settled", + "domain": "wallet", + "amount": 0, + "reservationId": "6b5b9c7a-a5bb-4d54-ba24-ad9d32ff3cdc", + "createdAt": 1785464456677 + }, + { + "seq": 156, + "kind": "usage_settled", + "domain": "wallet", + "amount": -15, + "reservationId": "6b5b9c7a-a5bb-4d54-ba24-ad9d32ff3cdc", + "createdAt": 1785464456677 + }, + { + "seq": 155, + "kind": "reservation_created", + "domain": "allowance", + "amount": 0, + "actionId": "tool.web_search", + "reservationId": "6b5b9c7a-a5bb-4d54-ba24-ad9d32ff3cdc", + "createdAt": 1785464454448 + }, + { + "seq": 154, + "kind": "reservation_created", + "domain": "allowance", + "amount": 0, + "actionId": "tool.web_search", + "reservationId": "6b5b9c7a-a5bb-4d54-ba24-ad9d32ff3cdc", + "createdAt": 1785464454448 + }, + { + "seq": 153, + "kind": "reservation_created", + "domain": "wallet", + "amount": 15, + "actionId": "tool.web_search", + "reservationId": "6b5b9c7a-a5bb-4d54-ba24-ad9d32ff3cdc", + "createdAt": 1785464454448 + }, + { + "seq": 152, + "kind": "reservation_settled", + "domain": "wallet", + "amount": 0, + "reservationId": "72383d9e-12f0-4c67-8bc8-0cb75bbc5ca0", + "createdAt": 1785464453963 + }, + { + "seq": 151, + "kind": "allowance_consumed", + "domain": "allowance", + "amount": -3, + "reservationId": "72383d9e-12f0-4c67-8bc8-0cb75bbc5ca0", + "createdAt": 1785464453963 + }, + { + "seq": 150, + "kind": "allowance_consumed", + "domain": "allowance", + "amount": -3, + "reservationId": "72383d9e-12f0-4c67-8bc8-0cb75bbc5ca0", + "createdAt": 1785464453962 + }, + { + "seq": 149, + "kind": "reservation_settled", + "domain": "wallet", + "amount": 0, + "reservationId": "78a6c08b-7a92-4578-8614-62453b11a46c", + "createdAt": 1785464452538 + }, + { + "seq": 148, + "kind": "allowance_consumed", + "domain": "allowance", + "amount": -1, + "reservationId": "78a6c08b-7a92-4578-8614-62453b11a46c", + "createdAt": 1785464452538 + }, + { + "seq": 147, + "kind": "allowance_consumed", + "domain": "allowance", + "amount": -1, + "reservationId": "78a6c08b-7a92-4578-8614-62453b11a46c", + "createdAt": 1785464452538 + }, + { + "seq": 146, + "kind": "reservation_created", + "domain": "allowance", + "amount": 224, + "actionId": "model.invoke.chat", + "reservationId": "72383d9e-12f0-4c67-8bc8-0cb75bbc5ca0", + "createdAt": 1785464452360 + }, + { + "seq": 145, + "kind": "reservation_created", + "domain": "allowance", + "amount": 224, + "actionId": "model.invoke.chat", + "reservationId": "72383d9e-12f0-4c67-8bc8-0cb75bbc5ca0", + "createdAt": 1785464452359 + }, + { + "seq": 144, + "kind": "reservation_created", + "domain": "wallet", + "amount": 0, + "actionId": "model.invoke.chat", + "reservationId": "72383d9e-12f0-4c67-8bc8-0cb75bbc5ca0", + "createdAt": 1785464452359 + }, + { + "seq": 143, + "kind": "reservation_settled", + "domain": "wallet", + "amount": 0, + "reservationId": "cb5ae08a-fe3e-482c-a13d-e8212cf93e8d", + "createdAt": 1785464452040 + }, + { + "seq": 142, + "kind": "usage_settled", + "domain": "wallet", + "amount": -15, + "reservationId": "cb5ae08a-fe3e-482c-a13d-e8212cf93e8d", + "createdAt": 1785464452040 + }, + { + "seq": 141, + "kind": "reservation_created", + "domain": "allowance", + "amount": 0, + "actionId": "tool.web_search", + "reservationId": "cb5ae08a-fe3e-482c-a13d-e8212cf93e8d", + "createdAt": 1785464449976 + }, + { + "seq": 140, + "kind": "reservation_created", + "domain": "allowance", + "amount": 0, + "actionId": "tool.web_search", + "reservationId": "cb5ae08a-fe3e-482c-a13d-e8212cf93e8d", + "createdAt": 1785464449976 + }, + { + "seq": 139, + "kind": "reservation_created", + "domain": "wallet", + "amount": 15, + "actionId": "tool.web_search", + "reservationId": "cb5ae08a-fe3e-482c-a13d-e8212cf93e8d", + "createdAt": 1785464449976 + }, + { + "seq": 138, + "kind": "reservation_settled", + "domain": "wallet", + "amount": 0, + "reservationId": "87b0706d-d865-46f7-af6c-61dc21b5c5bf", + "createdAt": 1785464448601 + }, + { + "seq": 137, + "kind": "allowance_consumed", + "domain": "allowance", + "amount": -3, + "reservationId": "87b0706d-d865-46f7-af6c-61dc21b5c5bf", + "createdAt": 1785464448600 + }, + { + "seq": 136, + "kind": "allowance_consumed", + "domain": "allowance", + "amount": -3, + "reservationId": "87b0706d-d865-46f7-af6c-61dc21b5c5bf", + "createdAt": 1785464448600 + }, + { + "seq": 135, + "kind": "reservation_created", + "domain": "allowance", + "amount": 224, + "actionId": "model.invoke.chat", + "reservationId": "78a6c08b-7a92-4578-8614-62453b11a46c", + "createdAt": 1785464446655 + }, + { + "seq": 134, + "kind": "reservation_created", + "domain": "allowance", + "amount": 224, + "actionId": "model.invoke.chat", + "reservationId": "78a6c08b-7a92-4578-8614-62453b11a46c", + "createdAt": 1785464446655 + }, + { + "seq": 133, + "kind": "reservation_created", + "domain": "wallet", + "amount": 0, + "actionId": "model.invoke.chat", + "reservationId": "78a6c08b-7a92-4578-8614-62453b11a46c", + "createdAt": 1785464446655 + }, + { + "seq": 132, + "kind": "reservation_created", + "domain": "allowance", + "amount": 224, + "actionId": "model.invoke.chat", + "reservationId": "87b0706d-d865-46f7-af6c-61dc21b5c5bf", + "createdAt": 1785464446433 + }, + { + "seq": 131, + "kind": "reservation_created", + "domain": "allowance", + "amount": 224, + "actionId": "model.invoke.chat", + "reservationId": "87b0706d-d865-46f7-af6c-61dc21b5c5bf", + "createdAt": 1785464446433 + }, + { + "seq": 130, + "kind": "reservation_created", + "domain": "wallet", + "amount": 0, + "actionId": "model.invoke.chat", + "reservationId": "87b0706d-d865-46f7-af6c-61dc21b5c5bf", + "createdAt": 1785464446433 + } + ] }, - "at": "2026-07-28T02:05:59.928Z" + "at": "2026-07-31T02:20:51.926Z" }, { "id": "P3.1", @@ -318,7 +691,7 @@ "status": 401, "body": "{\"error\":{\"message\":\"unauthorized\"}}" }, - "at": "2026-07-28T02:06:00.570Z" + "at": "2026-07-31T02:20:52.669Z" }, { "id": "P3.2", @@ -331,7 +704,7 @@ "status": 400, "body": "{\"error\":{\"message\":\"query required\"}}" }, - "at": "2026-07-28T02:06:01.090Z" + "at": "2026-07-31T02:20:52.904Z" }, { "id": "P3.3", @@ -344,7 +717,7 @@ "status": 400, "body": "{\"error\":{\"message\":\"invalid json\"}}" }, - "at": "2026-07-28T02:06:01.298Z" + "at": "2026-07-31T02:20:53.200Z" }, { "id": "P3.4", @@ -357,7 +730,7 @@ "coveredAtL1": "packages/opencode/test/tool/alpha-websearch-failure.test.ts (403 → forbidden, error.code preserved)" }, "required": false, - "at": "2026-07-28T02:06:01.298Z" + "at": "2026-07-31T02:20:53.200Z" }, { "id": "P3.5", @@ -370,7 +743,7 @@ "coveredAtL1": "packages/opencode/test/tool/alpha-websearch-failure.test.ts (502 → upstream)" }, "required": false, - "at": "2026-07-28T02:06:01.298Z" + "at": "2026-07-31T02:20:53.200Z" }, { "id": "P3.6", @@ -383,7 +756,7 @@ "coveredAtL1": "packages/opencode/test/tool/alpha-websearch-failure.test.ts (statusKind default → unexpected_status)" }, "required": false, - "at": "2026-07-28T02:06:01.298Z" + "at": "2026-07-31T02:20:53.200Z" }, { "id": "P3.7", @@ -393,10 +766,10 @@ "criterion": "POST /v1/tools/web_search with a valid bearer → HTTP 402 when the account is out of quota+wallet; a 200 proves the account is funded (402 not producible today without draining the owner's balance, which would also invalidate P2.1/P2.2/P2.3); any other status is unexpected and loud", "observed": { "status": 200, - "body": "{\"query\":\"alpha-code e7 402 preauth probe 2026-07-28T02:05:45.876Z\",\"results\":\"[tavily] alpha-code e7 402 preauth probe 2026-07-28T02:05:45.876Z\\n- 2026-01-01 Health Care Services Requiring Preauthorization (https://senderohealth.com/files/Health%20Care%20Services%20Requiring%20Preauthorization%20-%20Effective%2001012026.pdf)\\n Description Effective Date 2026-01-01 Health Care Services Requiring Preauthorization Code Policy/Guideline 2 Policy/Guideline 1 Comment Evidence of Coverage (EOC, Plan coverage document) InterQual® Evidence-Based Criteria & Guidelines 0031U CYP1A2 (CYTOCHROME P450 FAM", + "body": "{\"query\":\"alpha-code e7 402 preauth probe 2026-07-31T02:20:38.799Z\",\"results\":\"[tavily] alpha-code e7 402 preauth probe 2026-07-31T02:20:38.799Z\\n- New Prior Authorization Requirement for Certain Procedure Codes (https://www.fostercaretx.com/newsroom/eff-04012026-new-pa-auth-req-for-certain-procedure-codes-12232025.html)\\n | Q5117 | INJECTION TRASTUZUMAB-ANNS BIOSIMILAR 10 MG |\\n| Q5118 | INJECTION BEVACIZUMAB-BVZR BIOSIMILAR 10 MG |\\n| Q5120 | INJ PEGFILGRASTIM-BMEZ BIOSIMLR ZIEXTENZO 0.5 MG |\\n| Q5122 | INJECTION PEGFILGRASTIM APGF BIOSIMILAR 0.5 MG |\\n| Q5126 | INJ BEVACIZUMAB-MALY BIOSIMI", "accountShape": { - "balanceFen": 100000, - "walletUsedFen": 0, + "balanceFen": 99925, + "walletUsedFen": 75, "plan": { "id": "pro", "status": "active" @@ -408,22 +781,40 @@ }, "required": false, "note": "not-producible today — 不是绿,也不是跳过:账户有额度/余额,402 的唯一可达臂(accountPreauth)不会触发。要真采 402 需把账户打到余额+额度双空,那是破坏性计费变更且与本轮 AC1/计费证据互斥。映射本身由 L1 覆盖。", - "at": "2026-07-28T02:06:03.204Z" + "at": "2026-07-31T02:20:56.635Z" }, { "id": "P3.8", - "ac": "AC3 defect 消失 / 主权拒绝 LOUD", - "title": "the denied local websearch surfaces a discernible tool error, not a crash", - "status": "fail", - "criterion": "the tool part exists with state.status === \"error\" and a model-visible message containing \"denied by alpha sovereignty\" (or starting with \"Web search failed:\") — no unhandled defect", + "ac": "AC3 defect 消失 / 主权拒绝", + "title": "platform-pays makes the denied local websearch impossible for the model to attempt", + "status": "pass", + "criterion": "GET /experimental/tool for the selected platform model omits \"websearch\"; the user tools map cannot add an omitted tool, while discernible direct-denial wording remains an L1 responsibility", "observed": { - "sessionID": "ses_059873039ffeOlmHFCJDINoHn5", - "toolPartsSeen": [], - "message": "" + "platformProviderID": "alpha", + "platformModelID": "deepseek-v4-flash", + "builtinToolIDs": [ + "invalid", + "question", + "bash", + "read", + "glob", + "grep", + "edit", + "write", + "task", + "webfetch", + "todowrite", + "skill", + "alpha_reload", + "alpha_register", + "alpha_echo", + "alpha_ping" + ], + "modelCanAttemptLocalWebSearch": false, + "coveredAtL1": "packages/opencode/test/tool/alpha-websearch-failure.test.ts" }, "required": true, - "note": "the model declined to call the denied tool — re-run; the check needs the tool to actually be attempted", - "at": "2026-07-28T02:06:07.246Z" + "at": "2026-07-31T02:20:56.635Z" }, { "id": "P3.9", @@ -436,7 +827,7 @@ "baseline": "docs/design/2026-07-22-e7-cloud-web-search-baseline.md §1 二次更正 (2026-07-25)" }, "required": false, - "at": "2026-07-28T02:06:07.246Z" + "at": "2026-07-31T02:20:56.635Z" } ] } diff --git a/docs/verification/2026-07-27-e7-packaged-live/results/logged-in-20260731T021405Z.json b/docs/verification/2026-07-27-e7-packaged-live/results/logged-in-20260731T021405Z.json new file mode 100644 index 000000000000..f8ed88528409 --- /dev/null +++ b/docs/verification/2026-07-27-e7-packaged-live/results/logged-in-20260731T021405Z.json @@ -0,0 +1,805 @@ +{ + "schema": "alpha-code/e7-packaged-live/v1", + "issue": "alpha-code#643", + "phase": "logged-in", + "capturedAt": "2026-07-31T02:14:05.752Z", + "finishedAt": "2026-07-31T02:15:11.405Z", + "build": { + "app": "/Applications/alpha-code.app", + "asarSha256": "dded6b38f023e2bbaba3c152032a857f80221d280949c6838374741aea6f42b9", + "appMtime": "2026-07-31T02:12:02.218Z", + "pinnedCommit": "b8f030e0c", + "appVersion": "1.17.13" + }, + "engine": { + "url": "http://127.0.0.1:50152", + "version": "1.17.13" + }, + "auth": { + "status": "logged-in", + "mode": "platform", + "account": {}, + "expiresAt": 1785464921725, + "platformStatus": "ready" + }, + "endpoints": { + "web": "https://alphacodeone.com", + "platform": "https://alpha-gateway.tidelabs.click", + "account": "https://account.alphacodeone.com", + "cloud": "https://alpha-cloud.tidelabs.click", + "mcp": "https://alpha-cloud.tidelabs.click/mcp" + }, + "cloudMcpUrl": "https://alpha-cloud.tidelabs.click/mcp", + "remoteWebSearchToolName": "cloud_web_search", + "derivedEngineToolId": "cloud_cloud_web_search", + "checks": [ + { + "id": "P0.1", + "ac": "preflight", + "title": "app under test is the build this evidence directory was cut for", + "status": "pass", + "criterion": "sha256(app.asar) === dded6b38f023e2bbaba3c152032a857f80221d280949c6838374741aea6f42b9 (alpha @ b8f030e0c)", + "observed": { + "asarPath": "/Applications/alpha-code.app/Contents/Resources/app.asar", + "asarSha": "dded6b38f023e2bbaba3c152032a857f80221d280949c6838374741aea6f42b9", + "appMtime": "2026-07-31T02:12:02.218Z", + "pinnedCommit": "b8f030e0c" + }, + "required": true, + "at": "2026-07-31T02:14:05.873Z" + }, + { + "id": "P0.2", + "ac": "preflight", + "title": "CDP reachable on the packaged app", + "status": "pass", + "criterion": "http://127.0.0.1:9222/json lists a renderer page target", + "observed": { + "port": 9222 + }, + "required": true, + "at": "2026-07-31T02:14:05.897Z" + }, + { + "id": "P0.3", + "ac": "preflight", + "title": "sidecar engine healthy", + "status": "pass", + "criterion": "GET /global/health → { healthy: true }", + "observed": { + "url": "http://127.0.0.1:50152", + "health": { + "healthy": true, + "version": "1.17.13" + } + }, + "required": true, + "at": "2026-07-31T02:14:05.909Z" + }, + { + "id": "P0.4", + "ac": "preflight", + "title": "登录态(平台代付)—— 取证的硬前提", + "status": "pass", + "criterion": "auth.getState() === {status:\"logged-in\", mode:\"platform\"} AND /alpha-secrets/ALPHA_CLOUD_TOKEN exists", + "observed": { + "auth": { + "status": "logged-in", + "mode": "platform", + "account": {}, + "expiresAt": 1785464921725, + "platformStatus": "ready" + }, + "cloudTokenFile": { + "path": "/Users/tide/Library/Application Support/ai.opencode.desktop.dev/alpha-secrets/ALPHA_CLOUD_TOKEN", + "present": true + }, + "apiKeyFile": { + "path": "/Users/tide/Library/Application Support/ai.opencode.desktop.dev/alpha-secrets/ALPHA_API_KEY", + "present": true + }, + "endpoints": { + "web": "https://alphacodeone.com", + "platform": "https://alpha-gateway.tidelabs.click", + "account": "https://account.alphacodeone.com", + "cloud": "https://alpha-cloud.tidelabs.click", + "mcp": "https://alpha-cloud.tidelabs.click/mcp" + } + }, + "required": true, + "at": "2026-07-31T02:14:05.912Z" + }, + { + "id": "P1.1", + "ac": "AC1 packaged 登录态 listTools", + "title": "engine registered the alpha cloud MCP server and it is connected", + "criterion": "GET /mcp → [\"cloud\"].status === \"connected\"", + "required": true, + "status": "pass", + "observed": { + "cloud": { + "status": "connected" + } + }, + "at": "2026-07-31T02:14:07.920Z" + }, + { + "id": "P1.2", + "ac": "AC1 packaged 登录态 listTools", + "title": "the source-side cloud MCP definition uses a {file:} token reference and resolves to the expected endpoint", + "criterion": "the packaged sidecar's raw OPENCODE_CONFIG_CONTENT has mcp.cloud.url === \"https://alpha-cloud.tidelabs.click/mcp\" AND Authorization === \"Bearer {file:/alpha-secrets/ALPHA_CLOUD_TOKEN}\"; GET /config separately resolves that definition to the same URL and the secret-file value", + "required": true, + "status": "fail", + "observed": { + "expectedMcpUrl": "https://alpha-cloud.tidelabs.click/mcp", + "sourceProcessCandidates": 0, + "sourceObservationError": "OPENCODE_CONFIG_CONTENT was not observable on 1 packaged sidecar process(es)", + "sourceUsesFileReference": false, + "resolvedUrl": "https://alpha-cloud.tidelabs.click/mcp", + "resolvedAuthorizationMatchesSecret": true, + "secretFileMode": "600", + "secretDirectoryMode": "700" + }, + "at": "2026-07-31T02:14:08.105Z" + }, + { + "id": "P1.3", + "ac": "AC1 packaged 登录态 listTools", + "title": "LIVE-PATH catalog gate — the deployed cloud worker advertises a web-search tool", + "criterion": "MCP tools/list on the app-resolved endpoint contains a tool matching /web[_-]?search/; this anonymous catalog check makes no account-authorization claim (P2.2 does)", + "required": true, + "status": "pass", + "observed": { + "httpStatus": 200, + "remoteToolNames": [ + "cloud_dispatch", + "cloud_status", + "cloud_await", + "cloud_artifacts", + "cloud_schedule_create", + "cloud_schedule_list", + "cloud_web_search", + "cloud_schedule_delete" + ], + "matched": "cloud_web_search", + "accountBound": false, + "accountBoundGate": "P2.2 tools/call with the app-issued cloud bearer" + }, + "at": "2026-07-31T02:14:08.722Z" + }, + { + "id": "P1.4", + "ac": "AC1 packaged 登录态 listTools", + "title": "engine-visible id for the cloud web-search tool (MCP catalog prefixes the server name)", + "status": "pass", + "criterion": "engine id = sanitize(\"cloud\") + \"_\" + sanitize() per packages/opencode/src/mcp/catalog.ts:117-119 — record it, do not assume \"cloud_web_search\"", + "observed": { + "remoteName": "cloud_web_search", + "derivedEngineId": "cloud_cloud_web_search", + "alphaPinnedId": "cloud_web_search" + }, + "required": true, + "note": "MISMATCH: alpha pins \"cloud_web_search\" but the engine will register \"cloud_cloud_web_search\" — every alpha gate keyed on the pinned id is currently an EMPTY gate (it watches a tool id the engine never registers). Re-check each one against the real id and open a CODE ticket; README §7 lists them and the judgement method.", + "at": "2026-07-31T02:14:08.722Z" + }, + { + "id": "P1.5", + "ac": "AC1 云优先(本地 keyless 被抑制)", + "title": "every runtime agent's effective permission decision denies local keyless websearch", + "criterion": "for every GET /agent entry, Permission.evaluate(\"websearch\", \"*\", ruleset) using engine-equivalent Wildcard.match + findLast semantics returns action=deny; a later user-agent allow must make this gate fail", + "required": true, + "status": "pass", + "observed": { + "agentsWithoutEffectiveDeny": [], + "decisions": [ + { + "name": "build", + "action": "deny", + "matchedRuleIndex": 33, + "matchedRule": { + "permission": "websearch", + "pattern": "*", + "action": "deny" + } + }, + { + "name": "alpha-automation", + "action": "deny", + "matchedRuleIndex": 46, + "matchedRule": { + "permission": "websearch", + "pattern": "*", + "action": "deny" + } + }, + { + "name": "alpha-automation-standard", + "action": "deny", + "matchedRuleIndex": 73, + "matchedRule": { + "permission": "websearch", + "pattern": "*", + "action": "deny" + } + }, + { + "name": "alpha-readonly", + "action": "deny", + "matchedRuleIndex": 43, + "matchedRule": { + "permission": "websearch", + "pattern": "*", + "action": "deny" + } + }, + { + "name": "compaction", + "action": "deny", + "matchedRuleIndex": 32, + "matchedRule": { + "permission": "websearch", + "pattern": "*", + "action": "deny" + } + }, + { + "name": "explore", + "action": "deny", + "matchedRuleIndex": 61, + "matchedRule": { + "permission": "websearch", + "pattern": "*", + "action": "deny" + } + }, + { + "name": "general", + "action": "deny", + "matchedRuleIndex": 32, + "matchedRule": { + "permission": "websearch", + "pattern": "*", + "action": "deny" + } + }, + { + "name": "plan", + "action": "deny", + "matchedRuleIndex": 38, + "matchedRule": { + "permission": "websearch", + "pattern": "*", + "action": "deny" + } + }, + { + "name": "summary", + "action": "deny", + "matchedRuleIndex": 32, + "matchedRule": { + "permission": "websearch", + "pattern": "*", + "action": "deny" + } + }, + { + "name": "title", + "action": "deny", + "matchedRuleIndex": 32, + "matchedRule": { + "permission": "websearch", + "pattern": "*", + "action": "deny" + } + } + ] + }, + "at": "2026-07-31T02:14:08.740Z" + }, + { + "id": "P1.6", + "ac": "AC1 packaged 真调", + "title": "a platform-gateway model with tool-calling is available for the real call", + "status": "pass", + "criterion": "/config/providers has a provider whose options.baseURL starts with https://alpha-gateway.tidelabs.click and at least one model with capabilities.toolcall", + "observed": { + "platformProviderID": "alpha", + "platformModelID": "deepseek-v4-flash", + "providerIDs": [ + "deepseek-byok", + "zhipuai-byok", + "alpha" + ] + }, + "required": true, + "at": "2026-07-31T02:14:08.742Z" + }, + { + "id": "P1.7", + "ac": "AC1 云优先(本地 keyless 被抑制)", + "title": "the built-in tool set offered for a platform model no longer contains the local websearch", + "criterion": "GET /experimental/tool?provider&model does NOT contain \"websearch\"", + "required": true, + "status": "pass", + "observed": { + "builtinToolIDs": [ + "invalid", + "question", + "bash", + "read", + "glob", + "grep", + "edit", + "write", + "task", + "webfetch", + "todowrite", + "skill", + "alpha_reload", + "alpha_register", + "alpha_echo", + "alpha_ping" + ] + }, + "at": "2026-07-31T02:14:08.756Z" + }, + { + "id": "P2.1", + "ac": "AC1 packaged 真调返回 {query,results}", + "title": "the packaged app really invoked the cloud web-search tool through its own engine", + "criterion": "a real model turn produced a tool part whose id is the cloud web-search tool, state.status === \"completed\", and whose output parses to an object with `query` and `results`", + "required": true, + "status": "pass", + "observed": { + "sessionID": "ses_04a0cb2b6ffeS25KmUqLQWJAU5", + "toolPartsSeen": [ + { + "tool": "cloud_cloud_web_search", + "status": "completed" + } + ], + "calledTool": "cloud_cloud_web_search", + "state": "completed", + "outputKeys": [ + "query", + "results" + ] + }, + "at": "2026-07-31T02:14:17.891Z" + }, + { + "id": "P2.2", + "ac": "AC1 packaged 真调返回 {query,results}", + "title": "LIVE-PATH GATE ② — one real call on the app-resolved endpoint returns {query, results}", + "criterion": "MCP tools/call with the app's own cloud bearer returns isError !== true and a text payload parsing to an object with `query` and `results`", + "required": true, + "status": "pass", + "observed": { + "httpStatus": 200, + "isError": false, + "payloadKeys": [ + "query", + "results" + ] + }, + "at": "2026-07-31T02:14:21.962Z" + }, + { + "id": "P2.3", + "ac": "AC3 计费(ledger/settle)证据", + "title": "waitUntil-backed web-search settlement reaches this tenant's append-only ledger", + "criterion": "within 45000ms after the real calls, the account ledger appends a fact with kind=\"usage_settled\" AND actionId=\"tool.web_search\" and a seq absent from the pre-call page", + "required": true, + "status": "fail", + "observed": { + "settlementObservedAfterMs": 45537, + "settlementTimeoutMs": 45000, + "walletUsedFenBefore": 0, + "walletUsedFenAfter": 30, + "balanceFenBefore": 100000, + "balanceFenAfter": 99970, + "walletDelta": 30, + "balanceDelta": 30, + "settledWebSearchFacts": [], + "newLedgerFacts": [ + { + "seq": 124, + "kind": "reservation_settled", + "domain": "wallet", + "amount": 0, + "reservationId": "a0eca739-e6f2-4c73-8be7-865756a53832", + "createdAt": 1785464066781 + }, + { + "seq": 123, + "kind": "usage_settled", + "domain": "wallet", + "amount": -15, + "reservationId": "a0eca739-e6f2-4c73-8be7-865756a53832", + "createdAt": 1785464066781 + }, + { + "seq": 122, + "kind": "reservation_created", + "domain": "allowance", + "amount": 0, + "actionId": "tool.web_search", + "reservationId": "a0eca739-e6f2-4c73-8be7-865756a53832", + "createdAt": 1785464063583 + }, + { + "seq": 121, + "kind": "reservation_created", + "domain": "allowance", + "amount": 0, + "actionId": "tool.web_search", + "reservationId": "a0eca739-e6f2-4c73-8be7-865756a53832", + "createdAt": 1785464063583 + }, + { + "seq": 120, + "kind": "reservation_created", + "domain": "wallet", + "amount": 15, + "actionId": "tool.web_search", + "reservationId": "a0eca739-e6f2-4c73-8be7-865756a53832", + "createdAt": 1785464063583 + }, + { + "seq": 119, + "kind": "reservation_settled", + "domain": "wallet", + "amount": 0, + "reservationId": "f4d5f923-fa73-4940-8627-2659422d4f95", + "createdAt": 1785464062712 + }, + { + "seq": 118, + "kind": "allowance_consumed", + "domain": "allowance", + "amount": -3, + "reservationId": "f4d5f923-fa73-4940-8627-2659422d4f95", + "createdAt": 1785464062712 + }, + { + "seq": 117, + "kind": "allowance_consumed", + "domain": "allowance", + "amount": -3, + "reservationId": "f4d5f923-fa73-4940-8627-2659422d4f95", + "createdAt": 1785464062712 + }, + { + "seq": 116, + "kind": "reservation_created", + "domain": "allowance", + "amount": 224, + "actionId": "model.invoke.chat", + "reservationId": "f4d5f923-fa73-4940-8627-2659422d4f95", + "createdAt": 1785464060894 + }, + { + "seq": 115, + "kind": "reservation_created", + "domain": "allowance", + "amount": 224, + "actionId": "model.invoke.chat", + "reservationId": "f4d5f923-fa73-4940-8627-2659422d4f95", + "createdAt": 1785464060894 + }, + { + "seq": 114, + "kind": "reservation_created", + "domain": "wallet", + "amount": 0, + "actionId": "model.invoke.chat", + "reservationId": "f4d5f923-fa73-4940-8627-2659422d4f95", + "createdAt": 1785464060894 + }, + { + "seq": 113, + "kind": "reservation_settled", + "domain": "wallet", + "amount": 0, + "reservationId": "41b5ac3f-a26f-4cae-abce-fbc3ff4cb15e", + "createdAt": 1785464060238 + }, + { + "seq": 112, + "kind": "allowance_consumed", + "domain": "allowance", + "amount": -1, + "reservationId": "41b5ac3f-a26f-4cae-abce-fbc3ff4cb15e", + "createdAt": 1785464060238 + }, + { + "seq": 111, + "kind": "allowance_consumed", + "domain": "allowance", + "amount": -1, + "reservationId": "41b5ac3f-a26f-4cae-abce-fbc3ff4cb15e", + "createdAt": 1785464060238 + }, + { + "seq": 110, + "kind": "reservation_settled", + "domain": "wallet", + "amount": 0, + "reservationId": "9aef1613-dfa3-40e7-af48-5f40a9a104a7", + "createdAt": 1785464060223 + }, + { + "seq": 109, + "kind": "usage_settled", + "domain": "wallet", + "amount": -15, + "reservationId": "9aef1613-dfa3-40e7-af48-5f40a9a104a7", + "createdAt": 1785464060223 + }, + { + "seq": 108, + "kind": "reservation_created", + "domain": "allowance", + "amount": 0, + "actionId": "tool.web_search", + "reservationId": "9aef1613-dfa3-40e7-af48-5f40a9a104a7", + "createdAt": 1785464057802 + }, + { + "seq": 107, + "kind": "reservation_created", + "domain": "allowance", + "amount": 0, + "actionId": "tool.web_search", + "reservationId": "9aef1613-dfa3-40e7-af48-5f40a9a104a7", + "createdAt": 1785464057802 + }, + { + "seq": 106, + "kind": "reservation_created", + "domain": "wallet", + "amount": 15, + "actionId": "tool.web_search", + "reservationId": "9aef1613-dfa3-40e7-af48-5f40a9a104a7", + "createdAt": 1785464057802 + }, + { + "seq": 105, + "kind": "reservation_settled", + "domain": "wallet", + "amount": 0, + "reservationId": "bd873c0b-ac82-4461-b0b5-e449c79b28f8", + "createdAt": 1785464057160 + }, + { + "seq": 104, + "kind": "allowance_consumed", + "domain": "allowance", + "amount": -3, + "reservationId": "bd873c0b-ac82-4461-b0b5-e449c79b28f8", + "createdAt": 1785464057160 + }, + { + "seq": 103, + "kind": "allowance_consumed", + "domain": "allowance", + "amount": -3, + "reservationId": "bd873c0b-ac82-4461-b0b5-e449c79b28f8", + "createdAt": 1785464057160 + }, + { + "seq": 102, + "kind": "reservation_created", + "domain": "allowance", + "amount": 224, + "actionId": "model.invoke.chat", + "reservationId": "41b5ac3f-a26f-4cae-abce-fbc3ff4cb15e", + "createdAt": 1785464055347 + }, + { + "seq": 101, + "kind": "reservation_created", + "domain": "allowance", + "amount": 224, + "actionId": "model.invoke.chat", + "reservationId": "41b5ac3f-a26f-4cae-abce-fbc3ff4cb15e", + "createdAt": 1785464055347 + }, + { + "seq": 100, + "kind": "reservation_created", + "domain": "wallet", + "amount": 0, + "actionId": "model.invoke.chat", + "reservationId": "41b5ac3f-a26f-4cae-abce-fbc3ff4cb15e", + "createdAt": 1785464055347 + }, + { + "seq": 99, + "kind": "reservation_created", + "domain": "allowance", + "amount": 224, + "actionId": "model.invoke.chat", + "reservationId": "bd873c0b-ac82-4461-b0b5-e449c79b28f8", + "createdAt": 1785464055109 + }, + { + "seq": 98, + "kind": "reservation_created", + "domain": "allowance", + "amount": 224, + "actionId": "model.invoke.chat", + "reservationId": "bd873c0b-ac82-4461-b0b5-e449c79b28f8", + "createdAt": 1785464055109 + }, + { + "seq": 97, + "kind": "reservation_created", + "domain": "wallet", + "amount": 0, + "actionId": "model.invoke.chat", + "reservationId": "bd873c0b-ac82-4461-b0b5-e449c79b28f8", + "createdAt": 1785464055109 + }, + { + "seq": 96, + "kind": "allowance_window_rolled", + "domain": "allowance", + "amount": 3, + "createdAt": 1785464055108 + } + ] + }, + "at": "2026-07-31T02:15:07.500Z" + }, + { + "id": "P3.1", + "ac": "AC3 真实失败集 401", + "title": "gateway web_search is fail-closed without a bearer", + "criterion": "POST /v1/tools/web_search with no Authorization → HTTP 401", + "required": true, + "status": "pass", + "observed": { + "status": 401, + "body": "{\"error\":{\"message\":\"unauthorized\"}}" + }, + "at": "2026-07-31T02:15:08.207Z" + }, + { + "id": "P3.2", + "ac": "AC3 真实失败集 400", + "title": "gateway web_search rejects a missing query with 400", + "criterion": "POST /v1/tools/web_search with a valid bearer and body {} → HTTP 400", + "required": true, + "status": "pass", + "observed": { + "status": 400, + "body": "{\"error\":{\"message\":\"query required\"}}" + }, + "at": "2026-07-31T02:15:08.425Z" + }, + { + "id": "P3.3", + "ac": "AC3 真实失败集 400", + "title": "gateway web_search rejects malformed JSON with 400", + "criterion": "POST /v1/tools/web_search with a valid bearer and a malformed body → HTTP 400", + "required": true, + "status": "pass", + "observed": { + "status": 400, + "body": "{\"error\":{\"message\":\"invalid json\"}}" + }, + "at": "2026-07-31T02:15:08.638Z" + }, + { + "id": "P3.4", + "ac": "AC3 真实失败集 403", + "title": "403 (action_forbidden / job_not_enforceable)", + "status": "not-producible", + "criterion": "would require a token whose scope excludes model.invoke, or a non-enforceable job — neither is mintable from a normal desktop login", + "observed": { + "reason": "the desktop only ever holds route-purpose-bound tokens for model.invoke and cloud.dispatch", + "coveredAtL1": "packages/opencode/test/tool/alpha-websearch-failure.test.ts (403 → forbidden, error.code preserved)" + }, + "required": false, + "at": "2026-07-31T02:15:08.638Z" + }, + { + "id": "P3.5", + "ac": "AC3 真实失败集 502", + "title": "502 (no search backend configured)", + "status": "not-producible", + "criterion": "would require the deployed gateway to have neither TAVILY_API_KEY nor BRAVE_API_KEY", + "observed": { + "reason": "both secrets are configured on the deployed gateway (docs/verification/2026-07-22-e7-deploy-probe.md §4); removing them is a production mutation, out of bounds for a probe", + "coveredAtL1": "packages/opencode/test/tool/alpha-websearch-failure.test.ts (502 → upstream)" + }, + "required": false, + "at": "2026-07-31T02:15:08.638Z" + }, + { + "id": "P3.6", + "ac": "AC3 真实失败集 意外状态 LOUD", + "title": "unexpected non-2xx status maps to a loud, discernible failure", + "status": "not-producible", + "criterion": "would require the deployed gateway to emit a status outside {400,401,402,403,502} on this route", + "observed": { + "reason": "no request shape reachable from the desktop elicits one; the mapping itself is exercised at L1", + "coveredAtL1": "packages/opencode/test/tool/alpha-websearch-failure.test.ts (statusKind default → unexpected_status)" + }, + "required": false, + "at": "2026-07-31T02:15:08.638Z" + }, + { + "id": "P3.7", + "ac": "AC3 真实失败集 402", + "title": "402 / 余额(accountPreauth 余额或额度不足臂)", + "status": "not-producible", + "criterion": "POST /v1/tools/web_search with a valid bearer → HTTP 402 when the account is out of quota+wallet; a 200 proves the account is funded (402 not producible today without draining the owner's balance, which would also invalidate P2.1/P2.2/P2.3); any other status is unexpected and loud", + "observed": { + "status": 200, + "body": "{\"query\":\"alpha-code e7 402 preauth probe 2026-07-31T02:14:05.752Z\",\"results\":\"[tavily] alpha-code e7 402 preauth probe 2026-07-31T02:14:05.752Z\\n- Effective 4/1/26: New Prior Authorization Requirement for Certain Procedure Codes (https://www.fostercaretx.com/newsroom/eff-04012026-new-pa-auth-req-for-certain-procedure-codes-12232025.html)\\n | 22848 | PELVIC FIXATION OTHER THAN SACRUM |\\n| 22849 | REINSERTION SPINAL FIXA DEVICE |\\n| 25447 | ARTHROPLAS INTERPOSIT-INTERCARP/CARPOMETACARP JT |\\n| 27702 | ARTHROPLASTY ANK; W/IMPLNT (TOT ANK) |\\n| 28285 | CORRECT HAMMERTOE |\\n| 28296 | CORRECTION H", + "accountShape": { + "balanceFen": 99970, + "walletUsedFen": 30, + "plan": { + "id": "pro", + "status": "active" + } + }, + "armA": "unreachable by construction — the desktop holds a route-purpose JWT (via:\"jwt\"), never a job token, so perJobPrecall returns pass/enforced:false", + "armB": "did not fire — the account was preauthorised, i.e. it is not out of quota+wallet today", + "coveredAtL1": "packages/opencode/test/tool/alpha-websearch-failure.test.ts (402 → payment_required, both arms: 预授权拒绝 and per-job budget exceeded)" + }, + "required": false, + "note": "not-producible today — 不是绿,也不是跳过:账户有额度/余额,402 的唯一可达臂(accountPreauth)不会触发。要真采 402 需把账户打到余额+额度双空,那是破坏性计费变更且与本轮 AC1/计费证据互斥。映射本身由 L1 覆盖。", + "at": "2026-07-31T02:15:11.403Z" + }, + { + "id": "P3.8", + "ac": "AC3 defect 消失 / 主权拒绝", + "title": "platform-pays makes the denied local websearch impossible for the model to attempt", + "status": "pass", + "criterion": "GET /experimental/tool for the selected platform model omits \"websearch\"; the user tools map cannot add an omitted tool, while discernible direct-denial wording remains an L1 responsibility", + "observed": { + "platformProviderID": "alpha", + "platformModelID": "deepseek-v4-flash", + "builtinToolIDs": [ + "invalid", + "question", + "bash", + "read", + "glob", + "grep", + "edit", + "write", + "task", + "webfetch", + "todowrite", + "skill", + "alpha_reload", + "alpha_register", + "alpha_echo", + "alpha_ping" + ], + "modelCanAttemptLocalWebSearch": false, + "coveredAtL1": "packages/opencode/test/tool/alpha-websearch-failure.test.ts" + }, + "required": true, + "at": "2026-07-31T02:15:11.403Z" + }, + { + "id": "P3.9", + "ac": "AC3 已登记缺口", + "title": "cloud-side failures are loud but not classifiable (alpha-platform#105)", + "status": "not-producible", + "criterion": "the cloud MCP shell drops r.status (packages/gateway/src/cloud-mcp.ts), so a cloud failure reaches the client with no HTTP status and no error.code", + "observed": { + "registeredGap": "alpha-platform#105", + "baseline": "docs/design/2026-07-22-e7-cloud-web-search-baseline.md §1 二次更正 (2026-07-25)" + }, + "required": false, + "at": "2026-07-31T02:15:11.403Z" + } + ] +} diff --git a/docs/verification/2026-07-27-e7-packaged-live/results/logged-in-20260731T022038Z.json b/docs/verification/2026-07-27-e7-packaged-live/results/logged-in-20260731T022038Z.json new file mode 100644 index 000000000000..f607a04d8a6c --- /dev/null +++ b/docs/verification/2026-07-27-e7-packaged-live/results/logged-in-20260731T022038Z.json @@ -0,0 +1,833 @@ +{ + "schema": "alpha-code/e7-packaged-live/v1", + "issue": "alpha-code#643", + "phase": "logged-in", + "capturedAt": "2026-07-31T02:20:38.799Z", + "finishedAt": "2026-07-31T02:20:56.635Z", + "build": { + "app": "/Applications/alpha-code.app", + "asarSha256": "dded6b38f023e2bbaba3c152032a857f80221d280949c6838374741aea6f42b9", + "appMtime": "2026-07-31T02:12:02.218Z", + "pinnedCommit": "b8f030e0c", + "appVersion": "1.17.13" + }, + "engine": { + "url": "http://127.0.0.1:50152", + "version": "1.17.13" + }, + "auth": { + "status": "logged-in", + "mode": "platform", + "account": {}, + "expiresAt": 1785464921725, + "platformStatus": "ready" + }, + "endpoints": { + "web": "https://alphacodeone.com", + "platform": "https://alpha-gateway.tidelabs.click", + "account": "https://account.alphacodeone.com", + "cloud": "https://alpha-cloud.tidelabs.click", + "mcp": "https://alpha-cloud.tidelabs.click/mcp" + }, + "cloudMcpUrl": "https://alpha-cloud.tidelabs.click/mcp", + "remoteWebSearchToolName": "cloud_web_search", + "derivedEngineToolId": "cloud_cloud_web_search", + "checks": [ + { + "id": "P0.1", + "ac": "preflight", + "title": "app under test is the build this evidence directory was cut for", + "status": "pass", + "criterion": "sha256(app.asar) === dded6b38f023e2bbaba3c152032a857f80221d280949c6838374741aea6f42b9 (alpha @ b8f030e0c)", + "observed": { + "asarPath": "/Applications/alpha-code.app/Contents/Resources/app.asar", + "asarSha": "dded6b38f023e2bbaba3c152032a857f80221d280949c6838374741aea6f42b9", + "appMtime": "2026-07-31T02:12:02.218Z", + "pinnedCommit": "b8f030e0c" + }, + "required": true, + "at": "2026-07-31T02:20:38.908Z" + }, + { + "id": "P0.2", + "ac": "preflight", + "title": "CDP reachable on the packaged app", + "status": "pass", + "criterion": "http://127.0.0.1:9222/json lists a renderer page target", + "observed": { + "port": 9222 + }, + "required": true, + "at": "2026-07-31T02:20:38.923Z" + }, + { + "id": "P0.3", + "ac": "preflight", + "title": "sidecar engine healthy", + "status": "pass", + "criterion": "GET /global/health → { healthy: true }", + "observed": { + "url": "http://127.0.0.1:50152", + "health": { + "healthy": true, + "version": "1.17.13" + } + }, + "required": true, + "at": "2026-07-31T02:20:38.943Z" + }, + { + "id": "P0.4", + "ac": "preflight", + "title": "登录态(平台代付)—— 取证的硬前提", + "status": "pass", + "criterion": "auth.getState() === {status:\"logged-in\", mode:\"platform\"} AND /alpha-secrets/ALPHA_CLOUD_TOKEN exists", + "observed": { + "auth": { + "status": "logged-in", + "mode": "platform", + "account": {}, + "expiresAt": 1785464921725, + "platformStatus": "ready" + }, + "cloudTokenFile": { + "path": "/Users/tide/Library/Application Support/ai.opencode.desktop.dev/alpha-secrets/ALPHA_CLOUD_TOKEN", + "present": true + }, + "apiKeyFile": { + "path": "/Users/tide/Library/Application Support/ai.opencode.desktop.dev/alpha-secrets/ALPHA_API_KEY", + "present": true + }, + "endpoints": { + "web": "https://alphacodeone.com", + "platform": "https://alpha-gateway.tidelabs.click", + "account": "https://account.alphacodeone.com", + "cloud": "https://alpha-cloud.tidelabs.click", + "mcp": "https://alpha-cloud.tidelabs.click/mcp" + } + }, + "required": true, + "at": "2026-07-31T02:20:38.948Z" + }, + { + "id": "P1.1", + "ac": "AC1 packaged 登录态 listTools", + "title": "engine registered the alpha cloud MCP server and it is connected", + "criterion": "GET /mcp → [\"cloud\"].status === \"connected\"", + "required": true, + "status": "pass", + "observed": { + "cloud": { + "status": "connected" + } + }, + "at": "2026-07-31T02:20:38.960Z" + }, + { + "id": "P1.2", + "ac": "AC1 packaged 登录态 listTools", + "title": "the source-side cloud MCP definition uses a {file:} token reference and resolves to the expected endpoint", + "criterion": "a short-lived child spawned by the packaged engine inherits raw OPENCODE_CONFIG_CONTENT with mcp.cloud.url === \"https://alpha-cloud.tidelabs.click/mcp\" AND Authorization === \"Bearer {file:/alpha-secrets/ALPHA_CLOUD_TOKEN}\"; GET /config separately resolves that definition to the same URL and the secret-file value", + "required": true, + "status": "pass", + "observed": { + "expectedMcpUrl": "https://alpha-cloud.tidelabs.click/mcp", + "sourceObservation": { + "present": true, + "parsed": true, + "urlMatches": true, + "authorizationMatchesFileReference": true + }, + "sourceUsesFileReference": true, + "resolvedUrl": "https://alpha-cloud.tidelabs.click/mcp", + "resolvedAuthorizationMatchesSecret": true, + "secretFileMode": "600", + "secretDirectoryMode": "700" + }, + "at": "2026-07-31T02:20:39.045Z" + }, + { + "id": "P1.3", + "ac": "AC1 packaged 登录态 listTools", + "title": "LIVE-PATH catalog gate — the deployed cloud worker advertises a web-search tool", + "criterion": "MCP tools/list on the app-resolved endpoint contains a tool matching /web[_-]?search/; this anonymous catalog check makes no account-authorization claim (P2.2 does)", + "required": true, + "status": "pass", + "observed": { + "httpStatus": 200, + "remoteToolNames": [ + "cloud_dispatch", + "cloud_status", + "cloud_await", + "cloud_artifacts", + "cloud_schedule_create", + "cloud_schedule_list", + "cloud_web_search", + "cloud_schedule_delete" + ], + "matched": "cloud_web_search", + "accountBound": false, + "accountBoundGate": "P2.2 tools/call with the app-issued cloud bearer" + }, + "at": "2026-07-31T02:20:40.100Z" + }, + { + "id": "P1.4", + "ac": "AC1 packaged 登录态 listTools", + "title": "engine-visible id for the cloud web-search tool (MCP catalog prefixes the server name)", + "status": "pass", + "criterion": "engine id = sanitize(\"cloud\") + \"_\" + sanitize() per packages/opencode/src/mcp/catalog.ts:117-119 — record it, do not assume \"cloud_web_search\"", + "observed": { + "remoteName": "cloud_web_search", + "derivedEngineId": "cloud_cloud_web_search", + "alphaPinnedId": "cloud_web_search" + }, + "required": true, + "note": "MISMATCH: alpha pins \"cloud_web_search\" but the engine will register \"cloud_cloud_web_search\" — every alpha gate keyed on the pinned id is currently an EMPTY gate (it watches a tool id the engine never registers). Re-check each one against the real id and open a CODE ticket; README §7 lists them and the judgement method.", + "at": "2026-07-31T02:20:40.100Z" + }, + { + "id": "P1.5", + "ac": "AC1 云优先(本地 keyless 被抑制)", + "title": "every runtime agent's effective permission decision denies local keyless websearch", + "criterion": "for every GET /agent entry, Permission.evaluate(\"websearch\", \"*\", ruleset) using engine-equivalent Wildcard.match + findLast semantics returns action=deny; a later user-agent allow must make this gate fail", + "required": true, + "status": "pass", + "observed": { + "agentsWithoutEffectiveDeny": [], + "decisions": [ + { + "name": "build", + "action": "deny", + "matchedRuleIndex": 33, + "matchedRule": { + "permission": "websearch", + "pattern": "*", + "action": "deny" + } + }, + { + "name": "alpha-automation", + "action": "deny", + "matchedRuleIndex": 46, + "matchedRule": { + "permission": "websearch", + "pattern": "*", + "action": "deny" + } + }, + { + "name": "alpha-automation-standard", + "action": "deny", + "matchedRuleIndex": 73, + "matchedRule": { + "permission": "websearch", + "pattern": "*", + "action": "deny" + } + }, + { + "name": "alpha-readonly", + "action": "deny", + "matchedRuleIndex": 43, + "matchedRule": { + "permission": "websearch", + "pattern": "*", + "action": "deny" + } + }, + { + "name": "compaction", + "action": "deny", + "matchedRuleIndex": 32, + "matchedRule": { + "permission": "websearch", + "pattern": "*", + "action": "deny" + } + }, + { + "name": "explore", + "action": "deny", + "matchedRuleIndex": 61, + "matchedRule": { + "permission": "websearch", + "pattern": "*", + "action": "deny" + } + }, + { + "name": "general", + "action": "deny", + "matchedRuleIndex": 32, + "matchedRule": { + "permission": "websearch", + "pattern": "*", + "action": "deny" + } + }, + { + "name": "plan", + "action": "deny", + "matchedRuleIndex": 38, + "matchedRule": { + "permission": "websearch", + "pattern": "*", + "action": "deny" + } + }, + { + "name": "summary", + "action": "deny", + "matchedRuleIndex": 32, + "matchedRule": { + "permission": "websearch", + "pattern": "*", + "action": "deny" + } + }, + { + "name": "title", + "action": "deny", + "matchedRuleIndex": 32, + "matchedRule": { + "permission": "websearch", + "pattern": "*", + "action": "deny" + } + } + ] + }, + "at": "2026-07-31T02:20:40.108Z" + }, + { + "id": "P1.6", + "ac": "AC1 packaged 真调", + "title": "a platform-gateway model with tool-calling is available for the real call", + "status": "pass", + "criterion": "/config/providers has a provider whose options.baseURL starts with https://alpha-gateway.tidelabs.click and at least one model with capabilities.toolcall", + "observed": { + "platformProviderID": "alpha", + "platformModelID": "deepseek-v4-flash", + "providerIDs": [ + "deepseek-byok", + "zhipuai-byok", + "alpha" + ] + }, + "required": true, + "at": "2026-07-31T02:20:40.111Z" + }, + { + "id": "P1.7", + "ac": "AC1 云优先(本地 keyless 被抑制)", + "title": "the built-in tool set offered for a platform model no longer contains the local websearch", + "criterion": "GET /experimental/tool?provider&model does NOT contain \"websearch\"", + "required": true, + "status": "pass", + "observed": { + "builtinToolIDs": [ + "invalid", + "question", + "bash", + "read", + "glob", + "grep", + "edit", + "write", + "task", + "webfetch", + "todowrite", + "skill", + "alpha_reload", + "alpha_register", + "alpha_echo", + "alpha_ping" + ] + }, + "at": "2026-07-31T02:20:40.115Z" + }, + { + "id": "P2.1", + "ac": "AC1 packaged 真调返回 {query,results}", + "title": "the packaged app really invoked the cloud web-search tool through its own engine", + "criterion": "a real model turn produced a tool part whose id is the cloud web-search tool, state.status === \"completed\", and whose output parses to an object with `query` and `results`", + "required": true, + "status": "pass", + "observed": { + "sessionID": "ses_04a06ba39ffezzNQSdJIUR6r1K", + "toolPartsSeen": [ + { + "tool": "cloud_cloud_web_search", + "status": "completed" + } + ], + "calledTool": "cloud_cloud_web_search", + "state": "completed", + "outputKeys": [ + "query", + "results" + ] + }, + "at": "2026-07-31T02:20:49.163Z" + }, + { + "id": "P2.2", + "ac": "AC1 packaged 真调返回 {query,results}", + "title": "LIVE-PATH GATE ② — one real call on the app-resolved endpoint returns {query, results}", + "criterion": "MCP tools/call with the app's own cloud bearer returns isError !== true and a text payload parsing to an object with `query` and `results`", + "required": true, + "status": "pass", + "observed": { + "httpStatus": 200, + "isError": false, + "payloadKeys": [ + "query", + "results" + ] + }, + "at": "2026-07-31T02:20:51.851Z" + }, + { + "id": "P2.3", + "ac": "AC3 计费(ledger/settle)证据", + "title": "waitUntil-backed web-search settlement reaches this tenant's append-only ledger", + "criterion": "within 45000ms after the real calls, the account ledger appends reservation_created(actionId=\"tool.web_search\"), usage_settled, and reservation_settled facts sharing one new reservationId", + "required": true, + "status": "pass", + "observed": { + "settlementObservedAfterMs": 75, + "settlementTimeoutMs": 45000, + "walletUsedFenBefore": 45, + "walletUsedFenAfter": 75, + "balanceFenBefore": 99955, + "balanceFenAfter": 99925, + "walletDelta": 30, + "balanceDelta": 30, + "webSearchSettlements": [ + { + "reservationId": "6b5b9c7a-a5bb-4d54-ba24-ad9d32ff3cdc", + "reservationSeqs": [ + 155, + 154, + 153 + ], + "usageSeqs": [ + 156 + ], + "terminalSeqs": [ + 157 + ], + "amountSettled": -15 + }, + { + "reservationId": "cb5ae08a-fe3e-482c-a13d-e8212cf93e8d", + "reservationSeqs": [ + 141, + 140, + 139 + ], + "usageSeqs": [ + 142 + ], + "terminalSeqs": [ + 143 + ], + "amountSettled": -15 + } + ], + "newLedgerFacts": [ + { + "seq": 157, + "kind": "reservation_settled", + "domain": "wallet", + "amount": 0, + "reservationId": "6b5b9c7a-a5bb-4d54-ba24-ad9d32ff3cdc", + "createdAt": 1785464456677 + }, + { + "seq": 156, + "kind": "usage_settled", + "domain": "wallet", + "amount": -15, + "reservationId": "6b5b9c7a-a5bb-4d54-ba24-ad9d32ff3cdc", + "createdAt": 1785464456677 + }, + { + "seq": 155, + "kind": "reservation_created", + "domain": "allowance", + "amount": 0, + "actionId": "tool.web_search", + "reservationId": "6b5b9c7a-a5bb-4d54-ba24-ad9d32ff3cdc", + "createdAt": 1785464454448 + }, + { + "seq": 154, + "kind": "reservation_created", + "domain": "allowance", + "amount": 0, + "actionId": "tool.web_search", + "reservationId": "6b5b9c7a-a5bb-4d54-ba24-ad9d32ff3cdc", + "createdAt": 1785464454448 + }, + { + "seq": 153, + "kind": "reservation_created", + "domain": "wallet", + "amount": 15, + "actionId": "tool.web_search", + "reservationId": "6b5b9c7a-a5bb-4d54-ba24-ad9d32ff3cdc", + "createdAt": 1785464454448 + }, + { + "seq": 152, + "kind": "reservation_settled", + "domain": "wallet", + "amount": 0, + "reservationId": "72383d9e-12f0-4c67-8bc8-0cb75bbc5ca0", + "createdAt": 1785464453963 + }, + { + "seq": 151, + "kind": "allowance_consumed", + "domain": "allowance", + "amount": -3, + "reservationId": "72383d9e-12f0-4c67-8bc8-0cb75bbc5ca0", + "createdAt": 1785464453963 + }, + { + "seq": 150, + "kind": "allowance_consumed", + "domain": "allowance", + "amount": -3, + "reservationId": "72383d9e-12f0-4c67-8bc8-0cb75bbc5ca0", + "createdAt": 1785464453962 + }, + { + "seq": 149, + "kind": "reservation_settled", + "domain": "wallet", + "amount": 0, + "reservationId": "78a6c08b-7a92-4578-8614-62453b11a46c", + "createdAt": 1785464452538 + }, + { + "seq": 148, + "kind": "allowance_consumed", + "domain": "allowance", + "amount": -1, + "reservationId": "78a6c08b-7a92-4578-8614-62453b11a46c", + "createdAt": 1785464452538 + }, + { + "seq": 147, + "kind": "allowance_consumed", + "domain": "allowance", + "amount": -1, + "reservationId": "78a6c08b-7a92-4578-8614-62453b11a46c", + "createdAt": 1785464452538 + }, + { + "seq": 146, + "kind": "reservation_created", + "domain": "allowance", + "amount": 224, + "actionId": "model.invoke.chat", + "reservationId": "72383d9e-12f0-4c67-8bc8-0cb75bbc5ca0", + "createdAt": 1785464452360 + }, + { + "seq": 145, + "kind": "reservation_created", + "domain": "allowance", + "amount": 224, + "actionId": "model.invoke.chat", + "reservationId": "72383d9e-12f0-4c67-8bc8-0cb75bbc5ca0", + "createdAt": 1785464452359 + }, + { + "seq": 144, + "kind": "reservation_created", + "domain": "wallet", + "amount": 0, + "actionId": "model.invoke.chat", + "reservationId": "72383d9e-12f0-4c67-8bc8-0cb75bbc5ca0", + "createdAt": 1785464452359 + }, + { + "seq": 143, + "kind": "reservation_settled", + "domain": "wallet", + "amount": 0, + "reservationId": "cb5ae08a-fe3e-482c-a13d-e8212cf93e8d", + "createdAt": 1785464452040 + }, + { + "seq": 142, + "kind": "usage_settled", + "domain": "wallet", + "amount": -15, + "reservationId": "cb5ae08a-fe3e-482c-a13d-e8212cf93e8d", + "createdAt": 1785464452040 + }, + { + "seq": 141, + "kind": "reservation_created", + "domain": "allowance", + "amount": 0, + "actionId": "tool.web_search", + "reservationId": "cb5ae08a-fe3e-482c-a13d-e8212cf93e8d", + "createdAt": 1785464449976 + }, + { + "seq": 140, + "kind": "reservation_created", + "domain": "allowance", + "amount": 0, + "actionId": "tool.web_search", + "reservationId": "cb5ae08a-fe3e-482c-a13d-e8212cf93e8d", + "createdAt": 1785464449976 + }, + { + "seq": 139, + "kind": "reservation_created", + "domain": "wallet", + "amount": 15, + "actionId": "tool.web_search", + "reservationId": "cb5ae08a-fe3e-482c-a13d-e8212cf93e8d", + "createdAt": 1785464449976 + }, + { + "seq": 138, + "kind": "reservation_settled", + "domain": "wallet", + "amount": 0, + "reservationId": "87b0706d-d865-46f7-af6c-61dc21b5c5bf", + "createdAt": 1785464448601 + }, + { + "seq": 137, + "kind": "allowance_consumed", + "domain": "allowance", + "amount": -3, + "reservationId": "87b0706d-d865-46f7-af6c-61dc21b5c5bf", + "createdAt": 1785464448600 + }, + { + "seq": 136, + "kind": "allowance_consumed", + "domain": "allowance", + "amount": -3, + "reservationId": "87b0706d-d865-46f7-af6c-61dc21b5c5bf", + "createdAt": 1785464448600 + }, + { + "seq": 135, + "kind": "reservation_created", + "domain": "allowance", + "amount": 224, + "actionId": "model.invoke.chat", + "reservationId": "78a6c08b-7a92-4578-8614-62453b11a46c", + "createdAt": 1785464446655 + }, + { + "seq": 134, + "kind": "reservation_created", + "domain": "allowance", + "amount": 224, + "actionId": "model.invoke.chat", + "reservationId": "78a6c08b-7a92-4578-8614-62453b11a46c", + "createdAt": 1785464446655 + }, + { + "seq": 133, + "kind": "reservation_created", + "domain": "wallet", + "amount": 0, + "actionId": "model.invoke.chat", + "reservationId": "78a6c08b-7a92-4578-8614-62453b11a46c", + "createdAt": 1785464446655 + }, + { + "seq": 132, + "kind": "reservation_created", + "domain": "allowance", + "amount": 224, + "actionId": "model.invoke.chat", + "reservationId": "87b0706d-d865-46f7-af6c-61dc21b5c5bf", + "createdAt": 1785464446433 + }, + { + "seq": 131, + "kind": "reservation_created", + "domain": "allowance", + "amount": 224, + "actionId": "model.invoke.chat", + "reservationId": "87b0706d-d865-46f7-af6c-61dc21b5c5bf", + "createdAt": 1785464446433 + }, + { + "seq": 130, + "kind": "reservation_created", + "domain": "wallet", + "amount": 0, + "actionId": "model.invoke.chat", + "reservationId": "87b0706d-d865-46f7-af6c-61dc21b5c5bf", + "createdAt": 1785464446433 + } + ] + }, + "at": "2026-07-31T02:20:51.926Z" + }, + { + "id": "P3.1", + "ac": "AC3 真实失败集 401", + "title": "gateway web_search is fail-closed without a bearer", + "criterion": "POST /v1/tools/web_search with no Authorization → HTTP 401", + "required": true, + "status": "pass", + "observed": { + "status": 401, + "body": "{\"error\":{\"message\":\"unauthorized\"}}" + }, + "at": "2026-07-31T02:20:52.669Z" + }, + { + "id": "P3.2", + "ac": "AC3 真实失败集 400", + "title": "gateway web_search rejects a missing query with 400", + "criterion": "POST /v1/tools/web_search with a valid bearer and body {} → HTTP 400", + "required": true, + "status": "pass", + "observed": { + "status": 400, + "body": "{\"error\":{\"message\":\"query required\"}}" + }, + "at": "2026-07-31T02:20:52.904Z" + }, + { + "id": "P3.3", + "ac": "AC3 真实失败集 400", + "title": "gateway web_search rejects malformed JSON with 400", + "criterion": "POST /v1/tools/web_search with a valid bearer and a malformed body → HTTP 400", + "required": true, + "status": "pass", + "observed": { + "status": 400, + "body": "{\"error\":{\"message\":\"invalid json\"}}" + }, + "at": "2026-07-31T02:20:53.200Z" + }, + { + "id": "P3.4", + "ac": "AC3 真实失败集 403", + "title": "403 (action_forbidden / job_not_enforceable)", + "status": "not-producible", + "criterion": "would require a token whose scope excludes model.invoke, or a non-enforceable job — neither is mintable from a normal desktop login", + "observed": { + "reason": "the desktop only ever holds route-purpose-bound tokens for model.invoke and cloud.dispatch", + "coveredAtL1": "packages/opencode/test/tool/alpha-websearch-failure.test.ts (403 → forbidden, error.code preserved)" + }, + "required": false, + "at": "2026-07-31T02:20:53.200Z" + }, + { + "id": "P3.5", + "ac": "AC3 真实失败集 502", + "title": "502 (no search backend configured)", + "status": "not-producible", + "criterion": "would require the deployed gateway to have neither TAVILY_API_KEY nor BRAVE_API_KEY", + "observed": { + "reason": "both secrets are configured on the deployed gateway (docs/verification/2026-07-22-e7-deploy-probe.md §4); removing them is a production mutation, out of bounds for a probe", + "coveredAtL1": "packages/opencode/test/tool/alpha-websearch-failure.test.ts (502 → upstream)" + }, + "required": false, + "at": "2026-07-31T02:20:53.200Z" + }, + { + "id": "P3.6", + "ac": "AC3 真实失败集 意外状态 LOUD", + "title": "unexpected non-2xx status maps to a loud, discernible failure", + "status": "not-producible", + "criterion": "would require the deployed gateway to emit a status outside {400,401,402,403,502} on this route", + "observed": { + "reason": "no request shape reachable from the desktop elicits one; the mapping itself is exercised at L1", + "coveredAtL1": "packages/opencode/test/tool/alpha-websearch-failure.test.ts (statusKind default → unexpected_status)" + }, + "required": false, + "at": "2026-07-31T02:20:53.200Z" + }, + { + "id": "P3.7", + "ac": "AC3 真实失败集 402", + "title": "402 / 余额(accountPreauth 余额或额度不足臂)", + "status": "not-producible", + "criterion": "POST /v1/tools/web_search with a valid bearer → HTTP 402 when the account is out of quota+wallet; a 200 proves the account is funded (402 not producible today without draining the owner's balance, which would also invalidate P2.1/P2.2/P2.3); any other status is unexpected and loud", + "observed": { + "status": 200, + "body": "{\"query\":\"alpha-code e7 402 preauth probe 2026-07-31T02:20:38.799Z\",\"results\":\"[tavily] alpha-code e7 402 preauth probe 2026-07-31T02:20:38.799Z\\n- New Prior Authorization Requirement for Certain Procedure Codes (https://www.fostercaretx.com/newsroom/eff-04012026-new-pa-auth-req-for-certain-procedure-codes-12232025.html)\\n | Q5117 | INJECTION TRASTUZUMAB-ANNS BIOSIMILAR 10 MG |\\n| Q5118 | INJECTION BEVACIZUMAB-BVZR BIOSIMILAR 10 MG |\\n| Q5120 | INJ PEGFILGRASTIM-BMEZ BIOSIMLR ZIEXTENZO 0.5 MG |\\n| Q5122 | INJECTION PEGFILGRASTIM APGF BIOSIMILAR 0.5 MG |\\n| Q5126 | INJ BEVACIZUMAB-MALY BIOSIMI", + "accountShape": { + "balanceFen": 99925, + "walletUsedFen": 75, + "plan": { + "id": "pro", + "status": "active" + } + }, + "armA": "unreachable by construction — the desktop holds a route-purpose JWT (via:\"jwt\"), never a job token, so perJobPrecall returns pass/enforced:false", + "armB": "did not fire — the account was preauthorised, i.e. it is not out of quota+wallet today", + "coveredAtL1": "packages/opencode/test/tool/alpha-websearch-failure.test.ts (402 → payment_required, both arms: 预授权拒绝 and per-job budget exceeded)" + }, + "required": false, + "note": "not-producible today — 不是绿,也不是跳过:账户有额度/余额,402 的唯一可达臂(accountPreauth)不会触发。要真采 402 需把账户打到余额+额度双空,那是破坏性计费变更且与本轮 AC1/计费证据互斥。映射本身由 L1 覆盖。", + "at": "2026-07-31T02:20:56.635Z" + }, + { + "id": "P3.8", + "ac": "AC3 defect 消失 / 主权拒绝", + "title": "platform-pays makes the denied local websearch impossible for the model to attempt", + "status": "pass", + "criterion": "GET /experimental/tool for the selected platform model omits \"websearch\"; the user tools map cannot add an omitted tool, while discernible direct-denial wording remains an L1 responsibility", + "observed": { + "platformProviderID": "alpha", + "platformModelID": "deepseek-v4-flash", + "builtinToolIDs": [ + "invalid", + "question", + "bash", + "read", + "glob", + "grep", + "edit", + "write", + "task", + "webfetch", + "todowrite", + "skill", + "alpha_reload", + "alpha_register", + "alpha_echo", + "alpha_ping" + ], + "modelCanAttemptLocalWebSearch": false, + "coveredAtL1": "packages/opencode/test/tool/alpha-websearch-failure.test.ts" + }, + "required": true, + "at": "2026-07-31T02:20:56.635Z" + }, + { + "id": "P3.9", + "ac": "AC3 已登记缺口", + "title": "cloud-side failures are loud but not classifiable (alpha-platform#105)", + "status": "not-producible", + "criterion": "the cloud MCP shell drops r.status (packages/gateway/src/cloud-mcp.ts), so a cloud failure reaches the client with no HTTP status and no error.code", + "observed": { + "registeredGap": "alpha-platform#105", + "baseline": "docs/design/2026-07-22-e7-cloud-web-search-baseline.md §1 二次更正 (2026-07-25)" + }, + "required": false, + "at": "2026-07-31T02:20:56.635Z" + } + ] +} From f38ff46c97f4eac1f94bcaefa706723cce6fef76 Mon Sep 17 00:00:00 2001 From: jinjunnn Date: Thu, 30 Jul 2026 22:28:26 -0400 Subject: [PATCH 5/5] docs(verification): disclose full probe charge --- docs/verification/2026-07-27-e7-packaged-live/README.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/verification/2026-07-27-e7-packaged-live/README.md b/docs/verification/2026-07-27-e7-packaged-live/README.md index 1ab32c8e70f3..688c8a67eb41 100644 --- a/docs/verification/2026-07-27-e7-packaged-live/README.md +++ b/docs/verification/2026-07-27-e7-packaged-live/README.md @@ -187,6 +187,10 @@ cd ~/app/alpha-code && bun docs/verification/2026-07-27-e7-packaged-live/probe.t 该失败记录保留不改,用于证明判据为何必须修正。修正后的 [`logged-in-20260731T022038Z.json`](results/logged-in-20260731T022038Z.json) 在同一 `app.asar` 上 23 项、0 个必需失败;两次调用继续产生两笔各 15 分的精确结算。 +P3.7 为证明有余额账户真实返回 200,会在 P2.3 测量窗口之后再调用一次,因此每次完整相位 +实际为 3 笔 / 45 分。保留的假红运行与最终运行合计 6 笔 / 90 分;最终只读账户摘要为 +`walletUsedFen=90`、`balanceFen=99910`(运行前为 100000)。P2.3 表中的 30 分是它刻意圈定的 +两条 AC 路径差分,不是整轮总费用。 本次生产被测端点对应部署版本: