fix(local_browser): drop final_script.py workflow and bound ARIA history#16
Merged
Merged
Conversation
local_browser.yaml's prompts were copy-pasted from base.yaml's bash_command-driven workspace flow, which doesn't fit a live persistent Playwright session. The agent was told to launch its own browser inside final_script.py while also being told never to import or launch Playwright; and it was told to shell out via `run_command(...)`, a helper that does not exist anywhere in the codebase. Reframe the live-browser mode around its actual capabilities: - Drop final_script.py, plan.md, self_reflect_config.json, final_runs/, image_qa, and self_reflection from the prompts. There is no workspace directory in this mode; the agent drives the live `page` and reports its answer in `final_response` when done. - Override `agent.require_self_reflection_success: false` so the completion gate (which checks for `final_runs/run_<id>/self_reflect_result.json`) does not block `done: true` in this mode. - Override `agent.summary_user_prompt` to talk about live browser state (URL, controls, selectors) instead of workspace artifacts (`plan.md`, `final_script.py`, `final_runs/`) that don't exist. - Document in the yaml header what each step's observation contains (URL / title / stdout / ARIA snapshot / screenshot path) and note that the screenshot is NOT visually attached by default — flipping `model.attach_observation_screenshot: true` enables multimodal image input. Add a context-bounding knob in `AgentConfig`: - `keep_last_n_observations` (default -1, disabled) strips the ARIA snapshot payload from observation messages older than the most recent N, both in the rendered text content and the `extra` observation dict. URL / title / printed stdout are preserved so the agent still knows what it did. local_browser.yaml opts in with N=1. Validated end-to-end against "Search for flights from SEA to JFK on 2026-08-15 to 2026-08-20" via base.yaml + local_browser.yaml + model_openai.yaml: the agent reached the Google Flights results page (round trip, SEA -> JFK, Aug 15 -> Aug 20) in 7 steps with N=1 pruning (vs 9 steps without), and the saved trajectory shows ARIA stripped from the first 6 observations and preserved on the 7th. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
local_browser.yaml's prompts were copy-pasted frombase.yaml'sbash_command-driven workspace flow, which doesn't fit a live persistent Playwright session. Two concrete bugs:final_script.pywhile also being told never to import or launch Playwright.run_command(...)— a helper that doesn't exist anywhere in the codebase. Any agent step calling it would die withNameError.This PR reframes the live-browser mode around its actual capabilities and adds a context-bounding knob so ARIA snapshots don't dominate token usage as the loop runs.
Changes
local_browser.yaml(−221 / +127)final_script.py,plan.md,self_reflect_config.json,final_runs/,image_qa, andself_reflectionfrom the prompts. There is no workspace directory in this mode; the agent drives the livepageand reports its answer infinal_responsewhen done.agent.require_self_reflection_success: falseso the completion gate (which checks forfinal_runs/run_<id>/self_reflect_result.json) does not blockdone: true.agent.summary_user_promptto talk about live browser state (URL, controls, selectors) instead of workspace artifacts (plan.md,final_script.py,final_runs/) that don't exist in this mode.model.attach_observation_screenshot: trueenables multimodal image input.agent.keep_last_n_observations: 1.AgentConfig.keep_last_n_observations(+33 / -1)agents/default.py. Default-1(disabled) — opt in per config.extra.observationdict. URL / title / printed stdout are preserved so the agent still knows what it did.self.messagesafter eachadd_messagescall, finds observation messages byextra.observation, replaces the savedaria_snapshotstring in the text content with(ARIA snapshot pruned; see most recent observation).Validation
Live end-to-end run against task: "Search for flights from SEA to JFK on 2026-08-15 to 2026-08-20" via
base.yaml + local_browser.yaml + model_openai.yaml.SubmittedSubmittedgoogle.com/travel/flights/search?tfs=…Trajectory inspection of the N=1 run confirms: observation #1–6 have ARIA stripped from both the text content and
extra.observation.aria_snapshot; only observation #7 retains its full 15,282-char ARIA snapshot.Test plan
model_openai.yamlsucceeds and reaches the results page in 7 steps with N=1.persistent_browser.yaml,persistent_browser_always.yaml,crafted_cli.yaml, andtask_showcase.yaml(samerun_command/final_script.pyissues are copy-pasted there).