fix(agent): preserve attachment context in text-generation prompts - #1475
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ce97946e07
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| if (convertedPart) { | ||
| parts.push(convertedPart); | ||
| continue; | ||
| } |
There was a problem hiding this comment.
Preserve single attachment annotation across round trips
Re-processing a provider message that already contains an <uploaded_files> text annotation now appends a second annotation, because converted file/image parts no longer short-circuit before being collected into attachmentReferences. This happens in the child-fork flow (prepareHostedChildForkRuntimeStepMessages converts agent messages to provider messages and back), so attachment metadata can be duplicated on each round trip, inflating prompt size and increasing token pressure for long conversations with files.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Addressed in c12f800: provider messages that already contain an <uploaded_files> text annotation now skip adding the fallback attachment-context part, while still preserving the native file/image part. Added a provider round-trip regression test that asserts only one annotation remains.
ce97946 to
c12f800
Compare
8d12544 to
a29ba1a
Compare
a29ba1a to
dec60bf
Compare
Summary
image/fileparts while also adding provider-visible uploaded-file text context.<uploaded_files>metadata for native attachments instead of dropping them.<uploaded_files>annotations during provider round trips.0.1.426to0.1.427indeno.jsonandsrc/utils/version-constant.ts.Addresses https://github.com/veryfront/veryfront-studio/issues/3510 and the upstream attachment-context portion of https://github.com/veryfront/veryfront-studio/issues/3499.
Why this is needed
#1469 preserved native file/image parts through the AgentRuntimeMessage pipeline, but the text-generation runtime path still needed provider-visible attachment context. Without that context, the model can respond as if no attachment was present even though the persisted chat message contains a file part.
Regression coverage
<uploaded_files>annotation.Test plan
deno fmt --check deno.json src/utils/version-constant.ts src/agent/agent-runtime-message-adapter.ts src/agent/agent-runtime-message-adapter.test.ts src/agent/runtime-message-preparation.test.ts src/agent/runtime/text-generation-runtime-message-converter.ts src/agent/runtime/text-generation-runtime-message-converter.test.tsDENO_NO_PACKAGE_JSON=1 deno lint src/utils/version-constant.ts src/agent/agent-runtime-message-adapter.ts src/agent/agent-runtime-message-adapter.test.ts src/agent/runtime-message-preparation.test.ts src/agent/runtime/text-generation-runtime-message-converter.ts src/agent/runtime/text-generation-runtime-message-converter.test.tsdeno test --no-check --allow-all --unstable-worker-options --unstable-net src/agent/agent-runtime-message-adapter.test.ts src/agent/runtime-message-preparation.test.ts src/agent/runtime/text-generation-runtime-message-converter.test.ts— 7 passed / 31 steps / 0 faileddeno check src/agent/agent-runtime-message-adapter.ts src/agent/runtime-message-preparation.ts src/agent/runtime/text-generation-runtime-message-converter.tsFollow-up required after merge
After this package version is published,
veryfront-agentneeds a dependency-only PR bumpingveryfrontto the published version containing this change, followed by staging rollout and end-to-end attachment retest.