Skip to content

[bug] Coding agents get NO length rule in the prompt — lengthRule is computed for all and emitted only inside plain-speech, and the Behaviour tab shows "balanced" for a field that is unset #430

Description

@serge-ivo

A coding agent has NO length rule in its prompt at all, and the Behaviour tab shows settings that are not in effect

Raised from a live instance (bd43f4de-…, "Chess coder 2") whose owner asked why replies are so technical and so long when the Behaviour tab shows verbosity balanced.

The agent is behaving exactly as configured. The configuration plumbing is the defect.

Live state of that instance

GET /behaviour     → { "technicality": 100 }          ← the ONLY field set
                     templateDefault: {}               ← creator set nothing
GET /settings      → { engine: claude, autonomy: normal, merge_policy: merge }
GET /instructions  → ""                                ← Rules & Tips empty

technicality: 100 renders as "Assume senior-engineer familiarity. Be precise and specific: cite real file paths, function names, and short snippets, and skip introductory explanation." So the file paths are requested. That part is working.

verbosity is absent. And that is where three defects compound.


Defect 1 — lengthRule is computed for every agent and emitted for almost none

agent-think.ts:645 computes it unconditionally, with a comment stating the intent:

// The 2-sentence cap is a LENGTH rule that happened to live inside the plain-speech block. A
// subscriber who asked for thorough answers has already overruled it, so honour that rather than
// telling the model both things in the same prompt.
lengthRule: behaviour.verbosity ? fieldPrompt(behaviourField("verbosity")!, behaviour.verbosity)
                                : "MAXIMUM 2 sentences. Shorter is better.",

But in lib/agent-style-prompt.ts, lengthRule is destructured at :146 and used at exactly one site — :205, inside if (plainSpeech).

resolveResponseStyle (agent-behaviour.ts:784):

return { codingContext, technical, styleReminder, plainSpeech: !codingContext && !technical };

codingContext = repoChatStyle || hasCodingContexttrue for any instance with attached repos. So plainSpeech is false, and lengthRule never reaches the prompt.

The refactor lifted the value out of the plain-speech block but left the usage inside it. Net effect for a coding agent: no length instruction anywhere in the system prompt — not the subscriber's verbosity, not even the 2-sentence fallback. That is the whole explanation for unbounded replies.

Verified there is no length rule elsewhere: behaviourStyleReminder (agent-behaviour.ts:646-656) loops ["technicality", "verbosity"] but includes each only if (id in behaviour). With verbosity unset, it contributes nothing. styleReminder is injected at agent-think.ts:1014 and :1034; on this instance it carries technicality only.

Consequence worth stating plainly: setting verbosity would work on a coding agent (via behaviourStyleReminderstyleReminder), but only if explicitly set. Unset is not "balanced" — it is unconstrained.

Defect 2 — the Behaviour tab shows a value that is not in effect

store/console/src/tabs/BehaviourTab.tsx:278:

const effective = value ?? field.default;

GET /v1/instances/behaviour-schema gives verbosity a default of balanced. So an unset verbosity renders as "balanced" — the control is parked there, and a small grey default label sits beside it (:311).

But the behaviour of unset is not "balanced". Per Defect 1 it is "no length rule at all" on a coding agent, and "MAXIMUM 2 sentences" on a plain one. Neither is balanced, and both differ from what the screen says.

The schema's default is documented as "where the UI parks a control", but the UI presents it as the effective value. The default badge is doing a lot of work: it reads as "this is the platform default" (true) rather than "this value is not applied" (also true, and the part that matters). A user checking their settings sees balanced and reasonably concludes the agent was told to be balanced.

Defect 3 — a coding agent can never be made non-technical

plainSpeech = !codingContext && !technical, and technical = prefersTechnical(behaviour) ?? codingContext.

Setting technicality to 0 makes prefersTechnical false, so technical is false — but codingContext is still true, so plainSpeech stays false. The plain-speech rules, including the one the owner actually wants —

"Never mention filenames, paths, line numbers, git status, CSS classes, function names, or code." (agent-style-prompt.ts:206)

— are unreachable for any instance with a repo attached, at every slider position.

This is the capability/preference conflation #223-#226 set out to remove. It was removed from styleReminder (which does honour preference, agent-behaviour.ts:776-782) but not from plainSpeech, where capability still vetoes preference outright. The Behaviour tab offers the owner a control that is partly inert on exactly the agent type where it is most visible.


What to do

1. Emit lengthRule on every branch, not just plain-speech. The four styleGuidance branches each end with a style list; the length rule belongs on all of them. Cheapest correct form: append it after the branch returns, so a new branch cannot forget it.

2. Make the fallback honest per branch. "MAXIMUM 2 sentences" is right for read-aloud plain speech and wrong for a code explainer. Give the coding branches a sane unset default (a few short paragraphs, or bullet points) rather than nothing — an unbounded default is the current bug.

3. Let preference reach plainSpeech. When the owner has explicitly set a low technicality, honour it: plainSpeech = !technical with codingContext deciding only which grounding block applies, not the language level. That is the separation the field table was built for. Guard against the known failure mode — a plain-speech coding agent must still not be told it has a vector index or a Repo tab.

4. Show unset as unset. Render the control in an indeterminate state, or label it not set — platform default applies and say what that default actually does. Showing balanced for a field that produces no instruction is the part that made this a support question.

Alternatives considered and rejected

  • Set verbosity on this instance and close it. That fixes one user, not the class: every coding instance created since Agent Behaviour: a real home for character, replacing three hardcoded style heuristics #223 has the same unbounded default, and the console will keep telling all of them they are "balanced".
  • Store the schema defaults on every instance at creation. Rejected — it destroys the "unset is a first-class state" design (agent-behaviour.ts), which is what lets the creator default and the platform heuristic work at all.
  • Delete the default from the schema so nothing is displayed. Rejected — a slider needs a resting position; the fix is labelling it honestly, not removing it.
  • Only fix the UI label. Rejected — the agent would still have no length rule, which is the substantive half.

Acceptance criteria

  • A coding-capable instance with verbosity unset receives a length instruction in its system prompt.
  • Setting verbosity to brief measurably shortens replies on a coding agent.
  • Setting technicality to its lowest value on a coding agent stops it citing file paths.
  • The Behaviour tab does not display a value that is not applied; an unset field is visibly distinct from one set to the same value.
  • A test asserts every styleGuidance branch contains a length rule (so branch Multi-provider repos: private clone + identity for GitLab / Bitbucket #5 cannot silently omit it).

Regression risk

Related: #223-#226 (the behaviour field table this is meant to serve), #254/#255 (what happens when the style branch and the agent's real capabilities disagree), #429 (same instance, same live session).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions