feat(agents): add beta end call tool - #1474
Merged
Merged
Conversation
🦋 Changeset detectedLatest commit: 27383c8 The changes in this PR will be included in the next version bump. This PR includes changesets to release 34 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
… Toolset
`Agent({ tools })` now accepts a list of `FunctionTool | ProviderDefinedTool
| Toolset` entries instead of a `Record<string, FunctionTool>` map, matching
the shape of the Python `ToolContext`. `llm.tool({ ... })` now requires an
explicit `name`. `ToolContext` is exported as a class with the same surface
as the Python one (`functionTools`, `providerTools`, `toolsets`, `flatten()`,
`getFunctionTool()`, `updateTools()`, `copy()`, `equals()`), backed by a
flat `_tools` list plus denormalized name map / provider list / toolset list.
A `Toolset` base class is included as the unit of stateful tool composition
with `setup()` / `aclose()` lifecycle hooks.
This is the foundational refactor; the standalone Toolset PR will rebase on
top of this once it lands.
- Removed `Toolset` from this PR — it lives in the stacked Toolset PR on top instead. `ToolContext` only handles `FunctionTool` and `ProviderDefinedTool` here. - `updateTools()` now uses later-wins semantics for duplicate function names instead of throwing. - Fixed every remaining call site that iterated `ToolContext` like a record (`Object.keys/entries(toolCtx)`): inference LLM, OpenAI chat / WS / responses, MistralAI, Baseten, and AgentActivity's initial-tools snapshot all now go through `.functionTools`.
…rray UX
- `updateTools()`: dropped redundant name-empty check (the `tool()` factory
already enforces a non-empty name at construction).
- `ToolContext.hasTool(name)`: new helper that matches function tools by
`name` and provider tools by `id`, so chat-context filtering remains
correct when provider tool call items are added later.
- `ChatContext.copy({ toolCtx })`: switched the filter predicate from
`getFunctionTool(name)` to `hasTool(name)` for that future-proofing.
- `AgentActivity` initial-tools snapshot: include both function tool names
and provider tool ids in the `AgentConfigUpdate.toolsAdded` payload.
- Inference LLM tool builder: iterates `toolCtx.flatten()` and discriminates
on tool type instead of only `functionTools`; provider tools are skipped
until AJS-112 wires them up properly.
- `ToolCtxInput` + `toToolContext()` helpers: LLM `chat({ toolCtx })` and
`LLMStream` now accept either a `ToolContext` or a raw
`(FunctionTool | ProviderDefinedTool)[]` array. Callers can write
`toolCtx: [tool1, tool2]` directly; `run_result.ts` updated to use it.
…n amd typing
- Removed `toolListFromRecord` — `tool()` requires `name` so the legacy
`Record<string, FunctionTool>` shape never gets produced in practice.
- `Agent.toolCtx` now returns `this._toolCtx.copy()` so external callers
can't mutate the agent's internal state.
- `amd.ts` imports `ToolContextEntry` properly at the top instead of
using the inline `import('../llm/index.js').ToolContextEntry` syntax.
rosetta-livekit-bot
Bot
force-pushed
the
hater-breaking-evoked
branch
from
May 17, 2026 23:40
43f6089 to
5ee8112
Compare
|
|
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Brings the public tool surface in line with `livekit-agents` Python:
- Tool gains an `id: string` field on the base interface. `FunctionTool.id`
mirrors `name`; for `ProviderTool` it's the provider tool id. `ToolContext`
keys and equality use `tool.id` consistently
(matches Python's `_fnc_tools_map[tool.info.name]`).
- `ProviderDefinedTool` → `ProviderTool`, converted from an interface to
an `abstract class`. Plugins now subclass it
(`class WebSearch extends ProviderTool { ... }`) to attach
provider-specific fields and serializers, mirroring Python's
`OpenAITool(ProviderTool, ABC)` / `WebSearch(OpenAITool)` chain.
The unused `config: Record<string, unknown>` field is dropped, and the
`tool({ id })` factory overload is removed — bare instantiation of the
abstract base is now a compile error.
- `ToolType` literal `'provider-defined'` → `'provider'`.
- `Toolset` now carries a `TOOLSET_SYMBOL` marker; added `isToolset()`
guard alongside `isFunctionTool` / `isProviderTool`. Replaced the one
remaining `instanceof Toolset` check in `agent_activity` for
cross-realm safety (consistent with how the other tool kinds are
detected).
- Plugin TODO comments in google / mistralai / openai / phonic updated
from "provider-defined tools" → "provider tools".
Tests, types, and changeset updated. All 312 LLM tests pass and the 31
publishable packages build clean.
Co-authored-by: u9g <jason.lernerman@livekit.io> Co-authored-by: rosetta-livekit-bot[bot] <282703043+rosetta-livekit-bot[bot]@users.noreply.github.com>
rosetta-livekit-bot
Bot
force-pushed
the
hater-breaking-evoked
branch
from
May 26, 2026 21:32
afe2a7c to
982ff6c
Compare
rosetta-livekit-bot
Bot
changed the base branch from
brian/toolset
to
brian/toolsets-create
May 26, 2026 21:46
toubatbrian
reviewed
May 27, 2026
toubatbrian
left a comment
Contributor
There was a problem hiding this comment.
Left a few comments. Agree with the overall direction, but since this would create behavior that differs from Python, I’d recommend having @theomonnom take another pass.
toubatbrian
reviewed
May 27, 2026
Move per-turn re-resolution into a DynamicToolset subclass so the runtime can detect it via instanceof and skip building a fresh ToolContext each turn when no dynamic toolset is active.
u9g
force-pushed
the
hater-breaking-evoked
branch
2 times, most recently
from
May 28, 2026 20:01
853187b to
45c4de8
Compare
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
u9g
force-pushed
the
hater-breaking-evoked
branch
2 times, most recently
from
May 28, 2026 20:19
dbc6f12 to
436cf01
Compare
…dateTools Drops the DynamicToolset class and resolveEachTurn option. A toolset's tools factory runs once at activation, after a new optional setup hook for one-time init (connecting, wiring listeners). Toolsets whose tools change at runtime push the new list via ToolsetContext.updateTools; the runtime re-flattens and re-advertises without re-running the factory or polling each turn.
u9g
force-pushed
the
hater-breaking-evoked
branch
from
May 28, 2026 20:23
436cf01 to
fff6713
Compare
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
toubatbrian
reviewed
May 29, 2026
Use the per-invocation abortSignal from ToolOptions instead of the toolset teardownSignal so close listeners and timers clean up promptly when the tool call is cancelled.
Toolset tools are now always a static list. Dynamic toolsets can still push tools after activation via ToolsetContext.updateTools from a setup listener. Inline end_call's tool implementation directly into its execute callback now that no factory-level closure is needed.
This reverts commit a360cfb.
The end_call tool no longer needs a tools factory — its session reference is reachable via ctx.session, so build it directly in the tools array and move the implementation into the execute callback.
Match the naming convention for factory functions returning a Toolset.
Drop session/signal from ToolsetContext — toolsets no longer get lexical access to the session or a teardown signal. Drop the ToolsetFactoryFn form of Toolset.create: anything a factory did can be expressed as a setup hook that calls ctx.updateTools, so tools is now always a static list.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Co-authored-by: Cursor <cursoragent@cursor.com> # Conflicts: # agents/src/inference/llm.ts # agents/src/llm/index.ts # agents/src/llm/tool_context.test.ts # agents/src/llm/tool_context.ts # agents/src/voice/agent.test.ts # agents/src/voice/agent.ts # agents/src/voice/agent_activity.ts # agents/src/voice/agent_v2.ts # agents/src/voice/generation.ts # agents/src/voice/generation_tools.test.ts # agents/src/voice/remote_session.ts # plugins/google/src/index.ts # plugins/google/src/llm.ts # plugins/google/src/realtime/realtime_api.ts # plugins/google/src/utils.ts # plugins/mistralai/src/llm.ts # plugins/openai/src/responses/llm.ts # plugins/openai/src/ws/llm.ts
toubatbrian
force-pushed
the
hater-breaking-evoked
branch
from
June 1, 2026 19:13
cf03aca to
27383c8
Compare
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
- agent_activity: route onToolsetToolsChanged through updateTools so a dynamic toolset push refreshes history + chat context (not just the realtime session), and add a regression test proving the push records an AgentConfigUpdate - end_call: guard the fire-and-forget delayedSessionShutdown with .catch() to avoid unhandled rejections - aiplatform_llm: accept ToolCtxInput in chat() and normalize via toToolContext(), matching the other LLM plugins - tool_context: return a defensive copy from the providerTools getter - inference/llm: drop a duplicated provider-tool comment
toubatbrian
approved these changes
Jun 8, 2026
tinalenguyen
added a commit
that referenced
this pull request
Jun 23, 2026
Squashed local branch work, reconciled with the latest remote state: - Refactor ToolContext to parity class taking a list of Tool | Toolset (#1517) - feat(agents): add Toolset support to ToolContext and AgentActivity (#1525) - feat(agents): add beta end call tool (#1474) - fix(agents): thread FlushSentinel through Agent.create llmNode types - fix(elevenlabs): end server vad turns (#1745) - Add Inworld delivery mode inference TTS option (#1749) - Don't retain recorded events when recording is disabled (#1750) - fix(deepgram): validate updated model with stored language on updateOptions
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
EndCallToolported from Python for ending calls from a tool invocation.JobContext.deleteRoom()support and export worker credentials to child job env so room deletion works when options are passed programmatically.Testing
pnpm --filter @livekit/agents buildpnpm --filter @livekit/agents lint(passes with existing warnings)pnpm format:checkNotes
pnpm --filter @livekit/agents api:checkis currently blocked by API Extractor failing on existingexport * as ___syntax indist/index.d.tsbefore checking this API surface.