Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions examples/avatar/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,12 @@ Try it in the [LiveKit Playground](https://agents.livekit.io/?example=avatar).

> **Inference variant:** [`inference_agent.py`](./inference_agent.py) is a
> minimal version that provisions the avatar through **LiveKit Inference**
> instead of the BYOK plugin — the agent needs only `LIVEKIT_API_KEY` /
> `LIVEKIT_API_SECRET` (no `LEMONSLICE_API_KEY`); the gateway creates the
> instead of the BYOK plugin — no `LEMONSLICE_API_KEY` is needed; the agent
> authenticates with your LiveKit credentials and the gateway creates the
> provider session with LiveKit's wholesale key. Requires the
> `avatar_lemonslice` feature flag on your project. Run with
> `python inference_agent.py dev` and set `LEMONSLICE_IMAGE_URL`.
> `python inference_agent.py dev` and set `LIVEKIT_URL`, `LIVEKIT_API_KEY`,
> `LIVEKIT_API_SECRET`, and `LEMONSLICE_IMAGE_URL`.

## What's in here

Expand Down
40 changes: 27 additions & 13 deletions examples/avatar/inference_agent.py
Original file line number Diff line number Diff line change
@@ -1,13 +1,19 @@
"""Avatar agent provisioned through LiveKit Inference (no provider key).

Unlike agent.py (which uses the BYOK lemonslice plugin with a LEMONSLICE_API_KEY),
this starts the avatar with inference.AvatarSession: the agent authenticates only
with LIVEKIT_API_KEY / LIVEKIT_API_SECRET, and the Inference gateway creates the
LemonSlice session using LiveKit's wholesale key. Media and lip-sync still flow
in-room over DataStream, exactly as the BYOK path does.
this starts the avatar with inference.AvatarSession: no provider key is needed —
the agent authenticates with its LiveKit credentials (LIVEKIT_API_KEY /
LIVEKIT_API_SECRET, or LIVEKIT_INFERENCE_API_KEY / LIVEKIT_INFERENCE_API_SECRET
when set), and the Inference gateway creates the LemonSlice session using
LiveKit's wholesale key. Agent audio and the playback RPCs still flow in-room
over DataStream and the avatar publishes its video as a normal track, exactly as
the BYOK path does.

Requires the `avatar_lemonslice` feature flag to be enabled for your project on
the Inference gateway.
the Inference gateway; without it the gateway returns HTTP 403 "Inference Avatar
is not enabled for this project".

Env: LIVEKIT_URL, LIVEKIT_API_KEY, LIVEKIT_API_SECRET, LEMONSLICE_IMAGE_URL.

Run:
python inference_agent.py dev
Expand Down Expand Up @@ -44,19 +50,27 @@ async def entrypoint(ctx: JobContext) -> None:
tts=inference.TTS("cartesia/sonic-3"),
)

# Avatar provisioning goes through LiveKit Inference: only LIVEKIT_API_KEY /
# LIVEKIT_API_SECRET are needed (no provider key). The gateway creates the
# LemonSlice session with LiveKit's wholesale key; media stays in-room.
#
# Pass a catalog agent id instead of an image with
# inference.AvatarSession("lemonslice/<agent_id>", ...).
avatar_image_url = os.getenv("LEMONSLICE_IMAGE_URL")
if not avatar_image_url:
raise ValueError("LEMONSLICE_IMAGE_URL must be set")

# Provider-specific options go through extra_kwargs; LemonSliceOptions types
# the keys the gateway accepts (image_url / prompt / idle_prompt /
# idle_timeout).
#
# To use a pre-built LemonSlice agent instead of an image, pass its catalog id
# in the model string and drop image_url — the two are mutually exclusive:
#
# inference.AvatarSession(
# "lemonslice/<agent_id>",
# extra_kwargs=inference.LemonSliceOptions(prompt="..."),
# )
avatar = inference.AvatarSession(
"lemonslice",
image_url=avatar_image_url,
prompt="Be expressive in your movements and use your hands while talking.",
extra_kwargs=inference.LemonSliceOptions(
image_url=avatar_image_url,
prompt="Be expressive in your movements and use your hands while talking.",
),
)
await avatar.start(session, room=ctx.room)
await avatar.wait_for_join()
Expand Down