diff --git a/README.md b/README.md index 23ff037d..cb099db2 100644 --- a/README.md +++ b/README.md @@ -28,13 +28,55 @@ aai login # store your API key (browser-assisted) aai transcribe --sample # transcribe the hosted wildfires.mp3 sample ``` +## API key & security + +`aai` resolves your key in this order: + +1. The `ASSEMBLYAI_API_KEY` environment variable. +2. The OS keyring (macOS Keychain, Windows Credential Manager, Linux Secret + Service), written only when you run `aai login`. + +Two things worth knowing: the key is **never stored in a plaintext dotfile** — +`aai login` puts it in the OS keyring, and the only on-disk config (`config.toml`) +holds just profile names. And there is **no `--api-key` flag on run commands** +(`transcribe`, `stream`, …), so a key can't leak into `ps` output or shell history +via a command's arguments. + +**Prefer not to persist the key at all?** Skip `aai login` and set the environment +variable instead — it's checked *before* the keyring, so nothing is ever written to +disk: + +```sh +ASSEMBLYAI_API_KEY=sk_... aai transcribe call.mp3 +``` + +Prefixing it on a single command (rather than `export`-ing it) scopes the secret to +that one process. To also keep it out of your shell history, inject it from a secret +manager at call time: + +```sh +# 1Password CLI +ASSEMBLYAI_API_KEY=$(op read "op://Private/AssemblyAI/api key") aai transcribe call.mp3 +op run -- aai transcribe call.mp3 # …or wrap the whole command + +# HashiCorp Vault +ASSEMBLYAI_API_KEY=$(vault kv get -field=key secret/assemblyai) aai stream + +# macOS Keychain (a generic-password item you manage) +ASSEMBLYAI_API_KEY=$(security find-generic-password -w -s assemblyai -a "$USER") aai transcribe call.mp3 +``` + +In CI, set `ASSEMBLYAI_API_KEY` as a masked secret — nothing is stored. The env var +also overrides a stored key for one-off use; `aai logout` purges the keyring entry, +and `aai whoami` / `aai doctor` confirm which source is active without printing the key. + ## Commands | Command | What it does | | --- | --- | | `aai login` / `logout` / `whoami` | Manage the stored API key. | | `aai doctor` | Check your environment is ready (API key, network, ffmpeg, microphone, agent tooling). | -| `aai transcribe ` | Transcribe an audio file, URL, or YouTube URL (`--sample` for a demo, `--llm-gateway-prompt` to transform the result, `--show-code` to print the equivalent Python). | +| `aai transcribe ` | Transcribe an audio file, URL, or YouTube URL (`--sample` for a demo, `--llm` to transform the result through LLM Gateway, `--show-code` to print the equivalent Python). | | `aai transcripts list` / `get ` | Browse and fetch past transcripts. | | `aai stream [file]` | Real-time transcription from a file or the microphone. | | `aai agent` | Live two-way voice conversation with a voice agent. | @@ -128,19 +170,33 @@ aai stream --sample \ ## Live transcript → live LLM -`aai stream -o text` writes one finalized turn per line and flushes immediately, so it -can drive `aai llm` turn by turn. Add `--follow` (`-f`) to `aai llm` to keep re-running -your prompt over the *growing* transcript, refreshing the answer in place on every turn: +`aai stream --llm "PROMPT"` runs a prompt over the live transcript through LLM Gateway, +refreshing the answer on every finalized turn — one command, no pipe to wire up: + +```sh +aai stream --llm "summarize action items as I talk" +``` + +It's repeatable, so prompts chain — each runs on the previous one's response: + +```sh +aai stream --llm "extract action items" --llm "rewrite them as a checklist" +``` + +On a terminal you watch one evolving panel; piped onward it emits one JSON object per +refresh (`{"turns": N, "output": "…"}`). Ctrl-C to stop. + +**Prefer the pipe?** The same thing composes from the primitives: `aai stream -o text` +writes one finalized turn per line, and `aai llm -f` (`--follow`) re-runs your prompt +over the *growing* transcript. Reach for this when you want a `--system` prompt or other +tools in the pipeline: ```sh aai stream -o text | aai llm -f --system "You are a meeting scribe" "summarize action items as I talk" ``` -On a terminal you watch one evolving summary; piped onward it emits one JSON object per -refresh (`{"turns": N, "output": "…"}`). Each finalized turn triggers a fresh call over -the full transcript, so the answer is always current. Ctrl-C to stop. Without `--follow`, -`aai llm` stays one-shot — it reads stdin to EOF and answers once (`cat notes | aai llm -"summarize"`). +Without `--follow`, `aai llm` stays one-shot — it reads stdin to EOF and answers once +(`cat notes | aai llm "summarize"`). ## Voice agent @@ -173,16 +229,18 @@ aai agent --voice ivy --show-code # the full-duplex ag ``` The generated transcribe code includes result handling for the analysis features you -enabled. With `--llm-gateway-prompt` (repeatable — each prompt runs on the previous -response), it emits the chained LLM Gateway calls too: +enabled. With `--llm` (repeatable — each prompt runs on the previous response), it emits +the chained LLM Gateway calls too: ```sh aai transcribe call.mp3 \ - --llm-gateway-prompt "summarize" \ - --llm-gateway-prompt "translate the summary to Spanish" \ + --llm "summarize" \ + --llm "translate the summary to Spanish" \ --show-code > summarize_then_translate.py ``` +`aai stream --llm "…" --show-code` likewise emits the live transcribe→LLM-per-turn loop. + ## Pipelines `aai` is built to compose with the rest of your shell. Output is machine-clean @@ -207,23 +265,24 @@ curl -sL https://example.com/ep.mp3 | aai transcribe - # no temp file ffmpeg -i in.mp4 -f s16le -ac 1 -ar 16000 - | aai stream - # live, from a pipe ``` -**Feed transcripts into the LLM Gateway** (`aai llm` reads piped stdin): +**Feed text into the LLM Gateway** (`aai llm` reads piped stdin). For a transcript, +`aai transcribe --llm "…"` does it in one step — the pipe is for any *other* text: ```sh -aai transcribe call.mp3 -o text | aai llm "summarize, then list action items" cat notes.txt | aai llm "turn these into a changelog" ``` -**Stream, then summarize.** Piped `stream`/`agent` emit clean transcript lines with -`-o text`. A Ctrl-C in a pipe hits both sides, so to stop the producer *and* let the +**Pipe a live stream into other tools.** For live LLM summaries use `aai stream --llm` +(above) — one process, clean Ctrl-C. To pipe the live transcript into a *different* tool, +note that a Ctrl-C in a pipe hits both sides, so to stop the producer and let the consumer finish, signal only the producer — or end the stream on its own: ```sh # end after 30s by signaling just the producer (macOS: brew install coreutils, use gtimeout) -timeout -s INT 30s aai stream -o text | aai llm "summarize" +timeout -s INT 30s aai stream -o text | grep -i "action item" # or end on a natural pause (server-side inactivity timeout, in seconds) -aai stream -o text --inactivity-timeout 5 | aai llm "summarize the call" +aai stream -o text --inactivity-timeout 5 > call.txt # capture then process (most robust) aai stream -o text > call.txt # Ctrl-C to stop @@ -235,13 +294,6 @@ aai llm "summarize" < call.txt A cookbook of `aai` composed with common Unix tools. macOS shown; on Linux swap `pbcopy`/`pbpaste` → `xclip -sel clip`/`xclip -o` and `say` → `spd-say`. -**Live meeting scribe** — `-o text` streams one finalized turn per line; `aai llm -f` -re-summarizes the growing transcript in place on every turn (Ctrl-C to stop): - -```sh -aai stream -o text | aai llm -f --model claude-haiku-4-5-20251001 "summarize todos as I talk" -``` - **Chain `aai llm` into other tools** with `-o text` — it prints just the answer, so it pipes onward cleanly (no `jq` needed): @@ -260,7 +312,7 @@ cat error.log | aai llm "what's the root cause and the one-line fix?" for the pipeline you described, and `aai llm` rewrites it in another language: ```sh -aai transcribe --sample --llm-gateway-prompt "translate to french" --show-code | aai llm "rewrite in rust" +aai transcribe --sample --llm "translate to french" --show-code | aai llm "rewrite in rust" ``` **Mine the analysis JSON with `jq`** — enable a feature, then slice `-o json`: diff --git a/assemblyai_cli/code_gen/__init__.py b/assemblyai_cli/code_gen/__init__.py index 01a249b5..4f918fff 100644 --- a/assemblyai_cli/code_gen/__init__.py +++ b/assemblyai_cli/code_gen/__init__.py @@ -20,6 +20,15 @@ def transcribe( return _transcribe.render(merged, source, llm_gateway=llm_gateway) -def stream(merged: dict[str, object]) -> str: - """Generate runnable Python that reproduces this streaming invocation.""" - return _stream.render(merged) +def stream( + merged: dict[str, object], + *, + llm: dict[str, object] | None = None, +) -> str: + """Generate runnable Python that reproduces this streaming invocation. + + With `llm` (a dict of ``prompts``/``model``/``max_tokens``), the script refreshes a + prompt-chain over the growing transcript on every finalized turn — the live sibling + of `transcribe --llm` — mirroring how `stream --llm` runs. + """ + return _stream.render(merged, llm=llm) diff --git a/assemblyai_cli/code_gen/stream.py b/assemblyai_cli/code_gen/stream.py index 44fef592..bab581a2 100644 --- a/assemblyai_cli/code_gen/stream.py +++ b/assemblyai_cli/code_gen/stream.py @@ -1,5 +1,8 @@ from __future__ import annotations +from typing import cast + +from assemblyai_cli import llm as gateway from assemblyai_cli.code_gen import serialize # Streaming-class imports always used by the generated scaffold. SpeechModel is added @@ -30,6 +33,54 @@ def on_turn(client: StreamingClient, event: TurnEvent) -> None: print() +client = StreamingClient( + StreamingClientOptions(api_key=API_KEY, api_host="streaming.assemblyai.com") +) +client.on(StreamingEvents.Turn, on_turn) +""" + +_LLM_PREAMBLE = """import os + +import assemblyai as aai +from assemblyai.streaming.v3 import ( +{imports} +) +from openai import OpenAI + +# Export your key first: export ASSEMBLYAI_API_KEY="" +API_KEY = os.environ["ASSEMBLYAI_API_KEY"] +aai.settings.api_key = API_KEY + +# Run the prompts over the LLM Gateway (OpenAI-compatible). Each prompt runs on the +# previous response; the first runs on the transcript accumulated so far. +gateway = OpenAI(api_key=API_KEY, base_url={base_url!r}) +PROMPTS = [ +{prompts} +] +transcript: list[str] = [] + + +def run_chain(text: str) -> str: + result = text + for i, prompt in enumerate(PROMPTS): + source = text if i == 0 else result + response = gateway.chat.completions.create( + model={model!r}, + messages=[{{"role": "user", "content": prompt + "\\n\\nTranscript:\\n" + source}}], + max_tokens={max_tokens}, + ) + result = response.choices[0].message.content + return result + + +def on_turn(client: StreamingClient, event: TurnEvent) -> None: + # Refresh the answer on every finalized turn, over the growing transcript. + if not event.end_of_turn or not event.transcript: + return + transcript.append(event.transcript) + print(run_chain(" ".join(transcript))) + + client = StreamingClient( StreamingClientOptions(api_key=API_KEY, api_host="streaming.assemblyai.com") ) @@ -45,13 +96,29 @@ def on_turn(client: StreamingClient, event: TurnEvent) -> None: """ -def render(merged: dict[str, object]) -> str: - """Generate a runnable microphone-streaming script with the given params.""" +def render(merged: dict[str, object], *, llm: dict[str, object] | None = None) -> str: + """Generate a runnable microphone-streaming script with the given params. + + With `llm`, the script transforms the live transcript through the LLM Gateway, + refreshing a prompt chain on every finalized turn (the live sibling of + `transcribe --llm`). + """ names = list(_BASE_IMPORTS) if "speech_model" in merged: names.append("SpeechModel") imports = "\n".join(f" {name}," for name in sorted(names)) - preamble = _PREAMBLE.format(imports=imports) + + if llm: + prompts = "\n".join(f" {p!r}," for p in cast("list[str]", llm["prompts"])) + preamble = _LLM_PREAMBLE.format( + imports=imports, + base_url=gateway.GATEWAY_BASE_URL, + prompts=prompts, + model=llm["model"], + max_tokens=llm["max_tokens"], + ) + else: + preamble = _PREAMBLE.format(imports=imports) # Mic capture rate must match StreamingParameters.sample_rate, else audio is corrupt. rate = merged.get("sample_rate", 16000) diff --git a/assemblyai_cli/commands/llm.py b/assemblyai_cli/commands/llm.py index 72b9dc0f..bff1a5e3 100644 --- a/assemblyai_cli/commands/llm.py +++ b/assemblyai_cli/commands/llm.py @@ -1,59 +1,17 @@ from __future__ import annotations import typer -from rich.live import Live from rich.markup import escape -from rich.panel import Panel from assemblyai_cli import config, output, stdio from assemblyai_cli import llm as gateway from assemblyai_cli.context import AppState, run_command from assemblyai_cli.errors import UsageError +from assemblyai_cli.follow import FollowRenderer app = typer.Typer() -class _FollowRenderer: - """Render a live transcript transform that refreshes on every turn. - - On a terminal, the latest answer is redrawn in place inside a Rich panel so a - human watches one evolving summary. When piped or run by an agent (json_mode), - each refresh is emitted as one NDJSON object so it stays machine-readable. - """ - - def __init__(self, *, json_mode: bool) -> None: - self.json_mode = json_mode - self._live: Live | None = None - self._last: Panel | None = None - - def __enter__(self) -> _FollowRenderer: - if not self.json_mode: - # screen=True draws into the terminal's alternate buffer (like less/htop). - # In the `aai stream -o text | aai llm -f` pipeline two processes share one - # TTY: stream writes status to stderr and the Ctrl-C "^C" echoes into our - # region, desyncing Rich's relative-cursor teardown and duplicating the top - # border. The alt buffer is isolated and restored verbatim on exit, so that - # noise is discarded; we reprint the final panel to the normal screen below. - self._live = Live(console=output.console, auto_refresh=False, screen=True) - self._live.start() - return self - - def __call__(self, answer: str, turns: int) -> None: - if self.json_mode: - output.emit_ndjson({"turns": turns, "output": answer}) - elif self._live is not None: - title = f"scribe · {turns} turn{'s' if turns != 1 else ''}" - self._last = Panel(escape(answer or "…"), title=title, border_style="aai.brand") - self._live.update(self._last, refresh=True) - - def __exit__(self, *exc: object) -> None: - if self._live is not None: - self._live.stop() # leaves the alt buffer, restoring the normal screen - self._live = None - if self._last is not None: - output.console.print(self._last) # leave the final summary as scrollback - - @app.command() def llm( ctx: typer.Context, @@ -125,7 +83,7 @@ def ask(transcript_text: str) -> str: ) return gateway.content_of(response) - with _FollowRenderer(json_mode=json_mode) as render: + with FollowRenderer(json_mode=json_mode) as render: transcript: list[str] = [] try: for turn in stdio.iter_piped_stdin_lines(): diff --git a/assemblyai_cli/commands/stream.py b/assemblyai_cli/commands/stream.py index 6ec5f882..328e2e14 100644 --- a/assemblyai_cli/commands/stream.py +++ b/assemblyai_cli/commands/stream.py @@ -9,6 +9,7 @@ from assemblyai_cli import client, code_gen, config, config_builder, llm, output, youtube from assemblyai_cli.context import AppState, run_command from assemblyai_cli.errors import UsageError +from assemblyai_cli.follow import FollowRenderer from assemblyai_cli.microphone import MicrophoneSource from assemblyai_cli.streaming.render import StreamRenderer from assemblyai_cli.streaming.sources import TARGET_RATE, FileSource, StdinSource @@ -84,10 +85,12 @@ def stream( config_file: str = typer.Option(None, "--config-file", help="JSON file of streaming fields."), # existing prompt: str = typer.Option(None, "--prompt", help="Bias the speech model (u3-pro)."), - llm_gateway_prompt: str = typer.Option( + llm_prompt: list[str] = typer.Option( None, - "--llm-gateway-prompt", - help="After streaming, transform the full transcript through LLM Gateway.", + "--llm", + help="Run a prompt over the live transcript through LLM Gateway, refreshing the " + "answer on every finalized turn. Repeatable: each prompt runs on the previous " + "one's response (a chain).", ), model: str = typer.Option(llm.DEFAULT_MODEL, "--model", help="LLM Gateway model."), max_tokens: int = typer.Option(llm.DEFAULT_MAX_TOKENS, "--max-tokens", help="Max tokens."), @@ -154,7 +157,12 @@ def make_flags(rate: int) -> dict[str, object]: overrides=list(config_kv or []), config_file=config_file, ) - output.print_code(code_gen.stream(merged)) + gateway = ( + {"prompts": list(llm_prompt), "model": model, "max_tokens": max_tokens} + if llm_prompt + else None + ) + output.print_code(code_gen.stream(merged, llm=gateway)) return api_key = config.resolve_api_key(profile=state.profile) @@ -166,16 +174,38 @@ def make_flags(rate: int) -> dict[str, object]: elif from_file and (sample_rate is not None or device is not None): raise UsageError("--sample-rate and --device apply only to microphone input.") + if llm_prompt and text_mode: + raise UsageError( + "--llm renders a live panel (or NDJSON when piped); it can't be combined " + "with -o text." + ) + renderer = StreamRenderer(json_mode=json_mode, text_mode=text_mode) - # Collect finalized turns so we can transform the full transcript at the end. - turns: list[str] = [] + # In --llm mode the answer is rendered live by a FollowRenderer instead of the + # raw turns; transcript accumulates the finalized turns we re-run the chain over. + follow = FollowRenderer(json_mode=json_mode) if llm_prompt else None + transcript: list[str] = [] def on_turn(event: object) -> None: - renderer.turn(event) - if llm_gateway_prompt and getattr(event, "end_of_turn", False): - text = getattr(event, "transcript", "") or "" - if text: - turns.append(text) + if follow is None: + renderer.turn(event) + return + # Live LLM mode: re-run the prompt chain over the growing transcript on every + # finalized turn, refreshing one evolving answer (partials are ignored). + if not getattr(event, "end_of_turn", False): + return + text = getattr(event, "transcript", "") or "" + if not text: + return + transcript.append(text) + answer = llm.run_chain( + api_key, + list(llm_prompt), + transcript_text=" ".join(transcript), + model=model, + max_tokens=max_tokens, + ) + follow(answer, len(transcript)) def run(audio: FileSource | MicrophoneSource | StdinSource, rate: int) -> None: merged = config_builder.merge_streaming_params( @@ -183,34 +213,35 @@ def run(audio: FileSource | MicrophoneSource | StdinSource, rate: int) -> None: ) params = config_builder.construct_streaming_params(merged) - try: - client.stream_audio( - api_key, - audio, - params=params, - on_begin=renderer.begin, - on_turn=on_turn, - on_termination=renderer.termination, - ) - except KeyboardInterrupt: - # Ctrl-C is a normal "user stopped" signal -> exit 0 (still transform below). - renderer.close() - renderer.stopped() - except BrokenPipeError: - # Downstream consumer (e.g. `| head`) closed the pipe; stop quietly. - raise typer.Exit(code=0) from None - finally: - renderer.close() - - if llm_gateway_prompt and turns: - transformed = llm.transform_transcript( - api_key, - prompt=llm_gateway_prompt, - model=model, - transcript_text=" ".join(turns), - max_tokens=max_tokens, - ) - renderer.llm(transformed) + def drive() -> None: + try: + client.stream_audio( + api_key, + audio, + params=params, + on_begin=None if follow is not None else renderer.begin, + on_turn=on_turn, + on_termination=None if follow is not None else renderer.termination, + ) + except KeyboardInterrupt: + # Ctrl-C is a normal "user stopped" signal -> exit 0. + if follow is None: + renderer.close() + renderer.stopped() + except BrokenPipeError: + # Downstream consumer (e.g. `| head`) closed the pipe; stop quietly. + raise typer.Exit(code=0) from None + finally: + if follow is None: + renderer.close() + + # The FollowRenderer is a context manager (it owns the live panel); enter it + # around the whole session so it stops cleanly and prints the final answer. + if follow is not None: + with follow: + drive() + else: + drive() if from_stdin: # Raw PCM16 mono piped on stdin (e.g. `ffmpeg … -f s16le - | aai stream -`). @@ -230,7 +261,10 @@ def run(audio: FileSource | MicrophoneSource | StdinSource, rate: int) -> None: # Announce "Listening…" only once the device is open and recording, # not when the session opens — so early speech isn't lost in the gap. mic = MicrophoneSource( - device=device, capture_rate=sample_rate, on_open=renderer.listening + device=device, + capture_rate=sample_rate, + # In --llm mode the FollowRenderer owns the screen, so skip the notice. + on_open=(lambda: None) if follow is not None else renderer.listening, ) run(mic, mic.sample_rate) diff --git a/assemblyai_cli/commands/transcribe.py b/assemblyai_cli/commands/transcribe.py index 1a56817f..a6979d80 100644 --- a/assemblyai_cli/commands/transcribe.py +++ b/assemblyai_cli/commands/transcribe.py @@ -109,10 +109,10 @@ def transcribe( None, "--config", help="Set any TranscriptionConfig field as KEY=VALUE (repeatable)." ), config_file: str = typer.Option(None, "--config-file", help="JSON file of config fields."), - # llm gateway transform (existing) - llm_gateway_prompt: list[str] = typer.Option( + # llm gateway transform + llm_prompt: list[str] = typer.Option( None, - "--llm-gateway-prompt", + "--llm", help="Transform the finished transcript through LLM Gateway. Repeatable: each " "prompt runs on the previous one's response (a chain), the first on the transcript.", ), @@ -197,8 +197,8 @@ def body(state: AppState, json_mode: bool) -> None: # yields a runnable file. audio = client.resolve_audio_source(source, sample=sample) gateway = ( - {"prompts": list(llm_gateway_prompt), "model": model, "max_tokens": max_tokens} - if llm_gateway_prompt + {"prompts": list(llm_prompt), "model": model, "max_tokens": max_tokens} + if llm_prompt else None ) output.print_code(code_gen.transcribe(merged, audio, llm_gateway=gateway)) @@ -232,12 +232,12 @@ def body(state: AppState, json_mode: bool) -> None: print(client.select_transcript_field(transcript, output_field)) return - if llm_gateway_prompt: + if llm_prompt: # Chain the prompts: the first runs over the transcript (injected server-side # via transcript_id); each subsequent prompt runs over the prior response. steps: list[dict[str, str]] = [] previous: str | None = None - for i, prompt_text in enumerate(llm_gateway_prompt): + for i, prompt_text in enumerate(llm_prompt): # First prompt runs over the transcript (by id); each later one over # the prior response. target = ( diff --git a/assemblyai_cli/follow.py b/assemblyai_cli/follow.py new file mode 100644 index 00000000..5eac2586 --- /dev/null +++ b/assemblyai_cli/follow.py @@ -0,0 +1,51 @@ +from __future__ import annotations + +from rich.live import Live +from rich.markup import escape +from rich.panel import Panel + +from assemblyai_cli import output + + +class FollowRenderer: + """Render a live transcript transform that refreshes on every turn. + + On a terminal, the latest answer is redrawn in place inside a Rich panel so a + human watches one evolving summary. When piped or run by an agent (json_mode), + each refresh is emitted as one NDJSON object so it stays machine-readable. + + Shared by `aai llm --follow` (turns piped on stdin) and `aai stream --llm` + (turns produced live by the streaming session). + """ + + def __init__(self, *, json_mode: bool) -> None: + self.json_mode = json_mode + self._live: Live | None = None + self._last: Panel | None = None + + def __enter__(self) -> FollowRenderer: + if not self.json_mode: + # screen=True draws into the terminal's alternate buffer (like less/htop). + # In the `aai stream -o text | aai llm -f` pipeline two processes share one + # TTY: stream writes status to stderr and the Ctrl-C "^C" echoes into our + # region, desyncing Rich's relative-cursor teardown and duplicating the top + # border. The alt buffer is isolated and restored verbatim on exit, so that + # noise is discarded; we reprint the final panel to the normal screen below. + self._live = Live(console=output.console, auto_refresh=False, screen=True) + self._live.start() + return self + + def __call__(self, answer: str, turns: int) -> None: + if self.json_mode: + output.emit_ndjson({"turns": turns, "output": answer}) + elif self._live is not None: + title = f"scribe · {turns} turn{'s' if turns != 1 else ''}" + self._last = Panel(escape(answer or "…"), title=title, border_style="aai.brand") + self._live.update(self._last, refresh=True) + + def __exit__(self, *exc: object) -> None: + if self._live is not None: + self._live.stop() # leaves the alt buffer, restoring the normal screen + self._live = None + if self._last is not None: + output.console.print(self._last) # leave the final summary as scrollback diff --git a/assemblyai_cli/llm.py b/assemblyai_cli/llm.py index bf290723..adadede7 100644 --- a/assemblyai_cli/llm.py +++ b/assemblyai_cli/llm.py @@ -10,7 +10,7 @@ # The LLM Gateway is OpenAI-compatible, so we talk to it through the OpenAI SDK # pointed at this base URL. (The synchronous gateway has no assemblyai-SDK client.) GATEWAY_BASE_URL = "https://llm-gateway.assemblyai.com/v1" -DEFAULT_MODEL = "claude-sonnet-4-6" +DEFAULT_MODEL = "claude-haiku-4-5-20251001" DEFAULT_MAX_TOKENS = 1000 # Exact tag the gateway substitutes with a transcript's text when `transcript_id` @@ -131,3 +131,27 @@ def transform_transcript( transcript_id=transcript_id, ) return content_of(response) + + +def run_chain( + api_key: str, + prompts: list[str], + *, + transcript_text: str, + model: str = DEFAULT_MODEL, + max_tokens: int = DEFAULT_MAX_TOKENS, +) -> str: + """Run a chain of prompts over inline transcript text and return the final output. + + The first prompt runs over `transcript_text`; each subsequent prompt runs over the + previous prompt's response. Used by live streaming (`stream --llm`), where there is + no transcript id to inject server-side, so the text is always inlined. + """ + output = "" + text = transcript_text + for prompt in prompts: + output = transform_transcript( + api_key, prompt=prompt, model=model, max_tokens=max_tokens, transcript_text=text + ) + text = output + return output diff --git a/assemblyai_cli/streaming/render.py b/assemblyai_cli/streaming/render.py index 27ea4f37..590129da 100644 --- a/assemblyai_cli/streaming/render.py +++ b/assemblyai_cli/streaming/render.py @@ -52,17 +52,6 @@ def termination(self, event: object) -> None: } ) - def llm(self, content: str) -> None: - """Render the LLM Gateway transform of the full transcript (shown last).""" - if not content: - return - if self.json_mode: - self._emit({"type": "llm", "content": content}) - elif self.text_mode: - self._write(content + "\n") - else: - self._line(Text("\N{ELECTRIC LIGHT BULB} " + content, style="aai.brand")) - def stopped(self) -> None: if self.text_mode: self._status("Stopped.") diff --git a/tests/e2e/test_cli_e2e.py b/tests/e2e/test_cli_e2e.py index 7838028a..09bf6488 100644 --- a/tests/e2e/test_cli_e2e.py +++ b/tests/e2e/test_cli_e2e.py @@ -136,7 +136,7 @@ def test_transcribe_prompt_transforms_via_gateway(real_api_key): [ "transcribe", "--sample", - "--llm-gateway-prompt", + "--llm", "Summarize this transcript in one short sentence.", "--json", ], @@ -163,7 +163,7 @@ def test_e2e_transcribe_analysis(real_api_key): assert payload.get("summary") or payload.get("chapters"), f"no analysis fields: {payload!r}" -def test_stream_prompt_transforms_at_end(real_api_key, kokoro_pipeline, tmp_path): +def test_stream_prompt_transforms_live(real_api_key, kokoro_pipeline, tmp_path): spoken = "the quick brown fox jumps over the lazy dog" wav = _synthesize_wav(kokoro_pipeline, spoken, tmp_path / "fox.wav") @@ -171,7 +171,7 @@ def test_stream_prompt_transforms_at_end(real_api_key, kokoro_pipeline, tmp_path [ "stream", str(wav), - "--llm-gateway-prompt", + "--llm", "Summarize the transcript in one short sentence.", "--json", ], @@ -179,6 +179,7 @@ def test_stream_prompt_transforms_at_end(real_api_key, kokoro_pipeline, tmp_path ) assert proc.returncode == 0, f"stderr:\n{proc.stderr}" events = _ndjson(proc.stdout) - # The full transcript is transformed once after streaming, emitted as a final llm event. - llm_events = [e for e in events if e.get("type") == "llm" and e.get("content")] - assert llm_events, f"no transcript transform came back; events={events}" + # Live mode re-runs the prompt over the growing transcript, emitting one refresh + # ({"turns": N, "output": ...}) per finalized turn. + refreshes = [e for e in events if e.get("output")] + assert refreshes, f"no live transform refresh came back; events={events}" diff --git a/tests/test_code_gen.py b/tests/test_code_gen.py index 59d691fe..36d28ea2 100644 --- a/tests/test_code_gen.py +++ b/tests/test_code_gen.py @@ -352,3 +352,30 @@ def test_agent_show_code_uses_single_full_duplex_stream(): assert "RawOutputStream" not in code # No audioop: it's deprecated and removed in Python 3.13, so the script stays portable. assert "audioop" not in code + + +def test_stream_show_code_includes_llm_follow_loop(): + code = code_gen.stream( + {"speech_model": "universal_streaming"}, + llm={ + "prompts": ["summarize", "translate to french"], + "model": "claude-haiku-4-5-20251001", + "max_tokens": 500, + }, + ) + ast.parse(code) + assert "from openai import OpenAI" in code + assert "llm-gateway.assemblyai.com" in code + # Both prompts appear, in order, for the chain. + assert code.index("summarize") < code.index("translate to french") + # Still streams from the mic, refreshing the answer on each finalized turn. + assert "MicrophoneStream" in code + assert "end_of_turn" in code + assert "claude-haiku-4-5-20251001" in code + + +def test_stream_show_code_without_llm_is_plain_scaffold(): + code = code_gen.stream({}) + ast.parse(code) + assert "from openai import OpenAI" not in code # no gateway when --llm absent + assert "MicrophoneStream" in code diff --git a/tests/test_llm.py b/tests/test_llm.py index c2ebe259..b16ec40d 100644 --- a/tests/test_llm.py +++ b/tests/test_llm.py @@ -133,3 +133,42 @@ def fake_complete(api_key, *, model, messages, max_tokens, transcript_id=None): assert out == "SUMMARY" assert seen["transcript_id"] == "t_9" assert llm.TRANSCRIPT_TAG in seen["messages"][0]["content"] + + +def test_run_chain_single_prompt_runs_over_transcript(monkeypatch): + seen = {} + + def fake_complete(api_key, *, model, messages, max_tokens, transcript_id=None): + seen["messages"] = messages + seen["transcript_id"] = transcript_id + return _response("SUMMARY") + + monkeypatch.setattr(llm, "complete", fake_complete) + out = llm.run_chain("sk", ["summarize"], transcript_text="hola mundo", model="m", max_tokens=50) + assert out == "SUMMARY" + # No transcript_id in live mode -> the text is inlined into the prompt. + assert seen["transcript_id"] is None + content = seen["messages"][-1]["content"] + assert "summarize" in content and "hola mundo" in content + + +def test_run_chain_threads_output_through_prompts(monkeypatch): + calls = [] + + def fake_complete(api_key, *, model, messages, max_tokens, transcript_id=None): + calls.append(messages[-1]["content"]) + return _response(f"out{len(calls)}") + + monkeypatch.setattr(llm, "complete", fake_complete) + out = llm.run_chain( + "sk", + ["summarize", "translate to french"], + transcript_text="hola mundo", + model="m", + max_tokens=50, + ) + assert out == "out2" # final step's output + assert len(calls) == 2 + assert "summarize" in calls[0] and "hola mundo" in calls[0] + # Second prompt runs over the FIRST step's output, not the transcript. + assert "translate to french" in calls[1] and "out1" in calls[1] diff --git a/tests/test_llm_command.py b/tests/test_llm_command.py index 8b34541d..820dfa61 100644 --- a/tests/test_llm_command.py +++ b/tests/test_llm_command.py @@ -53,7 +53,7 @@ def fake_complete(api_key, *, model, messages, max_tokens, transcript_id=None): assert result.exit_code == 0 data = json.loads(result.output) assert data["output"] == "4" - assert data["model"] == "claude-sonnet-4-6" + assert data["model"] == "claude-haiku-4-5-20251001" assert seen["transcript_id"] is None assert seen["messages"][0]["content"] == "What is 2+2?" diff --git a/tests/test_stream_command.py b/tests/test_stream_command.py index 840f6562..17915b71 100644 --- a/tests/test_stream_command.py +++ b/tests/test_stream_command.py @@ -238,9 +238,9 @@ def fake( assert {"type": "turn", "transcript": "from file", "end_of_turn": True} in lines -def test_stream_prompt_transforms_accumulated_transcript(monkeypatch): +def test_stream_llm_refreshes_live_over_growing_transcript(monkeypatch): config.set_api_key("default", "sk_live") - seen = {} + seen = {"texts": []} def fake(api_key, source, *, params, on_turn=None, **kwargs): if on_turn: @@ -248,20 +248,20 @@ def fake(api_key, source, *, params, on_turn=None, **kwargs): on_turn(types.SimpleNamespace(transcript="mundo", end_of_turn=True)) on_turn(types.SimpleNamespace(transcript="partial", end_of_turn=False)) # ignored - def fake_transform(api_key, *, prompt, model, transcript_text, max_tokens): - seen["prompt"] = prompt + def fake_run_chain(api_key, prompts, *, transcript_text, model, max_tokens): + seen["texts"].append(transcript_text) + seen["prompts"] = prompts seen["model"] = model - seen["transcript_text"] = transcript_text seen["max_tokens"] = max_tokens - return "hello world" + return f"answer:{transcript_text}" monkeypatch.setattr("assemblyai_cli.commands.stream.client.stream_audio", fake) - monkeypatch.setattr("assemblyai_cli.commands.stream.llm.transform_transcript", fake_transform) + monkeypatch.setattr("assemblyai_cli.commands.stream.llm.run_chain", fake_run_chain) result = runner.invoke( app, [ "stream", - "--llm-gateway-prompt", + "--llm", "translate to english", "--model", "gpt-4.1", @@ -271,12 +271,47 @@ def fake_transform(api_key, *, prompt, model, transcript_text, max_tokens): ], ) assert result.exit_code == 0 - # The full transcript (finalized turns only) is sent for one transform. - assert seen["transcript_text"] == "hola mundo" + # One refresh per finalized turn, over the growing transcript (partials ignored). + assert seen["texts"] == ["hola", "hola mundo"] + assert seen["prompts"] == ["translate to english"] assert seen["model"] == "gpt-4.1" assert seen["max_tokens"] == 50 lines = [json.loads(x) for x in result.output.splitlines() if x.strip()] - assert {"type": "llm", "content": "hello world"} in lines + assert {"turns": 1, "output": "answer:hola"} in lines + assert {"turns": 2, "output": "answer:hola mundo"} in lines + # Live mode replaces the raw turn envelopes; only follow refreshes reach stdout. + assert '"type"' not in result.output + + +def test_stream_llm_chains_multiple_prompts(monkeypatch): + config.set_api_key("default", "sk_live") + seen = {} + + def fake(api_key, source, *, params, on_turn=None, **kwargs): + if on_turn: + on_turn(types.SimpleNamespace(transcript="hi", end_of_turn=True)) + + def fake_run_chain(api_key, prompts, *, transcript_text, model, max_tokens): + seen["prompts"] = prompts + return "done" + + monkeypatch.setattr("assemblyai_cli.commands.stream.client.stream_audio", fake) + monkeypatch.setattr("assemblyai_cli.commands.stream.llm.run_chain", fake_run_chain) + result = runner.invoke( + app, ["stream", "--llm", "summarize", "--llm", "translate to french", "--json"] + ) + assert result.exit_code == 0 + assert seen["prompts"] == ["summarize", "translate to french"] + + +def test_stream_llm_rejects_output_text(monkeypatch): + config.set_api_key("default", "sk_live") + monkeypatch.setattr( + "assemblyai_cli.commands.stream.client.stream_audio", + lambda *a, **k: (_ for _ in ()).throw(AssertionError("must not stream")), + ) + result = runner.invoke(app, ["stream", "--llm", "summarize", "-o", "text"]) + assert result.exit_code == 2 # --llm renders a panel/NDJSON; -o text is contradictory def test_stream_without_prompt_does_not_transform(monkeypatch): @@ -287,15 +322,15 @@ def fake(api_key, source, *, params, on_turn=None, **kwargs): if on_turn: on_turn(types.SimpleNamespace(transcript="hi", end_of_turn=True)) - def fake_transform(*a, **k): + def fake_run_chain(*a, **k): called["ran"] = True return "x" monkeypatch.setattr("assemblyai_cli.commands.stream.client.stream_audio", fake) - monkeypatch.setattr("assemblyai_cli.commands.stream.llm.transform_transcript", fake_transform) + monkeypatch.setattr("assemblyai_cli.commands.stream.llm.run_chain", fake_run_chain) result = runner.invoke(app, ["stream", "--json"]) assert result.exit_code == 0 - assert called["ran"] is False # no --llm-gateway-prompt -> no gateway call + assert called["ran"] is False # no --llm -> no gateway call def test_stream_prompt_biases_speech_model(monkeypatch): @@ -479,3 +514,15 @@ def fake_stream_audio(api_key, source, *, params, on_begin=None, on_turn=None, * # Final turn only, plain text; partials and JSON envelopes are not on stdout. assert result.output.strip() == "hello world" assert '"type"' not in result.output + + +def test_stream_show_code_with_llm_emits_follow_loop(monkeypatch): + def _boom(*a, **k): + raise AssertionError("must not stream") + + monkeypatch.setattr("assemblyai_cli.commands.stream.client.stream_audio", _boom) + result = runner.invoke(app, ["stream", "--llm", "summarize", "--show-code"]) + assert result.exit_code == 0 + assert "from openai import OpenAI" in result.output + assert "summarize" in result.output + assert "run_chain" in result.output # the live transcribe->LLM-per-turn loop diff --git a/tests/test_streaming_render.py b/tests/test_streaming_render.py index 720cde98..fca7be05 100644 --- a/tests/test_streaming_render.py +++ b/tests/test_streaming_render.py @@ -62,13 +62,6 @@ def test_human_long_partial_clears_wrapped_rows(): assert "\x1b[1A" in buf.getvalue() # moved up over the wrapped rows to clear them -def test_human_llm_line_rendered(): - r, buf = _human() - r.turn(_turn("hola", True)) - r.llm("the summary") - assert "the summary" in buf.getvalue() - - def test_human_stopped_announced(): r, buf = _human() r.stopped() @@ -99,20 +92,6 @@ def test_termination_json_emits_duration(): assert json.loads(out.getvalue()) == {"type": "termination", "audio_duration_seconds": 12.5} -def test_llm_json_emits_event(): - out = io.StringIO() - r = StreamRenderer(json_mode=True, out=out) - r.llm("the summary") - assert json.loads(out.getvalue()) == {"type": "llm", "content": "the summary"} - - -def test_llm_ignores_empty_content(): - out = io.StringIO() - r = StreamRenderer(json_mode=True, out=out) - r.llm("") - assert out.getvalue() == "" - - def test_close_is_noop_in_json_mode(): out = io.StringIO() r = StreamRenderer(json_mode=True, out=out) @@ -159,12 +138,3 @@ def test_listening_is_silent_in_json_mode(): r = StreamRenderer(json_mode=True, out=out) r.listening() assert out.getvalue() == "" # the "Listening…" line is human-only - - -def test_human_llm_line_is_branded(): - r, buf = _human(color_system="truecolor") - r.turn(_turn("hola", True)) - r.llm("the summary") - out = buf.getvalue() - assert "the summary" in out - assert "\x1b[" in out # brand styling emits ANSI diff --git a/tests/test_transcribe.py b/tests/test_transcribe.py index 096afac7..2a76b9c5 100644 --- a/tests/test_transcribe.py +++ b/tests/test_transcribe.py @@ -171,9 +171,7 @@ def fake_transform(api_key, *, prompt, model, transcript_id, max_tokens): monkeypatch.setattr( "assemblyai_cli.commands.transcribe.llm.transform_transcript", fake_transform ) - result = runner.invoke( - app, ["transcribe", "audio.mp3", "--llm-gateway-prompt", "summarize", "--json"] - ) + result = runner.invoke(app, ["transcribe", "audio.mp3", "--llm", "summarize", "--json"]) assert result.exit_code == 0 data = json.loads(result.output) assert data["text"] == "hello world" # raw transcript still present in JSON @@ -181,7 +179,7 @@ def fake_transform(api_key, *, prompt, model, transcript_id, max_tokens): assert steps == [{"prompt": "summarize", "output": "a short summary"}] # The transform is injected server-side via the transcript id. assert seen["transcript_id"] == "t_1" - assert seen["model"] == "claude-sonnet-4-6" + assert seen["model"] == "claude-haiku-4-5-20251001" def test_transcribe_chains_multiple_gateway_prompts(monkeypatch): @@ -208,9 +206,9 @@ def fake_transform( "transcribe", "audio.mp3", "--json", - "--llm-gateway-prompt", + "--llm", "summarize", - "--llm-gateway-prompt", + "--llm", "translate", ], ) @@ -235,9 +233,7 @@ def test_transcribe_prompt_human_shows_only_transform(monkeypatch): "assemblyai_cli.commands.transcribe.llm.transform_transcript", lambda *a, **k: "TRANSFORMED", ) - result = runner.invoke( - app, ["transcribe", "audio.mp3", "--llm-gateway-prompt", "summarize"] - ) + result = runner.invoke(app, ["transcribe", "audio.mp3", "--llm", "summarize"]) assert result.exit_code == 0 assert "TRANSFORMED" in result.output assert "hello world" not in result.output # human mode shows the transform only @@ -387,7 +383,7 @@ def _boom(*a, **k): def test_transcribe_show_code_includes_llm_gateway_without_running(monkeypatch): - # --llm-gateway-prompt must be reflected in the generated code, still without + # --llm must be reflected in the generated code, still without # transcribing or calling the gateway. def _boom(*a, **k): raise AssertionError("must not call the API") @@ -396,7 +392,7 @@ def _boom(*a, **k): monkeypatch.setattr("assemblyai_cli.commands.transcribe.llm.transform_transcript", _boom) result = runner.invoke( app, - ["transcribe", "--sample", "--llm-gateway-prompt", "translate to spanish", "--show-code"], + ["transcribe", "--sample", "--llm", "translate to spanish", "--show-code"], ) assert result.exit_code == 0 assert "llm-gateway.assemblyai.com" in result.output