feat(cli): unify transcribe + stream LLM Gateway under --llm

README.md

@@ -28,13 +28,55 @@ aai login                 # store your API key (browser-assisted)
 aai transcribe --sample   # transcribe the hosted wildfires.mp3 sample
 ```
 
+## API key & security
+
+`aai` resolves your key in this order:
+
+1. The `ASSEMBLYAI_API_KEY` environment variable.
+2. The OS keyring (macOS Keychain, Windows Credential Manager, Linux Secret
+   Service), written only when you run `aai login`.
+
+Two things worth knowing: the key is **never stored in a plaintext dotfile** —
+`aai login` puts it in the OS keyring, and the only on-disk config (`config.toml`)
+holds just profile names. And there is **no `--api-key` flag on run commands**
+(`transcribe`, `stream`, …), so a key can't leak into `ps` output or shell history
+via a command's arguments.
+
+**Prefer not to persist the key at all?** Skip `aai login` and set the environment
+variable instead — it's checked *before* the keyring, so nothing is ever written to
+disk:
+
+```sh
+ASSEMBLYAI_API_KEY=sk_... aai transcribe call.mp3
+```
+
+Prefixing it on a single command (rather than `export`-ing it) scopes the secret to
+that one process. To also keep it out of your shell history, inject it from a secret
+manager at call time:
+
+```sh
+# 1Password CLI
+ASSEMBLYAI_API_KEY=$(op read "op://Private/AssemblyAI/api key") aai transcribe call.mp3
+op run -- aai transcribe call.mp3                          # …or wrap the whole command
+
+# HashiCorp Vault
+ASSEMBLYAI_API_KEY=$(vault kv get -field=key secret/assemblyai) aai stream
+
+# macOS Keychain (a generic-password item you manage)
+ASSEMBLYAI_API_KEY=$(security find-generic-password -w -s assemblyai -a "$USER") aai transcribe call.mp3
+```
+
+In CI, set `ASSEMBLYAI_API_KEY` as a masked secret — nothing is stored. The env var
+also overrides a stored key for one-off use; `aai logout` purges the keyring entry,
+and `aai whoami` / `aai doctor` confirm which source is active without printing the key.
+
 ## Commands
 
 | Command | What it does |
 | --- | --- |
 | `aai login` / `logout` / `whoami` | Manage the stored API key. |
 | `aai doctor` | Check your environment is ready (API key, network, ffmpeg, microphone, agent tooling). |
-| `aai transcribe <file\|url>` | Transcribe an audio file, URL, or YouTube URL (`--sample` for a demo, `--llm-gateway-prompt` to transform the result, `--show-code` to print the equivalent Python). |
+| `aai transcribe <file\|url>` | Transcribe an audio file, URL, or YouTube URL (`--sample` for a demo, `--llm` to transform the result through LLM Gateway, `--show-code` to print the equivalent Python). |
 | `aai transcripts list` / `get <id>` | Browse and fetch past transcripts. |
 | `aai stream [file]` | Real-time transcription from a file or the microphone. |
 | `aai agent` | Live two-way voice conversation with a voice agent. |
@@ -128,19 +170,33 @@ aai stream --sample \
 
 ## Live transcript → live LLM
 
-`aai stream -o text` writes one finalized turn per line and flushes immediately, so it
-can drive `aai llm` turn by turn. Add `--follow` (`-f`) to `aai llm` to keep re-running
-your prompt over the *growing* transcript, refreshing the answer in place on every turn:
+`aai stream --llm "PROMPT"` runs a prompt over the live transcript through LLM Gateway,
+refreshing the answer on every finalized turn — one command, no pipe to wire up:
+
+```sh
+aai stream --llm "summarize action items as I talk"
+```
+
+It's repeatable, so prompts chain — each runs on the previous one's response:
+
+```sh
+aai stream --llm "extract action items" --llm "rewrite them as a checklist"
+```
+
+On a terminal you watch one evolving panel; piped onward it emits one JSON object per
+refresh (`{"turns": N, "output": "…"}`). Ctrl-C to stop.
+
+**Prefer the pipe?** The same thing composes from the primitives: `aai stream -o text`
+writes one finalized turn per line, and `aai llm -f` (`--follow`) re-runs your prompt
+over the *growing* transcript. Reach for this when you want a `--system` prompt or other
+tools in the pipeline:
 
 ```sh
 aai stream -o text | aai llm -f --system "You are a meeting scribe" "summarize action items as I talk"
 ```
 
-On a terminal you watch one evolving summary; piped onward it emits one JSON object per
-refresh (`{"turns": N, "output": "…"}`). Each finalized turn triggers a fresh call over
-the full transcript, so the answer is always current. Ctrl-C to stop. Without `--follow`,
-`aai llm` stays one-shot — it reads stdin to EOF and answers once (`cat notes | aai llm
-"summarize"`).
+Without `--follow`, `aai llm` stays one-shot — it reads stdin to EOF and answers once
+(`cat notes | aai llm "summarize"`).
 
 ## Voice agent
 
@@ -173,16 +229,18 @@ aai agent --voice ivy --show-code                           # the full-duplex ag
 ```
 
 The generated transcribe code includes result handling for the analysis features you
-enabled. With `--llm-gateway-prompt` (repeatable — each prompt runs on the previous
-response), it emits the chained LLM Gateway calls too:
+enabled. With `--llm` (repeatable — each prompt runs on the previous response), it emits
+the chained LLM Gateway calls too:
 
 ```sh
 aai transcribe call.mp3 \
-  --llm-gateway-prompt "summarize" \
-  --llm-gateway-prompt "translate the summary to Spanish" \
+  --llm "summarize" \
+  --llm "translate the summary to Spanish" \
   --show-code > summarize_then_translate.py
 ```
 
+`aai stream --llm "…" --show-code` likewise emits the live transcribe→LLM-per-turn loop.
+
 ## Pipelines
 
 `aai` is built to compose with the rest of your shell. Output is machine-clean
@@ -207,23 +265,24 @@ curl -sL https://example.com/ep.mp3 | aai transcribe -  # no temp file
 ffmpeg -i in.mp4 -f s16le -ac 1 -ar 16000 - | aai stream -   # live, from a pipe
 ```
 
-**Feed transcripts into the LLM Gateway** (`aai llm` reads piped stdin):
+**Feed text into the LLM Gateway** (`aai llm` reads piped stdin). For a transcript,
+`aai transcribe --llm "…"` does it in one step — the pipe is for any *other* text:
 
 ```sh
-aai transcribe call.mp3 -o text | aai llm "summarize, then list action items"
 cat notes.txt | aai llm "turn these into a changelog"
 ```
 
-**Stream, then summarize.** Piped `stream`/`agent` emit clean transcript lines with
-`-o text`. A Ctrl-C in a pipe hits both sides, so to stop the producer *and* let the
+**Pipe a live stream into other tools.** For live LLM summaries use `aai stream --llm`
+(above) — one process, clean Ctrl-C. To pipe the live transcript into a *different* tool,
+note that a Ctrl-C in a pipe hits both sides, so to stop the producer and let the
 consumer finish, signal only the producer — or end the stream on its own:
 
 ```sh
 # end after 30s by signaling just the producer (macOS: brew install coreutils, use gtimeout)
-timeout -s INT 30s aai stream -o text | aai llm "summarize"
+timeout -s INT 30s aai stream -o text | grep -i "action item"
 
 # or end on a natural pause (server-side inactivity timeout, in seconds)
-aai stream -o text --inactivity-timeout 5 | aai llm "summarize the call"
+aai stream -o text --inactivity-timeout 5 > call.txt
 
 # capture then process (most robust)
 aai stream -o text > call.txt        # Ctrl-C to stop
@@ -235,13 +294,6 @@ aai llm "summarize" < call.txt
 A cookbook of `aai` composed with common Unix tools. macOS shown; on Linux swap
 `pbcopy`/`pbpaste` → `xclip -sel clip`/`xclip -o` and `say` → `spd-say`.
 
-**Live meeting scribe** — `-o text` streams one finalized turn per line; `aai llm -f`
-re-summarizes the growing transcript in place on every turn (Ctrl-C to stop):
-
-```sh
-aai stream -o text | aai llm -f --model claude-haiku-4-5-20251001 "summarize todos as I talk"
-```
-
 **Chain `aai llm` into other tools** with `-o text` — it prints just the answer, so it
 pipes onward cleanly (no `jq` needed):
 
@@ -260,7 +312,7 @@ cat error.log         | aai llm "what's the root cause and the one-line fix?"
 for the pipeline you described, and `aai llm` rewrites it in another language:
 
 ```sh
-aai transcribe --sample --llm-gateway-prompt "translate to french" --show-code | aai llm "rewrite in rust"
+aai transcribe --sample --llm "translate to french" --show-code | aai llm "rewrite in rust"
 ```
 
 **Mine the analysis JSON with `jq`** — enable a feature, then slice `-o json`:

assemblyai_cli/code_gen/__init__.py

@@ -20,6 +20,15 @@ def transcribe(
     return _transcribe.render(merged, source, llm_gateway=llm_gateway)
 
 
-def stream(merged: dict[str, object]) -> str:
-    """Generate runnable Python that reproduces this streaming invocation."""
-    return _stream.render(merged)
+def stream(
+    merged: dict[str, object],
+    *,
+    llm: dict[str, object] | None = None,
+) -> str:
+    """Generate runnable Python that reproduces this streaming invocation.
+
+    With `llm` (a dict of ``prompts``/``model``/``max_tokens``), the script refreshes a
+    prompt-chain over the growing transcript on every finalized turn — the live sibling
+    of `transcribe --llm` — mirroring how `stream --llm` runs.
+    """
+    return _stream.render(merged, llm=llm)

assemblyai_cli/code_gen/stream.py

@@ -1,5 +1,8 @@
 from __future__ import annotations
 
+from typing import cast
+
+from assemblyai_cli import llm as gateway
 from assemblyai_cli.code_gen import serialize
 
 # Streaming-class imports always used by the generated scaffold. SpeechModel is added
@@ -30,6 +33,54 @@ def on_turn(client: StreamingClient, event: TurnEvent) -> None:
         print()
 
 
+client = StreamingClient(
+    StreamingClientOptions(api_key=API_KEY, api_host="streaming.assemblyai.com")
+)
+client.on(StreamingEvents.Turn, on_turn)
+"""
+
+_LLM_PREAMBLE = """import os
+
+import assemblyai as aai
+from assemblyai.streaming.v3 import (
+{imports}
+)
+from openai import OpenAI
+
+# Export your key first:  export ASSEMBLYAI_API_KEY="<your key>"
+API_KEY = os.environ["ASSEMBLYAI_API_KEY"]
+aai.settings.api_key = API_KEY
+
+# Run the prompts over the LLM Gateway (OpenAI-compatible). Each prompt runs on the
+# previous response; the first runs on the transcript accumulated so far.
+gateway = OpenAI(api_key=API_KEY, base_url={base_url!r})
+PROMPTS = [
+{prompts}
+]
+transcript: list[str] = []
+
+
+def run_chain(text: str) -> str:
+    result = text
+    for i, prompt in enumerate(PROMPTS):
+        source = text if i == 0 else result
+        response = gateway.chat.completions.create(
+            model={model!r},
+            messages=[{{"role": "user", "content": prompt + "\\n\\nTranscript:\\n" + source}}],
+            max_tokens={max_tokens},
+        )
+        result = response.choices[0].message.content
+    return result
+
+
+def on_turn(client: StreamingClient, event: TurnEvent) -> None:
+    # Refresh the answer on every finalized turn, over the growing transcript.
+    if not event.end_of_turn or not event.transcript:
+        return
+    transcript.append(event.transcript)
+    print(run_chain(" ".join(transcript)))
+
+
 client = StreamingClient(
     StreamingClientOptions(api_key=API_KEY, api_host="streaming.assemblyai.com")
 )
@@ -45,13 +96,29 @@ def on_turn(client: StreamingClient, event: TurnEvent) -> None:
 """
 
 
-def render(merged: dict[str, object]) -> str:
-    """Generate a runnable microphone-streaming script with the given params."""
+def render(merged: dict[str, object], *, llm: dict[str, object] | None = None) -> str:
+    """Generate a runnable microphone-streaming script with the given params.
+
+    With `llm`, the script transforms the live transcript through the LLM Gateway,
+    refreshing a prompt chain on every finalized turn (the live sibling of
+    `transcribe --llm`).
+    """
     names = list(_BASE_IMPORTS)
     if "speech_model" in merged:
         names.append("SpeechModel")
     imports = "\n".join(f"    {name}," for name in sorted(names))
-    preamble = _PREAMBLE.format(imports=imports)
+
+    if llm:
+        prompts = "\n".join(f"    {p!r}," for p in cast("list[str]", llm["prompts"]))
+        preamble = _LLM_PREAMBLE.format(
+            imports=imports,
+            base_url=gateway.GATEWAY_BASE_URL,
+            prompts=prompts,
+            model=llm["model"],
+            max_tokens=llm["max_tokens"],
+        )
+    else:
+        preamble = _PREAMBLE.format(imports=imports)
 
     # Mic capture rate must match StreamingParameters.sample_rate, else audio is corrupt.
     rate = merged.get("sample_rate", 16000)

assemblyai_cli/commands/llm.py

@@ -1,59 +1,17 @@
 from __future__ import annotations
 
 import typer
-from rich.live import Live
 from rich.markup import escape
-from rich.panel import Panel
 
 from assemblyai_cli import config, output, stdio
 from assemblyai_cli import llm as gateway
 from assemblyai_cli.context import AppState, run_command
 from assemblyai_cli.errors import UsageError
+from assemblyai_cli.follow import FollowRenderer
 
 app = typer.Typer()
 
 
-class _FollowRenderer:
-    """Render a live transcript transform that refreshes on every turn.
-
-    On a terminal, the latest answer is redrawn in place inside a Rich panel so a
-    human watches one evolving summary. When piped or run by an agent (json_mode),
-    each refresh is emitted as one NDJSON object so it stays machine-readable.
-    """
-
-    def __init__(self, *, json_mode: bool) -> None:
-        self.json_mode = json_mode
-        self._live: Live | None = None
-        self._last: Panel | None = None
-
-    def __enter__(self) -> _FollowRenderer:
-        if not self.json_mode:
-            # screen=True draws into the terminal's alternate buffer (like less/htop).
-            # In the `aai stream -o text | aai llm -f` pipeline two processes share one
-            # TTY: stream writes status to stderr and the Ctrl-C "^C" echoes into our
-            # region, desyncing Rich's relative-cursor teardown and duplicating the top
-            # border. The alt buffer is isolated and restored verbatim on exit, so that
-            # noise is discarded; we reprint the final panel to the normal screen below.
-            self._live = Live(console=output.console, auto_refresh=False, screen=True)
-            self._live.start()
-        return self
-
-    def __call__(self, answer: str, turns: int) -> None:
-        if self.json_mode:
-            output.emit_ndjson({"turns": turns, "output": answer})
-        elif self._live is not None:
-            title = f"scribe · {turns} turn{'s' if turns != 1 else ''}"
-            self._last = Panel(escape(answer or "…"), title=title, border_style="aai.brand")
-            self._live.update(self._last, refresh=True)
-
-    def __exit__(self, *exc: object) -> None:
-        if self._live is not None:
-            self._live.stop()  # leaves the alt buffer, restoring the normal screen
-            self._live = None
-            if self._last is not None:
-                output.console.print(self._last)  # leave the final summary as scrollback
-
-
 @app.command()
 def llm(
     ctx: typer.Context,
@@ -125,7 +83,7 @@ def ask(transcript_text: str) -> str:
             )
             return gateway.content_of(response)
 
-        with _FollowRenderer(json_mode=json_mode) as render:
+        with FollowRenderer(json_mode=json_mode) as render:
             transcript: list[str] = []
             try:
                 for turn in stdio.iter_piped_stdin_lines():