Install-path testing: install.sh smoke tests, CI, and dev docs

.claude/agents/security-reviewer.md

@@ -0,0 +1,36 @@
+---
+name: security-reviewer
+description: Use to review changes to authentication, credential storage, environment selection, or subprocess shell-outs in the aai CLI. Focuses on secret handling and the auth/login flow. Invoke after editing aai_cli/auth/, config.py, environments.py, or any subprocess call.
+tools: Glob, Grep, LS, Read, NotebookRead, WebFetch, TodoWrite, WebSearch, KillShell, BashOutput
+---
+
+You are a security reviewer for the **AssemblyAI CLI** (`aai`). Your job is to find security regressions in the areas this CLI is most sensitive about, then report concrete findings — not generic advice.
+
+## What this CLI guarantees (don't let a change break these)
+
+- **The API key is never written to a plaintext dotfile.** It lives only in the OS keyring (`config.KEYRING_SERVICE = "assemblyai-cli"`) or comes from the `ASSEMBLYAI_API_KEY` env var. The only on-disk config (`config.toml`) holds profile names and a per-profile `env`, never the key.
+- **Run commands (`transcribe`, `stream`, `agent`, `llm`, …) expose no `--api-key` flag.** A key must never be acceptable as a command argument — that would leak it into `ps` output and shell history. Only validation/login paths may take a key, and only via stdin/env.
+- **Key resolution order is fixed:** `--api-key` (validation paths only) → `ASSEMBLYAI_API_KEY` env → keyring. A change must not reorder this or add a new on-disk source.
+- **A credential is only valid against the environment that minted it.** `environments.py` binds a profile to its `env`; `context.env_override_warning` must still fire when `--env` contradicts the stored env.
+- **Stytch tokens shipped in `environments.py` are *public* tokens only** (`public-token-*`). Flag any *secret*/private token, client secret, or non-public credential added to source.
+
+## Subprocess / shell-out review
+
+The CLI intentionally shells out to `claude` and `npx` (ruff `S603/S607` are project-ignored). For any `subprocess` change verify:
+
+- Arguments are a fixed list, never a shell string; `shell=True` is never introduced.
+- No untrusted/user-controlled value is interpolated into the argv without validation.
+- `stdin=subprocess.DEVNULL` is preserved where a child might otherwise prompt and hang.
+- A timeout backstop remains.
+
+## Auth flow (`aai_cli/auth/`)
+
+Browser-assisted login uses AMS + **Stytch B2B OAuth discovery** (not Connected Apps). For `discovery.py`/`flow.py`/`loopback.py`/`ams.py` changes, check:
+
+- The loopback redirect binds to localhost only and validates the `state` parameter against CSRF.
+- Tokens/codes are not logged to stdout/stderr or persisted to disk.
+- Error paths surface a clean `CLIError`, never a raw exception leaking a token.
+
+## Output
+
+Report findings ranked by severity. For each: the file:line, what guarantee it breaks, and the concrete fix. If you find nothing, say so plainly — do not invent issues. Only report problems you can point to in the diff or code.

.claude/agents/template-contract-reviewer.md

@@ -0,0 +1,26 @@
+---
+name: template-contract-reviewer
+description: Use after any change under aai_cli/init/templates/ (the `aai init` starter apps) to verify the scaffold still ships correctly and stays covered by the parametrized contract tests. Catches the renamed-dotfile and wheel-packaging gotchas.
+tools: Glob, Grep, LS, Read, NotebookRead, TodoWrite, KillShell, BashOutput
+---
+
+You review changes to the `aai init` starter templates in `aai_cli/init/templates/` (`transcribe/`, `stream/`, `agent/`). These ship inside the wheel and are scaffolded onto users' machines, so a broken template ships broken examples. Verify the following and report concrete gaps.
+
+## Packaging integrity
+
+- **Renamed dotfiles:** templates store `gitignore` (scaffolded → `.gitignore`) and `env.example`. Confirm any new dotfile follows the committed-under-a-safe-name convention and that `aai_cli/init/scaffold.py` / `templates.py` knows how to rename it on copy.
+- **Wheel inclusion:** templates are force-included via `[tool.hatch.build.targets.wheel] artifacts = ["aai_cli/init/templates/**"]`, with `__pycache__`/`*.pyc` excluded. A new file type must be reachable by that glob; a stray compiled artifact must not leak into the wheel.
+- **The `transcribe/` gitignore negation:** the repo root `.gitignore` ignores `transcribe/` but negates `!aai_cli/init/templates/transcribe/`. Confirm a new template path doesn't get silently ignored by a broad root rule.
+
+## Contract-test coverage
+
+Every template is exercised by parametrized tests (`tests/test_init_template_*.py`, `test_init_templates.py`, `test_init_packaging.py`). For each change verify:
+
+- A new template is added to the parametrization, not left untested.
+- The template still satisfies the shared contract the tests assert (required files present: `requirements.txt`, `index.html`, `api/index.py`, `vercel.json`, `README.md`, `env.example`, `gitignore`).
+- The key is written only to the git-ignored `.env`, never embedded in scaffolded source or sent to the browser.
+- If the template's runtime deps changed, `requirements.txt` reflects it and stays installable.
+
+## Output
+
+List each gap as file:line + what contract it breaks + the fix (e.g. "add `agent` to the params in test_init_templates.py:NN"). If the change is fully covered and packages correctly, say so. Don't fabricate issues.

.claude/commands/review-changes.md

@@ -0,0 +1,23 @@
+---
+description: Review the current changes with the aai CLI's specialized reviewers (security + template contract), scoped to what actually changed.
+argument-hint: "[git ref to diff against, default: HEAD]"
+allowed-tools: Bash(git diff *), Bash(git status *), Bash(git log *), Task
+---
+
+Review the current working changes using this project's specialized subagents. Be surgical: only run a reviewer if the diff actually touches its area.
+
+## 1. Scope the diff
+
+Run `git status --short` and `git diff --stat ${1:-HEAD}` (and `git diff ${1:-HEAD}` for detail) to see what changed.
+
+## 2. Dispatch the relevant reviewers (in parallel)
+
+- If the diff touches **`aai_cli/auth/`, `config.py`, `environments.py`, `client.py`, or any `subprocess` call** → dispatch the **`security-reviewer`** agent on those changes.
+- If the diff touches **`aai_cli/init/templates/`, `aai_cli/init/scaffold.py`, `aai_cli/init/templates.py`, or the wheel-packaging config in `pyproject.toml`** → dispatch the **`template-contract-reviewer`** agent on those changes.
+- If the diff touches neither sensitive area, say so and skip — don't manufacture a review.
+
+Pass each agent the exact list of changed files in its scope so it reviews the diff, not the whole repo.
+
+## 3. Synthesize
+
+Combine the findings into one ranked report (severity → file:line → fix). Call out anything that would break the CLI's security guarantees (key never on disk/argv, env↔credential binding) or the template packaging contract. If both reviewers come back clean, state that plainly.

.claude/settings.json

@@ -0,0 +1,69 @@
+{
+  "permissions": {
+    "allow": [
+      "Bash(uv run *)",
+      "Bash(uv sync *)",
+      "Bash(uv lock *)",
+      "Bash(uv build *)",
+      "Bash(uv pip *)",
+      "Bash(uvx twine *)",
+      "Bash(ruff check *)",
+      "Bash(ruff format *)",
+      "Bash(mypy *)",
+      "Bash(pytest *)",
+      "Bash(pre-commit run *)",
+      "Bash(./scripts/check.sh)",
+      "Bash(bash scripts/check.sh)",
+      "Bash(markdownlint *)",
+      "Bash(shellcheck *)",
+      "Bash(actionlint *)",
+      "Bash(command -v *)",
+      "Bash(git status *)",
+      "Bash(git diff *)",
+      "Bash(git log *)",
+      "Bash(git show *)",
+      "Bash(git branch *)",
+      "Bash(gh pr view *)",
+      "Bash(gh pr list *)",
+      "Bash(gh run view *)",
+      "Bash(gh run list *)",
+      "mcp__assemblyai-docs__search_docs",
+      "mcp__assemblyai-docs__get_pages",
+      "mcp__assemblyai-docs__get_api_reference",
+      "mcp__assemblyai-docs__list_sections"
+    ],
+    "deny": [
+      "Read(.env)",
+      "Read(**/.env)",
+      "Read(**/*.env)",
+      "Read(**/*.pem)",
+      "Read(**/id_rsa)",
+      "Read(**/id_ed25519)",
+      "Read(**/*.p12)"
+    ]
+  },
+  "hooks": {
+    "PreToolUse": [
+      {
+        "matcher": "Edit|Write",
+        "hooks": [
+          {
+            "type": "command",
+            "command": "f=$(jq -r '.tool_input.file_path // empty'); case \"$f\" in *uv.lock|*.env|*/.env) echo \"Blocked: $f is a protected lock/secret file. Edit it by hand if you really intend to.\" >&2; exit 2;; esac; exit 0"
+          }
+        ]
+      }
+    ],
+    "PostToolUse": [
+      {
+        "matcher": "Edit|Write",
+        "hooks": [
+          {
+            "type": "command",
+            "command": "f=$(jq -r '.tool_input.file_path // empty'); case \"$f\" in *.py) uv run ruff check --fix \"$f\" >/dev/null 2>&1; uv run ruff format \"$f\" >/dev/null 2>&1;; esac; exit 0"
+          }
+        ]
+      }
+    ]
+  }
+}

.claude/skills/check/SKILL.md

@@ -0,0 +1,35 @@
+---
+name: check
+description: Run the full local verification gate for the aai CLI (the same checks CI runs). Use before pushing or opening a PR.
+disable-model-invocation: true
+---
+
+# check
+
+Run the project's canonical verification gate and report the result.
+
+## Steps
+
+1. Run the full gate:
+
+   ```sh
+   ./scripts/check.sh
+   ```
+
+   This runs, in order: `ruff check` → `ruff format --check` → `mypy` (src + tests) → `markdownlint` (excludes generated `docs/`) → `shellcheck install.sh` → `pytest` with a **90% branch-coverage gate** (`--cov-fail-under=90`, excluding `e2e` and `install_script` markers) → `uv build` + `twine check --strict`. Everything runs through `uv run` against the locked environment.
+
+2. If anything fails, fix it and re-run `./scripts/check.sh` until it passes. Do not claim success until the script prints `All checks passed.`
+
+## Optional, opt-in suites (not run by check.sh)
+
+Run these only when relevant — they are slow and/or need credentials:
+
+```sh
+uv run pytest -m e2e             # real-API end-to-end; needs ASSEMBLYAI_API_KEY + kokoro
+uv run pytest -m install_script  # builds a wheel and runs install.sh for real; needs network + uv/pipx
+```
+
+## Notes
+
+- If `shellcheck` isn't installed locally, `check.sh` skips it with a notice (CI still runs it) — that's expected, not a failure.
+- Report the final outcome with the actual tail of the output, not a summary from memory.

.claude/skills/release-prep/SKILL.md

@@ -0,0 +1,46 @@
+---
+name: release-prep
+description: Prepare an aai CLI release — bump the version, run the full gate, build and validate the wheel/sdist, and smoke-test the real install. Use when cutting a new release.
+disable-model-invocation: true
+---
+
+# release-prep
+
+Drive an `aai` release to a verified, publishable state. Stop and report at the first failure — never push or publish on a red check.
+
+## 1. Version bump
+
+- Update `version` in `pyproject.toml` (`[project]`). Confirm `aai_cli/__init__.py` `__version__` stays in sync (the `version` command and install smoke test read it).
+- Decide the bump (patch/minor/major) from what changed since the last tag; ask the user if it's ambiguous.
+
+## 2. Full gate
+
+```sh
+./scripts/check.sh
+```
+
+Must end with `All checks passed.` (ruff, mypy, markdownlint, shellcheck, pytest+coverage, build, `twine check --strict`).
+
+## 3. Real install smoke test
+
+```sh
+uv run pytest -q -m install_script
+```
+
+This builds the wheel and runs `install.sh` for real (pipx + pip --user), asserting `aai` runs. Needs network + uv/pipx.
+
+## 4. Build + metadata validation
+
+```sh
+rm -rf dist && uv build && uvx twine check --strict dist/*
+```
+
+Confirm both an sdist and a wheel are produced and the README renders for PyPI.
+
+## Distribution caveat
+
+The PyPI name **`assemblyai-cli` is squatted by a third party** — do **not** assume `pip install assemblyai-cli` resolves to this project. Publishing/distribution currently goes through `install.sh` (git install via pipx / pip --user) and any Homebrew tap, not that PyPI name. Flag this if a release step assumes the squatted name.
+
+## Output
+
+Report the version bumped to, the gate result (with output tail), and confirm `dist/` contains a validated wheel + sdist. Only then is the release ready to tag/push.

.github/workflows/ci.yml

@@ -4,7 +4,6 @@ on:
   pull_request:
     branches: [main]
   push:
-    branches: [main]
 
 # Least privilege: CI only needs to read the repo. Actions are pinned to commit
 # SHAs (a moved tag can't silently change what runs); Dependabot keeps them current.
@@ -98,3 +97,41 @@ jobs:
           python -m pip install -e . pip-audit
           # Append `--ignore-vuln <ID>` to accept an unfixable transitive advisory.
           python -m pip_audit
+
+  install-smoke:
+    name: install.sh real install (${{ matrix.os }})
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          - os: ubuntu-latest
+            kfilter: ""             # both branches: pipx + pip --user
+          - os: macos-latest
+            kfilter: "-k pipx"      # pipx only — PEP 668 makes pip --user flaky on macOS
+    runs-on: ${{ matrix.os }}
+    steps:
+      - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
+      - uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
+        with:
+          python-version: "3.12"
+          cache: pip
+
+      # `aai version` imports the package, which pulls in sounddevice (needs
+      # PortAudio) and ffmpeg-backed sources. Match the other jobs' system deps.
+      - name: System deps (Linux)
+        if: runner.os == 'Linux'
+        run: sudo apt-get update && sudo apt-get install -y libportaudio2 ffmpeg
+      - name: System deps (macOS)
+        if: runner.os == 'macOS'
+        run: brew install portaudio ffmpeg
+
+      # Use the system interpreter (no virtualenv) so install.sh's `pip --user`
+      # branch is allowed. Editable install makes `aai_cli` importable for the
+      # test's __version__ check; uv builds the wheel; pipx drives the pipx branch.
+      - name: Tooling
+        run: |
+          python -m pip install --upgrade pip  # need pip >= 25.1 for --group
+          python -m pip install -e . --group dev uv pipx
+
+      - name: Real install smoke
+        run: python -m pytest -q -m install_script ${{ matrix.kfilter }}

.gitignore

@@ -10,9 +10,9 @@ build/
 .coverage
 htmlcov/
 
-# Editor/agent and local planning artifacts
-.claude/
-docs/
+# Editor/agent local artifacts: keep personal settings local, but track the
+# team-shared bits (.claude/settings.json, agents/, skills/).
+.claude/settings.local.json
 
 # Local scratch scripts (often contain live keys)
 transcribe/

.mcp.json

@@ -0,0 +1,11 @@
+{
+  "mcpServers": {
+    "github": {
+      "type": "http",
+      "url": "https://api.githubcopilot.com/mcp/",
+      "headers": {
+        "Authorization": "Bearer ${GITHUB_PAT}"
+      }
+    }
+  }
+}