feat(index): add bulk indexing for many media items

[Hasnain2430]

Related issue

Closes #68.

Summary

vidxp index create handles one media item per invocation, so indexing a repository meant driving it once per video and tracking results by hand. This adds a transport-neutral planning operation and a thin CLI adapter over it.

• Application.plan_bulk_index takes a PlanBulkIndexCommand and returns a BulkIndexPlan of per-media targets, each pending or skipped with a reason. The plan is read-only, so any interface can show the decision before committing to work, and the HTTP, MCP, and Desktop adapters can reuse it unchanged.
• vidxp index bulk indexes every registered media item, or a selection passed with repeated --media-id. --plan-only shows the decision without indexing, --reindex plans covered media anyway, and --json emits the same payload for machine callers.

Skip rule. A media item is skipped when the active snapshot holds a generation for it, that generation covers every requested modality, and its recorded input_sha256 still matches the registered media checksum. Replacing a video's content or requesting a modality the generation lacks therefore plans it again. Media that is not in the ready state is reported as skipped with media_not_ready rather than silently dropped.

No new indexing behavior. Each pending target goes through the existing submit_index durable job, one job per media item. That mirrors how IngestionCoordinator already sequences ingestion, and it is what gives the batch its guarantees rather than new bookkeeping:

• a failure isolates to its own media, because each item is its own job;
• earlier successes stay committed, because each job commits its own generation;
• rerunning the command retries only what is still missing, because completed media is then skipped by the rule above;
• an individual job can be retried through the existing vidxp jobs retry <job-id>.

The command exits non-zero when any media failed, so it composes in scripts.

User-visible. New CLI command and new application operation; no existing contract changes, and no storage or model change, so existing repositories do not need rebuilding.

Notes for review

• I chose a new bulk subcommand rather than adding --all to index create, because create takes a required positional media identifier and overloading it would change an existing contract. Happy to reshape this if you would rather it be one command.
• I did not add a new job kind. A bulk_index workflow would have needed entries in JobKind, WORKFLOW_NAMES, the JobRequest and JobResult unions, dbos_workflows.py, and dbos_jobs.py, and it would have duplicated sequencing that per-item jobs already provide. max_concurrent_indexing defaults to 1, so per-item jobs serialize on their own rather than contending.
• The non-indexable-capability guard in plan_bulk_index mirrors the one in create_index. Every capability in the current registry has a collection name, so that branch is not reachable today; the test covers the reachable guard (an unknown capability is rejected) instead of inventing a fake capability to reach it.
• tests/test_bulk_index.py is written as a unittest.TestCase deliberately. CI discovers tests with python -m unittest discover, which does not collect module-level pytest functions, so a pytest-style file would not have run in CI at all (reported as [Bug]: CI skips 85 tests because unittest discovery cannot collect module-level pytest functions #156).

Validation

Windows 11, Python 3.12.14, scene model artifacts prepared. Every command below was run without piping, so the reported exit codes are the commands' own.

Unit tests — uv run --no-sync python -m unittest discover -s tests -p "test_bulk_index.py" → Ran 10 tests ... OK. Covers the skip rule, a generation missing a requested modality, a changed media checksum, --reindex, non-ready media, explicit versus repository-wide selection, cursor pagination across every page, and rejection of an unknown capability.

Lint — uvx "ruff~=0.16.1" check src tests → all checks passed. Also --select RUF100 to confirm no unused suppressions.

Documentation — npx --yes markdownlint-cli2@0.23.2 "INSTALLATION_GUIDE.md" → exit 0. No links added, so I did not run lychee locally; please let the documentation workflow cover it.

Real media, real model, real index. Two generated 3-second H.264 clips imported into an isolated --data-dir, indexed with the scene capability against google/siglip2-base-patch16-224. This exercised actual FFmpeg decoding, the model provider, the durable job boundary, and committed snapshot storage — not mocks.

Step	Command	Result
Plan before indexing	index bulk --modality scene --plan-only --json	planned 2, skipped 0, 2 items listed, nothing indexed, exit 0
Index the batch	index bulk --modality scene --json	indexed 2, skipped 0, failed 0, exit 0
Rerun unchanged	index bulk --modality scene --json	planned 0, skipped 2 both already_indexed with their generation ids, exit 0
Force reindex	index bulk --modality scene --reindex --json	indexed 2, skipped 0, exit 0
Explicit selection	index bulk --media-id <one> --modality scene --plan-only	1 target considered, the other ignored
Index intact afterwards	index status --json	state ready, media_count 2, modalities ['scene']
Content is searchable	search scene "colour bars test pattern" --json	2 moments, hits spanning both indexed media ids

Failure isolation, verified rather than assumed. In a second isolated repository I imported both clips and deleted the stored content object for one of them, then ran the batch:

Step	Result
index bulk --modality scene --json	indexed 1, failed 1, exit 1. one.mp4 -> indexed; two.mp4 -> failed: The requested media was not found.
Rerun the same command	skipped 1, planned 1, failed 1, exit 1. one.mp4 -> skipped already_indexed; only the failed item was retried.

That is the issue's core guarantee shown end to end: the successful item stayed committed, the failure was reported per media with its reason, and the rerun retried only what was still missing.

I also ran an earlier batch before preparing model artifacts. Both items failed with Model artifacts for the scene capability are not available locally, the batch continued rather than aborting on the first error, and the command exited 1 — the unhappy path behaves the same way.

Full suite — uv run --no-sync python -m pytest -q on this branch:

2 failed, 743 passed, 5 skipped, 106 subtests passed in 447.55s (0:07:27)

Both failures are the two Windows path assertions this branch does not contain a fix for, since it branches from upstream/main. They are test_codex_plugin.py::test_export_codex_plugin_materializes_the_canonical_skill_bundle and test_mcp.py::MCPTests::test_clip_submission_and_lazy_artifact_download, both sent separately as #154. Cherry-picking that commit onto this branch and rerunning the two affected files gives 60 passed, 7 subtests passed, so this change introduces no failures. I have not merged #154 into this branch, to keep the diff to one outcome.

One process note in case it saves someone else the confusion: an earlier full-suite run on this branch reported six failures, including four in test_native_ingestion.py. That was my own fault — I was running real-media indexing concurrently with the suite, and those end-to-end tests assert that autonomous indexing completes without status polling, which does not hold when the machine is saturated and models are being loaded by another process. Run on their own they pass (4 passed in 98.74s), and the clean full-suite run above is the one to trust.