feat(mcp): full self-service MCP surface for scoped users - #77
Merged
Conversation
Expand the embedded MCP server from a handful of read tools into a complete self-service plane so a scoped user can run EmbedBase end-to-end from an AI chat with their own key, each tool enforcing that user's grants exactly like its REST counterpart. - mcp: presigned two-step upload (request_upload/confirm_upload), download, document status/chunks/reprocess/delete, workspace/collection/tag CRUD, ingestion jobs & stats, and rate-limit introspection. - permissions: new grantable `manage_tags` capability (REST + MCP + console) so non-admins can manage tags in workspaces they can see. - ingestion: per-file `retention_days` (1-30) replacing the global temp knob. - mcp: live tool catalogue (GET /mcp-tools) drives the console tool list and the generated SKILL.md from the registered tools so they never drift; SKILL.md documents accepted upload formats and PDF-to-full-text conversion (markdown tables, image description + transcription). - mcp: per-user request rate limit, configurable per user in the console (users.rate_limit_rpm; 0 = inherit the global mcp.rate_limit_rpm), enforced by a post-auth per-user token bucket. - users: email is now optional (an account is identified by its username). Adds Alembic 0010 (users.rate_limit_rpm) and 0011 (users.email nullable).
…e the agent skill Three additions to the self-service MCP surface (PR #77): - Attach an original source file to a document — request_original_upload / confirm_original_upload store the raw source under a ".orig" key; it is never embedded and is fetched via download_document(original=true). Worker delete removes both the parse object and the original. - Validate content-type at both presigned confirm steps — Storage.read_head + upload.validate_content magic-byte-check the uploaded object, returning 415 and purging it on a type mismatch, so a reserved .pdf can't be satisfied with a PNG. - Deliver the agent skill as a progressive-disclosure bundle — a lean SKILL.md plus references/{searching,uploading,tools}.md, downloaded as a zip (dependency-free store-only writer), so a read-only session never loads the upload rules. search_documents is documented as the primary retrieval tool; get_document_chunks as a follow-up for specific chunk_ids.
…rvice # Conflicts: # api/services/documents.py # api/services/upload.py # tests/unit/test_upload.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Turns the embedded MCP server into a complete self-service plane: a scoped user can run EmbedBase end-to-end from an AI chat using their own API key, with every tool enforcing that user's grants exactly like the REST API. Admin-plane concerns (app config, user/key management) stay in the console.
What's included
request_upload→ PUT →confirm_upload),download_document, document status/chunks/reprocess/delete, workspace/collection/tag CRUD,list_ingestion_jobs/get_ingestion_stats, andget_rate_limit.manage_tagscapability — a new grantable privilege honoured by REST, MCP, and the console, so a non-admin can manage tags in workspaces they can see (gated byreadon the workspace +writeon the collection/document for assignment).retention_days(1–30; omit = permanent) on every upload path, superseding the global temp-retention knob.GET /mcp-toolsrenders the Settings → MCP tool list and the downloadable SKILL.md from the actually-registered tools, so they can never drift; a drift-guard test asserts the catalogue equals the live tool set. SKILL.md also documents the accepted upload formats (.pdf,.md/.markdown,.txt) and how to convert a structured PDF to full text — markdown tables for tables, description + transcription for images.users.rate_limit_rpm;0= inherit the globalmcp.rate_limit_rpm), enforced by a post-auth per-user token bucket keyed by user id (exact under concurrency; the pre-auth guard still protects the auth path).username);UNIQUE(email)still forbids duplicate addresses while permitting many no-email accounts.Migrations:
0010(users.rate_limit_rpm),0011(users.email nullable, via batch mode for SQLite).Testing
ruff✅ ·mypy✅ (122 files) ·pytest✅ 917 passed, 13 skippedtsc --noEmit✅docs/openapi.yamlregenerated/standards-checkclean; independent adversarial correctness reviews on the rate-limit and optional-email changes (all findings fixed and re-verified)