Skip to content

feat: add cross-application multi-namespace workflow benchmarks (#112) - #153

Draft
askmy-stack wants to merge 1 commit into
mainfrom
cursor/cross-app-benchmarks-c2a5
Draft

askmy-stack wants to merge 1 commit into
mainfrom
cursor/cross-app-benchmarks-c2a5

Conversation

@askmy-stack

Copy link
Copy Markdown
Owner

Summary

Implements #112 — cross-application / multi-namespace workflow benchmarks with failure attribution.

Changes

  • New tool_semantics.cross_app module:
    • ToolNamespace, CrossAppTask, CrossAppTrace
    • Evaluate multi-namespace fixture workflows (no live MCP)
    • Attribute failures to the namespace most likely responsible (structural diffs → missing required tools → failed hop)
  • Fixture: examples/cross_app/drive_gmail_calendar_crm.json (Drive → Gmail → Calendar → CRM)
  • Docs: docs/cross-app.md

Test plan

  • pytest tests/test_cross_app.py
  • ruff check / mypy
  • CI green

Closes #112

Open in Web Open in Cursor 

Support Drive→Gmail→Calendar→CRM style tasks with namespace tagging and
failure attribution to the interface change most likely responsible.

Co-authored-by: Abhinaysai Kamineni  <askmy-stack@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[P2] Cross-application / multi-namespace workflow benchmarks

2 participants