Summary
When creating multiple concurrent sessions with a specific model specified in the session config, the SDK calls list_models on each send_and_wait() call rather than caching the result. This causes 429 rate limiting errors at moderate concurrency levels (~100 sessions).
Environment
- SDK: GitHub Copilot Python SDK (
copilot package)
- Python: 3.11+
- OS: Windows 11
Steps to Reproduce
import asyncio
import os
from copilot import CopilotClient
MODEL = "claude-opus-4.5"
NUM_SESSIONS = 100
async def create_session(client, index):
# Step 1: Create session
try:
session = await client.create_session({"model": MODEL})
except Exception as e:
return False, f"[create_session] {e}"
# Step 2: Send and wait
try:
await session.send_and_wait({"prompt": "Say hello"}, timeout=120)
except Exception as e:
return False, f"[send_and_wait] {e}"
# Step 3: Cleanup
try:
await session.destroy()
except:
pass
return True, None
async def main():
client = CopilotClient()
await client.start()
print(f"Creating {NUM_SESSIONS} sessions with model={MODEL}...")
results = await asyncio.gather(*[
create_session(client, i) for i in range(NUM_SESSIONS)
])
await client.stop()
success = sum(1 for ok, _ in results if ok)
failed = [err for ok, err in results if not ok]
print(f"\nResults: {success}/{NUM_SESSIONS} succeeded")
if failed:
print(f"Errors ({len(failed)}):")
for err in set(failed):
print(f" [{failed.count(err)}x] {err[:80]}")
if __name__ == "__main__":
asyncio.run(main())Output
Creating 100 sessions with model=claude-opus-4.5...
Results: 66/100 succeeded
Errors (34):
[34x] [send_and_wait] Session error: Execution failed: Error: Failed to list models: 429
Potential Expected Behavior
- Use a cached list of models after the first
session.send_and_wait or bound the list of models upon client.create_session({"model": MODEL})
Summary
When creating multiple concurrent sessions with a specific
modelspecified in the session config, the SDK callslist_modelson eachsend_and_wait()call rather than caching the result. This causes 429 rate limiting errors at moderate concurrency levels (~100 sessions).Environment
copilotpackage)Steps to Reproduce
Output
Potential Expected Behavior
session.send_and_waitor bound the list of models uponclient.create_session({"model": MODEL})