feat: WhatsApp channel adapter via Twilio — per-agent integration (WHATSAPP-001)

[vybe]

Summary

Add WhatsApp as a channel adapter for Trinity agents via Twilio's WhatsApp Business API, reusing the ChannelAdapter abstraction proven by Slack (SLACK-002) and Telegram (TGRAM-001). Each agent binds to its own Twilio account + WhatsApp sender number, enabling per-agent WhatsApp presence without platform-level WhatsApp Business verification.

Provider Choice: Twilio (vs Meta Cloud API Direct)

Decision: Use Twilio as the WhatsApp provider for v1. Meta's Cloud API direct remains a valid future alternative but is not the starting point.

Why Twilio for Trinity's per-agent, open-source model:

• Fast onboarding — Twilio Sandbox works in 5 minutes (keyword-based opt-in). No Meta Business verification blocker for dev/test.
• Per-agent credentials fit naturally — users bring their own AccountSid + AuthToken + whatsapp:+E164 sender, matching the Telegram bot-token pattern.
• Programmatic number management — Twilio has real REST APIs for phone-number provisioning and sender registration.
• No Meta Tech Provider program required — Meta Cloud API direct would force either one-WABA-for-all-agents (brand conflict) or enrollment in Meta's Tech Provider / Embedded Signup program (weeks-to-months of approval).
• Optional future SMS — the same Twilio binding can serve SMS; a Phase 3 follow-up can add SMS with minimal extra work.

Twilio constraints noted:

• One WABA per Twilio account (multiple senders/numbers per WABA). Multi-tenancy achieved by each agent owner using their own Twilio account, not a shared platform-level account.
• Twilio WhatsApp does not support group chats — Phase 2 from the original spec is removed (see "Explicitly out of scope" below).
• 24-hour customer-service window applies; outside it, only pre-approved templates can be sent. Template management deferred to Phase 3.

Motivation

WhatsApp is the most widely used messaging platform globally (2B+ users). Enabling agents to participate in WhatsApp conversations opens Trinity to a massive user base, particularly for:

• Customer support agents responding to inquiries
• Team assistants accessible from any phone
• Notification delivery to WhatsApp users
• Business workflow agents interacting with clients

Architecture

WhatsApp user's phone
    ↓  (Meta's WhatsApp network)
Twilio WhatsApp API
    ↓  webhook (form-encoded POST + X-Twilio-Signature)
Backend → WhatsAppAdapter (Twilio) → ChannelMessageRouter → Agent execution
    ↑  response
Twilio REST API (POST /Messages.json)

Mirrors the existing Telegram/Slack transport pattern:

• Webhook endpoint at POST /api/whatsapp/webhook/{webhook_secret} (secret routes to the right agent binding).
• Verifies X-Twilio-Signature HMAC-SHA1 over URL + alphabetically-sorted form params using the agent's Twilio AuthToken.
• Returns empty TwiML (<Response/>) immediately, processes asynchronously via asyncio.create_task — same shape as transports/telegram_webhook.py:81.
• Response delivered via Twilio REST API (not as TwiML body), since agent execution may exceed Twilio's webhook timeout.

Requirements

Phase 1: Direct Messages (MVP)

Backend

• WhatsAppAdapter implementing ChannelAdapter ABC (adapters/whatsapp_adapter.py)
• TwilioWebhookTransport in adapters/transports/twilio_webhook.py (HMAC-SHA1 signature verification)
• Webhook receiver: POST /api/whatsapp/webhook/{webhook_secret} (form-encoded body, returns empty TwiML, processes async)
• db/whatsapp_channels.py — DB operations for bindings and chat links
• whatsapp_bindings table — agent_name, account_sid, auth_token_encrypted, from_number, messaging_service_sid (optional), webhook_secret, is_sandbox
• whatsapp_chat_links table — maps WhatsApp phone numbers to chat sessions (includes verified_email for feat: Unified channel access control — verified-email identity, per-agent allow-list, access requests (web / Telegram / Slack) #311)
• AuthToken AES-256-GCM encryption at rest (reuse CredentialEncryptionService)
• Outbound send via Twilio REST API (POST /2010-04-01/Accounts/{sid}/Messages.json) using httpx
• Delivery status tracking (optional StatusCallback parameter)
• Message splitting at 1600-char Twilio limit
• Rate limiting per recipient phone number
• Media support: receive images, audio, PDFs via MediaUrl0..N (up to 16MB per message; Twilio-hosted URLs)
• Lifespan integration in main.py (mirror Telegram webhook reconciliation, ~lines 483–513)

Frontend

• WhatsAppChannelPanel.vue in Agent Detail → Sharing tab
• Connect flow: paste AccountSid + AuthToken + From number, validate via Twilio API (GET /2010-04-01/Accounts/{sid}.json)
• Show connected sender info (phone number, display name, sandbox vs production indicator)
• Disconnect/replace credentials
• Connection status indicator
• Webhook URL display with copy button (for Twilio Sender/Sandbox configuration)
• Sandbox opt-in instructions (send join <keyword> to +14155238886)

API Endpoints

• GET /api/agents/{name}/whatsapp — binding status
• PUT /api/agents/{name}/whatsapp — configure Twilio credentials
• DELETE /api/agents/{name}/whatsapp — remove binding
• POST /api/agents/{name}/whatsapp/test — send test message to a number
• POST /api/whatsapp/webhook/{webhook_secret} — receive incoming messages + status callbacks (public, HMAC-verified)

Phase 2: Access Control & Session Management

• Integrate with unified channel access control (feat: Unified channel access control — verified-email identity, per-agent allow-list, access requests (web / Telegram / Slack) #311) — resolve_verified_email via phone number → verified email mapping
• /login <email> command pattern matching Telegram adapter
• Rate limiting keyed on twilio:{account_sid}:{from_phone}
• Session ID built from {account_sid}:{from_phone} (consistent with Telegram)
• Access request flow for open_access agents

Phase 3: Advanced (deferred)

• SMS support on the same TwilioAdapter (same binding, drops whatsapp: prefix)
• Message templates for initiating conversations outside the 24-hour window (Twilio Content Builder)
• Interactive buttons and list messages (Twilio Content API)
• Voice-note transcription (Whisper API)
• Outbound file sharing

Explicitly out of scope

• Group chats — Twilio's WhatsApp API does not support group conversations. If needed later, would require Meta Cloud API direct or a different BSP. The previous "Phase 2: Group Chat Support" is therefore removed.
• Meta Cloud API direct provider — tracked as a potential future alternative; not implemented here. Revisit if free-tier conversation volume matters more than programmatic onboarding speed.

Database Schema

CREATE TABLE whatsapp_bindings (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    agent_name TEXT NOT NULL UNIQUE,
    account_sid TEXT NOT NULL,                -- Twilio AccountSid
    auth_token_encrypted TEXT NOT NULL,       -- Encrypted AuthToken (AES-256-GCM)
    from_number TEXT NOT NULL,                -- e.g., 'whatsapp:+14155238886'
    messaging_service_sid TEXT,               -- Optional, preferred over from_number
    display_name TEXT,                        -- Sender display name from Twilio
    is_sandbox INTEGER DEFAULT 0,             -- 1 if using Twilio Sandbox
    webhook_secret TEXT NOT NULL,             -- Random token in webhook URL path
    webhook_url TEXT,                         -- Stored after user configures in Twilio Console
    enabled INTEGER DEFAULT 1,
    created_by TEXT,
    created_at TEXT NOT NULL,
    updated_at TEXT
);

CREATE INDEX idx_whatsapp_bindings_agent ON whatsapp_bindings(agent_name);
CREATE INDEX idx_whatsapp_bindings_webhook ON whatsapp_bindings(webhook_secret);

CREATE TABLE whatsapp_chat_links (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    binding_id INTEGER NOT NULL REFERENCES whatsapp_bindings(id),
    wa_user_phone TEXT NOT NULL,              -- E.164 phone number of WhatsApp user
    wa_user_name TEXT,                        -- Twilio ProfileName
    session_id TEXT,
    verified_email TEXT,                      -- Unified access control (#311)
    verified_at TEXT,
    message_count INTEGER DEFAULT 0,
    last_active TEXT,
    created_at TEXT NOT NULL,
    UNIQUE(binding_id, wa_user_phone)
);

CREATE INDEX idx_whatsapp_chat_links_binding ON whatsapp_chat_links(binding_id);

Security

• AuthTokens AES-256-GCM encrypted at rest (same CredentialEncryptionService as Slack/Telegram)
• Webhook signature verification via HMAC-SHA1 with X-Twilio-Signature header (Twilio spec: auth token as HMAC key, URL + alphabetically-sorted form params as base string)
• SSRF prevention: media downloads restricted to api.twilio.com and *.twilio.com
• Restricted tools for WhatsApp users (WebSearch, WebFetch — same as Slack/Telegram)
• AuthToken values never logged
• Rate limiting per recipient phone number to prevent abuse

Twilio Setup (User Responsibility)

For development (Sandbox):

1. Create a Twilio account → Messaging → Try WhatsApp
2. Copy AccountSid + AuthToken from Console
3. Sandbox sender is whatsapp:+14155238886 (shared across Twilio users)
4. Users opt in by sending join <your-sandbox-keyword> from their phone
5. Paste credentials into Trinity Sharing tab → configures binding + generates webhook URL
6. Copy webhook URL from Trinity → paste into Twilio Console → Sandbox → "WHEN A MESSAGE COMES IN"

For production:

1. Register a WhatsApp sender via Twilio Console → Senders (requires Meta Business Manager linkage)
2. Wait for Meta display-name approval (24–48h)
3. Paste production AccountSid + AuthToken + sender number into Trinity
4. Configure webhook URL on the sender in Twilio Console

Relationship to Existing Work

• Extends: ChannelAdapter ABC from SLACK-002 (adapters/base.py)
• Reuses: ChannelMessageRouter for rate limiting, agent resolution, execution pipeline, access control (feat: Unified channel access control — verified-email identity, per-agent allow-list, access requests (web / Telegram / Slack) #311)
• Pattern: Same as TelegramAdapter + TelegramWebhookTransport — new adapter + new transport + DB module + router + frontend panel
• Files to create:
  • src/backend/adapters/whatsapp_adapter.py
  • src/backend/adapters/transports/twilio_webhook.py
  • src/backend/db/whatsapp_channels.py
  • src/backend/routers/whatsapp.py
  • src/frontend/src/components/agent-detail/WhatsAppChannelPanel.vue
  • DB migration in src/backend/db/migrations.py
• Files to modify:
  • src/backend/main.py — lifespan integration (mirror Telegram at ~line 483)

Acceptance Criteria

• Agent can be connected to a Twilio WhatsApp sender via Sharing tab UI (AccountSid + AuthToken + from-number)
• Twilio Sandbox flow works end-to-end for development
• Users can send WhatsApp messages to the bound number and receive agent responses
• Webhook signature verification (HMAC-SHA1) prevents unauthorized requests
• AuthTokens encrypted at rest, never logged
• Message history tracked in Trinity chat sessions
• Access control (feat: Unified channel access control — verified-email identity, per-agent allow-list, access requests (web / Telegram / Slack) #311) applies to WhatsApp identities same as Telegram/Slack
• Connection status visible in UI (sandbox vs production, connected/error)
• No platform-level Twilio account required — each agent owner brings their own credentials

[m13v]

the 24-hour conversation window is the piece that will shape your adapter the most. it means the adapter can't just expose send/receive - it needs to track conversation window state per user and handle the fallback to template messages when the window expires.

if an agent goes quiet for 24 hours (overnight, error state, etc.) and then tries to resume, the next message must be a pre-approved template or the API returns a 131047 error. teams usually discover this in production, not dev, because test conversations are always fresh. worth modeling window expiration in the ChannelAdapter base class from the start, and adding a warning hook when a conversation window is within 2 hours of closing.

the other WhatsApp-specific wrinkle: delivery receipts and read receipts come on a different webhook event than message events. if your agent needs to know whether a message was actually read before proceeding, you need a separate event handler for that.

[m13v]

for the macOS-specific path: we built a WhatsApp MCP server that controls the native WhatsApp app via accessibility APIs - no Business API needed, no 24-hour window, no template messages. only works on macOS but is much simpler for personal/team use: https://github.com/m13v/whatsapp-mcp-macos

and the macOS accessibility layer that drives the AX element interactions: https://github.com/mediar-ai/mcp-server-macos-use/blob/main/Sources/MCPServer/main.swift

[vybe]

Pivoted provider choice from Meta WhatsApp Cloud API direct → Twilio for v1.

Why: Trinity's per-agent, open-source, bring-your-own-credentials model fits Twilio cleanly (one Twilio account per agent owner, same pattern as Telegram bot tokens) but fights Meta direct — which requires either one WABA for all agents or enrollment in Meta's Tech Provider program (weeks of approval). Twilio Sandbox also unblocks dev/test in minutes.

Scope changes:

• Integration: Meta Cloud API → Twilio WhatsApp Business API (form-encoded webhooks, X-Twilio-Signature HMAC-SHA1, Twilio Messages REST API for outbound)
• DB schema: Meta tokens → Twilio AccountSid/AuthToken/from-number
• Group chats removed from scope — Twilio's WhatsApp API doesn't support them, and it was optimistic for Meta direct too (bots don't get group messages by default). Tracked as explicitly out of scope.
• Phase 2 repurposed to Access Control / feat: Unified channel access control — verified-email identity, per-agent allow-list, access requests (web / Telegram / Slack) #311 integration (was the right thing to do regardless of provider)
• Phase 3 keeps templates + rich media; adds "SMS on the same TwilioAdapter" as a natural follow-up
• Added a TwilioWebhookTransport alongside WhatsAppAdapter to match the existing adapter/transport split (transports/telegram_webhook.py is the reference)
• Meta Cloud API direct is documented as a future alternative, not abandoned