Skip to content

Fix CLI sync, Telegram error handling, and memU max_tokens - #3

Merged
pufit merged 26 commits into
ClickHouse:mainfrom
constkolesnyak:fix/cli-sync-ingest-result
Mar 22, 2026
Merged

Fix CLI sync, Telegram error handling, and memU max_tokens#3
pufit merged 26 commits into
ClickHouse:mainfrom
constkolesnyak:fix/cli-sync-ingest-result

Conversation

@constkolesnyak

Copy link
Copy Markdown
Contributor

Summary

  • CLI sync: Fix broken nerve sync command after IngestResult refactor — update field names and remove stale engine argument from build_source_runners
  • Telegram channel: Add global error handler, log incoming messages, and wrap error replies in try/except to prevent silent failures
  • memU bridge: Default max_tokens to 4096 when omitted, preventing Anthropic API 400 errors

Test plan

  • Run nerve sync and verify it completes without errors
  • Send a message via Telegram and confirm it's logged and processed
  • Trigger an error in the Telegram handler and verify the error reply is sent (or gracefully logged if reply fails)
  • Verify memU extraction works without explicitly passing max_tokens

🤖 Generated with Claude Code

- Add global error handler to Telegram bot polling pipeline
- Log incoming messages with truncated preview
- Wrap error reply in try/except to prevent secondary failures
- Default max_tokens to 4096 in memU bridge when omitted, preventing
  Anthropic API 400 errors

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@constkolesnyak

Copy link
Copy Markdown
Contributor Author

у меня телеграм бот помер. теперь воскрес. надеюсь до утра доживёт

constkolesnyak and others added 25 commits March 20, 2026 15:55
The polling watchdog previously gave up after 10 consecutive restart
failures, leaving the bot permanently dead until a server restart.
Now it enters a 5-minute cooldown and retries indefinitely. Also
protects the watchdog itself from crashing on unexpected health-check
exceptions, and fixes the server lifecycle to properly await
channel start/stop.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The previous watchdog only checked if the polling asyncio task was
alive, but the task can be alive with a hung connection — silently
stuck, not actually receiving updates. Now the watchdog:

1. Tracks last update time across all handlers
2. After 10min with no updates, probes Telegram API (bot.get_me)
3. If the probe times out or fails, force-restarts polling
4. Logs a heartbeat every ~5min so we can verify it's running

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The active probe (bot.get_me) uses a new HTTP connection, so it can
succeed even when the polling long-poll connection is silently hung.
Now the watchdog force-restarts polling after 15min of staleness
regardless of probe results.

Also: if updater.stop() hangs during restart (stuck TCP connection),
we time out after 15s and rebuild the entire PTB Application from
scratch instead of waiting forever.

Timeline: 5min stale → probe API → 15min stale → force restart →
if stop() hangs → rebuild Application.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Two actual bugs fixed, replacing the previous band-aid watchdog:

1. Shutdown order: telegram was stopped AFTER background task
   cancellation. Starlette runs the lifespan in an anyio context,
   so cancelling background tasks propagated through cancel scopes
   and killed the polling task before clean shutdown. Now telegram
   stops first.

2. Watchdog checked wrong thing: only monitored the Updater (fetches
   from Telegram), not the Application (processes updates → handlers).
   If the Application's internal update-fetcher task crashed, updates
   piled up in the queue but no handlers fired — bot appeared dead
   while Updater looked healthy. Now checks both, plus queue depth.

Also:
- TCP keepalive on polling connection (SO_KEEPALIVE + TCP_KEEPIDLE=60s)
  prevents NAT/firewall from silently dropping the long-poll connection
- bootstrap_retries=-1 for infinite retry on transient startup errors
- Explicit read/connect timeouts on the polling HTTP client
- Simplified watchdog: detect → rebuild, no complex backoff chains

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Streaming messages were sent without parse_mode, causing markdown
(bold, links, etc.) to render as plain text. Added ParseMode.MARKDOWN
with a fallback to plain text if parsing fails.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Telegram's ParseMode.MARKDOWN (v1) doesn't support **bold** — only
*bold*. Since LLMs emit standard Markdown, messages silently fell
back to plain text. Switch to ParseMode.HTML with a md-to-HTML
converter that handles bold, italic, code, code fences, and links.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When the final streaming edit has identical content to the last
intermediate edit, Telegram returns "message is not modified". The
fallback was replacing the already-formatted HTML message with plain
text, stripping all formatting. Now we detect this case and return
early instead of falling back.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Downloads and base64-encodes images from Telegram, passes them through
the channel router to the agent engine as multi-modal content blocks.
Handles both single photos and media groups (albums) by buffering
album parts before sending as one message.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…densation

The Anthropic SDK prepends /v1 internally, so including it in base_url
caused /v1/v1/messages 404 errors. Also adds a proxy-compatible
condensation path using httpx for OpenAI-compatible endpoints.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Two tests leaked host state by not mocking ~/.claude/.credentials.json,
causing them to find the real OAuth token instead of testing the intended
waterfall paths.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Instead of showing repeated `[tool: X]` lines, group them as
`[tool: X] x3` for cleaner streaming output.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… running sessions

PTB processes updates sequentially by default, which means /stop gets
queued behind the active message handler and only runs after it finishes.
concurrent_updates(True) dispatches each update in its own asyncio task.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When /stop arrives before _run_inner() finishes creating the SDK client,
stop_session() found nothing to interrupt and returned False. Now it sets
a deferred stop flag that _run_inner() checks right after client creation,
aborting the turn before sending the message to the agent.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
/new now stops the running session before creating a replacement,
preventing orphaned sessions.  `nerve restart` spawns a detached helper
so it works even when invoked from inside the daemon (e.g. via Telegram).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…essions

interrupt() only signals the CLI to stop — it doesn't guarantee
receive_response() will return. Without also cancelling the task,
_run_inner hangs forever waiting for a ResultMessage that never comes.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When /stop cancels a turn, ResultMessage (the normal source of
sdk_session_id) never arrives.  Two changes:

1. Early-capture sdk_session_id from the first message that carries it
   (e.g. StreamEvent) instead of waiting for ResultMessage.
2. In the CancelledError handler, write the captured sdk_session_id to
   the DB before marking the session stopped.

Without this, new sessions stopped before their first turn completes
lose their sdk_session_id and cannot be resumed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
After /stop, a deferred-stop flag could linger from the previous turn
and immediately kill the next user message. Now engine.run() clears
stale flags before marking a new turn as running.

Also: wrap engine.run() in router as a Task so /stop can cancel it
reliably, and reorder CancelledError cleanup to persist sdk_session_id
before non-critical work (message saving, broadcast, memorize).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
After interrupt(), wait up to 5s for the turn to complete gracefully
via ResultMessage. This keeps the SDK client alive so the next message
continues instantly without reconnecting. Only falls back to task
cancellation if interrupt doesn't work within the timeout.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove redundant "tool:" prefix, collapse consecutive tool labels
without extra blank lines, and add a separator when transitioning
from tool block back to text.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When a user replies to a message or selects a quote, prepend structured
context so the model knows what is being referenced. Works for both
single messages and album (media group) messages.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add reaction handler that forwards emoji reactions to the agent as
text messages, with an LRU cache (200 entries) to include the original
message context. Fix allowed_updates to explicitly request all update
types — the default set excludes message_reaction and auto-detection
only works via Application.run_polling(), not Updater.start_polling().

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Introduces a `react` tool that sets emoji reactions on the user's last
message.  Adds REACTIONS capability to the channel model, tracks inbound
message context in the router, and implements Telegram's
setMessageReaction API.  Also resolves premium custom emoji IDs to their
base emoji in reaction-update handling.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@pufit
pufit merged commit 9c6042b into ClickHouse:main Mar 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants