Shared synchronous Session may cause 300-second stalls on silently stale pooled connections

[Jinx2000]

Summary

DashScope SDK 1.26.7 uses a process-wide shared synchronous requests.Session.

In a long-running Dify plugin process, embedding requests that use the SDK-managed shared Session intermittently take approximately 301 seconds, while normal requests complete in approximately 145–180 ms.

The affected request eventually succeeds because the caller retries the first transport failure. The initial exception is therefore not surfaced in the final Dify response.

Environment

• DashScope SDK: 1.26.7
• Python: 3.12
• requests: 2.34.2
• Integration: Dify Tongyi plugin 0.2.15
• API: TextEmbedding.call
• Model: text-embedding-v4
• Deployment: long-running Docker plugin processes

Observed behavior

Normal embedding invocation:

latency: approximately 145–180 ms
status: HTTP 200

Intermittently affected invocation:

latency: 301327 ms
status: HTTP 200 after caller-level retry

The corresponding knowledge-retrieval operation took 301.722 seconds.

The duration is consistent with:

1. the SDK's 300-second default read timeout;
2. the caller's 1-second retry backoff;
3. a fast successful retry.

The first exception is caught by the integration's retry helper, and the plugin runtime does not forward the Python warning to its container log. Therefore, we do not currently have the original ReadTimeout traceback and do not claim that the exception type has been directly captured.

Why this appears related to the shared Session

When no explicit Session is supplied, independent embedding invocations use the SDK-managed process-wide Session and connection pool.

The same deployment does not show this behavior on the Tongyi LLM path after that integration was changed to create and close a dedicated Session for every invocation.

We have implemented the same scoped mitigation for embedding calls:

• create one Session per embedding invocation;
• reuse it within that invocation's batches and retries;
• close it when the invocation completes;
• prevent pooled connections from crossing invocation boundaries.

Related Dify reports and changes:

• Embedding report: Tongyi text embedding: shared DashScope session can cause intermittent 300-second stalls langgenius/dify-official-plugins#3815
• Embedding mitigation PR: fix(tongyi): isolate DashScope sessions per embedding invocation langgenius/dify-official-plugins#3817
• Previous LLM Session isolation: fix(tongyi): isolate DashScope sessions per LLM invocation langgenius/dify-official-plugins#3659

DashScope PR #175 already addressed pooled connections that fail with a pre-response ConnectionError, but deliberately excludes ReadTimeout:

#175

Requested investigation

Please investigate whether the process-wide shared synchronous Session can reuse silently stale pooled connections and block until the default read timeout.

The SDK should ensure that stale pooled connections do not cause otherwise normal requests to stall for approximately 300 seconds. The specific connection-lifecycle strategy is left to the maintainers.

Expected behavior

A connection that has become silently stale should not block a normal embedding request until the full 300-second read timeout, and long-running applications should have a supported way to control the lifecycle of the SDK-managed shared connection pool.

[luk384090-cloud]

Thank you for this exceptionally detailed report — the latency breakdown (≈301 s ≈ 300 s default read timeout + 1 s retry backoff), the comparison against the per-invocation Session isolation in the Dify Tongyi plugin, and the links to the related Dify issues/PRs made it much easier for us to pinpoint the root cause.We confirmed that the process-wide shared synchronous requests.Session could reuse silently stale pooled connections, causing otherwise normal requests to block until the full 300-second read timeout.A fix has been merged in #187 and will ship in the next release:
TCP keepalive is now enabled on the shared sync session's connection pool (with platform-specific fallbacks), so silently dropped idle connections are detected and discarded instead of stalling a request until the read timeout.
A new public API, dashscope.close_shared_sync_session(), lets long-running applications explicitly close and reset the SDK-managed shared connection pool; it is also closed automatically at interpreter exit.
Your per-invocation session isolation in the Dify plugin remains a valid mitigation and is fully compatible with this fix — you can keep it as is, or simplify back to the shared session after upgrading.Thanks again for the thorough investigation and for contributing the mitigation upstream to Dify. If you still observe stalls after upgrading to the next release, please feel free to reopen this issue or file a new one.