Repository navigation
Conversation
The Node OTLP exporter streams with Transfer-Encoding: chunked and sends no Content-Length. The Receiver read Content-Length only, so every Node export came back 400 with zero spans; two TH-8103 children had to put a de-chunking relay in front of it. Receiver now accepts both framings. It also records one entry per accepted export (path, lower-cased headers, flattened resource attributes) via requests(), so a contract test can assert X-Api-Key/X-Secret-Key and project_name through the shared harness instead of a private recorder. Verified: 6 harness tests pass, and the TanStack example's real Node exporter delivered 4 chunked exports (4 spans, collector path, both auth headers, project_name/project_type) with no relay. Refs: TH-8339, TH-8103
TH-8327 step 1 (D-8327: measure first, wrapper only if the bare client emits nothing). Two measurements run through harness.run against a loopback Tavily fake and export with the real fi_instrumentation.register() to harness.Receiver. No call to api.tavily.com; placeholder keys only. Measured with tavily-python 0.8.4, langchain-community 0.4.2, langchain-core 1.5.2, langgraph 1.2.2 on Python 3.11: - Bare client (AC-02): TavilyClient and AsyncTavilyClient search and extract, register() only (also as the global provider): 4 real HTTP calls, 0 Tavily spans; only the measurement.control span arrives. Same with traceAI-langchain instrumented. tavily-python 0.8.4 contains no OpenTelemetry code. - LangGraph path (AC-01): the example's TavilySearchResults run by a ToolNode with traceAI-langchain: 1 TOOL span (gen_ai.span.kind=TOOL, name tavily_search_results_json) with 14 attribute keys. Its output.value carries the result text; with FI_HIDE_OUTPUTS the tool output is masked but the text still reaches the next agent/router input.value; FI_HIDE_INPUTS plus FI_HIDE_OUTPUTS keeps it in-process. Decision: the bare client emits nothing, so a minimal traceAI-tavily wrapper for TavilyClient/AsyncTavilyClient search and extract follows. approved: Nikhil 2026-10-03 blanket Refs: TH-8327
…ract The measurement (previous commit) found 0 spans from the bare client, so D-8327 releases a minimal wrapper. traceAI-tavily wraps search and extract on TavilyClient and AsyncTavilyClient (tavily-python 0.8.4 defines both in each class body; the deprecated tavily.Client inherits them). One span per call, tavily.search / tavily.extract, through FITracer: gen_ai.span.kind=TOOL (the key traceAI-langchain uses on the Tavily tool span), gen_ai.tool.name, input.value (search query, or extract's optional rerank query, also read positionally), tavily.url_count (extract), tavily.result_count and tavily.failed_result_count on success; status OK. URLs and response content are never recorded. The span is current while the SDK sends, so requests/httpx activity nests under it. tavily-python is declared >=0.8.4,<1 in pyproject and _instruments; fi-instrumentation-otel >=1.1.0; classifiers 3.10/3.11/3.13. Tests first: test_spans.py and test_packaging.py failed at collection (ModuleNotFoundError: traceai_tavily); 20 pass after. approved: Nikhil 2026-10-03 blanket Refs: TH-8327
A vendor exception (InvalidAPIKeyError, BadRequestError, TavilyKeylessLimitError, requests.HTTPError, httpx.HTTPStatusError, ...) records one exception event, sets ERROR "<Type>: <message>" and is re-raised unchanged. No retry: a keyless-limit error reaches the fake once (PRD J4). No result count on a failed span. A cancelled AsyncTavilyClient call (asyncio.CancelledError, or concurrent.futures.CancelledError) sets ERROR "cancelled" and tavily.cancelled=true, with no exception event. Tests first: test_errors.py had 8 of 9 failing (status UNSET, no event, no tavily.cancelled); all 30 non-LangChain tests pass after. approved: Nikhil 2026-10-03 blanket Refs: TH-8327
…es at 1 KB tavily-python 0.8.4 keeps the key in TavilyClient.api_key, TavilyClient.headers["Authorization"] and the requests session headers; AsyncTavilyClient has no api_key attribute and keeps it only in its httpx client's Authorization header. A caller's own session or httpx client can carry a key api_key does not. Every key found there is replaced by [redacted] in input.value, the ERROR status description, and the exception event's message and stacktrace (an error body can repeat the query). The caller's exception and the request are unchanged. input.value and the status description are cut to 1024 UTF-8 bytes on a character boundary, after redaction, so a key cut at the limit leaves no prefix. Tests first: test_privacy.py had 9 of 9 failing (key in input.value, status and event; uncapped queries); 39 non-LangChain tests pass after. approved: Nikhil 2026-10-03 blanket Refs: TH-8327
Reading the client's key, reading the arguments, starting the span, making it current, reading the result, recording the error and ending the span are each guarded and logged at debug level. A failure never replaces the vendor's result or exception; if the span cannot start, the call runs untraced; every started span ends once. Key discovery fails closed: if a place where the SDK keeps the key cannot be read, no free text is recorded (no input.value; the ERROR description is the exception type and the exception event's message/stacktrace are [redacted]), because a key there could not be removed. A response whose results/failed_results is not a list records no count (unknown is absent, never 0). Tests first: test_isolation.py had 10 of 12 failing (the injected fault reached the caller); 51 non-LangChain tests pass after. approved: Nikhil 2026-10-03 blanket Refs: TH-8327
…uninstrument instrument(config=TraceConfig(...)) is accepted explicitly; the default reads FI_HIDE_* from the environment. hide_inputs masks input.value to __REDACTED__ through FITracer; counts stay. A config that is not a TraceConfig raises TypeError and wraps nothing. using_session/using_user context reaches the span and suppress_tracing records nothing (FITracer). A tavily-python release that moved TavilyClient/AsyncTavilyClient or one of the methods is logged as a WARNING naming it and skipped; the rest is still instrumented and instrument() never raises into host startup. uninstrument() restores the exact original functions and is safe when nothing was wrapped; a no-op tracer provider is accepted. Tests first: test_instrumentor.py had 4 of 10 failing (no TypeError, KeyError/AttributeError on a moved API, and a leaked instrumented state that broke the next test); 61 non-LangChain tests pass after. approved: Nikhil 2026-10-03 blanket Refs: TH-8327
The real TavilyClient and AsyncTavilyClient call the loopback fake and export through the real fi_instrumentation.register() into harness.Receiver. Every export goes to /tracer/v1/traces with x-api-key and x-secret-key (no authorization header) and resource project_name/project_type=observe. Five calls give five TOOL spans under the caller's agent-turn span; the 401 is ERROR with one exception event; input.value carries the redacted query; counts are on the wire. No Tavily key, tvly- prefix, requested URL or fake response text reaches the wire, with a control showing the same text reached the caller. This verifies behaviour from earlier commits, so it passed on its first run; mutation check: disabling redaction fails both tests, and writing fi.span.kind instead of gen_ai.span.kind fails the contract test. approved: Nikhil 2026-10-03 blanket Refs: TH-8327
….run examples/search.py registers an OBSERVE project, instruments Tavily and runs one TavilyClient.search; TAVILY_BASE_URL optionally points the client at another endpoint. test_example.py runs it as a subprocess through harness.run against the Receiver and the loopback fake: one tavily.search TOOL span with the query and result count, the collector path, both auth headers and the project resource; no key or response text on the wire. Test first: it failed because examples/search.py did not exist; it passes after. approved: Nikhil 2026-10-03 blanket Refs: TH-8327
Re-runs the TH-8327 measurement scripts with traceAI-tavily instrumented: - Bare client: one TOOL span per call (search, extract, async search, async extract), each a child of the caller's span. - LangGraph path with TavilySearchResults: still exactly one TOOL span and the same seven graph spans (AC-04). langchain-community posts to the Tavily API with requests itself and never calls TavilyClient. - A LangChain @tool written over TavilyClient.search: the LangChain TOOL span plus one tavily.search span. traceAI-langchain does not make its spans current (by design, see its _tracer.py), so tavily.search starts its own trace; wrapping the call in trace.use_span(traceai_langchain.get_current_span()) makes it a child. The custom-tool test failed first (the --custom-tool option did not exist; then the nesting assertion showed the root span); 8 measurement tests pass with langchain-community 0.4.2 / langgraph 1.2.2. approved: Nikhil 2026-10-03 blanket Refs: TH-8327
python/frameworks/tavily/README.md states the measured result and the boundary (AC-08): the LangChain tool path is traceAI-langchain's (one TOOL span, result text exported unless FI_HIDE_INPUTS and FI_HIDE_OUTPUTS are both set) and this package adds nothing to it; the bare client emits 0 spans without this package. It documents the custom-LangChain-tool case (two spans, tavily.search in its own trace unless the LangChain span is made current), the attributes, key redaction locations and fail-closed behaviour, URL/content exclusion, the 1 KB cap, TraceConfig hiding, errors, cancellation, uninstrument, version checks and limits. Adds the traceAI-tavily row to the root README's Python "Tools and Libraries" table and to python/README.md's "Tools & Integrations" table. approved: Nikhil 2026-10-03 blanket Refs: TH-8327
…suite The measurement and example subprocesses put the repo's python/ directory on PYTHONPATH, so a run against the published fi-instrumentation-otel wheel still exported through the source tree. They now add the directory the test process imported fi_instrumentation from: the source tree in a repo run, site-packages in a published-release run. 71 tests pass on Python 3.11 with fi-instrumentation-otel==1.1.0 from PyPI and python/ off PYTHONPATH. approved: Nikhil 2026-10-03 blanket Refs: TH-8327
…cannot be built Building the redacted stacktrace is guarded on its own, so a failure there records the event with a [redacted] stacktrace instead of dropping it. No behaviour change on the tested paths; 65 non-LangChain tests pass. Mutation checks on this head (each reverted): dropping the OK status fails 10 tests, not making the span current 2, dropping tavily.cancelled 2, not restoring on uninstrument 6, not reading the httpx client's key 3, capping characters instead of bytes 1. approved: Nikhil 2026-10-03 blanket Refs: TH-8327
ruff check --isolated --select E,F,W --line-length 100 is clean on python/frameworks/tavily; 71 tests pass with the LangChain extras. approved: Nikhil 2026-10-03 blanket Refs: TH-8327
poetry-core adds a "Programming Language :: Python :: 3.x" classifier for every minor the python constraint (>=3.10,<3.14) admits, so the wheel built from the previous head listed 3.12 although only 3.10, 3.11 and 3.13 had been run (checked: uv build, then the installed wheel's metadata). The suite now also runs on 3.12 (66 passed, 6 LangChain skips; 72 passed with the LangChain extras), and pyproject lists 3.10-3.13. A packaging test asserts the python constraint admits exactly the classified versions, so the built metadata and pyproject cannot drift. It failed first (2 failures: 3.12 missing); 8 packaging tests pass after. approved: Nikhil 2026-10-03 blanket Refs: TH-8327
…de current The README and the custom-tool measurement made traceAI-langchain's tool span current with trace.use_span(get_current_span(), end_on_exit=False). use_span records the exception on that span and traceAI-langchain's on_tool_error records it again, so a failing call left two exception events on the tool span. Both now pass record_exception=False and set_status_on_exception=False. Tests: the measurement script gains --fail (the fake's scripted 401 with ToolNode(handle_tool_errors=True)); a LangGraph run of the custom tool asserts one exception event on the tool span and one on its tavily.search child. A plain-OpenTelemetry test pins the same pattern without extras. approved: Nikhil 2026-10-03 blanket Refs: TH-8327
…ys a new trace The custom-tool paragraph said tavily.search starts its own trace because traceAI-langchain does not make its spans current. It is a child of whatever OpenTelemetry span is current and starts its own trace only when none is; it is just never the LangChain tool span's child unless the tool makes that span current. Pinned by test_span_is_a_child_of_the_active_span, test_sync_search_emits_one_tool_span (no parent) and the custom-tool measurement. approved: Nikhil 2026-10-03 blanket Refs: TH-8327
…tion extract's rerank query was read from positional index 7, which holds only for tavily-python 0.8.4's argument order. Both wrappers now bind the call with inspect.signature(...).bind_partial(self, *args, **kwargs) and take "query" by name. The signature is read once per wrapped function and cached on the wrapper; the plain function is taken from the bound method or from wrapt's partial proxy (a TavilyClient.extract(client, ...) call), whose own signature still lists self under wrapt 1.x. If the signature cannot be read or the arguments do not bind, no query is recorded and nothing is raised. Tests: keyword, positional and class-level calls on sync and async extract; a stub with query moved to second place (keyword, positional, every argument positional, class-level); arguments that do not bind; a failing inspect.signature; one signature read for four calls. approved: Nikhil 2026-10-03 blanket Refs: TH-8327
With TraceConfig(hide_inputs=True) or FI_HIDE_INPUTS=true, input.value was __REDACTED__ but an error that repeats the query (the fake's echo-400: "Bad query: <query>") still carried it in the status description, exception.message and exception.stacktrace. The instrumentor now passes hide_inputs to the wrappers; with it on, the key is removed first and then each verbatim occurrence of the (key-redacted) query is replaced by __REDACTED__. A keyless client now gets an explicit, scrubbed exception event too. An empty query removes nothing. With hide_inputs off nothing changes. Tests: sync and async, TraceConfig and FI_HIDE_INPUTS, keyed and keyless clients; hide_inputs off keeps the query; an empty query leaves the error text readable. approved: Nikhil 2026-10-03 blanket Refs: TH-8327
…commands The README now gives both test commands, run from the repository root: the base suite, and the same with wrapt<2 plus langchain-community 0.4.2, langchain-core 1.5.2 and langgraph 1.2.2. It says the base command skips the 7 LangChain/LangGraph measurement tests (AC-01, AC-04, the bare client with traceAI-langchain, the two custom-tool runs) and that they live in this package rather than under frameworks/langchain, a deviation from PRD section 13 to keep the change in one package. Tests: test_packaging reads the section, checks both commands, and checks the stated count against the measurement tests that importorskip a LangChain extra. approved: Nikhil 2026-10-03 blanket Refs: TH-8327
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
TH-8327's spec says to measure first and add a wrapper only if the bare Tavily client emits nothing. This PR does both.
Measurement (committed as tests; tavily-python 0.8.4, langchain-community 0.4.2, langchain-core 1.5.2, langgraph 1.2.2):
TavilyClient/AsyncTavilyClient: sync and async,searchandextract, produced 0 Tavily spans, both withregister()as the global provider and with traceAI-langchain instrumented. A control span arrived, so the export path works. tavily-python has no OpenTelemetry code.TavilySearchResultstool: exactly 1 TOOL span. By default it carries the tool's result text.FI_HIDE_OUTPUTSmasks only the tool span, and the text still appears in the next agent'sinput.value.Wrapper (
python/frameworks/tavily, built because the bare client emits nothing):tavily.search/tavily.extract, onTavilyClientandAsyncTavilyClient, withgen_ai.span.kind=TOOL. The span is current during the vendor call.input.value, capped at 1024 UTF-8 bytes on a character boundary; URL, result and failed-result counts. A count is omitted when unknown, never written as 0. URLs and response content are never recorded.api_key,headers, the requests session, and the async httpx client.cancelledwithtavily.cancelled=true.uninstrument()restores everything exactly.tavily-python>=0.8.4,<1,fi-instrumentation-otel>=1.1.0, classifiers for Python 3.10-3.13 (all four run; a test keeps the classifiers in step with thepythonconstraint).Deviations from the spec (with evidence)
frameworks/langchain(PRD 13), to keep the change in one package.gen_ai.span.kind, the repo constant that traceAI-langchain emits. fi-collector reads it as well asfi.span.kind, which Exa and Firecrawl use.trace.use_span(traceai_langchain.get_current_span(), end_on_exit=False, record_exception=False, set_status_on_exception=False)to nest it.Review round 1 and fixes (pr-reviewer t_c5312487 APPROVE; pr-verifier t_56ed8da2 VERIFIED at
33c22ca)Non-blocking findings fixed test-first before the demo (approved: Nikhil 2026-10-03 blanket):
use_spannesting recorded each exception twice4be8759record_exception=False, set_status_on_exception=False; error-path test added.9567907tavily.searchparents to the current span; it starts a new trace only when none is current.queryread from fixed position 7fb51f5chide_inputs66d4aa1FI_HIDE_INPUTS, the query is replaced by__REDACTED__in error status and exception text; the fake echoes it in its 400 body.7aefd72## Testswith the base and LangChain-extras commands.Tests
Head
7aefd72, coordinator rerun from the repo root on a clean tree (tickets/TH-8327/exact-head-7aefd72.txt):No files changed outside the package, and no secret-like lines were added.
33c22cathe implementer also ran C (publishedfi-instrumentation-otel1.1.0) and D (OTel 1.29.0 floor), and the built wheel in a fresh venv; 8 deliberate breakages were caught. These were not repeated at7aefd72.wrapt<2because traceAI-langchain fails on wrapt 2.x. That is pre-existing; published installs cap wrapt below 2.register()into the sharedReceiver.Limits
langchain-tavilywas read from source, not run.Video demo
Narrated terminal demo, 5:59, recorded at head
7aefd72, the head verified in round 2 (pr-verifier t_a91d0d34). 1080p H.264/AAC, 9 chapters. sha25618469ea24f7014e26539f9af5a2f2cf512df20fdcaa7cb618c82164661fee15f.The video, captions, transcript, chapter list, preview and media check are attached privately to Linear TH-8327 (Future AGI workspace access needed).
Every run uses the real
tavily-python0.8.4 client against the package's loopback fake Tavily API, with placeholder keys and the shared harness receiver. There is no vendor call and no live fi-collector.7aefd72: branch and remote at the same SHA, 0 uncommitted files, 19 commits on3eaadc8; outside the package only one README line each/tracer/v1/traces; both auth header names shown (values not shown);project_type=observe;tavily.searchTOOL span under the app spanagent-turn.input.valueis'agent asked about [redacted]'andtavily.result_countis 2. The export contains no key, no response content and no result URLsFI_HIDE_INPUTS=trueand a 400 that echoes the query,input.value, status and exception message read__REDACTED__, while the caller still gets the full error. CONTROL (hide off) shows the query. BEFOREfb51f5c, the query leaked into status and exceptionqueryis bound by name across 4 real call forms. On a stub that movesqueryto second place, BEFORE9567907records nothing and HEAD records ituse_spansnippet,tavily.searchnests under the LangChain tool spantavily_web_searchinside LangGraph. On a failure, each span has exactly 1 exception event; CONTROL at4be8759^gives the tool span 2_querybinds by name;_scrubremoves the key first, then the hidden queryOnly frozen stretches were shortened; on-screen output matches the coordinator-verified driver output in all 7 terminal chapters.
Coordinator checks:
Stacked on #203 (shared harness,
3eaadc8), basedev. Not merged.Linear: TH-8327. approved: Nikhil 2026-10-03 blanket