fix: give native graph spans their LaunchDarkly identity by apucacao · Pull Request #121 · launchdarkly/python-ai-sdk

apucacao · 2026-09-30T22:49:03Z

Summary

Spans from the native graph adapters (to_openai_agents, to_claude_agents, to_lang_graph) had nothing tying them to the graph's AI Config, so Monitoring couldn't find them by config key.

The launchdarkly.graph span now gets set_ld_span_attributes, like every other handler: config key, run id, context keys and the feature_flag event. New helper: make_graph_track_data.
Graph and node tracking events now carry the environment id when LD_ENVIRONMENT_ID is set.
Graph-level events ($ld:ai:graph:invocation_success and friends) now use the graph key as configKey, matching the span and graph(). They used the root node's key.
The graph() runner's own launchdarkly.graph span gets the same identity.
make_graph_track_data stays internal (imported from launchdarkly_ai_server.utils, not in __all__). TELEMETRY-CONTRACT.md section 10 says which config key graph spans and events carry.
A setup error (an agent that fails to build, a graph that fails to compile) now fails the run like a run error does: every adapter opens the span before setup, records the exception with ERROR status, fires $ld:ai:graph:invocation_failure and ends the span once. Before, OpenAI and LangChain exported no span, and no adapter tracked the failure.

JS twin: launchdarkly/js-ai-sdk#101.

Testing

Adapter and client tests cover the span attributes, the event and the environment id.
Staging: graph run with this branch.

🤖 Generated with Claude Code

Note

Overview
Native graph runs (to_openai_agents, to_claude_agents, to_lang_graph) and the SDK graph() runner now stamp launchdarkly.graph spans with the same LaunchDarkly identity as other handlers via set_ld_span_attributes and a new make_graph_track_data helper, so AI Config Monitoring can correlate traces by config key (including the feature_flag span event).

Graph-level track events ($ld:ai:graph:invocation_success, failure, duration, total tokens) now use the graph key as configKey instead of the root node’s key; node events are unchanged. Track payloads also include environmentId when LD_ENVIRONMENT_ID is set.

Each native adapter wraps setup and execution in one try/finally so the graph span ends exactly once, setup failures are recorded as ERROR spans with $ld:ai:graph:invocation_failure, and OpenAI/LangChain no longer skip span export on pre-run errors. TELEMETRY-CONTRACT.md documents the parity rules (including empty variationKey on native adapters).

^{Reviewed by Cursor Bugbot for commit 8c67043. Bugbot is set up for automated code reviews on this repo. Configure here.}

Every $ld:ai:* event that execute_and_track sends carries the LaunchDarkly environment id, because AI Config Monitoring needs it to match a trace to the config that produced it. The two payloads built outside that path did not: make_track_data, used by every native graph adapter, and the graph-level graph_track_data in graph.py. Both now read the id through the same _try_get_environment_id helper, and omit the key when there is no id to report. Co-Authored-By: Claude Opus 5.5 <[email protected]>

A native graph adapter has no track data of its own: make_track_data describes one node, and the run as a whole is the graph flag. Add make_graph_track_data, which names the graph flag as both the config key and the graph key, the same choice the SDK's own graph runner makes. Adapters pass it to set_ld_span_attributes in the next commit. Co-Authored-By: Claude Opus 5.5 <[email protected]>

to_openai_agents, to_claude_agents and to_lang_graph each open a launchdarkly.graph span that carried only the graph key and token counts. The AI Config Monitoring tab finds a trace by a feature_flag span event and the config key, so none of these runs appeared against their AI Config. Tag the span through set_ld_span_attributes, the same helper every handler already uses, so the graph span now carries launchdarkly.config.key, launchdarkly.variation.key, launchdarkly.run.id, the context keys, and the feature_flag event with feature_flag.set.id. Co-Authored-By: Claude Opus 5.5 <[email protected]>

apucacao · 2026-10-01T14:10:20Z

bugbot run

cursor

Stale Bugbot comment from a previous run.

jeffdupont

Reviewed with the 1.0 freeze in mind. The fix is right: native graph spans now carry the config identity Monitoring needs, and the environmentId handling matches execute_and_track. Tests pass locally on 6c19b28 (1407 passed, 11 skipped).

Two things I'd like settled before GA, because they are hard to change once 1.0 ships:

New public name. make_graph_track_data is exported from the package root. The 1.0 API-surface plan makes make_track_data internal and renames it make_node_track_data, so this one should be internal too. Same for makeGraphTrackData in launchdarkly/js-ai-sdk#101. Details inline.
Graph span and graph events disagree on the config. The span now says config.key = <graph key>. The $ld:ai:graph:invocation_success / duration:total / total_tokens events in all three adapters still use make_track_data(root, ...), so their configKey is the root node's key. The SDK's own graph() uses the graph key for those events (graph.py _build_graph). That's not new in this PR, but event payloads are a data contract with Monitoring, and changing them after GA would quietly shift customers' dashboards. Could we pick one, either here or in a follow-up before GA? The monorepo TESTING.md doesn't say which configKey graph events carry, so it should be written down there too.

Smaller notes:

The SDK's own graph() span (graph.py:919 invoke, :1135 stream) still doesn't get set_ld_span_attributes, and neither does JS graph.ts:722. After this PR, native-adapter graph spans can be found by config key but graph() spans can't. Is that intended, or does Monitoring reach graph() traces through the node spans?
launchdarkly.graph.key is now set twice on the native span: once directly, then again by set_ld_span_attributes. It's harmless; the direct call could go.
Not from this PR: in openai-agents the span starts before the try that ends it, so a ValueError during agent setup (the "not built" checks) leaves the span un-ended and never exported. JS #101 adds an "ends the span once" test. Python may want the same.
I didn't find a matching ai-sdks-monorepo spec PR. The span attributes and feature_flag event on launchdarkly.graph are worth adding to TESTING.md so both SDKs stay in step.

jeffdupont · 2026-10-02T15:56:34Z

    "init_evaluations",
    # utils
    "create_handler",
+    "make_graph_track_data",


This adds a new name to the public surface that freezes at 1.0. Only the three adapter packages call it, so it's internal plumbing. The API-surface plan moves make_track_data to internal (as make_node_track_data) for the same reason. Could the adapters import it from launchdarkly_ai_server.utils instead, so it stays out of __all__? JS #101 has the same export in index.ts.

jeffdupont · 2026-10-02T15:56:34Z

+    track_data: dict[str, Any] = {
+        "runId": run_id,
+        "configKey": graph_key,
+        "variationKey": "",


variationKey is always "" here (and version always 1), while graph() reports the real values from the graph flag's meta. Native-graph spans will always have an empty launchdarkly.variation.key, so per-variation Monitoring views won't work for them. If GraphDefinition exposes meta later, this signature (graph_key, run_id) has to change. That's one more reason to keep the helper private until then.

jeffdupont · 2026-10-02T15:56:34Z

+            set_ld_span_attributes(
+                span,
+                {
+                    "__ld": make_graph_track_data(def_obj.key, run_id),


The span's config key is now the graph key, but invocation_success / duration:total / total_tokens below (lines ~279 and ~316) still use make_track_data(root, ...), so those events carry the root node's key. Same in claude-agents and langchain-agents. The SDK graph() runner keys those events to the graph. Should the native adapters do the same, so a graph run means the same thing whichever runner you use?

jeffdupont · 2026-10-02T15:56:34Z

        **model_stamps_from_meta(meta),
        "graphKey": key,
    }
+    _environment_id = _try_get_environment_id()


Good to see environmentId here. The launchdarkly.graph span this runner opens (lines ~919 and ~1135) still doesn't call set_ld_span_attributes, though, so graph() spans don't carry launchdarkly.config.key or the feature_flag event that native-adapter spans now have. Is that intended?

The launchdarkly.graph span that graph() opens, on invoke and on stream, carried only launchdarkly.graph.key. It now goes through set_ld_span_attributes with the same track data its graph events use, so it has the config key, variation, run id, context keys and the feature_flag event, like the native adapters' graph span. TELEMETRY-CONTRACT.md section 10 now says which config key the graph span and the graph-level events carry. Co-Authored-By: Claude Opus 5.5 <[email protected]>

…ternal The $ld:ai:graph:invocation_success, invocation_failure, duration:total and total_tokens events from the three native adapters carried the root node's config key, while their graph span and graph() use the graph key. They now use make_graph_track_data, so a graph run reports the same config key whichever runner produced it. make_graph_track_data is no longer exported from the package root. Only the adapters use it, and its signature will change if GraphDefinition gains the graph's variation metadata, so it stays out of the 1.0 surface. The adapters no longer set launchdarkly.graph.key by hand, since set_ld_span_attributes already does. to_openai_agents and to_lang_graph now open the graph span only after the agents and graph are built. A setup error such as a child agent that was not built used to leave a started span that was never ended or exported. Co-Authored-By: Claude Opus 5.5 <[email protected]>

apucacao · 2026-10-02T17:13:32Z

bugbot run

cursor

Stale Bugbot comment from a previous run.

A setup error in a native graph adapter (an agent that fails to build, a sub-agent tool that fails to build, a graph that fails to compile) was not reported as a graph failure. to_openai_agents and to_lang_graph opened the launchdarkly.graph span only after setup, so no span was exported at all, and none of the three adapters tracked $ld:ai:graph:invocation_failure for it. to_claude_agents also recorded a run error and ended its span twice, once in the run's own handler and again in the outer one. All three adapters now open the span, tag it with set_ld_span_attributes, and run setup and the run in one try. Any error records the exception on the span, sets status ERROR with the error message, tracks invocation_failure with the graph's track data when a context is present, and re-raises. The span ends exactly once, in a finally, so a failed setup still leaves no span open.

apucacao · 2026-10-05T22:45:55Z

bugbot run

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 8c67043. Configure here.}

apucacao and others added 3 commits September 30, 2026 18:46

apucacao mentioned this pull request Sep 30, 2026

fix: give native graph spans their LaunchDarkly identity launchdarkly/js-ai-sdk#101

Open

cursor Bot reviewed Oct 1, 2026

View reviewed changes

apucacao marked this pull request as ready for review October 1, 2026 14:27

apucacao requested review from andrewklatzke and jeffdupont October 2, 2026 13:48

jeffdupont reviewed Oct 2, 2026

View reviewed changes

apucacao and others added 2 commits October 2, 2026 12:26

cursor Bot reviewed Oct 2, 2026

View reviewed changes

This was referenced Oct 2, 2026

fix: pass AI Config model parameters through to every provider handler #107

Open

feat(AIC-3408): add Vercel AI SDK messages and agents adapters #103

Open

feat: add LiteLLM messages and agents integrations #102

Open

cursor Bot reviewed Oct 5, 2026

View reviewed changes

Navigation Menu

fix: give native graph spans their LaunchDarkly identity - #121

fix: give native graph spans their LaunchDarkly identity#121
apucacao wants to merge 6 commits into
mainfrom
fix/native-graph-span-identity

apucacao commented Sep 30, 2026 •

edited

Loading

Uh oh!

apucacao commented Oct 1, 2026

Uh oh!

cursor Bot left a comment •

edited

Loading

Uh oh!

jeffdupont left a comment

Uh oh!

jeffdupont Oct 2, 2026

Uh oh!

jeffdupont Oct 2, 2026

Uh oh!

jeffdupont Oct 2, 2026

Uh oh!

jeffdupont Oct 2, 2026

Uh oh!

apucacao commented Oct 2, 2026

Uh oh!

cursor Bot left a comment •

edited

Loading

Uh oh!

apucacao commented Oct 5, 2026

Uh oh!

cursor Bot left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

apucacao commented Sep 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Testing

Uh oh!

apucacao commented Oct 1, 2026

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jeffdupont left a comment

Choose a reason for hiding this comment

Uh oh!

jeffdupont Oct 2, 2026

Choose a reason for hiding this comment

Uh oh!

jeffdupont Oct 2, 2026

Choose a reason for hiding this comment

Uh oh!

jeffdupont Oct 2, 2026

Choose a reason for hiding this comment

Uh oh!

jeffdupont Oct 2, 2026

Choose a reason for hiding this comment

Uh oh!

apucacao commented Oct 2, 2026

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

apucacao commented Oct 5, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

apucacao commented Sep 30, 2026 •

edited

Loading

cursor Bot left a comment •

edited

Loading

cursor Bot left a comment •

edited

Loading