Built In Metrics

Agent Metrics Instrumentation

You're using a skill that wires LaunchDarkly agent metrics around an existing provider call. Your job is to audit what's already there, pick the right tier from the ladder below, and implement it with the least ceremony that still captures the metrics the Monitoring tab needs (duration, input/output tokens, success/error, plus TTFT when streaming).

The single most important thing to get right: default to the highest tier that fits the shape of the call. Going lower ("just write the manual tracker calls") looks flexible but costs you drift, missed metrics, and legacy patterns the SDKs have moved past.

The four-tier ladder

This is the order the official SDK READMEs (Python core, Node core, and every provider package) recommend. Walk from the top and stop at the first tier that fits:

Every provider — OpenAI, LangChain, Vercel, Bedrock, Anthropic, Gemini, custom HTTP — uses the same generic shape: tracker.trackMetricsOf(getAIMetricsFromResponse, () => providerCall()) in Node, tracker.track_metrics_of(get_ai_metrics_from_response, provider_call) in Python. The extractor is the only thing that changes per provider: import getAIMetricsFromResponse from the matching @launchdarkly/server-sdk-ai-<provider> (or ldai_<provider>) package, or write a small custom function that returns LDAIMetrics. There are no provider-specific tracker methods.

Workflow

1. Explore the existing call site

Before picking a tier, find the provider call and answer these questions:

• [ ] Shape? Is it a chat loop (history + turn-based), a one-shot completion, an agent step, or something else? → drives Tier 1 vs 2.
• [ ] Framework? Raw provider SDK? LangChain / LangGraph? Vercel AI SDK? CrewAI? Strands? → drives which Tier-2 provider package (if any) applies.
• [ ] Provider? OpenAI, Anthropic, Bedrock, Gemini, Azure, custom HTTP? → cross-reference with the package availability matrix below.
• [ ] Streaming? If yes, you'll need TTFT tracking, which means Tier 4 for the TTFT part even if the rest is Tier 2.
• [ ] Language? Python or Node? Provider-package coverage differs between them.
• [ ] Already using a config? If not, route to configs-create first — tracking requires a tracker, which is obtained by calling create_tracker() / createTracker() on the config object returned by completion_config() / completionConfig() / createModel().
• [ ] On the current SDK API? If the call site uses aiclient.config(...) / aiClient.config(...) or constructs an AIConfig(...) / LDAIConfig default, it's on the pre-0.20 surface. Migrate it as part of this work before adding tracking:
• aiclient.config(...) → aiclient.completion_config(...) for one-shot/chat or aiclient.agent_config(...) for agent mode (mirror the call signature). Node is the same with camelCase.
• AIConfig(...) default → AICompletionConfigDefault(...) or AIAgentConfigDefault(...) (Node: LDAICompletionConfigDefault / LDAIAgentConfigDefault). AIConfig is the base class the SDK returns; it isn't a valid default-value constructor — the typed *Default variants are.
• If the result was being tuple-unpacked (config, tracker = aiclient.config(...)), drop the unpack — the new methods return a single config object. Obtain the tracker via config.create_tracker() / aiConfig.createTracker().
• For deeper rewrites (call sites with hardcoded model/prompt as well), hand off to migrate instead of doing the full migration here.

2. Look up your Tier-2 option

Use this matrix to decide whether Tier 2 (provider package) is available for your situation. If it's not, drop to Tier 3 (custom extractor). If the shape is chat-loop, go to Tier 1 first regardless of what's in this matrix.

3. Implement from the matching reference

Once you know the tier and the provider, open the reference file and follow the pattern. The references are written so Tier 1 is always the first example, Tier 2/3 next, and Tier 4 last. Stop at the first tier that matches the app's shape.

Guardrails that apply to every tier:

1. Always check `config.enabled` before making the tracked call. A disabled config means the user has flagged the feature off — you should short-circuit to whatever fallback the app uses (cached response, error, degraded path) rather than making the provider call at all.
2. Wrap the existing call, don't rewrite it. Tier 2 and Tier 3 are designed to slot around an unmodified provider call. If you find yourself rewriting the call to fit the tracker, you're at the wrong tier — drop down one.
3. Errors are handled inside `trackMetricsOf`. The wrapper catches exceptions, records trackError() internally, and re-raises — do not add except: tracker.trackError() on top, it's a noop that also trips the at-most-once guard. Tier 1 handles both paths automatically. At Tier 4 (manual, streaming, track_duration_of) the caller does own the error-tracking call.