Sentry Setup AI Monitoring

All Skills > Feature Setup > AI Monitoring

Setup Sentry AI Agent Monitoring

Configure Sentry to track LLM calls, agent executions, tool usage, and token consumption.

Invoke This Skill When

• User asks to "monitor AI/LLM calls" or "track OpenAI/Anthropic usage"
• User wants "AI observability" or "agent monitoring"
• User asks about token usage, model latency, or AI costs

Important: The SDK versions, API names, and code samples below are examples. Always verify against docs.sentry.io before implementing, as APIs and minimum versions may have changed.

Prerequisites

AI monitoring requires tracing enabled (tracesSampleRate > 0).

If the app has multi-turn chats, set a conversation ID by default anywhere it makes sense to identify a chat session. Sentry uses gen_ai.conversation.id to group related AI spans into Conversations. Some integrations infer it automatically, but many setups need to set it explicitly.

Data Capture Warning

Prompt and output recording captures user content that is likely PII. In JavaScript, genAI input/output capture is on by default (governed by dataCollection.genAI); in Python it is enabled via send_default_pii=True; in Laravel it is enabled via SENTRY_SEND_DEFAULT_PII=true. Before relying on this capture (or per-integration overrides — recordInputs/recordOutputs in JS, include_prompts in Python), confirm:

• The application's privacy policy permits capturing user prompts and model responses
• Captured data complies with applicable regulations (GDPR, CCPA, etc.)
• Sentry data retention settings are appropriate for the sensitivity of the data

Ask the user whether they want prompt/output capture enabled. Do not enable prompt/output capture without explicit confirmation. Use tracesSampleRate: 1.0 only in development; in production, use a lower value or a tracesSampler function.

Detection First

Always detect installed AI SDKs before configuring:

# JavaScript
grep -E '"(openai|@anthropic-ai/sdk|ai|@langchain|@google/genai)"' package.json

# Python
grep -E '(openai|anthropic|langchain|huggingface)' requirements.txt pyproject.toml 2>/dev/null

# PHP / Laravel
grep -E '"(laravel/ai|openai-php|openai/|anthropic|llm)' composer.json 2>/dev/null
ls artisan 2>/dev/null && echo "Laravel detected"

Sampling Check

After detecting AI SDKs, check the current sampling configuration:

# JavaScript
grep -E 'tracesSampleRate|tracesSampler' sentry.*.config.* instrument.* src/instrument.* app/instrument.* 2>/dev/null

# Python
grep -E 'traces_sample_rate|traces_sampler' *.py **/*.py 2>/dev/null

# PHP / Laravel
grep -E 'SENTRY_TRACES_SAMPLE_RATE|traces_sample_rate|traces_sampler' .env config/sentry.php 2>/dev/null

If `tracesSampleRate` / `traces_sample_rate` is below 1.0 AND no `tracesSampler` / `traces_sampler` is configured:

Ask the user:

"Your current sample rate is {rate}. Agent runs are sampled as complete span trees — if the root span is dropped, all child gen_ai spans are lost. For full AI visibility, gen_ai-related transactions should be sampled at 100%. Would you like me to set up a tracesSampler that keeps AI traces at 100% while sampling other traffic at your current rate?"

If user confirms, read ${SKILL_ROOT}/references/sampling.md for implementation patterns.

Supported SDKs

JavaScript

*Vercel AI: 10.53.0+ required. Requires experimental_telemetry per-call.

Python

Integrations auto-enable when the AI package is installed — no explicit registration needed:

PHP / Laravel

Laravel AI support requires Laravel 12.x or later, sentry/sentry-laravel 4.27.0 or later, and tracing enabled.

JavaScript Configuration

Node.js — auto-enabled integrations

Just ensure tracing is enabled. Integrations auto-enable when the AI package is installed:

Sentry.init({
  dsn: "YOUR_DSN",
  tracesSampleRate: 1.0, // Lower in production (e.g., 0.1)
  // OpenAI, Anthropic, Google GenAI, LangChain integrations auto-enable in Node.js
});

To customize (e.g., enable prompt capture after user confirmation — see Data Capture Warning):

Sentry.init({
  dsn: "YOUR_DSN",
  tracesSampleRate: 1.0,
  dataCollection: {
    // To disable sending user data and HTTP bodies, uncomment the lines below. For more info visit:
    // https://docs.sentry.io/platforms/javascript/configuration/options/#dataCollection
    // userInfo: false,
    // httpBodies: [],
  },
  integrations: [
    Sentry.openAIIntegration({
      // recordInputs/recordOutputs default to true (governed by dataCollection.genAI)
    }),
  ],
});

Cloudflare Workers (no runtime patching)

The Workers runtime (workerd) does not support monkey-patching, so Node.js-style auto-instrumentation does not apply. Workers AI (env.AI) is auto-instrumented by withSentry (v10.67.0+); openai, @anthropic-ai/sdk, @google/genai, and ai need either the build-time Sentry Cloudflare Vite plugin (v10.68.0+, experimental) or manual client wrapping; LangChain/LangGraph are manual-only. Read ${SKILL_ROOT}/../../references/sdks/cloudflare/ai-monitoring.md for the full setup.

Browser / Next.js OpenAI (manual wrapping required)

In browser-side code or Next.js meta-framework apps, auto-instrumentation is not available. Wrap the client manually:

import OpenAI from "openai";
import * as Sentry from "@sentry/nextjs"; // or @sentry/react, @sentry/browser

const openai = Sentry.instrumentOpenAiClient(new OpenAI());
// Use 'openai' client as normal

LangChain / LangGraph (auto-enabled)

Sentry.init({
  dsn: "YOUR_DSN",
  tracesSampleRate: 1.0,
  dataCollection: {
    // To disable sending user data and HTTP bodies, uncomment the lines below. For more info visit:
    // https://docs.sentry.io/platforms/javascript/configuration/options/#dataCollection
    // userInfo: false,
    // httpBodies: [],
  },
  integrations: [
    Sentry.langChainIntegration(),
    Sentry.langGraphIntegration(),
  ],
});

Vercel AI SDK

Add to sentry.edge.config.ts for Edge runtime:

Sentry.init({
  dsn: "YOUR_DSN",
  tracesSampleRate: 1.0,
  dataCollection: {
    // To disable sending user data and HTTP bodies, uncomment the lines below. For more info visit:
    // https://docs.sentry.io/platforms/javascript/configuration/options/#dataCollection
    // userInfo: false,
    // httpBodies: [],
  },
  integrations: [Sentry.vercelAIIntegration()],
});

Enable telemetry per-call:

await generateText({
  model: openai("gpt-4o"),
  prompt: "Hello",
  experimental_telemetry: {
    isEnabled: true,
    recordInputs: true,
    recordOutputs: true,
  },
});

Python Configuration

Integrations auto-enable — just init with tracing. Only add explicit imports to customize options:

import sentry_sdk