OpenLIT Overview

LLM observability and AI agent observability for agents built with CrewAI, LangGraph, or any agent framework. OpenLIT brings OpenTelemetry-native instrumentation so you can analyze how your agents perform in production with full stack visibility:

Tracing - Instrument once via OpenTelemetry-native SDKs for 90+ LLMs and agent frameworks to produce OpenTelemetry traces.
Lowest level transparency - Exceptions Monitoring surfaces errors with full stack traces and span context so you know exactly where and why something failed. All telemetry is automatically redacted for sensitive data before it leaves your stack.
Understand cost and latency - Track token consumption, spend per model, provider, and request. Monitor GPU utilization alongside LLM call latency in custom dashboards built on your raw telemetry.

Prompt management, prompt versioning, and model testing tools for teams building AI applications:

Prompt versioning and deployment - Prompt Hub treats prompts as versioned artifacts. Edit, version, and deploy to any environment without code changes. Roll back instantly when needed.
Collaborate on prompts - Edit prompts as a team with full change history. Every version is tracked and linked to the evaluations and traces it produced, giving you full traceability from prompt to output.
Model comparison and A/B testing - OpenGround runs side-by-side LLM prompt testing across models, comparing cost, latency, and output quality so you pick the right model before you ship.

Coding agent observability for Claude Code, Cursor, and Codex. Understand token consumption, AI spend, and developer productivity across your entire engineering team:

One-command setup - Install the openlit CLI and wire any vendor in under a minute. No SDK, no code changes in your repos.
Session monitoring - Track cost per session, token usage, lines added and removed, model acceptance rate, commits, and PRs per user and per team.
AI spend visibility - See exactly how much each developer, team, and project spends on AI coding tools. Break down cost by model and vendor across Claude Code, Cursor, and Codex in one place.
Privacy controls - Automatic secret redaction, usage breakdowns by repo, and cohort-floor anonymization so individual data stays protected.

LLM evaluation in production and development using the same evaluators. Close the loop between what you ship and what you measure:

Online LLM evaluation - LLM-as-a-judge runs automatically on your live traces in production. Create custom evaluators with your own scoring criteria and quality gates. No setup required.
Offline evaluation and CI/CD testing - Use the same evaluators via the programmatic evaluations API for regression testing in CI pipelines and local development. Catch quality regressions before they reach production.
Traceability - Every evaluation score is linked back to the exact prompt version, model, and trace that produced it so you know what changed and why quality shifted.

Manage the agents, keys, and pricing behind your AI stack from one place:

Track and manage agents - See every agent version running in production, monitor health, and manage deployments from the /agents page.
Vault - Centrally store and rotate LLM API keys that agents retrieve at runtime. No restarts, no code changes, no secrets in your codebase.
AI cost tracking - Costs under Monitor combines spend analytics, Manage models, and Auto Pricing. OpenLIT ships with pricing for popular models and applies it across traces, dashboards, and cost reports.

Otter is a built-in AI copilot that works across all of OpenLIT. Query your telemetry, analyze traces, improve prompts, and generate dashboards through natural language:

Natural language queries - Ask questions about traces, costs, and tokens. Otter converts them to SQL and runs them inline.
Trace and span analysis - Get structured AI analysis across performance, reliability, cost, token efficiency, and execution path.
Prompt improvement - Ask Otter to review and suggest improvements to any Prompt Hub prompt.
Dashboard generation - Describe the dashboard you want and Otter generates a complete importable layout with working queries.
Resource management - Create rules, contexts, vault secrets, and custom models through conversation.

Getting Started

Monitor

Develop

Dashboards

Otter (AI Copilot)

Platform

Resources

OpenLIT Overview

Get Started

Instrument an AI Agent

Observe coding agents

Deploy OpenLIT

Evaluate LLM responses

Frequently asked questions

​Get Started