Get started with AI Observability

OpenLIT automatically instruments LLMs, VectorDBs, MCP, and frameworks by default.

Get real-time cost tracking, token usage monitoring, and latency optimization for your AI applications.

Select where your AI application is running

Kubernetes
Docker
Linux
Local Development

Deploy OpenLIT with the Controller

Deploy OpenLIT and the Controller together using the Helm chart. The Controller automatically discovers your services and lets you enable LLM observability from the dashboard — no code changes needed.

helm repo add openlit https://openlit.github.io/helm/
helm repo update
helm install openlit openlit/openlit --set openlit-controller.enabled=true

The Controller’s DaemonSet, RBAC, and configuration are all handled by the chart. The dashboard URL and OTLP endpoint are auto-derived from the Helm release name.

If you’ve created an API key in Settings → API Keys, pass it to the Controller so it authenticates with the dashboard:

helm install openlit openlit/openlit \
  --set openlit-controller.enabled=true \
  --set openlit-controller.apiKey="YOUR_OPENLIT_API_KEY"

You can also set the OPENLIT_API_KEY environment variable. See Controller configuration for details.

For detailed configuration options, see the Installation guide and Controller configuration.

Enable observability from the Agents page

Open the OpenLIT dashboard and navigate to Agents
Your services making LLM API calls will appear automatically
Click Enable next to any service to start collecting traces and metrics

The Controller uses eBPF for zero-code LLM observability and can also inject the OpenLIT Python SDK for agent framework traces — no code changes, image rebuilds, or redeployments required. For deeper application-level tracing, you can also use the OpenLIT SDKs.

Deploy OpenLIT with Docker Compose

git clone git@github.com:openlit/openlit.git

From the root directory of the OpenLIT Repo, add the Controller service to your docker-compose.yml:

openlit-controller:
  image: ghcr.io/openlit/controller:latest
  privileged: true
  pid: "host"
  volumes:
    - /proc:/host/proc:ro
    - /sys/kernel/debug:/sys/kernel/debug:ro
    - /sys/fs/bpf:/sys/fs/bpf:rw
    - /var/run/docker.sock:/var/run/docker.sock
  environment:
    OPENLIT_URL: "http://openlit:3000"
    OPENLIT_PROC_ROOT: "/host/proc"
    OTEL_EXPORTER_OTLP_ENDPOINT: "http://openlit:4318"
    # OPENLIT_API_KEY: "your-api-key"  # Recommended: see tip below
  depends_on:
    - openlit

Then start everything:

docker compose up -d

If you’ve created an API key in Settings → API Keys, add it to the Controller’s environment to authenticate requests:

environment:
  OPENLIT_API_KEY: "your-openlit-api-key"

See Controller configuration for all available options.

Enable observability from the Agents page

Open the OpenLIT dashboard at http://127.0.0.1:3000
Navigate to Agents
Your containers making LLM API calls will appear automatically
Click Enable next to any service to start collecting traces and metrics

The Controller uses eBPF for zero-code LLM observability and can also inject the OpenLIT Python SDK for agent framework traces — no code changes or image rebuilds required. For deeper application-level tracing, you can also use the OpenLIT SDKs.

Deploy OpenLIT

git clone git@github.com:openlit/openlit.git

From the root directory of the OpenLIT Repo:

docker compose up -d

Install the OpenLIT Controller

x86_64 (amd64)
ARM64 (aarch64)

curl -fsSL https://github.com/openlit/openlit/releases/latest/download/openlit-controller-linux-amd64 \
  -o /usr/local/bin/openlit-controller
chmod +x /usr/local/bin/openlit-controller

curl -fsSL https://github.com/openlit/openlit/releases/latest/download/openlit-controller-linux-arm64 \
  -o /usr/local/bin/openlit-controller
chmod +x /usr/local/bin/openlit-controller

Verify the installation:

openlit-controller --version

Create a systemd service with the configuration passed as environment variables:

/etc/systemd/system/openlit-controller.service

[Unit]
Description=OpenLIT Controller
After=network.target

[Service]
Environment="OPENLIT_URL=http://your-openlit-host:3000"
Environment="OTEL_EXPORTER_OTLP_ENDPOINT=http://your-openlit-host:4318"
Environment="OPENLIT_POLL_INTERVAL=60s"
Environment="OPENLIT_ENVIRONMENT=production"
# Environment="OPENLIT_API_KEY=your-api-key"  # Recommended: see tip below
ExecStart=/usr/local/bin/openlit-controller
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target

systemctl daemon-reload
systemctl enable --now openlit-controller

If you’ve created an API key in Settings → API Keys, add it to the systemd service to authenticate Controller requests:

Environment="OPENLIT_API_KEY=your-openlit-api-key"

Then reload: systemctl daemon-reload && systemctl restart openlit-controller

Enable observability from the Agents page

Open the OpenLIT dashboard and navigate to Agents
Your processes making LLM API calls will appear automatically
Click Enable next to any service to start collecting traces and metrics

The Controller uses eBPF for zero-code LLM observability and can also inject the OpenLIT Python SDK for agent framework traces — no code changes required. For deeper application-level tracing, you can also use the OpenLIT SDKs.

Deploy OpenLIT

git clone git@github.com:openlit/openlit.git

From the root directory of the OpenLIT Repo:

docker compose up -d

Install the OpenLIT SDK

Python
Typescript

pip install openlit

npm install openlit

Instrument your AI application

Python
Typescript

Not sure which method to choose? Check out Instrumentation Methods to understand the differences.

Zero-code instrumentation
Manual instrumentation

openlit-instrument \
  --service-name my-ai-app \
  --environment development \
  --otlp-endpoint http://127.0.0.1:4318 \
  python your_app.py

import openlit

openlit.init(otlp_endpoint="http://127.0.0.1:4318")

Examples:

from openai import OpenAI
import openlit

openlit.init(otlp_endpoint="http://127.0.0.1:4318")

client = OpenAI(api_key="YOUR_OPENAI_KEY")

chat_completion = client.chat.completions.create(
    messages=[
        {
            "role": "user",
            "content": "What is LLM Observability?",
        }
    ],
    model="gpt-4o-mini",
)

import Openlit from "openlit"

Openlit.init({ otlpEndpoint: "http://127.0.0.1:4318" })

Example with OpenAI:

import Openlit from "openlit"

Openlit.init({ otlpEndpoint: "http://127.0.0.1:4318" })

async function main() {
    const OpenAI = await import("openai").then((e) => e.default);
    const openai = new OpenAI({ apiKey: YOUR_OPENAI_KEY });
    const completion = await openai.chat.completions.create({
        model: "gpt-4o-mini",
        messages: [{ role: "system", content: "Who are you?" }],
    });
    console.log(completion?.choices?.[0]);
}

main();

Refer to OpenLIT Python SDK repository or Typescript SDK repository for more advanced configurations and use cases.

Monitor and optimize your AI applications

With real-time LLM observability data now flowing to OpenLIT, visualize comprehensive AI performance metrics including token costs, latency patterns, hallucination rates, and model accuracy to optimize your production AI applications. Head over to OpenLIT at 127.0.0.1:3000 on your browser to start exploring. You can login using the default credentials:

Email: user@openlit.io
Password: openlituser

If you have any questions or need support, reach out to our community.

Quickstart: LLM Evaluations

Get started with evaluating your LLM responses in 2 simple steps

Integrations

60+ AI integrations with automatic instrumentation and performance tracking

Create a dashboard

Create custom visualizations with flexible widgets, queries, and real-time AI monitoring

Getting Started

Observability

Evaluations

Dashboards

Otter (AI Copilot)

Pricing & Models

Prompts and Experiments

Developer Resources

Get started with AI Observability

Select where your AI application is running

Deploy OpenLIT with the Controller

Deploy OpenLIT with Docker Compose

Deploy OpenLIT

Install the OpenLIT Controller

Deploy OpenLIT

Monitor and optimize your AI applications

Quickstart: LLM Evaluations

Integrations

Create a dashboard

Getting Started

Observability

Evaluations

Dashboards

Otter (AI Copilot)

Pricing & Models

Prompts and Experiments

Developer Resources

Documentation Index

​Select where your AI application is running

Deploy OpenLIT with the Controller

Deploy OpenLIT with Docker Compose

Deploy OpenLIT

Install the OpenLIT Controller

Deploy OpenLIT

​Monitor and optimize your AI applications

Quickstart: LLM Evaluations

Integrations

Create a dashboard

Select where your AI application is running

Monitor and optimize your AI applications