OpenLIT automatically instruments LLMs, VectorDBs, MCP, and frameworks by default.
This guide demonstrates how to implement real-time cost tracking, token usage monitoring, hallucination detection, and latency optimization for your AI applications with OpenTelemetry traces and metrics.
1

Deploy OpenLIT

1

Git clone OpenLIT repository

git clone git@github.com:openlit/openlit.git
2

Start Docker Compose

From the root directory of the OpenLIT Repo, Run the below command:
docker compose up -d
2

Install OpenLIT SDK

Kubernetes

Running in Kubernetes? Try the OpenLIT Operator

Automatically inject instrumentation into existing workloads without modifying pod specs, container images, or application code.
pip install openlit
3

Instrument your AI application

Not sure which method to choose? Check out Instrumentation Methods to understand the differences.
# Install OpenLIT
pip install openlit

# Configure via CLI arguments
openlit-instrument \
  --service-name my-ai-app \
  --environment production \
  --otlp-endpoint http://127.0.0.1:4318 \
  python your_app.py
Refer to OpenLIT Python SDK repository or Typescript SDK repository for more advanced configurations and use cases.
4

Monitor, debug and test the quality of your AI applications

With real-time LLM observability data now flowing to OpenLIT, visualize comprehensive AI performance metrics including token costs, latency patterns, hallucination rates, and model accuracy to optimize your production AI applications.Just head over to OpenLIT at 127.0.0.1:3000 on your browser to start exploring. You can login using the default credentials
  • Email: user@openlit.io
  • Password: openlituser
You’re all set! Your AI applications now have observability with real-time performance monitoring, cost tracking, and AI safety evaluations. Send Observability telemetry to other OpenTelemetry backends If you wish to send telemetry directly from the SDK to another backend, you can stop the current Docker services by using the command below. For more details on sending the data to your existing OpenTelemetry backends, checkout our Supported Destinations guide. 
docker compose down
If you have any questions or need support, reach out to our community.

Quickstart: LLM Evaluations

Get started with evaluating your LLM responses in 2 simple steps

Integrations

60+ AI integrations with automatic instrumentation and performance tracking

Destinations

Send telemetry to Datadog, Grafana, New Relic, and other observability stacks