Monitor NVIDIA GPUs using OpenTelemetry

Using the SDK

Collect and send GPU performance metrics directly from your application to an OpenTelemetry endpoint.

Using the Collector

Install the OpenTelemetry GPU Collector as a Docker container to collect and send GPU performance metrics to an OpenTelemetry endpoint.

Using the SDK

Install OpenLIT

Open your command line or terminal and run:

pip install openlit

Initialize OpenLIT in your Application

Python
Typescript

Zero Code Instrumentation
One-Line Instrumentation

Perfect for existing applications - no code modifications needed:

Via CLI Arguments
Via Environment Variables

# Configure via CLI arguments
openlit-instrument \
  --service-name my-ai-app \
  --environment production \
  --otlp-endpoint YOUR_OTEL_ENDPOINT \
  python your_app.py

# Configure via environment variables
export OTEL_SERVICE_NAME=my-ai-app
export OTEL_DEPLOYMENT_ENVIRONMENT=production
export OTEL_EXPORTER_OTLP_ENDPOINT=YOUR_OTEL_ENDPOINT

# Run with zero code changes
openlit-instrument python your_app.py

Perfect for: Legacy applications, production systems where code changes need approval, quick testing, or when you want to add observability without touching existing code.

Via Function Parameters
Via Environment Variables

import openlit

openlit.init(otlp_endpoint="YOUR_OTEL_ENDPOINT")

Add the following two lines to your application code:

import openlit

openlit.init()

Then, configure the your OTLP endpoint using environment variable:

export OTEL_EXPORTER_OTLP_ENDPOINT=YOUR_OTEL_ENDPOINT

One-Line Instrumentation (SDK)

Via Function Parameters
Via Environment Variables

import openlit from "openlit"

openlit.init({ otlpEndpoint: "YOUR_OTEL_ENDPOINT" })

Add the following two lines to your application code:

import openlit from "openlit"

openlit.init()

Then, configure the your OTLP endpoint using environment variable:

export OTEL_EXPORTER_OTLP_ENDPOINT=YOUR_OTEL_ENDPOINT

Replace: YOUR_OTEL_ENDPOINT with the URL of your OpenTelemetry backend, such as http://127.0.0.1:4318 if you are using OpenLIT and a local OTel Collector.To send metrics and traces to other Observability tools, refer to the supported destinations.For more advanced configurations and application use cases, visit the OpenLIT Python repository or OpenLIT Typescript repository.

Using the Collector

Pull `otel-gpu-collector` Docker Image

You can quickly start using the OTel GPU Collector by pulling the Docker image:

docker pull ghcr.io/openlit/otel-gpu-collector:latest

Run `otel-gpu-collector` Docker container

You can quickly start using the OTel GPU Collector by pulling the Docker image: Here’s a quick example showing how to run the container with the required environment variables:

docker run --gpus all --pid=host \
    -e OTEL_SERVICE_NAME='chatbot' \
    -e OTEL_RESOURCE_ATTRIBUTES='deployment.environment=staging' \
    -e OTEL_EXPORTER_OTLP_ENDPOINT="YOUR_OTEL_ENDPOINT" \
    -e OTEL_EXPORTER_OTLP_HEADERS="YOUR_OTEL_HEADERS" \
    ghcr.io/openlit/otel-gpu-collector:latest

--pid=host is required for per-process GPU attribution (cmdline, PID, zombie state).For more advanced configurations of the collector, visit the OTel GPU Collector repository.Note: If you’ve deployed OpenLIT using Docker Compose, make sure to use the host’s IP address or add OTel GPU Collector to the Docker Compose:

Docker Compose: Add the following config under `services`

otel-gpu-collector:
  image: ghcr.io/openlit/otel-gpu-collector:latest
  pid: host
  environment:
    OTEL_SERVICE_NAME: 'chatbot'
    OTEL_RESOURCE_ATTRIBUTES: 'deployment.environment=staging'
    OTEL_EXPORTER_OTLP_ENDPOINT: "http://otel-collector:4318"
  device_requests:
  - driver: nvidia
    count: all
    capabilities: [gpu]
  depends_on:
  - otel-collector
  restart: always

Host IP: Use the Host IP to connect to OTel Collector

OTEL_EXPORTER_OTLP_ENDPOINT="http://192.168.10.15:4318"

Environment Variables

OTel GPU Collector uses standard OpenTelemetry environment variables for configuration:

Environment Variable	Description	Default Value
`OTEL_SERVICE_NAME`	Service/application name attached to all metrics	`default`
`OTEL_RESOURCE_ATTRIBUTES`	Resource attributes, e.g. `deployment.environment=production`	`deployment.environment=default`
`OTEL_EXPORTER_OTLP_ENDPOINT`	OpenTelemetry OTLP endpoint URL	(required)
`OTEL_EXPORTER_OTLP_HEADERS`	Headers for authenticating with the OTLP endpoint	Ignore if using OpenLIT
`OTEL_METRIC_EXPORT_INTERVAL`	Metric polling interval in milliseconds	`10000`

Collected Metrics

Details on the types of metrics collected and their descriptions.

Quickstart: LLM Observability

Production-ready AI monitoring setup in 2 simple steps with zero code changes

Configuration

Configure the OpenLIT SDK according to you requirements.

Destinations

Send telemetry to Datadog, Grafana, New Relic, and other observability stacks

Getting Started

Core Features

Integrations

Destinations

Monitor NVIDIA GPUs using OpenTelemetry

Get started

Using the SDK

Using the Collector

Environment Variables

Collected Metrics

Quickstart: LLM Observability

Configuration

Destinations

​Get started

Using the SDK

Using the Collector

​Environment Variables

Collected Metrics

Quickstart: LLM Observability

Configuration

Destinations

Get started

Environment Variables