GPU Performance Monitoring

OpenLIT SDK

Collect and send GPU performance metrics directly from your application to an OpenTelemetry endpoint.

OpenTelemetry GPU Collector

Install the OpenTelemetry GPU Collector as a Docker container to collect and send GPU performance metrics to an OpenTelemetry endpoint.

Using the SDK

Deploy OpenLIT

Git clone OpenLIT repository

git clone git@github.com:openlit/openlit.git

Start Docker Compose

From the root directory of the OpenLIT Repo, Run the below command:

docker compose up -d

Install OpenLIT

Open your command line or terminal and run:

pip install openlit

Initialize OpenLIT in your Application

Not sure which method to choose? Check out Instrumentation Methods to understand the differences.

Zero-code instrumentation
Manual instrumentation

Via CLI arguments
Via Environment Variables

# Start GPU monitoring instantly
openlit-instrument --collect-system-metrics python your_app.py

# With custom settings
openlit-instrument \
  --otlp-endpoint http://127.0.0.1:4318 \
  --service-name my-gpu-app \
  --environment production \
  --collect-system-metrics \
  python your_app.py

Set environment variables:

export OTEL_EXPORTER_OTLP_ENDPOINT="http://127.0.0.1:4318"
export OPENLIT_COLLECT_SYSTEM_METRICS=true
export OTEL_SERVICE_NAME=my-gpu-app

# Run your application
openlit-instrument python your_app.py

You can set up OpenLIT in your application using either function arguments directly in your code or by using environment variables.

Parameters
Environment Variables

Add the following two lines to your application code:

import openlit

openlit.init(
  otlp_endpoint="http:127.0.0.1:4318", 
  collect_system_metrics=True  # This enables GPU monitoring
)

Replace:

YOUR_OTEL_ENDPOINT with the URL of your OpenTelemetry backend, such as http://127.0.0.1:4318 if you are using OpenLIT.

Note: collect_system_metrics=True replaces the deprecated collect_gpu_stats=True

Configure your OTLP endpoint using environment variables:

export OTEL_EXPORTER_OTLP_ENDPOINT=""http://127.0.0.1:4318"
export OPENLIT_COLLECT_SYSTEM_METRICS=true

Add the following two lines to your application code:

import openlit

openlit.init(collect_system_metrics=True)

Replace:

YOUR_OTEL_ENDPOINT with the URL of your OpenTelemetry backend, such as http://127.0.0.1:4318 if you are using OpenLIT.

To send metrics to other Observability tools, refer to the supported destinations.For more advanced configurations and application use cases, visit the OpenLIT Python repository.

Using the Collector

Deploy OpenLIT

Git clone OpenLIT repository

git clone git@github.com:openlit/openlit.git

Start Docker Compose

From the root directory of the OpenLIT Repo, Run the below command:

docker compose up -d

Pull otel-gpu-collector Docker Image

You can quickly start using the OTel GPU Collector by pulling the Docker image:

docker pull ghcr.io/openlit/otel-gpu-collector:latest

Run otel-gpu-collector Docker container

You can quickly start using the OTel GPU Collector by pulling the Docker image: Here’s a quick example showing how to run the container with the required environment variables:

docker run --gpus all --pid=host \
    -e OTEL_SERVICE_NAME='chatbot' \
    -e OTEL_RESOURCE_ATTRIBUTES='deployment.environment=staging' \
    -e OTEL_EXPORTER_OTLP_ENDPOINT="http://127.0.0.1:4318" \
    ghcr.io/openlit/otel-gpu-collector:latest

--pid=host is required for per-process GPU attribution (cmdline, PID, zombie state).For more advanced configurations of the collector, visit the OTel GPU Collector repository.Note: If you’ve deployed OpenLIT using Docker Compose, make sure to use the host’s IP address or add OTel GPU Collector to the Docker Compose:

Docker Compose: Add the following config under services

otel-gpu-collector:
  image: ghcr.io/openlit/otel-gpu-collector:latest
  pid: host
  environment:
    OTEL_SERVICE_NAME: 'chatbot'
    OTEL_RESOURCE_ATTRIBUTES: 'deployment.environment=staging'
    OTEL_EXPORTER_OTLP_ENDPOINT: "http://openlit:4318"
  device_requests:
  - driver: nvidia
    count: all
    capabilities: [gpu]
  depends_on:
  - openlit
  restart: always

Host IP: Use the Host IP to connect to OpenLIT

OTEL_EXPORTER_OTLP_ENDPOINT="http://192.168.10.15:4318"

Integrations

60+ AI integrations with automatic instrumentation and performance tracking

Create a dashboard

Create custom visualizations with flexible widgets, queries, and real-time AI monitoring

Manage prompts

Version, deploy, and collaborate on prompts with centralized management and tracking

Getting Started

Concepts

Observability

Evaluations

Dashboards

Otter (AI Copilot)

Pricing & Models

Prompts and Experiments

Developer Resources

GPU Performance Monitoring

Get started

OpenLIT SDK

OpenTelemetry GPU Collector

Integrations

Create a dashboard

Manage prompts

​Get started

OpenLIT SDK

OpenTelemetry GPU Collector

Integrations

Create a dashboard

Manage prompts

Get started