Monitor llmman using OpenTelemetry

OpenLIT uses OpenTelemetry Auto-Instrumentation to help you monitor LLM applications built using models served by llmman. This includes tracking performance, token usage, and how users interact with the application. llmman is a local model runner that serves the Ollama API (alongside OpenAI- and Anthropic-compatible ones) on port 17434. There is no dedicated llmman client library: applications talk to it through the Ollama or OpenAI SDKs, so OpenLIT’s existing Ollama and OpenAI instrumentations pick it up automatically once the client is pointed at http://localhost:17434. The integration is compatible with

Ollama Python SDK client >=0.2.0
Ollama TypeScript/JavaScript SDK client >=0.5.0
OpenAI Python SDK client >=1.92.0
OpenAI TypeScript SDK client >=1.13.0

Get started

Install OpenLIT

Open your command line or terminal and run:

Python
TypeScript

pip install openlit

npm install openlit

Initialize OpenLIT in your Application

Python
Typescript

Zero Code Instrumentation
One-Line Instrumentation

Perfect for existing applications - no code modifications needed:

Via CLI Arguments
Via Environment Variables

# Configure via CLI arguments
openlit-instrument \
  --service-name my-ai-app \
  --environment production \
  --otlp-endpoint YOUR_OTEL_ENDPOINT \
  python your_app.py

# Configure via environment variables
export OTEL_SERVICE_NAME=my-ai-app
export OTEL_DEPLOYMENT_ENVIRONMENT=production
export OTEL_EXPORTER_OTLP_ENDPOINT=YOUR_OTEL_ENDPOINT

# Run with zero code changes
openlit-instrument python your_app.py

Perfect for: Legacy applications, production systems where code changes need approval, quick testing, or when you want to add observability without touching existing code.

Via Function Parameters
Via Environment Variables

import openlit

openlit.init(otlp_endpoint="YOUR_OTEL_ENDPOINT")

Add the following two lines to your application code:

import openlit

openlit.init()

Then, configure the your OTLP endpoint using environment variable:

export OTEL_EXPORTER_OTLP_ENDPOINT=YOUR_OTEL_ENDPOINT

One-Line Instrumentation (SDK)

Via Function Parameters
Via Environment Variables

import openlit from "openlit"

openlit.init({ otlpEndpoint: "YOUR_OTEL_ENDPOINT" })

Add the following two lines to your application code:

import openlit from "openlit"

openlit.init()

Then, configure the your OTLP endpoint using environment variable:

export OTEL_EXPORTER_OTLP_ENDPOINT=YOUR_OTEL_ENDPOINT

Replace: YOUR_OTEL_ENDPOINT with the URL of your OpenTelemetry backend, such as http://127.0.0.1:4318 if you are using OpenLIT and a local OTel Collector.To send metrics and traces to other Observability tools, refer to the supported destinations.For more advanced configurations and application use cases, visit the OpenLIT Python repository or OpenLIT Typescript repository.

Point your client at llmman

Start the server with llmman serve and pull a model with llmman pull gemma4. Then configure the SDK you already use to talk to http://localhost:17434 instead of Ollama’s default http://localhost:11434. No API key is required; if a client insists on one, any placeholder works.

import ollama
import openlit

openlit.init(otlp_endpoint="http://127.0.0.1:4318")

client = ollama.Client(host="http://localhost:17434")
response = client.chat(model="gemma4", messages=[
    {"role": "user", "content": "Why is the sky blue?"},
])

from openai import OpenAI
import openlit

openlit.init(otlp_endpoint="http://127.0.0.1:4318")

client = OpenAI(base_url="http://localhost:17434/v1", api_key="llmman")
response = client.chat.completions.create(
    model="gemma4",
    messages=[{"role": "user", "content": "Why is the sky blue?"}],
)

import Openlit from "openlit"

Openlit.init({ otlpEndpoint: "http://127.0.0.1:4318" })

async function main() {
  const { Ollama } = await import("ollama");
  const ollama = new Ollama({ host: "http://localhost:17434" });
  const response = await ollama.chat({
    model: "gemma4",
    messages: [{ role: "user", content: "Why is the sky blue?" }],
  });

  console.log(response);
}

main();

import Openlit from "openlit"

Openlit.init({ otlpEndpoint: "http://127.0.0.1:4318" })

async function main() {
  const OpenAI = await import("openai").then((e) => e.default);
  const client = new OpenAI({ baseURL: "http://localhost:17434/v1", apiKey: "llmman" });
  const response = await client.chat.completions.create({
    model: "gemma4",
    messages: [{ role: "user", content: "Why is the sky blue?" }],
  });

  console.log(response);
}

main();

The Ollama SDKs also honour OLLAMA_HOST, so export OLLAMA_HOST=127.0.0.1:17434 switches an existing Ollama application to llmman without code changes.

Quickstart: LLM Observability

Production-ready AI monitoring setup in 2 simple steps with zero code changes

Configuration

Configure the OpenLIT SDK according to you requirements.

Destinations

Send telemetry to Datadog, Grafana, New Relic, and other observability stacks

Getting Started

Core Features

Integrations

Destinations

Monitor llmman using OpenTelemetry

Get started

Quickstart: LLM Observability

Configuration

Destinations

​Get started

Quickstart: LLM Observability

Configuration

Destinations

Get started