Authentication

GitHub

Slack

Discord

OpenLIT

OpenAPI specification for API Endpoints in OpenLIT

Introduction

What is OpenLIT?

Quickly start monitoring your AI Applications in just a single line of code

Quickstart: AI Observability

Get started with AI Observability

Quickly evaluate your model responses for Hallucination, Bias, and Toxicity

Quickstart: Evaluations

Get started with Evaluations

Quickly secure your app from Prompt Injection, Sensitive Topics, and Topic Restriction

Quickstart: Guardrails

Get started with Guardrails

Self-host OpenLIT in your preferred environment

Self-hosting

SDK Configuration

UI Configuration

Overview

Automatically track your AI apps with OpenTelemetry to gain insights into performance and behavior

Tracing

Gather valuable metrics for your AI apps using OpenTelemetry auto instrumentation

Metrics

Monitor NVIDIA GPUs with key metrics like usage, temperature, and power using OpenTelemetry

GPU Monitoring

GPU Performance Monitoring

Use your own Pricing File to calculate LLM usage costs

Track Cost for Custom Models

Manage prompts centrally, fetch versions, and use variables for dynamic prompts

Prompt Hub

Store and access sensitive information like LLM API keys securely

Vault

Secure your app from Prompt Injection, Sensitive Topics, and Topic Restriction

Guardrails

Evaluate your model responses for Hallucination, Bias, and Toxicity

Programmatic Evaluations

Record Exceptions Automatically and See detailed Stack Traces

Exceptions Monitoring

Test and compare different LLMs side-by-side based on performance, cost, and other key metrics

OpenGround

Connect and switch between multiple ClickHouse databases in OpenLIT

Connect Multiple Databases

Track LLM Costs, Agent actions, Tokens, Performance along with User Interactions

Monitor AI Applications using OpenTelemetry

OpenAI

Monitor OpenAI using OpenTelemetry

Ollama

Monitor Ollama using OpenTelemetry

Anthropic

Monitor Anthropic using OpenTelemetry

GPT4All

Monitor GPT4All using OpenTelemetry

Cohere

Monitor Cohere using OpenTelemetry

Mistral

Monitor Mistral using OpenTelemetry

GitHub Models

Monitor GitHub Models using OpenTelemetry

Azure OpenAI

Monitor Azure OpenAI using OpenTelemetry

Azure AI Inference

Monitor Azure AI Inference using OpenTelemetry

HuggingFace

Monitor HuggingFace using OpenTelemetry

Amazon Bedrock

Monitor Amazon Bedrock using OpenTelemetry

Vertex AI

Monitor Vertex AI using OpenTelemetry

Google AI Studio

Monitor Google AI Studio using OpenTelemetry

Groq

Monitor Groq using OpenTelemetry

NVIDIA NIM

Monitor NVIDIA NIM using OpenTelemetry

Monitor xAI using OpenTelemetry

ElevenLabs

Monitor ElevenLabs using OpenTelemetry

AI21

Monitor AI21 using OpenTelemetry

Together

Monitor Together using OpenTelemetry

Assembly AI

Monitor Assembly AI using OpenTelemetry

Reka AI

Monitor Reka using OpenTelemetry

OLA Krutrim

Monitor Krutrim using OpenTelemetry

Titan ML

Monitor Titan ML using OpenTelemetry

Prem AI

Monitor Prem AI using OpenTelemetry

ChromaDB

Monitor ChromaDB using OpenTelemetry

Pinecone

Monitor Pinecone using OpenTelemetry

Qdrant

Monitor Qdrant using OpenTelemetry

Milvus

Monitor Milvus using OpenTelemetry

AstraDB

Monitor AstraDB using OpenTelemetry

LangChain

Monitor LangChain using OpenTelemetry

LiteLLM

Monitor LiteLLM using OpenTelemetry

CrewAI

Monitor CrewAI using OpenTelemetry

LlamaIndex

Monitor LlamaIndex using OpenTelemetry

DSPy

Monitor DSPy using OpenTelemetry

AutoGen / AG2

Monitor AutoGen using OpenTelemetry

Haystack

Monitor Haystack using OpenTelemetry

EmbedChain

Monitor EmbedChain using OpenTelemetry

mem0

Monitor mem0 using OpenTelemetry

Guardrails AI

Monitor Guardrails AI using OpenTelemetry

Phidata

Monitor Phidata using OpenTelemetry

MultiOn

Monitor MultiOn using OpenTelemetry

Julep AI

Monitor Julep AI using OpenTelemetry

Letta

Monitor Letta using OpenTelemetry

Crawl4AI

Monitor Crawl4AI using OpenTelemetry

FireCrawl

Monitor FireCrawl using OpenTelemetry

Dynamiq

Monitor Dynamiq using OpenTelemetry

ControlFlow

Monitor ControlFlow using OpenTelemetry

NVIDIA GPUs

Monitor NVIDIA GPUs using OpenTelemetry

AMD GPUs

Monitor AMD GPUs using OpenTelemetry

Send LLM Monitoring data to your existing Observablity Stack

Connect to your existing Observablity Tools

LLM Observability with OpenTelemetry Collector and OpenLIT

OpenTelemetry Collector

LLM Observability with Prometheus and Grafana Tempo using OpenLIT

Prometheus + Tempo

LLM Observability with Prometheus and Jaeger using OpenLIT

Prometheus + Jaeger

LLM Observability with Grafana Cloud and OpenLIT

Grafana Cloud

LLM Observability with New Relic and OpenLIT

New Relic

LLM Observability with Elastic and OpenLIT

Elastic

LLM Observability with HyperDX using OpenLIT

HyperDX

LLM Observability with DataDog and OpenLIT

DataDog

LLM Observability with SigNoz and OpenLIT

SigNoz

LLM Observability with OneUptime and OpenLIT

OneUptime

LLM Observability with Dynatrace and OpenLIT

Dynatrace

LLM Observability with OpenObserve and OpenLIT

OpenObserve

LLM Observability with Highlight.io and OpenLIT

Highlight.io

LLM Observability with Murnitur and OpenLIT

Murnitur

LLM Observability with Middleware and OpenLIT

Middleware

Fetches a compiled prompt using the provided prompt ID, version, and variables.

Get Prompt

Fetches secret(s) using the provided key or tags.

Get Secret(s)

Details about OpenLIT anonymous usage metrics reporting

Anonymous Usage Metrics

Introduction

Features

Integrations

Connections

API Reference

Privacy

Introduction

Prompt Hub API

Authentication

Introduction

Features

Integrations

Connections

API Reference

Privacy

Prompt Hub API

​Authentication

Authentication