LLM Application Metrics

OpenLIT offers automatic instrumentation with OpenTelemetry for various LLM providers, frameworks, and VectorDBs, enabling you to gain valuable insights into the behavior and performance of your LLM applications through metrics.

This documentation covers metrics settings, understanding semantic conventions, and interpreting metric attributes, empowering you to enhance the monitoring and observability of your LLM applications.

​

Disable Metrics

You have the option to disable the collection of metrics if needed. By default, metrics collection is enabled.

Example:

# Disable metrics collection
openlit.init(disable_metrics=True)

​

Using an existing OTel Metrics instance

You have the flexibility to integrate your existing OpenTelemetry (OTel) Metrics instance configuration with OpenLIT. If you already have an OTel Metrics instance instantiated in your application, you can pass it directly to openlit.init(meter=meter). This integration ensures that OpenLIT utilizes your custom OTel metrics instance settings, allowing for a unified metrics setup across your application.

Example:

# Instantiate an OpenTelemetry Metrics meter
meter = ...

# Pass the meter to OpenLIT
openlit.init(meter=meter)

​

Semantic Convention

This section outlines the OpenTelemetry metrics collected by OpenLIT from applications using LLMs and Vector Databases. These metrics offer a straightforward overview of application performance and resource usage. They serve as a supplement to the detailed data captured through tracing, aiding in the easy creation of dashboards for quick monitoring of system usage and performance.

​

GenAI/LLM Metrics

​

VectorDB Metrics

​

GPU Metrics