Gather valuable metrics for your AI apps using OpenTelemetry auto instrumentation
openlit.init(meter=meter)
.
This integration ensures that OpenLIT utilizes your custom OTel metrics instance settings, allowing for a unified metrics setup across your application.
Example:
OTEL_RESOURCE_ATTRIBUTES
environment variable allows you to provide additional OpenTelemetry resource attributes when starting your application with OpenLIT. OpenLIT already includes some default resource attributes:
telemetry.sdk.name: openlit
service.name: YOUR_SERVICE_NAME
deployment.environment: YOUR_ENVIRONMENT_NAME
OTEL_RESOURCE_ATTRIBUTES
variable. Your custom attributes will be added on top of the existing OpenLIT attributes, providing additional context to your telemetry data. Simply format your attributes as key1=value1,key2=value2
.
For example:
Metric Name | Description | Unit | Type | Attributes |
---|---|---|---|---|
gen_ai.total.requests | Number of requests to the LLM. | 1 | Counter | telemetry.sdk.name , gen_ai.application_name , gen_ai.system , gen_ai.environment , gen_ai.operation.name , gen_ai.request.model |
gen_ai.usage.input_tokens | Number of input tokens processed. | 1 | Counter | telemetry.sdk.name , gen_ai.application_name , gen_ai.system , gen_ai.environment , gen_ai.operation.name , gen_ai.request.model |
gen_ai.usage.output_tokens | Number of output tokens processed. | 1 | Counter | telemetry.sdk.name , gen_ai.application_name , gen_ai.system , gen_ai.environment , gen_ai.operation.name , gen_ai.request.model |
gen_ai.usage.total_tokens | Total number of tokens processed. | 1 | Counter | telemetry.sdk.name , gen_ai.application_name , gen_ai.system , gen_ai.environment , gen_ai.operation.name , gen_ai.request.model |
gen_ai.usage.cost | The cost distribution of LLM requests. | USD | Histogram | telemetry.sdk.name , gen_ai.application_name , gen_ai.system , gen_ai.environment , gen_ai.operation.name , gen_ai.request.model |
Metric Name | Description | Unit | Type | Attributes |
---|---|---|---|---|
db.total.requests | Number of requests to VectorDBs. | 1 | Counter | telemetry.sdk.name , gen_ai.application_name , gen_ai.environment |
Metric Name | Description | Unit | Type | Attributes |
---|---|---|---|---|
gpu.utilization | GPU Utilization in percentage | percent | Gauge | telemetry.sdk.name , gen_ai.application_name , gen_ai.environment , gpu_index , gpu_name , gpu_uuid |
gpu.enc.utilization | GPU encoder Utilization in percentage | percent | Gauge | telemetry.sdk.name , gen_ai.application_name , gen_ai.environment , gpu_index , gpu_name , gpu_uuid |
gpu.dec.utilization | GPU decoder Utilization in percentage | percent | Gauge | telemetry.sdk.name , gen_ai.application_name , gen_ai.environment , gpu_index , gpu_name , gpu_uuid |
gpu.temperature | GPU Temperature in Celsius | Celsius | Gauge | telemetry.sdk.name , gen_ai.application_name , gen_ai.environment , gpu_index , gpu_name , gpu_uuid |
gpu.fan_speed | GPU Fan Speed (0-100) as an integer | Integer | Gauge | telemetry.sdk.name , gen_ai.application_name , gen_ai.environment , gpu_index , gpu_name , gpu_uuid |
gpu.memory.available | Available GPU Memory in MB | MB | Gauge | telemetry.sdk.name , gen_ai.application_name , gen_ai.environment , gpu_index , gpu_name , gpu_uuid |
gpu.memory.total | Total GPU Memory in MB | MB | Gauge | telemetry.sdk.name , gen_ai.application_name , gen_ai.environment , gpu_index , gpu_name , gpu_uuid |
gpu.memory.used | Used GPU Memory in MB | MB | Gauge | telemetry.sdk.name , gen_ai.application_name , gen_ai.environment , gpu_index , gpu_name , gpu_uuid |
gpu.memory.free | Free GPU Memory in MB | MB | Gauge | telemetry.sdk.name , gen_ai.application_name , gen_ai.environment , gpu_index , gpu_name , gpu_uuid |
gpu.power.draw | GPU Power Draw in Watts | Watt | Gauge | telemetry.sdk.name , gen_ai.application_name , gen_ai.environment , gpu_index , gpu_name , gpu_uuid |
gpu.power.limit | GPU Power Limit in Watts | Watt | Gauge | telemetry.sdk.name , gen_ai.application_name , gen_ai.environment , gpu_index , gpu_name , gpu_uuid |