> ## Documentation Index
> Fetch the complete documentation index at: https://docs.openlit.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Oodle

> Send OpenLIT traces and metrics to Oodle for high-scale, cost-efficient agent observability

<Frame>
  <img src="https://mintcdn.com/openlit/8RAlALx3TRnpEsHy/images/oodle-agent-observability.png?fit=max&auto=format&n=8RAlALx3TRnpEsHy&q=85&s=20296422a19bb60f73ffe956b28d87ea" width="3436" height="1738" data-path="images/oodle-agent-observability.png" />
</Frame>

To send OpenTelemetry traces and metrics generated by OpenLIT from your AI Application to Oodle, follow the below steps.

### 1. Get your Oodle Credentials

1. **Sign in to your Oodle account**
2. **Get your Oodle credentials**:
   * **OODLE\_ENDPOINT**: Your Oodle OTLP ingestion endpoint
   * **INSTANCE\_ID**: Your Oodle instance identifier
   * **API\_KEY**: Your Oodle API key for authentication

### 2. Configure OpenTelemetry Collector

**Install OpenTelemetry Collector** (if not already running):
For detailed installation instructions, refer to the [OpenTelemetry Collector Documentation](https://opentelemetry.io/docs/collector/installation/).

**Configure the Collector** for Oodle integration:

<Accordion title="Complete Collector Configuration">
  ```yaml theme={"theme":{"light":"github-light","dark":"github-dark"}}
  receivers:
    otlp:
      protocols:
        grpc:
          endpoint: 0.0.0.0:4317
        http:
          endpoint: 0.0.0.0:4318

  processors:
    batch:

  exporters:
    otlphttp/oodle:
      endpoint: "https://OODLE_ENDPOINT"
      headers:
        X-OODLE-INSTANCE: "INSTANCE_ID"
        X-API-KEY: "API_KEY"

  service:
    pipelines:
      metrics:
        receivers: [otlp]
        processors: [batch]
        exporters: [otlphttp/oodle]
      traces:
        receivers: [otlp]
        processors: [batch]
        exporters: [otlphttp/oodle]
  ```
</Accordion>

Replace:

1. `OODLE_ENDPOINT` with your Oodle endpoint
2. `INSTANCE_ID` with your Oodle instance ID
3. `API_KEY` with your Oodle API key

### 3. Instrument your application

<Tabs>
  <Tab title="SDK">
    **For direct integration into your Python applications:**

    <Tabs>
      <Tab title="Function Arguments">
        ```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
        import openlit

        openlit.init(
          otlp_endpoint="YOUR_OTELCOL_URL:4318"
        )
        ```

        Replace:

        1. `YOUR_OTELCOL_URL:4318` with your OpenTelemetry Collector HTTP endpoint.
           * Example (Local): `http://127.0.0.1:4318`
           * Example (Remote): `http://otel-collector.company.com:4318`
      </Tab>

      <Tab title="Environment Variables">
        ```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
        import openlit

        openlit.init()
        ```

        Set these environment variables:

        ```shell theme={"theme":{"light":"github-light","dark":"github-dark"}}
        export OTEL_EXPORTER_OTLP_ENDPOINT="YOUR_OTELCOL_URL:4318"
        export OTEL_SERVICE_NAME="my-ai-service"
        export OTEL_DEPLOYMENT_ENVIRONMENT="production"
        ```

        Replace:

        1. `YOUR_OTELCOL_URL:4318` with your OpenTelemetry Collector HTTP endpoint.
           * Example (Local): `http://127.0.0.1:4318`
           * Example (Remote): `http://otel-collector.company.com:4318`
      </Tab>
    </Tabs>

    Refer to the OpenLIT [Python SDK repository](https://github.com/openlit/openlit/tree/main/sdk/python) for more advanced configurations and use cases.
  </Tab>

  <Tab title="CLI">
    **For zero-code auto-instrumentation via command line:**

    <Tabs>
      <Tab title="CLI Arguments">
        ```shell theme={"theme":{"light":"github-light","dark":"github-dark"}}
        # Using CLI arguments
        openlit-instrument \
          --otlp-endpoint "YOUR_OTELCOL_URL:4318" \
          --service-name "my-ai-service" \
          --deployment-environment "production" \
          python app.py
        ```

        Replace:

        1. `YOUR_OTELCOL_URL:4318` with your OpenTelemetry Collector HTTP endpoint.
           * Example (Local): `http://127.0.0.1:4318`
           * Example (Remote): `http://otel-collector.company.com:4318`
      </Tab>

      <Tab title="Environment Variables">
        ```shell theme={"theme":{"light":"github-light","dark":"github-dark"}}
        # Set environment variables (takes precedence over CLI args)
        export OTEL_EXPORTER_OTLP_ENDPOINT="YOUR_OTELCOL_URL:4318"
        export OTEL_SERVICE_NAME="my-ai-service"
        export OTEL_DEPLOYMENT_ENVIRONMENT="production"

        # Run your application
        openlit-instrument python app.py
        ```

        Replace:

        1. `YOUR_OTELCOL_URL:4318` with your OpenTelemetry Collector HTTP endpoint.
           * Example (Local): `http://127.0.0.1:4318`
           * Example (Remote): `http://otel-collector.company.com:4318`
      </Tab>
    </Tabs>

    Refer to the OpenLIT [Python SDK repository](https://github.com/openlit/openlit/tree/main/sdk/python) for more advanced configurations and use cases.
  </Tab>
</Tabs>

### 4. Import Oodle Dashboard

Once your LLM application is instrumented, you can visualize the metrics in Oodle using our pre-built dashboard:

1. **Log into your Oodle Instance**
2. **Navigate to Dashboards**: Click **Dashboards** in the primary menu
3. **Import Dashboard**: Click **New** → **Import** in the drop-down menu
4. **Copy the Dashboard JSON**: Use the JSON provided in the accordion below
5. **Paste and Import**: Paste the JSON directly into the text area and click **Import**
6. **Save the Dashboard**: Save the imported dashboard

<Accordion title="GenAI Observability Dashboard JSON">
  ```json theme={"theme":{"light":"github-light","dark":"github-dark"}}
  {
    "annotations": {
      "list": [
        {
          "builtIn": 1,
          "datasource": {
            "type": "grafana",
            "uid": "-- Grafana --"
          },
          "enable": true,
          "hide": true,
          "iconColor": "rgba(0, 211, 255, 1)",
          "name": "Annotations & Alerts",
          "type": "dashboard"
        }
      ]
    },
    "editable": true,
    "fiscalYearStartMonth": 0,
    "graphTooltip": 1,
    "id": 7995,
    "links": [],
    "panels": [
      {
        "fieldConfig": {
          "defaults": {},
          "overrides": []
        },
        "gridPos": {
          "h": 4,
          "w": 4,
          "x": 0,
          "y": 0
        },
        "id": 61,
        "options": {
          "code": {
            "language": "plaintext",
            "showLineNumbers": false,
            "showMiniMap": false
          },
          "content": "<img\n src=\"https://www.oodle.ai/img/logo.svg\"\n width=\"100%\"\n height=\"100%\">",
          "mode": "html"
        },
        "pluginVersion": "12.3.1",
        "title": "",
        "transparent": true,
        "type": "text"
      },
      {
        "datasource": {
          "type": "prometheus",
          "uid": "${metrics_datasource}"
        },
        "fieldConfig": {
          "defaults": {
            "color": {
              "fixedColor": "blue",
              "mode": "palette-classic-by-name"
            },
            "mappings": [],
            "thresholds": {
              "mode": "absolute",
              "steps": [
                {
                  "color": "red",
                  "value": 0
                },
                {
                  "color": "#EAB839",
                  "value": 10
                },
                {
                  "color": "#6ED0E0",
                  "value": 100
                }
              ]
            },
            "unit": "none"
          },
          "overrides": []
        },
        "gridPos": {
          "h": 4,
          "w": 5,
          "x": 4,
          "y": 0
        },
        "id": 22,
        "options": {
          "colorMode": "background",
          "graphMode": "area",
          "justifyMode": "auto",
          "orientation": "auto",
          "percentChangeColorMode": "standard",
          "reduceOptions": {
            "calcs": [
              "lastNotNull"
            ],
            "fields": "",
            "values": false
          },
          "showPercentChange": false,
          "textMode": "value_and_name",
          "wideLayout": true
        },
        "pluginVersion": "12.3.1",
        "targets": [
          {
            "datasource": {
              "type": "prometheus",
              "uid": "${metrics_datasource}"
            },
            "editorMode": "code",
            "expr": "sum(increase(gen_ai_client_operation_duration_count{telemetry_sdk_name=\"openlit\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__range]))",
            "legendFormat": "Total Requests",
            "range": true,
            "refId": "A"
          }
        ],
        "title": "",
        "transparent": true,
        "type": "stat"
      },
      {
        "datasource": {
          "type": "prometheus",
          "uid": "${metrics_datasource}"
        },
        "fieldConfig": {
          "defaults": {
            "color": {
              "fixedColor": "purple",
              "mode": "shades"
            },
            "mappings": [],
            "thresholds": {
              "mode": "absolute",
              "steps": [
                {
                  "color": "green",
                  "value": 0
                },
                {
                  "color": "red",
                  "value": 80
                }
              ]
            },
            "unit": "short"
          },
          "overrides": []
        },
        "gridPos": {
          "h": 4,
          "w": 5,
          "x": 9,
          "y": 0
        },
        "id": 3,
        "options": {
          "colorMode": "background",
          "graphMode": "area",
          "justifyMode": "auto",
          "orientation": "auto",
          "percentChangeColorMode": "standard",
          "reduceOptions": {
            "calcs": [
              "lastNotNull"
            ],
            "fields": "",
            "values": false
          },
          "showPercentChange": false,
          "textMode": "auto",
          "wideLayout": true
        },
        "pluginVersion": "12.3.1",
        "targets": [
          {
            "datasource": {
              "type": "prometheus",
              "uid": "${metrics_datasource}"
            },
            "editorMode": "code",
            "expr": "sum(increase(gen_ai_client_token_usage_sum{telemetry_sdk_name=\"openlit\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__range]))",
            "legendFormat": "Total Usage Tokens",
            "range": true,
            "refId": "A"
          }
        ],
        "title": "",
        "transparent": true,
        "type": "stat"
      },
      {
        "datasource": {
          "type": "prometheus",
          "uid": "${metrics_datasource}"
        },
        "fieldConfig": {
          "defaults": {
            "color": {
              "fixedColor": "blue",
              "mode": "shades"
            },
            "mappings": [],
            "thresholds": {
              "mode": "absolute",
              "steps": [
                {
                  "color": "green",
                  "value": 0
                },
                {
                  "color": "#EAB839",
                  "value": 0.5
                },
                {
                  "color": "red",
                  "value": 1
                }
              ]
            },
            "unit": "currencyUSD"
          },
          "overrides": []
        },
        "gridPos": {
          "h": 4,
          "w": 5,
          "x": 14,
          "y": 0
        },
        "id": 5,
        "options": {
          "colorMode": "background",
          "graphMode": "area",
          "justifyMode": "auto",
          "orientation": "auto",
          "percentChangeColorMode": "inverted",
          "reduceOptions": {
            "calcs": [
              "lastNotNull"
            ],
            "fields": "",
            "values": false
          },
          "showPercentChange": false,
          "textMode": "auto",
          "wideLayout": true
        },
        "pluginVersion": "12.3.1",
        "targets": [
          {
            "datasource": {
              "type": "prometheus",
              "uid": "${metrics_datasource}"
            },
            "editorMode": "code",
            "expr": "sum(last_over_time(gen_ai_usage_cost_sum{telemetry_sdk_name=\"openlit\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__interval])) \n/ \n(sum(last_over_time(gen_ai_usage_cost_count{telemetry_sdk_name=\"openlit\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__interval])))",
            "legendFormat": "Avg Usage Cost",
            "range": true,
            "refId": "A"
          }
        ],
        "title": "",
        "transparent": true,
        "type": "stat"
      },
      {
        "datasource": {
          "type": "prometheus",
          "uid": "${metrics_datasource}"
        },
        "fieldConfig": {
          "defaults": {
            "color": {
              "fixedColor": "blue",
              "mode": "shades"
            },
            "mappings": [],
            "thresholds": {
              "mode": "absolute",
              "steps": [
                {
                  "color": "green",
                  "value": 0
                },
                {
                  "color": "red",
                  "value": 80
                }
              ]
            },
            "unit": "currencyUSD"
          },
          "overrides": []
        },
        "gridPos": {
          "h": 4,
          "w": 5,
          "x": 19,
          "y": 0
        },
        "id": 2,
        "options": {
          "colorMode": "background",
          "graphMode": "area",
          "justifyMode": "auto",
          "orientation": "auto",
          "percentChangeColorMode": "standard",
          "reduceOptions": {
            "calcs": [
              "lastNotNull"
            ],
            "fields": "",
            "values": false
          },
          "showPercentChange": false,
          "textMode": "auto",
          "wideLayout": true
        },
        "pluginVersion": "12.3.1",
        "targets": [
          {
            "datasource": {
              "type": "prometheus",
              "uid": "${metrics_datasource}"
            },
            "editorMode": "code",
            "expr": "sum(last_over_time(gen_ai_usage_cost_sum{telemetry_sdk_name=\"openlit\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__interval]))",
            "legendFormat": "Total Usage Cost",
            "range": true,
            "refId": "A"
          }
        ],
        "title": "",
        "transparent": true,
        "type": "stat"
      },
      {
        "datasource": {
          "type": "prometheus",
          "uid": "${metrics_datasource}"
        },
        "fieldConfig": {
          "defaults": {
            "color": {
              "mode": "palette-classic"
            },
            "custom": {
              "axisBorderShow": false,
              "axisCenteredZero": false,
              "axisColorMode": "text",
              "axisLabel": "",
              "axisPlacement": "auto",
              "barAlignment": 0,
              "barWidthFactor": 0.6,
              "drawStyle": "line",
              "fillOpacity": 100,
              "gradientMode": "opacity",
              "hideFrom": {
                "legend": false,
                "tooltip": false,
                "viz": false
              },
              "insertNulls": false,
              "lineInterpolation": "linear",
              "lineWidth": 2,
              "pointSize": 5,
              "scaleDistribution": {
                "type": "linear"
              },
              "showPoints": "auto",
              "showValues": false,
              "spanNulls": false,
              "stacking": {
                "group": "A",
                "mode": "none"
              },
              "thresholdsStyle": {
                "mode": "off"
              }
            },
            "mappings": [],
            "min": 0,
            "thresholds": {
              "mode": "absolute",
              "steps": [
                {
                  "color": "green",
                  "value": 0
                },
                {
                  "color": "red",
                  "value": 80
                }
              ]
            },
            "unit": "currencyUSD"
          },
          "overrides": []
        },
        "gridPos": {
          "h": 7,
          "w": 12,
          "x": 0,
          "y": 4
        },
        "id": 55,
        "options": {
          "legend": {
            "calcs": [],
            "displayMode": "list",
            "placement": "bottom",
            "showLegend": true
          },
          "tooltip": {
            "hideZeros": true,
            "mode": "multi",
            "sort": "none"
          }
        },
        "pluginVersion": "12.3.1",
        "targets": [
          {
            "datasource": {
              "type": "prometheus",
              "uid": "${metrics_datasource}"
            },
            "editorMode": "code",
            "expr": "increase(gen_ai_usage_cost_sum{telemetry_sdk_name=\"openlit\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[1h]) / 1000",
            "legendFormat": "{{gen_ai_provider_name}} / {{gen_ai_request_model}}",
            "range": true,
            "refId": "A"
          }
        ],
        "title": "Cost Trend Analysis",
        "type": "timeseries"
      },
      {
        "datasource": {
          "type": "prometheus",
          "uid": "${metrics_datasource}"
        },
        "fieldConfig": {
          "defaults": {
            "color": {
              "mode": "palette-classic"
            },
            "custom": {
              "axisBorderShow": false,
              "axisCenteredZero": false,
              "axisColorMode": "text",
              "axisLabel": "",
              "axisPlacement": "auto",
              "barAlignment": 0,
              "barWidthFactor": 0.6,
              "drawStyle": "line",
              "fillOpacity": 100,
              "gradientMode": "opacity",
              "hideFrom": {
                "legend": false,
                "tooltip": false,
                "viz": false
              },
              "insertNulls": false,
              "lineInterpolation": "linear",
              "lineWidth": 2,
              "pointSize": 5,
              "scaleDistribution": {
                "type": "linear"
              },
              "showPoints": "auto",
              "showValues": false,
              "spanNulls": false,
              "stacking": {
                "group": "A",
                "mode": "none"
              },
              "thresholdsStyle": {
                "mode": "off"
              }
            },
            "mappings": [],
            "min": 0,
            "thresholds": {
              "mode": "absolute",
              "steps": [
                {
                  "color": "green",
                  "value": 0
                },
                {
                  "color": "red",
                  "value": 80
                }
              ]
            },
            "unit": "percent"
          },
          "overrides": []
        },
        "gridPos": {
          "h": 4,
          "w": 12,
          "x": 12,
          "y": 4
        },
        "id": 56,
        "options": {
          "legend": {
            "calcs": [],
            "displayMode": "list",
            "placement": "bottom",
            "showLegend": true
          },
          "tooltip": {
            "hideZeros": false,
            "mode": "multi",
            "sort": "none"
          }
        },
        "pluginVersion": "12.3.1",
        "targets": [
          {
            "datasource": {
              "type": "prometheus",
              "uid": "${metrics_datasource}"
            },
            "editorMode": "code",
            "expr": "sum(rate(gen_ai_client_operation_duration_count{http_status_code!~\"4..|5..\", telemetry_sdk_name=\"openlit\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__rate_interval])) / sum(rate(gen_ai_client_operation_duration_count{telemetry_sdk_name=\"openlit\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__rate_interval])) * 100",
            "hide": true,
            "legendFormat": "Total Requests",
            "range": true,
            "refId": "A"
          },
          {
            "datasource": {
              "type": "prometheus",
              "uid": "${metrics_datasource}"
            },
            "editorMode": "code",
            "expr": "(  \n  sum(increase(gen_ai_client_operation_duration_count{telemetry_sdk_name=\"openlit\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__rate_interval]))  \n  -  \n  (sum(increase(gen_ai_client_operation_duration_count{error_type=~\".+\", telemetry_sdk_name=\"openlit\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__rate_interval])) or vector(0))  \n)  \n/  \nsum(increase(gen_ai_client_operation_duration_count{telemetry_sdk_name=\"openlit\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__rate_interval]))  \n* 100  ",
            "hide": true,
            "legendFormat": "__auto",
            "range": true,
            "refId": "B"
          },
          {
            "datasource": {
              "type": "prometheus",
              "uid": "${metrics_datasource}"
            },
            "editorMode": "code",
            "expr": "sum(increase(gen_ai_client_operation_duration_count{telemetry_sdk_name=\"openlit\", error_type=~\".+\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__rate_interval]))  \n/  \nsum(increase(gen_ai_client_operation_duration_count{telemetry_sdk_name=\"openlit\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__rate_interval]))  \n* 100  ",
            "legendFormat": "__auto",
            "range": true,
            "refId": "C"
          },
          {
            "datasource": {
              "type": "prometheus",
              "uid": "${metrics_datasource}"
            },
            "editorMode": "code",
            "expr": "histogram_quantile(0.95, sum by(le, gen_ai_provider_name) (rate(gen_ai_client_operation_duration_bucket{service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__rate_interval])))  ",
            "hide": true,
            "legendFormat": "{{gen_ai_provider_name}}",
            "range": true,
            "refId": "D"
          }
        ],
        "title": "Error Rate %",
        "type": "timeseries"
      },
      {
        "datasource": {
          "type": "prometheus",
          "uid": "${metrics_datasource}"
        },
        "fieldConfig": {
          "defaults": {
            "color": {
              "mode": "palette-classic"
            },
            "custom": {
              "align": "left",
              "cellOptions": {
                "mode": "basic",
                "type": "gauge"
              },
              "footer": {
                "reducers": []
              },
              "inspect": false
            },
            "decimals": 0,
            "mappings": [],
            "min": 0,
            "thresholds": {
              "mode": "absolute",
              "steps": [
                {
                  "color": "green",
                  "value": 0
                },
                {
                  "color": "red",
                  "value": 80
                }
              ]
            },
            "unit": "short"
          },
          "overrides": [
            {
              "matcher": {
                "id": "byName",
                "options": "Metric"
              },
              "properties": [
                {
                  "id": "displayName",
                  "value": "Model"
                },
                {
                  "id": "custom.width",
                  "value": 150
                },
                {
                  "id": "custom.cellOptions",
                  "value": {
                    "type": "auto"
                  }
                }
              ]
            }
          ]
        },
        "gridPos": {
          "h": 7,
          "w": 12,
          "x": 12,
          "y": 8
        },
        "id": 65,
        "options": {
          "cellHeight": "md",
          "frameIndex": 0,
          "showHeader": true,
          "sortBy": [
            {
              "desc": true,
              "displayName": "Model"
            }
          ]
        },
        "pluginVersion": "12.3.1",
        "targets": [
          {
            "datasource": {
              "type": "prometheus",
              "uid": "${metrics_datasource}"
            },
            "editorMode": "code",
            "expr": "topk(5, sum by(gen_ai_request_model) (last_over_time(gen_ai_client_operation_duration_count{telemetry_sdk_name=\"openlit\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__range])))",
            "instant": true,
            "legendFormat": "{{gen_ai_request_model}}",
            "refId": "A"
          }
        ],
        "title": "Top 5 Models by Requests",
        "transformations": [
          {
            "id": "seriesToRows",
            "options": {}
          },
          {
            "id": "organize",
            "options": {
              "excludeByName": {
                "Time": true
              },
              "renameByName": {
                "Metric": "Model",
                "Value": "Requests"
              }
            }
          }
        ],
        "type": "table"
      },
      {
        "datasource": {
          "type": "prometheus",
          "uid": "${metrics_datasource}"
        },
        "fieldConfig": {
          "defaults": {
            "color": {
              "mode": "thresholds"
            },
            "mappings": [],
            "thresholds": {
              "mode": "absolute",
              "steps": [
                {
                  "color": "green",
                  "value": 0
                },
                {
                  "color": "yellow",
                  "value": 2
                },
                {
                  "color": "red",
                  "value": 5
                }
              ]
            },
            "unit": "s"
          },
          "overrides": []
        },
        "gridPos": {
          "h": 4,
          "w": 12,
          "x": 0,
          "y": 11
        },
        "id": 58,
        "options": {
          "colorMode": "value",
          "graphMode": "area",
          "justifyMode": "auto",
          "orientation": "auto",
          "percentChangeColorMode": "standard",
          "reduceOptions": {
            "calcs": [
              "lastNotNull"
            ],
            "fields": "",
            "values": false
          },
          "showPercentChange": false,
          "textMode": "auto",
          "wideLayout": true
        },
        "pluginVersion": "12.3.1",
        "targets": [
          {
            "datasource": {
              "type": "prometheus",
              "uid": "${metrics_datasource}"
            },
            "editorMode": "code",
            "expr": "histogram_quantile(0.95, sum by (le, gen_ai_provider_name) (rate(gen_ai_server_time_to_first_token_bucket{telemetry_sdk_name=\"openlit\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__rate_interval])))",
            "legendFormat": "{{gen_ai_provider_name}} P95",
            "range": true,
            "refId": "A"
          }
        ],
        "title": "Time to First Token",
        "type": "stat"
      },
      {
        "datasource": {
          "type": "prometheus",
          "uid": "${metrics_datasource}"
        },
        "fieldConfig": {
          "defaults": {
            "color": {
              "mode": "palette-classic"
            },
            "custom": {
              "axisBorderShow": false,
              "axisCenteredZero": false,
              "axisColorMode": "text",
              "axisLabel": "",
              "axisPlacement": "auto",
              "barAlignment": 0,
              "barWidthFactor": 0.6,
              "drawStyle": "line",
              "fillOpacity": 100,
              "gradientMode": "opacity",
              "hideFrom": {
                "legend": false,
                "tooltip": false,
                "viz": false
              },
              "insertNulls": false,
              "lineInterpolation": "smooth",
              "lineWidth": 2,
              "pointSize": 5,
              "scaleDistribution": {
                "type": "linear"
              },
              "showPoints": "auto",
              "showValues": false,
              "spanNulls": true,
              "stacking": {
                "group": "A",
                "mode": "none"
              },
              "thresholdsStyle": {
                "mode": "off"
              }
            },
            "mappings": [],
            "thresholds": {
              "mode": "absolute",
              "steps": [
                {
                  "color": "green",
                  "value": 0
                },
                {
                  "color": "red",
                  "value": 80
                }
              ]
            },
            "unit": "reqps"
          },
          "overrides": []
        },
        "gridPos": {
          "h": 10,
          "w": 24,
          "x": 0,
          "y": 15
        },
        "id": 64,
        "options": {
          "legend": {
            "calcs": [
              "mean",
              "max",
              "lastNotNull"
            ],
            "displayMode": "table",
            "placement": "bottom",
            "showLegend": true
          },
          "tooltip": {
            "hideZeros": false,
            "mode": "multi",
            "sort": "desc"
          }
        },
        "pluginVersion": "12.3.1",
        "targets": [
          {
            "datasource": {
              "type": "prometheus",
              "uid": "${metrics_datasource}"
            },
            "editorMode": "code",
            "expr": "sum by(gen_ai_provider_name, gen_ai_request_model) (rate(gen_ai_client_operation_duration_count[$__rate_interval]))",
            "legendFormat": "{{gen_ai_provider_name}} / {{gen_ai_request_model}}",
            "range": true,
            "refId": "A"
          }
        ],
        "title": "Request Rate by Platform & Model",
        "transparent": true,
        "type": "timeseries"
      },
      {
        "datasource": {
          "type": "prometheus",
          "uid": "${metrics_datasource}"
        },
        "fieldConfig": {
          "defaults": {
            "color": {
              "mode": "palette-classic"
            },
            "custom": {
              "axisBorderShow": false,
              "axisCenteredZero": false,
              "axisColorMode": "text",
              "axisLabel": "",
              "axisPlacement": "auto",
              "barAlignment": 0,
              "barWidthFactor": 0.6,
              "drawStyle": "bars",
              "fillOpacity": 100,
              "gradientMode": "opacity",
              "hideFrom": {
                "legend": false,
                "tooltip": false,
                "viz": false
              },
              "insertNulls": false,
              "lineInterpolation": "smooth",
              "lineWidth": 2,
              "pointSize": 5,
              "scaleDistribution": {
                "type": "linear"
              },
              "showPoints": "always",
              "showValues": false,
              "spanNulls": true,
              "stacking": {
                "group": "A",
                "mode": "normal"
              },
              "thresholdsStyle": {
                "mode": "off"
              }
            },
            "mappings": [],
            "thresholds": {
              "mode": "absolute",
              "steps": [
                {
                  "color": "green",
                  "value": 0
                },
                {
                  "color": "red",
                  "value": 80
                }
              ]
            },
            "unit": "none"
          },
          "overrides": []
        },
        "gridPos": {
          "h": 6,
          "w": 24,
          "x": 0,
          "y": 25
        },
        "id": 6,
        "options": {
          "legend": {
            "calcs": [],
            "displayMode": "list",
            "placement": "bottom",
            "showLegend": true
          },
          "tooltip": {
            "hideZeros": false,
            "mode": "single",
            "sort": "none"
          }
        },
        "pluginVersion": "12.3.1",
        "targets": [
          {
            "datasource": {
              "type": "prometheus",
              "uid": "${metrics_datasource}"
            },
            "editorMode": "code",
            "expr": "avg(gen_ai_client_token_usage_bucket{telemetry_sdk_name=\"openlit\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\", gen_ai_token_type=\"input\"})",
            "legendFormat": "Prompt Tokens",
            "range": true,
            "refId": "A"
          },
          {
            "datasource": {
              "type": "prometheus",
              "uid": "${metrics_datasource}"
            },
            "editorMode": "code",
            "expr": "avg(gen_ai_client_token_usage_bucket{telemetry_sdk_name=\"openlit\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\", gen_ai_token_type=\"output\"})",
            "legendFormat": "Completion Tokens",
            "range": true,
            "refId": "B"
          },
          {
            "datasource": {
              "type": "prometheus",
              "uid": "${metrics_datasource}"
            },
            "editorMode": "code",
            "expr": "avg(gen_ai_usage_cost_bucket{telemetry_sdk_name=\"openlit\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"})",
            "legendFormat": "Usage Cost",
            "range": true,
            "refId": "C"
          }
        ],
        "title": "Average Token Consumption vs. Average Usage Cost Comparison",
        "transparent": true,
        "type": "timeseries"
      },
      {
        "collapsed": false,
        "gridPos": {
          "h": 1,
          "w": 24,
          "x": 0,
          "y": 31
        },
        "id": 31,
        "panels": [],
        "title": "Cost",
        "type": "row"
      },
      {
        "datasource": {
          "type": "prometheus",
          "uid": "${metrics_datasource}"
        },
        "fieldConfig": {
          "defaults": {
            "color": {
              "mode": "palette-classic"
            },
            "custom": {
              "align": "left",
              "cellOptions": {
                "mode": "basic",
                "type": "gauge"
              },
              "footer": {
                "reducers": []
              },
              "inspect": false
            },
            "decimals": 4,
            "mappings": [],
            "min": 0,
            "thresholds": {
              "mode": "absolute",
              "steps": [
                {
                  "color": "green",
                  "value": 0
                },
                {
                  "color": "red",
                  "value": 80
                }
              ]
            },
            "unit": "currencyUSD"
          },
          "overrides": [
            {
              "matcher": {
                "id": "byName",
                "options": "Metric"
              },
              "properties": [
                {
                  "id": "displayName",
                  "value": "Model"
                },
                {
                  "id": "custom.width",
                  "value": 150
                },
                {
                  "id": "custom.cellOptions",
                  "value": {
                    "type": "auto"
                  }
                }
              ]
            }
          ]
        },
        "gridPos": {
          "h": 8,
          "w": 12,
          "x": 0,
          "y": 32
        },
        "id": 67,
        "options": {
          "cellHeight": "md",
          "frameIndex": 0,
          "showHeader": true,
          "sortBy": [
            {
              "desc": true,
              "displayName": "Last *"
            }
          ]
        },
        "pluginVersion": "12.3.1",
        "targets": [
          {
            "datasource": {
              "type": "prometheus",
              "uid": "${metrics_datasource}"
            },
            "editorMode": "code",
            "expr": "topk(5, sum by(gen_ai_provider_name) (last_over_time(gen_ai_usage_cost_sum{telemetry_sdk_name=\"openlit\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__range])))",
            "instant": true,
            "legendFormat": "{{gen_ai_provider_name}}",
            "refId": "A"
          }
        ],
        "title": "Top 5 Providers by Cost",
        "transformations": [
          {
            "id": "seriesToRows",
            "options": {}
          },
          {
            "id": "organize",
            "options": {
              "excludeByName": {
                "Time": true
              },
              "renameByName": {
                "Metric": "Model",
                "Value": "Cost (USD)"
              }
            }
          }
        ],
        "type": "table"
      },
      {
        "datasource": {
          "type": "prometheus",
          "uid": "${metrics_datasource}"
        },
        "fieldConfig": {
          "defaults": {
            "color": {
              "mode": "palette-classic"
            },
            "custom": {
              "align": "left",
              "cellOptions": {
                "mode": "basic",
                "type": "gauge"
              },
              "footer": {
                "reducers": []
              },
              "inspect": false
            },
            "decimals": 4,
            "mappings": [],
            "min": 0,
            "thresholds": {
              "mode": "absolute",
              "steps": [
                {
                  "color": "green",
                  "value": 0
                },
                {
                  "color": "red",
                  "value": 80
                }
              ]
            },
            "unit": "currencyUSD"
          },
          "overrides": [
            {
              "matcher": {
                "id": "byName",
                "options": "Metric"
              },
              "properties": [
                {
                  "id": "displayName",
                  "value": "Model"
                },
                {
                  "id": "custom.width",
                  "value": 150
                },
                {
                  "id": "custom.cellOptions",
                  "value": {
                    "type": "auto"
                  }
                }
              ]
            }
          ]
        },
        "gridPos": {
          "h": 8,
          "w": 12,
          "x": 12,
          "y": 32
        },
        "id": 66,
        "options": {
          "cellHeight": "md",
          "frameIndex": 0,
          "showHeader": true,
          "sortBy": [
            {
              "desc": true,
              "displayName": "Last *"
            }
          ]
        },
        "pluginVersion": "12.3.1",
        "targets": [
          {
            "datasource": {
              "type": "prometheus",
              "uid": "${metrics_datasource}"
            },
            "editorMode": "code",
            "expr": "topk(5, sum by(gen_ai_request_model) (last_over_time(gen_ai_usage_cost_sum{telemetry_sdk_name=\"openlit\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__range])))",
            "instant": true,
            "legendFormat": "{{gen_ai_request_model}}",
            "refId": "A"
          }
        ],
        "title": "Top 5 Models by Cost",
        "transformations": [
          {
            "id": "seriesToRows",
            "options": {}
          },
          {
            "id": "organize",
            "options": {
              "excludeByName": {
                "Time": true
              },
              "renameByName": {
                "Metric": "Model",
                "Value": "Cost (USD)"
              }
            }
          }
        ],
        "type": "table"
      },
      {
        "datasource": {
          "type": "prometheus",
          "uid": "${metrics_datasource}"
        },
        "fieldConfig": {
          "defaults": {
            "color": {
              "mode": "palette-classic"
            },
            "custom": {
              "axisBorderShow": false,
              "axisCenteredZero": false,
              "axisColorMode": "text",
              "axisLabel": "",
              "axisPlacement": "auto",
              "barAlignment": 0,
              "barWidthFactor": 0.6,
              "drawStyle": "line",
              "fillOpacity": 100,
              "gradientMode": "opacity",
              "hideFrom": {
                "legend": false,
                "tooltip": false,
                "viz": false
              },
              "insertNulls": false,
              "lineInterpolation": "linear",
              "lineWidth": 1,
              "pointSize": 5,
              "scaleDistribution": {
                "type": "linear"
              },
              "showPoints": "auto",
              "showValues": false,
              "spanNulls": false,
              "stacking": {
                "group": "A",
                "mode": "none"
              },
              "thresholdsStyle": {
                "mode": "off"
              }
            },
            "decimals": 4,
            "mappings": [],
            "min": 0,
            "thresholds": {
              "mode": "absolute",
              "steps": [
                {
                  "color": "green",
                  "value": 0
                },
                {
                  "color": "red",
                  "value": 80
                }
              ]
            },
            "unit": "currencyUSD"
          },
          "overrides": []
        },
        "gridPos": {
          "h": 8,
          "w": 12,
          "x": 0,
          "y": 40
        },
        "id": 29,
        "options": {
          "legend": {
            "calcs": [
              "median"
            ],
            "displayMode": "list",
            "placement": "bottom",
            "showLegend": true
          },
          "tooltip": {
            "hideZeros": false,
            "mode": "multi",
            "sort": "none"
          }
        },
        "pluginVersion": "12.3.1",
        "targets": [
          {
            "datasource": {
              "type": "prometheus",
              "uid": "${metrics_datasource}"
            },
            "editorMode": "code",
            "expr": "sum by(gen_ai_provider_name) (last_over_time(gen_ai_usage_cost_sum{telemetry_sdk_name=\"openlit\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__interval]))",
            "legendFormat": "{{gen_ai_provider_name}}",
            "range": true,
            "refId": "A"
          }
        ],
        "title": "Total Cost By System",
        "transparent": true,
        "type": "timeseries"
      },
      {
        "datasource": {
          "type": "prometheus",
          "uid": "${metrics_datasource}"
        },
        "fieldConfig": {
          "defaults": {
            "color": {
              "mode": "palette-classic"
            },
            "custom": {
              "axisBorderShow": false,
              "axisCenteredZero": false,
              "axisColorMode": "text",
              "axisLabel": "",
              "axisPlacement": "auto",
              "barAlignment": 0,
              "barWidthFactor": 0.6,
              "drawStyle": "line",
              "fillOpacity": 100,
              "gradientMode": "opacity",
              "hideFrom": {
                "legend": false,
                "tooltip": false,
                "viz": false
              },
              "insertNulls": false,
              "lineInterpolation": "linear",
              "lineWidth": 1,
              "pointSize": 5,
              "scaleDistribution": {
                "type": "linear"
              },
              "showPoints": "auto",
              "showValues": false,
              "spanNulls": false,
              "stacking": {
                "group": "A",
                "mode": "none"
              },
              "thresholdsStyle": {
                "mode": "off"
              }
            },
            "decimals": 2,
            "mappings": [],
            "thresholds": {
              "mode": "absolute",
              "steps": [
                {
                  "color": "green",
                  "value": 0
                },
                {
                  "color": "red",
                  "value": 80
                }
              ]
            },
            "unit": "currencyUSD"
          },
          "overrides": []
        },
        "gridPos": {
          "h": 8,
          "w": 12,
          "x": 12,
          "y": 40
        },
        "id": 32,
        "options": {
          "legend": {
            "calcs": [
              "median"
            ],
            "displayMode": "list",
            "placement": "bottom",
            "showLegend": true
          },
          "tooltip": {
            "hideZeros": false,
            "mode": "multi",
            "sort": "none"
          }
        },
        "pluginVersion": "12.3.1",
        "targets": [
          {
            "datasource": {
              "type": "prometheus",
              "uid": "${metrics_datasource}"
            },
            "editorMode": "code",
            "expr": "sum(increase(gen_ai_usage_cost_sum{telemetry_sdk_name=\"openlit\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__rate_interval])) by(gen_ai_request_model) / 1000",
            "legendFormat": "__auto",
            "range": true,
            "refId": "A"
          }
        ],
        "title": "Total Cost By Model",
        "transparent": true,
        "type": "timeseries"
      },
      {
        "datasource": {
          "type": "prometheus",
          "uid": "${metrics_datasource}"
        },
        "fieldConfig": {
          "defaults": {
            "color": {
              "mode": "thresholds"
            },
            "decimals": 0,
            "mappings": [],
            "thresholds": {
              "mode": "absolute",
              "steps": [
                {
                  "color": "green",
                  "value": 0
                },
                {
                  "color": "blue",
                  "value": 80
                }
              ]
            },
            "unit": "short"
          },
          "overrides": []
        },
        "gridPos": {
          "h": 6,
          "w": 8,
          "x": 0,
          "y": 48
        },
        "id": 70,
        "options": {
          "colorMode": "value",
          "graphMode": "area",
          "justifyMode": "auto",
          "orientation": "auto",
          "percentChangeColorMode": "standard",
          "reduceOptions": {
            "calcs": [
              "lastNotNull"
            ],
            "fields": "",
            "values": false
          },
          "showPercentChange": false,
          "textMode": "auto",
          "wideLayout": true
        },
        "pluginVersion": "12.3.1",
        "targets": [
          {
            "datasource": {
              "type": "prometheus",
              "uid": "${metrics_datasource}"
            },
            "editorMode": "code",
            "expr": "sum by(deployment_environment) (last_over_time(gen_ai_client_operation_duration_count{telemetry_sdk_name=\"openlit\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__range]))",
            "instant": true,
            "legendFormat": "{{deployment_environment}}",
            "refId": "A"
          }
        ],
        "title": "Requests by Environment",
        "transparent": true,
        "type": "stat"
      },
      {
        "datasource": {
          "type": "prometheus",
          "uid": "${metrics_datasource}"
        },
        "fieldConfig": {
          "defaults": {
            "color": {
              "fixedColor": "purple",
              "mode": "shades"
            },
            "decimals": 0,
            "mappings": [],
            "thresholds": {
              "mode": "absolute",
              "steps": [
                {
                  "color": "green",
                  "value": 0
                }
              ]
            },
            "unit": "short"
          },
          "overrides": []
        },
        "gridPos": {
          "h": 6,
          "w": 8,
          "x": 8,
          "y": 48
        },
        "id": 69,
        "options": {
          "colorMode": "value",
          "graphMode": "area",
          "justifyMode": "auto",
          "orientation": "auto",
          "percentChangeColorMode": "standard",
          "reduceOptions": {
            "calcs": [
              "lastNotNull"
            ],
            "fields": "",
            "values": false
          },
          "showPercentChange": false,
          "textMode": "auto",
          "wideLayout": true
        },
        "pluginVersion": "12.3.1",
        "targets": [
          {
            "datasource": {
              "type": "prometheus",
              "uid": "${metrics_datasource}"
            },
            "editorMode": "code",
            "expr": "sum by(gen_ai_provider_name) (last_over_time(gen_ai_client_operation_duration_count{telemetry_sdk_name=\"openlit\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__range]))",
            "instant": true,
            "legendFormat": "{{gen_ai_provider_name}}",
            "refId": "A"
          }
        ],
        "title": "Requests by Platform",
        "transparent": true,
        "type": "stat"
      },
      {
        "datasource": {
          "type": "prometheus",
          "uid": "${metrics_datasource}"
        },
        "fieldConfig": {
          "defaults": {
            "color": {
              "fixedColor": "green",
              "mode": "fixed"
            },
            "decimals": 0,
            "mappings": [],
            "thresholds": {
              "mode": "absolute",
              "steps": [
                {
                  "color": "green",
                  "value": 0
                },
                {
                  "color": "red",
                  "value": 80
                }
              ]
            },
            "unit": "short"
          },
          "overrides": []
        },
        "gridPos": {
          "h": 6,
          "w": 8,
          "x": 16,
          "y": 48
        },
        "id": 68,
        "options": {
          "colorMode": "value",
          "graphMode": "area",
          "justifyMode": "auto",
          "orientation": "auto",
          "percentChangeColorMode": "standard",
          "reduceOptions": {
            "calcs": [
              "lastNotNull"
            ],
            "fields": "",
            "values": false
          },
          "showPercentChange": false,
          "textMode": "auto",
          "wideLayout": true
        },
        "pluginVersion": "12.3.1",
        "targets": [
          {
            "datasource": {
              "type": "prometheus",
              "uid": "${metrics_datasource}"
            },
            "editorMode": "code",
            "expr": "sum by(gen_ai_operation_name) (last_over_time(gen_ai_client_operation_duration_count{telemetry_sdk_name=\"openlit\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__range]))",
            "instant": true,
            "legendFormat": "{{gen_ai_operation_name}}",
            "refId": "A"
          }
        ],
        "title": "Requests by Type",
        "transparent": true,
        "type": "stat"
      },
      {
        "datasource": {
          "type": "prometheus",
          "uid": "${metrics_datasource}"
        },
        "fieldConfig": {
          "defaults": {
            "color": {
              "mode": "continuous-GrYlRd"
            },
            "mappings": [],
            "thresholds": {
              "mode": "absolute",
              "steps": [
                {
                  "color": "green",
                  "value": 0
                },
                {
                  "color": "red",
                  "value": 80
                }
              ]
            }
          },
          "overrides": []
        },
        "gridPos": {
          "h": 8,
          "w": 24,
          "x": 0,
          "y": 54
        },
        "id": 57,
        "options": {
          "displayMode": "lcd",
          "legend": {
            "calcs": [],
            "displayMode": "list",
            "placement": "bottom",
            "showLegend": false
          },
          "maxVizHeight": 300,
          "minVizHeight": 16,
          "minVizWidth": 8,
          "namePlacement": "auto",
          "orientation": "horizontal",
          "reduceOptions": {
            "calcs": [
              "lastNotNull"
            ],
            "fields": "",
            "values": false
          },
          "showUnfilled": true,
          "sizing": "auto",
          "valueMode": "color"
        },
        "pluginVersion": "12.3.1",
        "targets": [
          {
            "datasource": {
              "type": "prometheus",
              "uid": "${metrics_datasource}"
            },
            "editorMode": "code",
            "expr": "histogram_quantile(0.95, sum(rate(gen_ai_client_operation_duration_bucket{telemetry_sdk_name=\"openlit\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__rate_interval])) by (le, gen_ai_provider_name))",
            "legendFormat": "{{gen_ai_provider_name}}",
            "range": true,
            "refId": "A"
          }
        ],
        "title": "Latency Distribution",
        "transparent": true,
        "type": "bargauge"
      },
      {
        "collapsed": false,
        "gridPos": {
          "h": 1,
          "w": 24,
          "x": 0,
          "y": 62
        },
        "id": 46,
        "panels": [],
        "title": "Latency",
        "type": "row"
      },
      {
        "fieldConfig": {
          "defaults": {},
          "overrides": []
        },
        "gridPos": {
          "h": 2,
          "w": 12,
          "x": 0,
          "y": 63
        },
        "id": 59,
        "options": {
          "code": {
            "language": "plaintext",
            "showLineNumbers": false,
            "showMiniMap": false
          },
          "content": "### By System",
          "mode": "markdown"
        },
        "pluginVersion": "12.3.1",
        "title": "",
        "type": "text"
      },
      {
        "fieldConfig": {
          "defaults": {},
          "overrides": []
        },
        "gridPos": {
          "h": 2,
          "w": 12,
          "x": 12,
          "y": 63
        },
        "id": 60,
        "options": {
          "code": {
            "language": "plaintext",
            "showLineNumbers": false,
            "showMiniMap": false
          },
          "content": "### By Model",
          "mode": "markdown"
        },
        "pluginVersion": "12.3.1",
        "title": "",
        "type": "text"
      },
      {
        "datasource": {
          "type": "prometheus",
          "uid": "${metrics_datasource}"
        },
        "fieldConfig": {
          "defaults": {
            "color": {
              "mode": "continuous-GrYlRd"
            },
            "decimals": 2,
            "mappings": [],
            "min": 0,
            "thresholds": {
              "mode": "absolute",
              "steps": [
                {
                  "color": "green",
                  "value": 0
                },
                {
                  "color": "red",
                  "value": 80
                }
              ]
            },
            "unit": "s"
          },
          "overrides": []
        },
        "gridPos": {
          "h": 8,
          "w": 12,
          "x": 0,
          "y": 65
        },
        "id": 38,
        "options": {
          "displayMode": "gradient",
          "legend": {
            "calcs": [],
            "displayMode": "list",
            "placement": "bottom",
            "showLegend": false
          },
          "maxVizHeight": 300,
          "minVizHeight": 16,
          "minVizWidth": 8,
          "namePlacement": "auto",
          "orientation": "auto",
          "reduceOptions": {
            "calcs": [
              "lastNotNull"
            ],
            "fields": "",
            "values": false
          },
          "showUnfilled": true,
          "sizing": "auto",
          "valueMode": "color"
        },
        "pluginVersion": "12.3.1",
        "targets": [
          {
            "datasource": {
              "type": "prometheus",
              "uid": "${metrics_datasource}"
            },
            "editorMode": "code",
            "expr": "histogram_quantile(0.95, \n  sum by(le, gen_ai_provider_name) (\n    increase(gen_ai_client_operation_duration_bucket{telemetry_sdk_name=\"openlit\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__range])\n  )\n)",
            "instant": true,
            "legendFormat": "__auto",
            "refId": "A"
          }
        ],
        "title": "Tokens Generation Duration (P95)",
        "transformations": [
          {
            "id": "seriesToRows",
            "options": {}
          },
          {
            "id": "sortBy",
            "options": {
              "sort": [
                {
                  "desc": true,
                  "field": "Value"
                }
              ]
            }
          }
        ],
        "transparent": true,
        "type": "bargauge"
      },
      {
        "datasource": {
          "type": "prometheus",
          "uid": "${metrics_datasource}"
        },
        "fieldConfig": {
          "defaults": {
            "color": {
              "mode": "continuous-GrYlRd"
            },
            "decimals": 2,
            "mappings": [],
            "min": 0,
            "thresholds": {
              "mode": "absolute",
              "steps": [
                {
                  "color": "green",
                  "value": 0
                },
                {
                  "color": "red",
                  "value": 80
                }
              ]
            },
            "unit": "s"
          },
          "overrides": []
        },
        "gridPos": {
          "h": 8,
          "w": 12,
          "x": 12,
          "y": 65
        },
        "id": 37,
        "options": {
          "displayMode": "gradient",
          "legend": {
            "calcs": [],
            "displayMode": "list",
            "placement": "bottom",
            "showLegend": false
          },
          "maxVizHeight": 300,
          "minVizHeight": 16,
          "minVizWidth": 8,
          "namePlacement": "auto",
          "orientation": "auto",
          "reduceOptions": {
            "calcs": [
              "lastNotNull"
            ],
            "fields": "",
            "values": false
          },
          "showUnfilled": true,
          "sizing": "auto",
          "valueMode": "color"
        },
        "pluginVersion": "12.3.1",
        "targets": [
          {
            "datasource": {
              "type": "prometheus",
              "uid": "${metrics_datasource}"
            },
            "editorMode": "code",
            "expr": "histogram_quantile(0.95, \n  sum by(le, gen_ai_request_model) (\n    increase(gen_ai_client_operation_duration_bucket{telemetry_sdk_name=\"openlit\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__range])\n  )\n)",
            "instant": true,
            "legendFormat": "__auto",
            "refId": "A"
          }
        ],
        "title": "Tokens Generation Duration (P95)",
        "transformations": [
          {
            "id": "seriesToRows",
            "options": {}
          },
          {
            "id": "sortBy",
            "options": {
              "sort": [
                {
                  "desc": true,
                  "field": "Value"
                }
              ]
            }
          }
        ],
        "transparent": true,
        "type": "bargauge"
      },
      {
        "datasource": {
          "type": "prometheus",
          "uid": "${metrics_datasource}"
        },
        "fieldConfig": {
          "defaults": {
            "color": {
              "mode": "palette-classic"
            },
            "custom": {
              "axisBorderShow": false,
              "axisCenteredZero": false,
              "axisColorMode": "text",
              "axisLabel": "",
              "axisPlacement": "auto",
              "barAlignment": 0,
              "barWidthFactor": 0.6,
              "drawStyle": "line",
              "fillOpacity": 0,
              "gradientMode": "none",
              "hideFrom": {
                "legend": false,
                "tooltip": false,
                "viz": false
              },
              "insertNulls": false,
              "lineInterpolation": "linear",
              "lineWidth": 1,
              "pointSize": 5,
              "scaleDistribution": {
                "type": "linear"
              },
              "showPoints": "auto",
              "showValues": false,
              "spanNulls": false,
              "stacking": {
                "group": "A",
                "mode": "none"
              },
              "thresholdsStyle": {
                "mode": "off"
              }
            },
            "mappings": [],
            "thresholds": {
              "mode": "absolute",
              "steps": [
                {
                  "color": "green",
                  "value": 0
                },
                {
                  "color": "red",
                  "value": 80
                }
              ]
            },
            "unit": "s"
          },
          "overrides": []
        },
        "gridPos": {
          "h": 8,
          "w": 12,
          "x": 0,
          "y": 73
        },
        "id": 41,
        "options": {
          "legend": {
            "calcs": [],
            "displayMode": "list",
            "placement": "bottom",
            "showLegend": true
          },
          "tooltip": {
            "hideZeros": false,
            "mode": "single",
            "sort": "none"
          }
        },
        "pluginVersion": "12.3.1",
        "targets": [
          {
            "datasource": {
              "type": "prometheus",
              "uid": "${metrics_datasource}"
            },
            "editorMode": "code",
            "expr": "histogram_quantile(0.95, sum(rate(gen_ai_server_time_to_first_token_bucket{telemetry_sdk_name=\"openlit\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__rate_interval])) by (gen_ai_provider_name, le))",
            "legendFormat": "__auto",
            "range": true,
            "refId": "A"
          }
        ],
        "title": "Time to first token",
        "transparent": true,
        "type": "timeseries"
      },
      {
        "datasource": {
          "type": "prometheus",
          "uid": "${metrics_datasource}"
        },
        "fieldConfig": {
          "defaults": {
            "color": {
              "mode": "palette-classic"
            },
            "custom": {
              "axisBorderShow": false,
              "axisCenteredZero": false,
              "axisColorMode": "text",
              "axisLabel": "",
              "axisPlacement": "auto",
              "barAlignment": 0,
              "barWidthFactor": 0.6,
              "drawStyle": "line",
              "fillOpacity": 0,
              "gradientMode": "none",
              "hideFrom": {
                "legend": false,
                "tooltip": false,
                "viz": false
              },
              "insertNulls": false,
              "lineInterpolation": "linear",
              "lineWidth": 1,
              "pointSize": 5,
              "scaleDistribution": {
                "type": "linear"
              },
              "showPoints": "auto",
              "showValues": false,
              "spanNulls": false,
              "stacking": {
                "group": "A",
                "mode": "none"
              },
              "thresholdsStyle": {
                "mode": "off"
              }
            },
            "mappings": [],
            "thresholds": {
              "mode": "absolute",
              "steps": [
                {
                  "color": "green",
                  "value": 0
                },
                {
                  "color": "red",
                  "value": 80
                }
              ]
            },
            "unit": "s"
          },
          "overrides": []
        },
        "gridPos": {
          "h": 8,
          "w": 12,
          "x": 12,
          "y": 73
        },
        "id": 42,
        "options": {
          "legend": {
            "calcs": [],
            "displayMode": "list",
            "placement": "bottom",
            "showLegend": true
          },
          "tooltip": {
            "hideZeros": false,
            "mode": "single",
            "sort": "none"
          }
        },
        "pluginVersion": "12.3.1",
        "targets": [
          {
            "datasource": {
              "type": "prometheus",
              "uid": "${metrics_datasource}"
            },
            "editorMode": "code",
            "expr": "histogram_quantile(0.95, sum(rate(gen_ai_server_time_to_first_token_bucket{telemetry_sdk_name=\"openlit\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__rate_interval])) by (gen_ai_request_model, le))",
            "legendFormat": "__auto",
            "range": true,
            "refId": "A"
          }
        ],
        "title": "Time to first token",
        "transparent": true,
        "type": "timeseries"
      },
      {
        "datasource": {
          "type": "prometheus",
          "uid": "${metrics_datasource}"
        },
        "fieldConfig": {
          "defaults": {
            "color": {
              "mode": "palette-classic"
            },
            "custom": {
              "axisBorderShow": false,
              "axisCenteredZero": false,
              "axisColorMode": "text",
              "axisLabel": "",
              "axisPlacement": "auto",
              "barAlignment": 0,
              "barWidthFactor": 0.6,
              "drawStyle": "line",
              "fillOpacity": 0,
              "gradientMode": "none",
              "hideFrom": {
                "legend": false,
                "tooltip": false,
                "viz": false
              },
              "insertNulls": false,
              "lineInterpolation": "linear",
              "lineWidth": 1,
              "pointSize": 5,
              "scaleDistribution": {
                "type": "linear"
              },
              "showPoints": "auto",
              "showValues": false,
              "spanNulls": false,
              "stacking": {
                "group": "A",
                "mode": "none"
              },
              "thresholdsStyle": {
                "mode": "off"
              }
            },
            "mappings": [],
            "thresholds": {
              "mode": "absolute",
              "steps": [
                {
                  "color": "green",
                  "value": 0
                },
                {
                  "color": "red",
                  "value": 80
                }
              ]
            },
            "unit": "s"
          },
          "overrides": []
        },
        "gridPos": {
          "h": 9,
          "w": 12,
          "x": 0,
          "y": 81
        },
        "id": 39,
        "options": {
          "legend": {
            "calcs": [],
            "displayMode": "list",
            "placement": "bottom",
            "showLegend": true
          },
          "tooltip": {
            "hideZeros": false,
            "mode": "single",
            "sort": "none"
          }
        },
        "pluginVersion": "12.3.1",
        "targets": [
          {
            "datasource": {
              "type": "prometheus",
              "uid": "${metrics_datasource}"
            },
            "editorMode": "code",
            "expr": "histogram_quantile(0.95, sum(rate(gen_ai_client_operation_duration_bucket{telemetry_sdk_name=\"openlit\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__rate_interval])) by (gen_ai_provider_name, le))",
            "legendFormat": "__auto",
            "range": true,
            "refId": "A"
          }
        ],
        "title": "Tokens Generation Duration",
        "transparent": true,
        "type": "timeseries"
      },
      {
        "datasource": {
          "type": "prometheus",
          "uid": "${metrics_datasource}"
        },
        "fieldConfig": {
          "defaults": {
            "color": {
              "mode": "palette-classic"
            },
            "custom": {
              "axisBorderShow": false,
              "axisCenteredZero": false,
              "axisColorMode": "text",
              "axisLabel": "",
              "axisPlacement": "auto",
              "barAlignment": 0,
              "barWidthFactor": 0.6,
              "drawStyle": "line",
              "fillOpacity": 0,
              "gradientMode": "none",
              "hideFrom": {
                "legend": false,
                "tooltip": false,
                "viz": false
              },
              "insertNulls": false,
              "lineInterpolation": "linear",
              "lineWidth": 1,
              "pointSize": 5,
              "scaleDistribution": {
                "type": "linear"
              },
              "showPoints": "auto",
              "showValues": false,
              "spanNulls": false,
              "stacking": {
                "group": "A",
                "mode": "none"
              },
              "thresholdsStyle": {
                "mode": "off"
              }
            },
            "mappings": [],
            "thresholds": {
              "mode": "absolute",
              "steps": [
                {
                  "color": "green",
                  "value": 0
                },
                {
                  "color": "red",
                  "value": 80
                }
              ]
            },
            "unit": "s"
          },
          "overrides": []
        },
        "gridPos": {
          "h": 9,
          "w": 12,
          "x": 12,
          "y": 81
        },
        "id": 43,
        "options": {
          "legend": {
            "calcs": [],
            "displayMode": "list",
            "placement": "bottom",
            "showLegend": true
          },
          "tooltip": {
            "hideZeros": false,
            "mode": "single",
            "sort": "none"
          }
        },
        "pluginVersion": "12.3.1",
        "targets": [
          {
            "datasource": {
              "type": "prometheus",
              "uid": "${metrics_datasource}"
            },
            "editorMode": "code",
            "expr": "histogram_quantile(0.95, sum(rate(gen_ai_client_operation_duration_bucket{telemetry_sdk_name=\"openlit\", service_name=~\"$service_name\", deployment_environment=~\"$deployment_environment\"}[$__rate_interval])) by (gen_ai_request_model, le))",
            "legendFormat": "__auto",
            "range": true,
            "refId": "A"
          }
        ],
        "title": "Tokens Generation Duration",
        "transparent": true,
        "type": "timeseries"
      }
    ],
    "preload": false,
    "refresh": "30s",
    "schemaVersion": 42,
    "tags": [
      "ai-observability-integration",
      "ai-generated"
    ],
    "templating": {
      "list": [
        {
          "current": {
            "text": "oodle",
            "value": "P4033CA274A2C2DFE"
          },
          "includeAll": false,
          "label": "Metrics data source",
          "name": "metrics_datasource",
          "options": [],
          "query": "prometheus",
          "refresh": 1,
          "regex": "(?!grafanacloud-usage|grafanacloud-ml-metrics).+",
          "type": "datasource"
        },
        {
          "current": {
            "text": "All",
            "value": [
              "$__all"
            ]
          },
          "datasource": {
            "type": "prometheus",
            "uid": "${metrics_datasource}"
          },
          "definition": "label_values(service_name)",
          "includeAll": true,
          "label": "service_name",
          "multi": true,
          "name": "service_name",
          "options": [],
          "query": {
            "qryType": 1,
            "query": "label_values(service_name)",
            "refId": "PrometheusVariableQueryEditor-VariableQuery"
          },
          "refresh": 2,
          "type": "query"
        },
        {
          "current": {
            "text": "All",
            "value": [
              "$__all"
            ]
          },
          "datasource": {
            "type": "prometheus",
            "uid": "${metrics_datasource}"
          },
          "definition": "label_values(deployment_environment)",
          "includeAll": true,
          "label": "deployment_environment",
          "multi": true,
          "name": "deployment_environment",
          "options": [],
          "query": {
            "qryType": 1,
            "query": "label_values(deployment_environment)",
            "refId": "PrometheusVariableQueryEditor-VariableQuery"
          },
          "refresh": 2,
          "sort": 1,
          "type": "query"
        }
      ]
    },
    "time": {
      "from": "now-24h",
      "to": "now"
    },
    "timepicker": {},
    "timezone": "browser",
    "title": "GenAI Observability",
    "uid": "cdiz9piuoa3ggc",
    "version": 1
  }
  ```
</Accordion>

Your OpenLIT-instrumented AI applications will appear in Oodle with comprehensive metrics including:

* **LLM Request Rates**: Monitor request volume and patterns
* **Usage Costs**: Track AI model costs and budget allocation
* **Token Consumption**: Analyze input/output token usage
* **Model Performance**: Compare performance across different models
* **VectorDB Operations**: Monitor database requests and operations
* **Application Breakdown**: View metrics by application and environment


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.