To directly send OpenTelemetry metrics and traces generated by OpenLIT SDK from your AI Application to Grafana Cloud, Follow the below steps.

1

Get your Grafana Cloud Credentials

  1. Sign in to Grafana Cloud Portal and select your Grafana Cloud Stack.
  2. Click Configure in the OpenTelemetry section.
  3. In the Password / API Token section, click on Generate now to create a new API token:
    • Give the API token a name, for example openlit
    • Click on Create token
    • Click on Close without copying the token
    • Copy and Save the value for OTEL_EXPORTER_OTLP_ENDPOINT and OTEL_EXPORTER_OTLP_HEADERS
      Replace the space after Basic with %20: OTEL_EXPORTER_OTLP_HEADERS=“Authorization=Basic%20[base64 instanceID:token]”
2

Add the following two lines to your application code:

import openlit

openlit.init(
  otlp_endpoint="YOUR_GRAFANA_OTEL_GATEWAY_URL", 
  otlp_headers="YOUR_GRAFANA_OTEL_GATEWAY_AUTH"
)

Replace:

  1. YOUR_GRAFANA_OTEL_GATEWAY_URL with the OTEL_EXPORTER_OTLP_ENDPOINT value you copied in Step 1.
  • Example - https://otlp-gateway-<zone>.grafana.net/otlp
  1. YOUR_GRAFANA_OTEL_GATEWAY_AUTH with the OTEL_EXPORTER_OTLP_HEADERS value you copied in Step 1.
  • Example - Authorization=Basic%20<base64 encoded Instance ID and API Token>

Refer to the OpenLIT Python SDK repository for more advanced configurations and use cases.

3

Import the pre-built Dashboard

  1. Log into your Grafana Cloud Instance.
  2. Click Dashboards in the primary menu.
  3. Click New and select Import in the drop-down menu.
  4. Copy the dashboard JSON provided in the accordion named Dashboard below.
    1. To get the GPU Monitoring dashboard, Copy the dashboard JSON provided in the accordion named GPU Dashboard
  5. Paste the dashboard JSON text directly into the text area.
  6. Click Import.
  7. Save the dashboard.

Integrations

Start Monitoring your LLM Application

Was this page helpful?

Prometheus + JaegerNew Relic