To directly send OpenTelemetry metrics and traces generated by OpenLIT from your AI Application to Grafana Cloud, follow the below steps.

​
1. Get your Credentials

  1. Sign in to Grafana Cloud Portal and select your Grafana Cloud Stack.
  2. Click Configure in the OpenTelemetry section.
  3. In the Password / API Token section, click on Generate now to create a new API token:
    • Give the API token a name, for example openlit
    • Click on Create token
    • Click on Close without copying the token
    • Copy and Save the value for OTEL_EXPORTER_OTLP_ENDPOINT and OTEL_EXPORTER_OTLP_HEADERS
      Replace the space after Basic with %20: OTEL_EXPORTER_OTLP_HEADERS=“Authorization=Basic%20[base64 instanceID:token]”

​
2. Instrument your application

For Kubernetes deployments with zero-code instrumentation: 
apiVersion: openlit.io/v1alpha1
kind: AutoInstrumentation
metadata:
  name: autoinstrumentation-cr-name
  namespace: default
spec:
  selector:
    matchLabels:
      instrument: "true"
  python:
    instrumentation:
      provider: "openlit"
      version: "latest"
  otlp:
    endpoint: "YOUR_GRAFANA_OTEL_GATEWAY_URL"
    headers: "YOUR_GRAFANA_OTEL_GATEWAY_AUTH"
    timeout: 30
  resource:
    environment: "production"
    serviceName: "my-ai-service"
Replace:
  1. YOUR_GRAFANA_OTEL_GATEWAY_URL with the OTEL_EXPORTER_OTLP_ENDPOINT value you copied in Step 1.
    • Example: https://otlp-gateway-<zone>.grafana.net/otlp
  2. YOUR_GRAFANA_OTEL_GATEWAY_AUTH with the OTEL_EXPORTER_OTLP_HEADERS value you copied in Step 1.
    • Example: Authorization=Basic%20<base64 encoded Instance ID and API Token>
Refer to the OpenLIT Operator Documentation for more advanced configurations and use cases.

​
3. Import the pre-built Dashboard

  1. Log into your Grafana Cloud Instance.
  2. Click Dashboards in the primary menu.
  3. Click New and select Import in the drop-down menu.
  4. Copy the dashboard JSON provided in the accordion named Dashboard below.
    1. To get the GPU Monitoring dashboard, Copy the dashboard JSON provided in the accordion named GPU Dashboard
  5. Paste the dashboard JSON text directly into the text area.
  6. Click Import.
  7. Save the dashboard.
{
    "annotations": {
    "list": [
        {
        "builtIn": 1,
        "datasource": {
            "type": "grafana",
            "uid": "-- Grafana --"
        },
        "enable": true,
        "hide": true,
        "iconColor": "rgba(0, 211, 255, 1)",
        "name": "Annotations & Alerts",
        "type": "dashboard"
        }
    ]
    },
    "editable": true,
    "fiscalYearStartMonth": 0,
    "graphTooltip": 1,
    "id": 0,
    "links": [
    {
        "asDropdown": false,
        "icon": "external link",
        "includeVars": false,
        "keepTime": false,
        "tags": [],
        "targetBlank": true,
        "title": "OpenLIT GitHub",
        "tooltip": "",
        "type": "link",
        "url": "https://github.com/openlit/openlit"
    }
    ],
    "liveNow": false,
    "panels": [
    {
        "datasource": {
        "type": "prometheus",
        "uid": "${DS_PROMETHEUS}"
        },
        "description": "This dashboard displays the usage of Large Language Models (LLM) and Vector Databases, tracking OpenTelemetry Traces and Metrics sent using [OpenLIT](https://github.com/openlit/openlit).",
        "fieldConfig": {
        "defaults": {
            "color": {
            "mode": "palette-classic"
            },
            "custom": {
            "hideFrom": {
                "legend": false,
                "tooltip": false,
                "vis": false
            }
            },
            "mappings": []
        },
        "overrides": []
        },
        "gridPos": {
        "h": 3,
        "w": 24,
        "x": 0,
        "y": 0
        },
        "id": 1,
        "options": {
        "code": {
            "language": "plaintext",
            "showLineNumbers": false,
            "showMiniMap": false
        },
        "content": "# GenAI Observability Dashboard\nThis dashboard displays the usage of Large Language Models (LLM) and Vector Databases, tracking OpenTelemetry Traces and Metrics sent using [OpenLIT](https://github.com/openlit/openlit).",
        "mode": "markdown"
        },
        "pluginVersion": "10.1.2",
        "targets": [
        {
            "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
            },
            "refId": "A"
        }
        ],
        "title": "GenAI Observability Dashboard",
        "type": "text"
    }
    ],
    "refresh": "30s",
    "schemaVersion": 38,
    "style": "dark",
    "tags": [
    "openlit",
    "genai",
    "observability"
    ],
    "templating": {
    "list": [
        {
        "current": {
            "selected": false,
            "text": "Prometheus",
            "value": "P1809F7CD0C75ACF3"
        },
        "hide": 0,
        "includeAll": false,
        "label": "Data Source",
        "multi": false,
        "name": "DS_PROMETHEUS",
        "options": [],
        "query": "prometheus",
        "refresh": 1,
        "regex": "",
        "skipUrlSync": false,
        "type": "datasource"
        }
    ]
    },
    "time": {
    "from": "now-1h",
    "to": "now"
    },
    "timepicker": {},
    "timezone": "",
    "title": "GenAI Observability",
    "uid": "genai-observability",
    "version": 1,
    "weekStart": ""
}

{
    "annotations": {
    "list": [
        {
        "builtIn": 1,
        "datasource": {
            "type": "grafana",
            "uid": "-- Grafana --"
        },
        "enable": true,
        "hide": true,
        "iconColor": "rgba(0, 211, 255, 1)",
        "name": "Annotations & Alerts",
        "type": "dashboard"
        }
    ]
    },
    "editable": true,
    "fiscalYearStartMonth": 0,
    "graphTooltip": 1,
    "id": 0,
    "links": [
    {
        "asDropdown": false,
        "icon": "external link",
        "includeVars": false,
        "keepTime": false,
        "tags": [],
        "targetBlank": true,
        "title": "OpenLIT GitHub",
        "tooltip": "",
        "type": "link",
        "url": "https://github.com/openlit/openlit"
    }
    ],
    "liveNow": false,
    "panels": [],
    "refresh": "30s",
    "schemaVersion": 38,
    "style": "dark",
    "tags": [
    "openlit",
    "gpu",
    "monitoring"
    ],
    "templating": {
    "list": []
    },
    "time": {
    "from": "now-1h",
    "to": "now"
    },
    "timepicker": {},
    "timezone": "",
    "title": "GPU Monitoring",
    "uid": "gpu-monitoring",
    "version": 1,
    "weekStart": ""
}