New Relic

To directly send OpenTelemetry metrics and traces generated by OpenLIT SDK from your AI Application to New Relic, Follow the below steps.

Get your New Relic Credentials

Go to the New Relic API Keys UI Page
Click Create a Key Button
In the Create an API Key Wizard:
- Set Key Type as Ingest - License
- Give the API Key Name, for example openlit and Optionally add any Notes
- Click on Create a Key
Copy the key by clicking on the three dots (…).

Add the following two lines to your application code:

import openlit

openlit.init(
  otlp_endpoint="https://otlp.nr-data.net:443", 
  otlp_headers="api-key=YOUR_NEWRELIC_LICENSE_KEY"
)

Replace:

YOUR_NEWRELIC_LICENSE_KEY with the API Key value you copied in Step 1.
- Example - "api-key=acxxxxxxxx"

Refer to the OpenLIT Python SDK repository for more advanced configurations and use cases.

Import the pre-built Dashboard

Go to one.newrelic.com > Dashboards.
In the top-right corner, click Import dashboard.
Copy the dashboard JSON provided in the accordion named Dashboard below.
Paste the dashboard JSON text directly into the text area.
Choose the account and permission settings for the dashboard. You can’t change the account once you’ve set it, but you can change the permissions at any time.
Click Import Dashboard.

Dashboard

{
  "name": "GenAI Observability",
  "description": null,
  "permissions": "PUBLIC_READ_WRITE",
  "pages": [
      {
      "name": "GenAI Observability",
      "description": null,
      "widgets": [
          {
          "title": "",
          "layout": {
              "column": 1,
              "row": 1,
              "width": 4,
              "height": 2
          },
          "linkedEntityGuids": null,
          "visualization": {
              "id": "viz.markdown"
          },
          "rawConfiguration": {
              "text": "# GenAI Observability Dashboard\nThis dashboard displays the usage of Large Language Models (LLM) and Vector Databases, tracking OpenTelemetry Traces and Metrics sent using [OpenLIT](https://github.com/openlit/openlit)."
          }
          },
          {
          "title": "",
          "layout": {
              "column": 5,
              "row": 1,
              "width": 4,
              "height": 2
          },
          "linkedEntityGuids": null,
          "visualization": {
              "id": "viz.billboard"
          },
          "rawConfiguration": {
              "facet": {
              "showOtherSeries": false
              },
              "nrqlQueries": [
              {
                  "accountIds": [
                  4060909
                  ],
                  "query": "SELECT count(`gen_ai.total.requests`) AS 'Total Successful LLM Requests' FROM Metric WHERE `telemetry.sdk.name` = 'openlit'"
              }
              ],
              "platformOptions": {
              "ignoreTimeRange": false
              }
          }
          },
          {
          "title": "",
          "layout": {
              "column": 9,
              "row": 1,
              "width": 4,
              "height": 2
          },
          "linkedEntityGuids": null,
          "visualization": {
              "id": "viz.billboard"
          },
          "rawConfiguration": {
              "facet": {
              "showOtherSeries": false
              },
              "nrqlQueries": [
              {
                  "accountIds": [
                  4060909
                  ],
                  "query": "SELECT count(`db.total.requests`) AS 'Total Successful VectorDB Requests' FROM Metric WHERE `telemetry.sdk.name` = 'openlit'"
              }
              ],
              "platformOptions": {
              "ignoreTimeRange": false
              }
          }
          },
          {
          "title": "",
          "layout": {
              "column": 1,
              "row": 3,
              "width": 4,
              "height": 2
          },
          "linkedEntityGuids": null,
          "visualization": {
              "id": "viz.billboard"
          },
          "rawConfiguration": {
              "dataFormatters": [],
              "facet": {
              "showOtherSeries": false
              },
              "nrqlQueries": [
              {
                  "accountIds": [
                  4060909
                  ],
                  "query": "SELECT average(gen_ai.usage.cost AS 'Usage Cost') FROM Span WHERE gen_ai.operation.name != 'vectordb' And gen_ai.operation.name != 'framework'"
              }
              ],
              "platformOptions": {
              "ignoreTimeRange": false
              }
          }
          },
          {
          "title": "",
          "layout": {
              "column": 5,
              "row": 3,
              "width": 4,
              "height": 2
          },
          "linkedEntityGuids": null,
          "visualization": {
              "id": "viz.billboard"
          },
          "rawConfiguration": {
              "dataFormatters": [],
              "facet": {
              "showOtherSeries": false
              },
              "nrqlQueries": [
              {
                  "accountIds": [
                  4060909
                  ],
                  "query": "SELECT sum(gen_ai.usage.cost AS 'Usage Cost') FROM Span WHERE gen_ai.operation.name != 'vectordb' And gen_ai.operation.name != 'framework'"
              }
              ],
              "platformOptions": {
              "ignoreTimeRange": false
              }
          }
          },
          {
          "title": "",
          "layout": {
              "column": 9,
              "row": 3,
              "width": 4,
              "height": 2
          },
          "linkedEntityGuids": null,
          "visualization": {
              "id": "viz.billboard"
          },
          "rawConfiguration": {
              "dataFormatters": [],
              "facet": {
              "showOtherSeries": false
              },
              "nrqlQueries": [
              {
                  "accountIds": [
                  4060909
                  ],
                  "query": "SELECT average(gen_ai.usage.total_tokens AS 'Total Tokens') FROM Span WHERE gen_ai.operation.name != 'vectordb' And gen_ai.operation.name != 'framework'"
              }
              ],
              "platformOptions": {
              "ignoreTimeRange": false
              }
          }
          },
          {
          "title": "Request Duration Dirstribution",
          "layout": {
              "column": 1,
              "row": 5,
              "width": 12,
              "height": 3
          },
          "linkedEntityGuids": null,
          "visualization": {
              "id": "viz.histogram"
          },
          "rawConfiguration": {
              "legend": {
              "enabled": true
              },
              "nrqlQueries": [
              {
                  "accountIds": [
                  4060909
                  ],
                  "query": "SELECT histogram(duration.ms / 1000, 1, 10) AS 'Duration' FROM Span"
              }
              ],
              "platformOptions": {
              "ignoreTimeRange": false
              },
              "yAxisLeft": {
              "zero": true
              }
          }
          },
          {
          "title": "Top GenAI Models by Usage",
          "layout": {
              "column": 1,
              "row": 8,
              "width": 6,
              "height": 6
          },
          "linkedEntityGuids": null,
          "visualization": {
              "id": "viz.bar"
          },
          "rawConfiguration": {
              "facet": {
              "showOtherSeries": false
              },
              "nrqlQueries": [
              {
                  "accountIds": [
                  4060909
                  ],
                  "query": "SELECT count(`gen_ai.total.requests`) FROM Metric FACET `gen_ai.request.model` WHERE `telemetry.sdk.name` = 'openlit' LIMIT 10"
              }
              ],
              "platformOptions": {
              "ignoreTimeRange": false
              }
          }
          },
          {
          "title": "GenAI Requests by Platform",
          "layout": {
              "column": 7,
              "row": 8,
              "width": 3,
              "height": 3
          },
          "linkedEntityGuids": null,
          "visualization": {
              "id": "viz.pie"
          },
          "rawConfiguration": {
              "facet": {
              "showOtherSeries": true
              },
              "legend": {
              "enabled": true
              },
              "nrqlQueries": [
              {
                  "accountIds": [
                  4060909
                  ],
                  "query": "SELECT count(`gen_ai.total.requests`) AS 'GenAI Requests by Platform' FROM Metric FACET `gen_ai.system` WHERE `telemetry.sdk.name` = 'openlit' LIMIT 5"
              }
              ],
              "platformOptions": {
              "ignoreTimeRange": false
              }
          }
          },
          {
          "title": "GenAI Requests by Type",
          "layout": {
              "column": 10,
              "row": 8,
              "width": 3,
              "height": 3
          },
          "linkedEntityGuids": null,
          "visualization": {
              "id": "viz.pie"
          },
          "rawConfiguration": {
              "facet": {
              "showOtherSeries": true
              },
              "legend": {
              "enabled": true
              },
              "nrqlQueries": [
              {
                  "accountIds": [
                  4060909
                  ],
                  "query": "SELECT count(`gen_ai.total.requests`) AS 'GenAI Requests by Type' FROM Metric FACET `gen_ai.operation.name` WHERE `telemetry.sdk.name` = 'openlit' LIMIT 5"
              }
              ],
              "platformOptions": {
              "ignoreTimeRange": false
              }
          }
          },
          {
          "title": "GenAI Requests by Environment",
          "layout": {
              "column": 7,
              "row": 11,
              "width": 3,
              "height": 3
          },
          "linkedEntityGuids": null,
          "visualization": {
              "id": "viz.pie"
          },
          "rawConfiguration": {
              "facet": {
              "showOtherSeries": true
              },
              "legend": {
              "enabled": true
              },
              "nrqlQueries": [
              {
                  "accountIds": [
                  4060909
                  ],
                  "query": "SELECT count(`gen_ai.total.requests`) AS 'GenAI Requests by Environment' FROM Metric FACET `gen_ai.environment` WHERE `telemetry.sdk.name` = 'openlit' LIMIT 5"
              }
              ],
              "platformOptions": {
              "ignoreTimeRange": false
              }
          }
          },
          {
          "title": "GenAI Requests by Application",
          "layout": {
              "column": 10,
              "row": 11,
              "width": 3,
              "height": 3
          },
          "linkedEntityGuids": null,
          "visualization": {
              "id": "viz.pie"
          },
          "rawConfiguration": {
              "facet": {
              "showOtherSeries": true
              },
              "legend": {
              "enabled": true
              },
              "nrqlQueries": [
              {
                  "accountIds": [
                  4060909
                  ],
                  "query": "SELECT count(`gen_ai.total.requests`) AS 'GenAI Requests by Application' FROM Metric FACET `gen_ai.application_name` WHERE `telemetry.sdk.name` = 'openlit' LIMIT 5"
              }
              ],
              "platformOptions": {
              "ignoreTimeRange": false
              }
          }
          },
          {
          "title": "Prompt Tokens vs Completion Tokens vs Usage Cost",
          "layout": {
              "column": 1,
              "row": 14,
              "width": 12,
              "height": 4
          },
          "linkedEntityGuids": null,
          "visualization": {
              "id": "viz.area"
          },
          "rawConfiguration": {
              "facet": {
              "showOtherSeries": false
              },
              "legend": {
              "enabled": true
              },
              "nrqlQueries": [
              {
                  "accountIds": [
                  4060909
                  ],
                  "query": "SELECT average(`gen_ai.usage.input_tokens`) AS 'Avg Input Tokens', average(`gen_ai.usage.output_tokens`) AS 'Avg Output Tokens', average(`gen_ai.usage.cost`) AS 'Avg Usage Cost' FROM Span WHERE `telemetry.sdk.name` = 'openlit' TIMESERIES "
              }
              ],
              "platformOptions": {
              "ignoreTimeRange": false
              }
          }
          },
          {
          "title": "GenAI Requests",
          "layout": {
              "column": 1,
              "row": 18,
              "width": 12,
              "height": 6
          },
          "linkedEntityGuids": null,
          "visualization": {
              "id": "viz.table"
          },
          "rawConfiguration": {
              "facet": {
              "showOtherSeries": false
              },
              "initialSorting": {
              "direction": "desc",
              "name": "timestamp"
              },
              "nrqlQueries": [
              {
                  "accountIds": [
                  4060909
                  ],
                  "query": "SELECT timestamp, trace.id, gen_ai.operation.name AS 'Category', gen_ai.environment AS 'Environment', gen_ai.application_name AS 'Application', gen_ai.request.model AS 'Model', gen_ai.usage.cost AS 'Usage Cost', duration.ms AS 'Duration' FROM Span WHERE gen_ai.operation.name != 'vectordb' And gen_ai.operation.name != 'framework' And telemetry.sdk.name = 'openlit'"
              }
              ],
              "platformOptions": {
              "ignoreTimeRange": false
              }
          }
          },
          {
          "title": "Top VectorDB Operations by Usage",
          "layout": {
              "column": 1,
              "row": 24,
              "width": 6,
              "height": 3
          },
          "linkedEntityGuids": null,
          "visualization": {
              "id": "viz.bar"
          },
          "rawConfiguration": {
              "facet": {
              "showOtherSeries": false
              },
              "nrqlQueries": [
              {
                  "accountIds": [
                  4060909
                  ],
                  "query": "SELECT count(`db.total.requests`) FROM Metric FACET `db.operation` WHERE `telemetry.sdk.name` = 'openlit'"
              }
              ],
              "platformOptions": {
              "ignoreTimeRange": false
              }
          }
          },
          {
          "title": "DB Requests by System",
          "layout": {
              "column": 7,
              "row": 24,
              "width": 2,
              "height": 3
          },
          "linkedEntityGuids": null,
          "visualization": {
              "id": "viz.pie"
          },
          "rawConfiguration": {
              "facet": {
              "showOtherSeries": true
              },
              "legend": {
              "enabled": true
              },
              "nrqlQueries": [
              {
                  "accountIds": [
                  4060909
                  ],
                  "query": "SELECT count(`db.total.requests`) AS 'DB Requests by System' FROM Metric FACET `db.system` WHERE `telemetry.sdk.name` = 'openlit' LIMIT 5"
              }
              ],
              "platformOptions": {
              "ignoreTimeRange": false
              }
          }
          },
          {
          "title": "DB Requests by Environment",
          "layout": {
              "column": 9,
              "row": 24,
              "width": 2,
              "height": 3
          },
          "linkedEntityGuids": null,
          "visualization": {
              "id": "viz.pie"
          },
          "rawConfiguration": {
              "facet": {
              "showOtherSeries": true
              },
              "legend": {
              "enabled": true
              },
              "nrqlQueries": [
              {
                  "accountIds": [
                  4060909
                  ],
                  "query": "SELECT count(`db.total.requests`) AS 'DB Requests by Environment' FROM Metric FACET `gen_ai.environment` WHERE `telemetry.sdk.name` = 'openlit' LIMIT 5"
              }
              ],
              "platformOptions": {
              "ignoreTimeRange": false
              }
          }
          },
          {
          "title": "DB Requests by Application",
          "layout": {
              "column": 11,
              "row": 24,
              "width": 2,
              "height": 3
          },
          "linkedEntityGuids": null,
          "visualization": {
              "id": "viz.pie"
          },
          "rawConfiguration": {
              "facet": {
              "showOtherSeries": true
              },
              "legend": {
              "enabled": true
              },
              "nrqlQueries": [
              {
                  "accountIds": [
                  4060909
                  ],
                  "query": "SELECT count(`db.total.requests`) AS 'DB Requests by Application' FROM Metric FACET `gen_ai.application_name` WHERE `telemetry.sdk.name` = 'openlit' LIMIT 5"
              }
              ],
              "platformOptions": {
              "ignoreTimeRange": false
              }
          }
          },
          {
          "title": "VectorDB Requests",
          "layout": {
              "column": 1,
              "row": 27,
              "width": 12,
              "height": 6
          },
          "linkedEntityGuids": null,
          "visualization": {
              "id": "viz.table"
          },
          "rawConfiguration": {
              "dataFormatters": [],
              "facet": {
              "showOtherSeries": false
              },
              "initialSorting": {
              "direction": "desc",
              "name": "timestamp"
              },
              "nrqlQueries": [
              {
                  "accountIds": [
                  4060909
                  ],
                  "query": "SELECT timestamp, trace.id, gen_ai.environment AS 'Environment', gen_ai.application_name AS 'Application', db.system AS 'System', db.operation AS 'Operation', duration.ms AS 'Duration' FROM Span SINCE 24 hours ago WHERE gen_ai.operation.name = 'vectordb'"
              }
              ],
              "platformOptions": {
              "ignoreTimeRange": false
              }
          }
          }
      ]
      }
  ],
  "variables": []
  }

Integrations

Start Monitoring your LLM Application

Introduction

Features

Integrations

Connections

API Reference

Privacy

Integrations