Tracing

OpenLIT provides OpenTelemetry Auto instrumentation for various LLM providers, frameworks, and VectorDBs, providing you with insights into the behavior and performance of your LLM applications.

This documentation covers tracing settings, understanding semantic conventions, and interpreting span attributes to enhance the monitoring and observability of your LLM applications.

​

Using an existing OTel Tracer

You have the flexibility to integrate your existing OpenTelemetry (OTel) tracer configuration with OpenLIT. If you already have an OTel tracer instantiated in your application, you can pass it directly to openlit.init(tracer=tracer). This integration ensures that OpenLIT utilizes your custom tracer settings, allowing for a unified tracing setup across your application.

Example:

# Instantiate an OpenTelemetry Tracer
tracer = ...

# Pass the tracer to OpenLIT
openlit.init(tracer=tracer)

​

Add custom resource attributes

The OTEL_RESOURCE_ATTRIBUTES environment variable allows you to provide additional OpenTelemetry resource attributes when starting your application with OpenLIT. OpenLIT already includes some default resource attributes:

• telemetry.sdk.name: openlit
• service.name: YOUR_SERVICE_NAME
• deployment.environment: YOUR_ENVIRONMENT_NAME

You can enhance these default resource attributes by adding your own using the OTEL_RESOURCE_ATTRIBUTES variable. Your custom attributes will be added on top of the existing OpenLIT attributes, providing additional context to your telemetry data. Simply format your attributes as key1=value1,key2=value2.

For example:

export OTEL_RESOURCE_ATTRIBUTES="service.instance.id=YOUR_SERVICE_ID,k8s.pod.name=K8S_POD_NAME,k8s.namespace.name=K8S_NAMESPACE,k8s.node.name=K8S_NODE_NAME"

​

Disable Tracing of Content

By default, OpenLIT adds the prompts and completions to Trace span attributes.

However, you may want to disable this logging for privacy reasons, as they may contain highly sensitive data from your users. You may also simply want to reduce the size of your traces.

Example:

openlit.init(trace_content=False)

​

Disable Batch

By default, the SDK batches spans using the OpenTelemetry batch span processor. When working locally, sometimes you may wish to disable this behavior. You can do that with this flag.

Example:

openlit.init(disable_batch=True)

​

Disable Instrumentations

By default, OpenLIT automatically detects which models and frameworks you are using and instruments them for you. You can override this and disable instrumentation for specific frameworks and models.

Example:

openlit.init(disabled_instrumentors=["anthropic", "langchain"])

​

Manual Tracing

Using openlit.trace, you get access to manually create traces, allowing you to record every process within a single function.

@openlit.trace
def generate_one_liner():
    completion = client.chat.completions.create(
        model="gpt-3.5-turbo",
        messages=[
            {
                "role": "system",
                "content": "Return a one liner from any movie for me to guess",
            }
        ],
    )

The trace function automatically groups any LLM function invoked within generate_one_liner, providing you with organized groupings right out of the box.

You can do more with traces by running the start_trace context generator:

with openlit.start_trace(name="<GIVE_TRACE_A_NAME>") as trace:
    # your code

Use trace.set_result('') to set the final result of the trace and trace.set_metadata({}) to add custom metadata.

Full Example

@openlit.trace
def generate_one_liner():
    completion = client.chat.completions.create(
        model="gpt-3.5-turbo",
        messages=[
            {
                "role": "system",
                "content": "Return a one liner from any movie for me to guess",
            }
        ],
    )

def guess_one_liner(one_liner: str):
    with openlit.start_trace("Guess One-liner") as trace:
        completion = client.chat.completions.create(
            model="gpt-3.5-turbo",
            messages=[
                {
                    "role": "user",
                    "content": f"Guess movie from this line: {one_liner}",
                }
            ],
        )
        trace.set_result(completion.choices[0].message.content)

​

Semantic Convention

This section outlines the OpenTelemetry traces collected by OpenLIT from applications utilizing Large Language Models (LLMs) and Vector Databases. The span attributes adhere to the GenAI Semantic Conventions established by the OpenTelemetry community, ensuring standardized data collection that enhances monitoring and debugging capabilities.

​

Span Definitions

This section outlines fundamental definitions related to span types and names used within the OpenTelemetry traces to identify the nature and source of requests.

​

General Span Attributes

General span attributes encapsulate the environmental and operational context of AI applications, offering insights into where and how the spans are generated.

​

GenAI/LLM Span Attributes

​

Request Attributes

These attributes detail the specifics of requests sent to LLMs, including configurations and parameters that define the behavior and responses of the LLM.

​

Response Attributes

Attributes in this category help trace and understand responses from LLMs, encapsulating data relevant to outcomes and any content generated as a result.

​

Usage Attributes

These attributes track the consumption and costs associated with LLM requests, facilitating operational analysis and resource management.

​

Content Attributes

Content attributes specifically relate to the data exchanged with LLMs during requests and responses, including initial prompts and resulting outputs.

​

VectorDB Span Attributes

This section addresses attributes related to interactions with Vector Databases supporting the LLM operations, providing insights into database operations and related parameters.