> ## Documentation Index
> Fetch the complete documentation index at: https://docs.openlit.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Monitor llmman using OpenTelemetry

> OpenLIT brings AI observability to llmman LLM apps via OpenTelemetry auto-instrumentation of the Ollama and OpenAI SDKs, tracing performance and token usage across Python and TypeScript.

OpenLIT uses OpenTelemetry Auto-Instrumentation to help you monitor LLM applications built using models served by [llmman](https://github.com/llmmanorg/llmman). This includes tracking performance, token usage, and how users interact with the application.

llmman is a local model runner that serves the Ollama API (alongside OpenAI- and Anthropic-compatible ones) on port `17434`. There is no dedicated llmman client library: applications talk to it through the Ollama or OpenAI SDKs, so OpenLIT's existing [Ollama](/latest/sdk/integrations/ollama) and [OpenAI](/latest/sdk/integrations/openai) instrumentations pick it up automatically once the client is pointed at `http://localhost:17434`.

The integration is compatible with

* Ollama Python SDK client `>=0.2.0`
* Ollama TypeScript/JavaScript SDK client `>=0.5.0`
* OpenAI Python SDK client `>=1.92.0`
* OpenAI TypeScript SDK client `>=1.13.0`

## Get started

<Steps>
  <Step title="Install OpenLIT">
    Open your command line or terminal and run:

    <Tabs>
      <Tab title="Python">
        ```shell theme={"theme":{"light":"github-light","dark":"github-dark"}}
        pip install openlit
        ```
      </Tab>

      <Tab title="TypeScript">
        ```shell theme={"theme":{"light":"github-light","dark":"github-dark"}}
        npm install openlit
        ```
      </Tab>
    </Tabs>
  </Step>

  <Step title="Initialize OpenLIT in your Application">
    <Tabs>
      <Tab title="Python">
        <Tabs>
          <Tab title="Zero Code Instrumentation">
            Perfect for existing applications - no code modifications needed:

            <Tabs>
              <Tab title="Via CLI Arguments">
                ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
                # Configure via CLI arguments
                openlit-instrument \
                  --service-name my-ai-app \
                  --environment production \
                  --otlp-endpoint YOUR_OTEL_ENDPOINT \
                  python your_app.py
                ```
              </Tab>

              <Tab title="Via Environment Variables">
                ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
                # Configure via environment variables
                export OTEL_SERVICE_NAME=my-ai-app
                export OTEL_DEPLOYMENT_ENVIRONMENT=production
                export OTEL_EXPORTER_OTLP_ENDPOINT=YOUR_OTEL_ENDPOINT

                # Run with zero code changes
                openlit-instrument python your_app.py
                ```
              </Tab>
            </Tabs>

            <Info>
              **Perfect for**: Legacy applications, production systems where code changes need approval, quick testing, or when you want to add observability without touching existing code.
            </Info>
          </Tab>

          <Tab title="One-Line Instrumentation">
            <Tabs>
              <Tab title="Via Function Parameters">
                ```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
                import openlit

                openlit.init(otlp_endpoint="YOUR_OTEL_ENDPOINT")
                ```
              </Tab>

              <Tab title="Via Environment Variables">
                Add the following two lines to your application code:

                ```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
                import openlit

                openlit.init()
                ```

                Then, configure the your OTLP endpoint using environment variable:

                ```shell theme={"theme":{"light":"github-light","dark":"github-dark"}}
                export OTEL_EXPORTER_OTLP_ENDPOINT=YOUR_OTEL_ENDPOINT
                ```
              </Tab>
            </Tabs>
          </Tab>
        </Tabs>
      </Tab>

      <Tab title="Typescript">
        <Tabs>
          <Tab title="One-Line Instrumentation (SDK)">
            <Tabs>
              <Tab title="Via Function Parameters">
                ```typescript theme={"theme":{"light":"github-light","dark":"github-dark"}}
                import openlit from "openlit"

                openlit.init({ otlpEndpoint: "YOUR_OTEL_ENDPOINT" })
                ```
              </Tab>

              <Tab title="Via Environment Variables">
                Add the following two lines to your application code:

                ```typescript theme={"theme":{"light":"github-light","dark":"github-dark"}}
                import openlit from "openlit"

                openlit.init()
                ```

                Then, configure the your OTLP endpoint using environment variable:

                ```shell theme={"theme":{"light":"github-light","dark":"github-dark"}}
                export OTEL_EXPORTER_OTLP_ENDPOINT=YOUR_OTEL_ENDPOINT
                ```
              </Tab>
            </Tabs>
          </Tab>
        </Tabs>
      </Tab>
    </Tabs>

    **Replace:** `YOUR_OTEL_ENDPOINT` with the URL of your OpenTelemetry backend, such as `http://127.0.0.1:4318` if you are using OpenLIT and a local OTel Collector.

    To send metrics and traces to other Observability tools, refer to the [supported destinations](/latest/sdk/destinations/overview).

    For more advanced configurations and application use cases, visit the [OpenLIT Python repository](https://github.com/openlit/openlit/tree/main/sdk/python) or [OpenLIT Typescript repository](https://github.com/openlit/openlit/tree/main/sdk/typescript).
  </Step>

  <Step title="Point your client at llmman">
    Start the server with `llmman serve` and pull a model with `llmman pull gemma4`. Then configure the SDK you already use to talk to `http://localhost:17434` instead of Ollama's default `http://localhost:11434`. No API key is required; if a client insists on one, any placeholder works.

    <CodeGroup>
      ```python Ollama SDK theme={"theme":{"light":"github-light","dark":"github-dark"}}
      import ollama
      import openlit

      openlit.init(otlp_endpoint="http://127.0.0.1:4318")

      client = ollama.Client(host="http://localhost:17434")
      response = client.chat(model="gemma4", messages=[
          {"role": "user", "content": "Why is the sky blue?"},
      ])
      ```

      ```python OpenAI SDK theme={"theme":{"light":"github-light","dark":"github-dark"}}
      from openai import OpenAI
      import openlit

      openlit.init(otlp_endpoint="http://127.0.0.1:4318")

      client = OpenAI(base_url="http://localhost:17434/v1", api_key="llmman")
      response = client.chat.completions.create(
          model="gemma4",
          messages=[{"role": "user", "content": "Why is the sky blue?"}],
      )
      ```

      ```typescript Ollama SDK theme={"theme":{"light":"github-light","dark":"github-dark"}}
      import Openlit from "openlit"

      Openlit.init({ otlpEndpoint: "http://127.0.0.1:4318" })

      async function main() {
        const { Ollama } = await import("ollama");
        const ollama = new Ollama({ host: "http://localhost:17434" });
        const response = await ollama.chat({
          model: "gemma4",
          messages: [{ role: "user", content: "Why is the sky blue?" }],
        });

        console.log(response);
      }

      main();
      ```

      ```typescript OpenAI SDK theme={"theme":{"light":"github-light","dark":"github-dark"}}
      import Openlit from "openlit"

      Openlit.init({ otlpEndpoint: "http://127.0.0.1:4318" })

      async function main() {
        const OpenAI = await import("openai").then((e) => e.default);
        const client = new OpenAI({ baseURL: "http://localhost:17434/v1", apiKey: "llmman" });
        const response = await client.chat.completions.create({
          model: "gemma4",
          messages: [{ role: "user", content: "Why is the sky blue?" }],
        });

        console.log(response);
      }

      main();
      ```
    </CodeGroup>

    <Tip>
      The Ollama SDKs also honour `OLLAMA_HOST`, so `export OLLAMA_HOST=127.0.0.1:17434` switches an existing Ollama application to llmman without code changes.
    </Tip>
  </Step>
</Steps>

***

<CardGroup cols={3}>
  <Card title="Quickstart: LLM Observability" href="/latest/sdk/quickstart-ai-observability" icon="bolt">
    Production-ready AI monitoring setup in 2 simple steps with zero code changes
  </Card>

  <Card title="Configuration" href="/latest/sdk/configuration" icon="bolt">
    Configure the OpenLIT SDK according to you requirements.
  </Card>

  <Card title="Destinations" href="/latest/sdk/destinations/overview" icon="link">
    Send telemetry to Datadog, Grafana, New Relic, and other observability stacks
  </Card>
</CardGroup>