Using the Default Pricing File
The default pricing file is recommended for general use as it is consistently updated to stay current with the pricing models of different LLM providers. Users should consider using a custom pricing file primarily when dealing with custom or fine-tuned models not covered by the default settings.Using a Custom Pricing File
Users can provide their ownpricing_json
variable, which can either be a file path or a URL pointing to a custom JSON file. This file should contain pricing details such as the model names and their associated costs in USD per 1000 tokens for chat-based models.
Example Usage:
-
To load pricing details from an external URL:
-
To load pricing details from a local file path:
Important Considerations
- JSON Structure: The custom pricing JSON must follow the same structure as the default pricing file provided by OpenLIT. If the structure does not match, OpenLIT will not be able to accurately calculate costs based on the custom pricing file.
- Initialization: OpenLIT SDK initializes by pulling data from the provided or default JSON source. It’s essential to ensure that the path or URL is accessible during initialization to fetch the necessary pricing data.
- Updates: If pricing details change, users need to update their custom JSON file accordingly. If using a URL, ensure that the file at the URL is updated; if using a local file, update the file directly on your system.
Deploy OpenLIT
Deployment options for scalable LLM monitoring infrastructure
Integrations
60+ AI integrations with automatic instrumentation and performance tracking
Destinations
Send telemetry to Datadog, Grafana, New Relic, and other observability stacks
Running in Kubernetes? Try the OpenLIT Operator
Automatically inject instrumentation into existing workloads without modifying pod specs, container images, or application code.