Setup
1
Install the plugin
openlit SDK. In headless installs, add --yes-deps.To install by hand, copy plugins/hermes to ~/.hermes/plugins/openlit and run hermes plugins enable openlit.2
Point it at OpenLIT
Save your endpoint, and your API key if OpenLIT requires one, to Hermes’s Use
~/.hermes/.env:~/.hermes/.env rather than a shell export. Chats run inside a background Hermes gateway, which doesn’t see variables set in your shell. Any variable in the table below can be set the same way.Without an endpoint the plugin stays disabled and logs a warning.
3
Restart and check
Stop the Hermes gateway so it reloads the plugin and the new settings:Your next Hermes chat starts the gateway again. If you run the gateway as a service (
hermes gateway install), run hermes gateway restart instead. Without a service, restart runs the gateway in the foreground of your terminal.Send a message to Hermes, then open the Agents page. hermes-agent appears on the Applications tab, and its turns are on the Traces page.Configuration
If OpenLIT runs in a container and Hermes runs on the host, or the other way around,
127.0.0.1 points at the wrong machine. Use an address both can reach, such as host.docker.internal on Docker Desktop.
What gets captured
Each user turn is one trace. Delegated subagent turns are nested inside the trace of the turn that started them. Turns in the same Hermes session sharegen_ai.conversation.id.
Auxiliary LLM calls, such as title generation and compression, are
chat spans with hermes.auxiliary_task set. A call that runs outside a turn gets its own trace.
When Hermes stops a turn early, for example on an API error it won’t retry, it doesn’t report the turn’s end. The plugin closes that turn when the next turn in the session starts, when the session ends, when Hermes exits, or, if nothing else happened in the turn, 60 seconds after an API error Hermes won’t retry or 3 minutes after its final retry fails (Hermes may still wait up to 2 minutes and try again). Any model call in that window, such as a fallback or a recovery attempt, keeps the turn open. The span ends at the turn’s last activity, hermes.turn.exit_reason is abandoned, and the turn is marked failed if its last LLM call failed.
Cost is calculated in gen_ai.usage.cost from OpenLIT’s pricing data. It is 0 for models that aren’t in that data, such as self-hosted models. Set OPENLIT_PRICING_JSON to price them.
Content capture
Prompts, responses, tool arguments, and tool results aren’t exported by default. To include them, add this to~/.hermes/.env and restart the gateway:
gen_ai.input.messages and gen_ai.output.messages on turn and LLM spans, including the tool calls a model requests and the tool results it receives, and gen_ai.tool.call.arguments and gen_ai.tool.call.result on tool spans. On LLM spans for providers that take the system prompt outside the messages, such as Anthropic, it also sets gen_ai.system_instructions; for other providers the system prompt is the first input message. Auxiliary LLM spans include the output only.
To limit how much is stored, set OTEL_SPAN_ATTRIBUTE_VALUE_LENGTH_LIMIT to a maximum attribute length.
Troubleshooting
No traces appear
No traces appear
Run
hermes plugins list and check that openlit is enabled. Then look for OpenLIT plugin disabled in ~/.hermes/logs/agent.log, which means no endpoint was found. Run hermes gateway stop (or hermes gateway restart for a gateway service) after any change to ~/.hermes/.env, because a running gateway keeps the old settings and plugin code.Some spans are missing
Some spans are missing
Look for
OpenLIT hook warnings in ~/.hermes/logs/agent.log. A hook that fails drops its span and logs the error there, and Hermes carries on.OpenLIT is unreachable
OpenLIT is unreachable
Hermes keeps working and turns aren’t slowed down. Spans that can’t be delivered are dropped.
Cost shows 0
Cost shows 0
The model isn’t in OpenLIT’s pricing data, or pricing hadn’t finished loading when Hermes started. Set
OPENLIT_PRICING_JSON for self-hosted models.
