Tokvera
Find out where your LLM inference is slow, failing or expensive, in two commands.
It grades the endpoint A–F and writes a self-contained report. It works with OpenAI-compatible servers (vLLM, TGI, Ollama, llama.cpp and others) and the OpenAI and Anthropic APIs.
What it measures
Time to first token and latency percentiles at rising concurrency, error rates, cost per 1,000 requests, and whether repeated prompts are worth caching.
It grades the endpoint A–F and writes a self-contained report. It works with OpenAI-compatible servers (vLLM, TGI, Ollama, llama.cpp and others) and the OpenAI and Anthropic APIs.
The grade is labelled provisional until it is calibrated. How the grade works.
tokvera scan --demoEfficiency Score: D (60.8 / 100, provisional scoring 0.9.0) capped at D: more than 5% of requests failed capped at C: time to first token collapses under load Provisional scoring. The weights and thresholds are provisional: they have not yet been calibrated against real pilot scans and may change before scoring 1.0.0.Requests: 60 sent, 55 succeeded. 1. [fail] Time to first token collapses under load: p99 201 ms at concurrency 16 against 11 ms at concurrency 1 2. [fail] Requests are failing: 5 of 60 requests failed (HTTP 429 ×5) 3. [fail] Many prompts repeat exactly: 16 of 40 prompts in the demo sample repeat an earlier one exactly 4. [warn] No Prometheus metrics at /metrics: no Prometheus exposition answered at /metrics on the target's originReport: tokvera-scan/report.htmlThen your own endpoint
A scan sends real load, so it shows the plan first and asks before sending.
Prompts and responses never appear in a report. Cost figures are estimates from a dated rate sheet, and every report shows its date and source.
No telemetry. Tokvera contacts only the target you specify, plus GitHub when you run update or doctor --check-update, and tokvera.halotrcer.com and GitHub for the plugin's first-run install.
tokvera scan --target http://localhost:8000/v1 --dry-run # what would be sent; contacts nothingtokvera scan --target http://localhost:8000/v1 # shows the plan again and asks before sendingWhere else it runs
In CI
The GitHub Action fails a pull request whose latency or cost regressed against a baseline report.
In Kubernetes
A Helm chart scans an in-cluster endpoint on a schedule and keeps the latest report.
In your coding agent
tokvera mcp is an MCP server for coding agents. It sends load only to an endpoint you named, and only after you agreed to it.
Estimate before you measure
The AI bill leak calculator estimates your monthly spend and the part that goes on repeated, uncached input, from your own volumes and prices. A scan then measures it.
Want it fixed?
The scan shows where the problems are. Fixing them, and auditing what your production traffic actually does, is the paid audit from Halotrcer.
AI Performance & Observability Audit: from $3,500, about 2 weeks.
Running AI agents too?
Sievant checks every tool call your agents make. Halotrcer runs both and writes one report.
Launching soon
Tokvera isn’t released yet
Install commands will appear here on release day. Until then, book a call to talk through your setup, or email us.