Metrics and cost accounting

How to read each metric, compare sessions, and test the accounting without confusing context size, token spend, or provider billing.

Use /metrics or /stats for a report overlay in the TUI, or a text report in the plain REPL. Both read the session's recorded data. A turn can make several model calls; each call can resend the conversation. That is why input-token spend can grow much faster than the context window.

The footer keeps short status messages beside the metrics. Longer messages move to wrapped rows below them, with space reserved so the composer stays visible.

Open a report

Use the full session ID printed at startup. In reference overlays, j/k, ↑/↓, or Ctrl+P/Ctrl+N move one row; PageUp/PageDown and Ctrl+U/Ctrl+D move ten rows. gg/G jump to the top/bottom. The mouse wheel also scrolls the overlay, leaving the transcript in place. Press / to type a substring filter, Enter or Esc to return to navigation with the filter preserved, and Esc while navigating to close. q also closes while navigating; in a filter it is ordinary text. Opening /metrics or /stats does not add transcript rows. It is a snapshot: reopen it to refresh the numbers while work continues.

Which work is counted?

The Summary, Latest Call, Averages, Context Breakdown, and Recent Calls sections describe the selected session's conversation model calls. They exclude helper requests and workers. Session Totals and Cost by Model include that session's helper requests, such as compaction, fan-out planning, and memory extraction. Helpers contribute spend without changing the reported conversation context.

Workers lists separately recorded child sessions linked to this parent. Parent + Available Workers adds their totals to the parent once each. Worker context does not get added to the parent's context percentage. A missing worker file produces a warning and is excluded from combined totals. Historical sessions without recorded worker links cannot recover that association.

Counts cover successfully returned requests for which Worksmith wrote a model_metrics event. Failed or interrupted requests that never return usage may still incur provider charges that this accounting cannot see. A transport attempt is not a completed model call. The separate usage event drives live UI counters; it is not added again to the metrics totals.

Request timing and context

Timings in JSON use milliseconds. Display values use ms or seconds; k means thousands of tokens rounded to one decimal place. Display rounding can hide small changes that remain visible in JSON.

Context Breakdown

These are local estimates of prompt components, not provider billing categories. They primarily estimate text bytes divided by four, counting tool-call names and arguments and serialized tool schemas where applicable. They do not reproduce the provider's tokenizer or chat-template framing, so the sum can differ from the provider count.

A rising history value often explains growing context. A large system, tools, or skills value indicates standing prompt overhead that compaction may not remove. Breakdown data can be absent in older sessions.

Tool execution timing

Completed TUI tool entries show their duration. In JSON mode, tool_result events include an optional elapsed_ms, paired with tool_call by id:

worksmith --think off --mode json "Search the web for Rust release notes"

A result might include:

{"type":"tool_result","id":"call_1","name":"web","ok":true,"output":"...","elapsed_ms":160}

This measures tool dispatch, including approval waits and retries inside the call. A checkpoint also includes follow-up model requests and time waiting for your answer. It excludes the initiating model request and result rendering. Do not add checkpoint time to overlapping follow-up model time as if they were separate parts of wall-clock duration.

Durations are also saved in session events. Older results and calls that never reach dispatch may omit the field; zero is valid for a sub-millisecond call. /stats model-time totals remain model-request totals, not aggregates of tool execution time.

Session totals, turns, and cache coverage

The JSON field names below apply to a totals object. Per-turn rows use the same token, call, time, and cost definitions within that turn.

For example, if one request reports 800 cached tokens out of 1,000 input tokens and a second request reports no cache information, the rate is 80% over 1/2 reporting calls. It does not mean 80% of all session input was cached. Cache writes are already in input totals and are never added to the hit numerator.

Per-Turn History and Highest Known Turn Costs

#1, #2, and so on identify reconstructed turns. Each row includes its usage, last and peak conversation context, and the latest available context breakdown. done is the loop's outcome; (validated) means the last recorded validation result passed. A bare done is not proof that a check ran. (validation failed) means the last recorded check failed. in progress / incomplete means no turn_complete event was recorded; the report cannot tell a live turn from an interrupted process solely from that absence.

Helper calls between turns contribute to session totals without creating a new turn or being charged to the next one. Helpers recorded during an active turn contribute to that turn. Attribution follows event order, not causal tracing of background requests. Per-turn totals therefore need not sum to session totals.

Highest Known Turn Costs shows up to five turns ranked by known cost. Unknown costs do not participate in the numeric ranking, so a partly unpriced turn could actually be the most expensive. Worker costs are separate; this ranking does not allocate a worker's spend to its originating parent turn.

Cost estimates and the footer

Prices in [models."provider/model"] are USD per million tokens. For each request, Worksmith records:

cost_usd = (input_tokens × input_price + output_tokens × output_price) / 1,000,000

Both prices must be known. For loopback endpoints, unset prices default to zero; for remote endpoints, unset prices remain unknown. Configured local prices take precedence. The model key and estimate are captured with the request, so changing /model or editing prices later does not reprice previous events.

These are estimates at standard rates, not invoices. Provider-specific cache read discounts, cache-write premiums, special token-category prices, and charges without returned usage are not calculated. Zero means known free only when the call was priced; a missing price is different from zero. Small known amounts can also round to $0.000 in the footer.

On resume, the cost is restored from recorded metrics, but the live conversation context/output counters start fresh. /stats is the source for historical counts. A reopened session's worker links also remain available to /stats, even though the new UI's worker manager is not tracking those old processes.

JSON and future providers

The offline JSON report has four top-level fields:

models maps the recorded model key to request-only totals. Tool calls, compactions, and nudges are counted at session/turn level, not assigned to a model in this map. Old events without model identity use unknown (legacy). Each turn includes number, outcome, validation, totals, peak_context, last_context, and last_context_breakdown. validation is true, false, or null for no recorded check.

Raw JSONL model_metrics events additionally carry session_id (absent in legacy events), model, purpose (agent or helper; empty for legacy events), cost_usd, normalized token counts, optional cache data and context_breakdown, total_ms, first_output_ms, prompt_tokens_per_second, and completion_tokens_per_second. Timing trends are available in those events and the text report; the offline JSON accounting report does not repeat every request's timings. Read a missing optional measurement as unavailable. A legacy session's missing prices, cache details, or worker links cannot be reconstructed by changing today's config.

Background helper jobs capture their originating session before launch. Their metrics stay in that session even if /new runs before the job finishes; late events from the old session do not update the new session's footer.

All provider adapters return the shared Usage contract: input includes cache reads and writes, output includes reasoning, and optional cache details are subsets. Adapters with inclusive totals populate it directly; adapters with disjoint counts can use Usage::from_parts. Usage.reported is true only when both input and output counts are available, including explicitly reported zeros. The default is false. Direct adapters must set it; from_parts sets it to true and must only be used with complete reported categories. Missing or partial usage has no cost estimate, even with configured prices; its numeric zero placeholders do not establish that the request was free. Wire field names and streaming usage accumulation belong in the adapter. New measurements extend the shared types with optional, serde-defaulted fields, keeping older sessions readable.

The eval harness in evals/run.py reads the offline JSON report's combined totals for headline accounting, including workers and helpers. Stats collection failures are recorded as metrics_error on the evaluation row; validation still runs and determines pass/fail. Failed stats collection leaves accounting incomplete. Its dollars per solved task is total recorded cost across attempts / externally validated passing attempts, including spend on failures in the numerator. It reports that ratio only when all rows have metrics, no unpriced calls, no worker warnings, and at least one pass. This differs from a session turn merely saying done. Eval elapsed is wall time around execution, stats collection, and validation; model_ms is summed request time.

Verify the accounting

Run these commands from the repository so cargo run uses the current checkout rather than an older installed binary. Replace placeholders with your configured model and the full session ID printed at startup.

cargo run -- --model <your-provider/model>

1. Ask it to explain a source file, then trace a related function. Open /metrics and check that calls, tool usage, tokens, and per-turn rows grow. Scroll the dashboard with page keys or the wheel; the transcript should stay in place. Try / filtering, Enter, and gg/G, then close with Esc.
2. Run /stats while thinking is streaming; closing it should reveal uninterrupted thinking without command output in the transcript. Compare its totals with a freshly opened /metrics. Cache data should show reported coverage or unavailable; cache hits depend on the provider and are not guaranteed by repeating a prompt.
3. Use /model to choose another configured model and send another prompt. Check the cost breakdown has both model keys and earlier costs stay fixed. A remote model without prices should show unpriced calls. A loopback model without configured prices should show zero cost.
4. Run /spawn Summarize src/validation.rs without changing files. Wait for it and any synthesis to finish. Check /stats for separate worker totals and the combined total; parent context should not become worker context.
5. After enough turns to have old history, run /compact. If it actually makes a helper request, its spend should appear in session totals. A completed compaction should increment the compaction count. There may be nothing old enough to compact in a short session.
6. Copy the session ID, quit, and run the offline commands below. Counts and prices should survive reopening. Editing configured prices afterwards must not change this session's recorded costs.

cargo run -- stats <session-id>
cargo run -- stats <session-id> --json
cargo run -- --plain --resume <session-id>

In the plain REPL, run /stats and /quit. The offline report can also be read when the model server is stopped.

These automated tests exercise inclusive and disjoint provider accounting without adding or contacting another provider:

cargo test disjoint_provider_usage_uses_the_same_events_costs_and_reports
cargo test inclusive_wire_counts_are_not_added_to_their_details
cargo test --test metrics
python3 -m unittest discover -s evals -p test_metrics.py

Before committing code changes, run the full cargo test suite and cargo clippy --all-targets -- -D warnings as well.