AI analytics in the Cockpit app

The AI section of Lumen mode reads Orkestia's AI telemetry: every model call, every agent session, the autonomous coding lane, MCP traffic and decision calls. These are the same views as on lumen.orkestia.dev.

Who can read them

The AI entries are always listed in the sidebar. All pages use the toolbar's time range and read GET /api/ai/*, GET /api/mcp/* or GET /api/decisions/*. Sections the server reports as unavailable show "Unavailable right now" instead of zeros.

AI overview

A header strip of five tiles (LLM cost, LLM calls, agent sessions, deliveries merged, MCP requests), then one card for each area. Each card shows its figures, a small trend, how it changed against the window before ("vs before"), and opens its page. The earlier window is read after the current one and is optional: if it cannot be read, the cards still show.

Needs attention lists what crosses a threshold, each line linking to its page:

A footer says what counts as low: an answer confidence under 0.7, a judge score of 2 or less.

LLM calls

Source: /api/ai/llm/summary and /api/ai/llm/breakdown.

• Filters by model and by caller, with a clear button. An empty window shows the hint that ai.chat.complete reports every call to project ltinteg-llm, and suggests a longer range.
• Tiles: calls (with cost per call), tokens in and out, cache hit rate (with read and written), latency p50 and p95, and calls with no price.
• Activity chart with a Tokens / Calls / Cost switch. Tokens are stacked in and out, with cost as a line on its own axis.
• Cost by model and Top spenders (the top 5 by the dimension below), each with its share.
• Models table: provider, calls, tokens, p95, tokens per second, cost, and an unpriced tag where the price table has no price for that model.
• Who spends it: break down by Actor, Caller kind, Org, Workflow or End user. Workflow adds Open workflows in Cockpit.

Cost comes from the server's price table; calls to a model it has no price for are counted but not priced. A missing value shows as an en dash, never as zero.

Agent sessions

Source: /api/ai/sessions/summary, /api/ai/sessions, /api/ai/sessions/{uuid}.

• Filters: Status (all, completed, failed, cancelled, running) and Judge score (any, 2 or less, 3 or less). Click a defect or an actor to filter to it; the filters show as chips you can remove, and Clear filters removes them all.
• Tiles: eleven, covering sessions, success rate, failed, running, mean and low quality, cost, average steps, tool failure rate and duration p50 and p95.
• Activity chart: sessions per step, failed in red, with a dashed line for low-quality sessions.
• Judge scores (1 to 5, how many of the sessions were rated), Defects (the judge's diagnosis of what went wrong) and Status split.
• Actors and Agent configs tables: sessions, success, quality, cost.
• Sessions table with the judge score, tool calls (and failures), tokens and duration, sortable by Newest, Cost, Judge score or Duration.
• Click a row for the session drawer: status, judge, cost and duration tiles, the task and any error, and the session's facts.

Quality is the workforce judge's score. See Staff and agents for how sessions are produced.

Delivery lane

Source: /api/ai/delivery/summary, /api/ai/delivery/deliveries, /api/ai/delivery/{uuid}. The autonomous coding lane from ticket to merge.

• Tiles: thirteen, covering deliveries, merged, blocked now, failed, merge rate, cost with cost per merge, and lead time.
• Intake health: whether the ticket intake is healthy, its last success and last failure, and a ticks chart. A server without the intake set up says it is not reporting.
• Funnel: deliveries that reached each state, as a share of the first stage.
• Time in state (p50 and p95) and Blocked reasons by failure class.
• Repositories: click one to filter the page to it.
• Deliveries list with a Stuck only toggle. Click one for its drawer: its transitions in order and a link to its ticket in Cockpit.

Running tickets through the lane: Run a ticket.

MCP

Source: /api/mcp/*. Reads your organization's MCP telemetry; see Explore and correlate.

• Servers and Clients with request bars; clients also show their error rate, protocol versions and last seen. Click a server to scope the page to it.
• Activity chart: requests ok and failed, with the p95 latency as a line on its own scale.
• Tools table: click a column header to sort, and filter by name. For one server it can also show tools that were called, never called, or all.
• Tool drawer: its clients with their error rates, and its slowest and recent calls; each call opens its trace in Lumen.
• Tool catalog: what clients are offered and what changed.

Decisions

Source: /api/decisions/summary, /activity, /purposes, /purpose, /actors. Decision calls (the typed-decision models) recorded as decision.* metrics.

• Tiles: calls, error rate, p95 latency, mean confidence, low confidence (answers below 0.70), tokens in and out. A vendor and model selector scopes the page.
• Activity chart: calls ok and failed, a dashed line for low-confidence answers, and p95 latency.
• Vendors and models and Purposes tables with call bars; Actors shows names where the organization knows them and a short id otherwise.
• Click a purpose for its drawer: per question, the answer mix as a coloured bar, a confidence histogram (buckets under 0.7 marked as warnings), the answer share over time, failure codes, and the low-confidence answers with their workflow id (with a copy button).