Orkestia
Blog
Lumen

Explore and correlate

Logs, error groups, traces, spans, metrics, projects, infrastructure and MCP in the Cockpit app, and the links that connect them to each other and to Cockpit

The Observe pages of Lumen mode are views over the Query API. Their point is the links: from any log you can reach its trace, its error group, its project and the other lines with the same fingerprint, without retyping an id. All pages use the toolbar's time range.

Logs

A stacked volume-by-level chart (GET /api/logs/pulse) above a paged table (100 per page, GET /api/logs).

FilterNotes
SearchMessage text (q), applied after a short pause or on Enter
ProjectFrom GET /api/projects
Leveldebug, info, warning, error, critical
Trace / Fingerprint chipsAppear when you arrive from a link; click the chip's ✕ to remove

Click a row for the drawer: full message, traceback, resolution notes, facts, and the context JSON. A Clear filters button appears when anything narrows the list; an empty list with no filters points you to Usage & API keys to create an ingest key.

Error groups

A group is the unit of triage (Use Lumen). The page shows summary tiles (groups, open, critical, new in 24h, events), then a paged table (50 per page).

FilterDefault
StatusOpen; also Regressed, Resolved, Ignored, All
SeverityAny (critical, high, medium, low)
SortLast seen; also by count or first seen
Unassigned onlyoff

In the drawer:

ActionAPIRule
Resolve…PATCH /api/error-groups/{id}/resolveNeeds both a root cause and a solution; the form refuses to send otherwise
IgnorePATCH …/ignore
ReopenPATCH …/reopen
Assign / UnassignPATCH …/assignThe app sends the existing team back with the assignee, because the API clears a field you leave out
SeverityPATCH …/severityA selector in the drawer: critical, high, medium or low

Tick the box on several rows to act on all of them at once: Bulk actions on error groups.

Only the actions that apply to the group's status are shown. A failure shows inline in the drawer and the list refreshes after each success.

Traces and spans

Traces lists traces (50 per page) filtered by project, free text (trace id, log message or operation) and errors only. A trace opens as a waterfall: spans nested under their parent, each bar placed on the trace's own time axis (red when it failed), the logs of each span under it, then logs outside any span and the trace's error groups. Spans with a missing parent are shown at the top level rather than dropped.

Spans shows window statistics (count, error rate, average, p50, p95, p99), a by-operation table, the slowest spans and the failing operations. Filter by project or by operation-name prefix; clicking a by-operation row filters to it, and a slow span opens its trace.

A trace only has a timeline when the original events carried trace_id / span_id: see Send data.

Metrics

Pick a metric, an aggregation (avg, sum, min, max, count, p50, p90, p95, p99), an optional grouping (by project or service) and a project. The chart shows the series with a y-axis and first/last time labels, above window statistics (points, min, avg, p50, p95, p99, max).

  • With no metric chosen, the page uses the first name the API returns and lists the names as chips.
  • Buckets are 5m for 15m–6h and 1h for 24h and 7d.
  • When more series exist than the API returns (20 by default), the smallest are combined as "other" and a note says so.
  • The page starts on the first project rather than "All projects", because the metric name list (GET /api/metrics/names) scans the organization's metrics when no project is chosen, which can take close to a minute or answer "Could not reach Lumen" on a very large organization. Pick another project, or look across all projects when the page offers it. If the chosen project has sent no metrics the page says so instead of claiming none were ever sent. You can also just type a name. Errors stay visible with Try again.

Projects

One health row per project: status (healthy, attention, error), workflows, a trend sparkline, error rate, p95, cost, open error groups, flags, total logs and last seen. Problem projects sort first. Health comes from GET /api/dashboard/projects for the selected range, so it is empty for a project that had no activity in the window ("no activity"); log totals come from GET /api/projects and are all-time.

A flag is the fingerprint of one of the project's top open errors, shown as its first 8 characters. Click it to open Error groups searched for that fingerprint. Click a row for the project drawer: its facts, its ten most recent error groups, and the Look closer links below.

Infrastructure

The Postgres fleet that reports through the AppData host: host cards (CPU and memory gauges, uptime, room for more instances), instance counts (running, paused, crashed), and a table with connections, memory, CPU, database size, wake p95 and last seen. Sorted with crashed instances first, then by connection pressure; gauges turn amber near a limit and red at 80% of connections or 90% of memory. Click a host card to scope the page to that host.

An instance's state comes from the orkestia.pghost.instance.state gauge when it reports one. Without it, an instance with fresh data counts as running, one with recorded exits as crashed, otherwise paused. The instance drawer has its own 1h / 6h / 24h select and charts for connections, backends, memory, CPU, database size, locks, wake latency, wire connections and OOM kills and exits.

The page needs a Lumen API that serves GET /api/metrics/latest (api-lumen 0.1.130 or later). On an older server it says so instead of showing an empty fleet.

MCP

Server picker (all, or one), request and tool-call tiles, an activity chart, and tables of servers, tools and clients. With one server selected the tools table also lists tools nobody called ("not called"), and a Tool catalog card shows tools added, removed or changed between catalog snapshots. A tool opens a drawer with its call chart, failure types, callers, and the recent and slowest calls. Server-side tool inventory: Lumen MCP.

Blue text opens the target with a filter or a record preselected. Back returns you (navigation).

FromTo
Logits trace, Logs of this trace, its error group, Same fingerprint, its project, all logs of the project
Error groupits project, Logs with this fingerprint, More errors in this project, each recent occurrence's trace, its ticket
TraceLogs of this trace, the root project, each span's project, each error group
Span / slow spanits trace, its project
Project drawerLogs, Error groups, Spans, Metrics for that project; Traces; each of its error groups
Project flagError groups searched for that fingerprint
Dashboard panelsMetrics (metric, aggregation, project), Logs, Error groups, Projects; rows open the group, project or log list they name
MCP tool drawerthe trace of a recent or slow call
Metrics pageLogs and Error groups for the chosen project

Into and out of Cockpit

Lumen and Cockpit share one window, so some links leave Lumen mode.

FromTo
Error group ticket_id that is a Cockpit ticket (a uuid)Open ticket in Cockpit: the ticket opens in the Tickets section (the Cockpit app). A ticket_url opens in the browser instead
Agent session with a ticket_uuidOpen ticket in Cockpit (see AI analytics)
LLM breakdown by workflow typeOpen workflows in Cockpit
An app's Health tab in Cockpit (apps with a Lumen project)Open <project> in Lumen: overview, logs, error groups, metrics, filtered to that project

Names instead of ids

A metric series or table key that is an organization id is shown as the organization's name when you belong to that organization. Ids of organizations you are not in stay as ids.