Orkestia
Blog
Lumen

Send data

Ingest contract — log/metric/Pulse JSON, batch semantics, fingerprint algorithm, status codes, Python SDK, Kafka lumen.logs

Base: https://lumen-api.orkestia.dev

Provision + key first — Enable. Query side: Query API. Cluster: Collector.

This is JSON HTTP, not OpenTelemetry protobuf. Unknown top-level log fields → 422.

Auth (write)

HeaderWhen
X-Api-Key: lumk_… (scope ingest)POST /api/logs/*, /api/metrics/*
X-Api-Key: lump_… (or Authorization: Bearer lump_…, or X-Lumen-Pulse-Write-Key)POST /api/product/*
OriginRequired for Pulse keys with client_kind=browser
X-Lumen-PulseMust not appear on ops ingest → 400 PRODUCT_SIGNAL_NOT_ALLOWED

Do not send an organization UUID. Customer keys are org-bound.

Responses (logs)

HTTPBodyMeaning
201{ "id": "<uuid>", "received_at": "<iso>" }Stored. id is the public log UUID
200{ "id": null, "dropped": true }Ingest rule drop / sample miss — not stored
429{ "id": null, "quota_exceeded": true } or INGEST_RATE_LIMITEDPlan cap or per-minute rate
403{ "code": "LUMEN_NOT_PROVISIONED" }Org not enabled or paused
401INGEST_AUTH_REQUIRED / INGEST_AUTH_INVALIDMissing or bad key
400PRODUCT_SIGNAL_NOT_ALLOWEDPulse payload/header on ops path
422validation messagesSchema

Rate-limited ingest also sets Retry-After, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset. Limit is the plan's ingest cap / minute.

Batch: { "items": [ … ] }, max 1000. Prefer batch over single.

Logs

POST /api/logs/ingest
POST /api/logs/ingest/batch

Required: project, message.

curl -X POST https://lumen-api.orkestia.dev/api/logs/ingest \
  -H "X-Api-Key: lumk_REPLACE_ME" \
  -H "Content-Type: application/json" \
  -d '{
    "project": "billing-api",
    "source": "api",
    "level": "ERROR",
    "message": "charge failed: card_declined",
    "environment": "production",
    "release": "v2.4.1",
    "traceback": "…",
    "context": { "request_id": "abc" },
    "trace_id": "11111111-1111-1111-1111-111111111111",
    "span_id": "22222222-2222-2222-2222-222222222222",
    "exception_class": "CardDeclined",
    "error_code": "card_declined"
  }'
FieldReqType / defaultNotes
projectyesstringFirst accepted event creates the project. GET /api/projects lists names that have ingested
messageyesstringBody. Dynamic tokens are collapsed for fingerprinting
sourcenostring, default "workflow"Collector sends kubernetes / kubernetes-event; Python SDK default python-sdk
levelnostringERROR, WARNING, CRITICAL, INFO, … If omitted, normalizer may set ERROR when exception evidence is present, else UNKNOWN. Omitted + no exception is not treated as ERROR
environmentnodefault "production"
releasenostringTag / commit
tracebacknostringStack text — used in fingerprint
contextnoobjectArbitrary JSON. PII-oriented keys are redacted; do not put secrets here
trace_id, span_id, parent_span_idnoUUIDRequired for a real span tree. Other IDs → external_trace_id
external_trace_idnostringUpstream non-UUID trace
received_atnoISO-8601Default: server now
service, operation, request_idnostring
workflow_id, workflow_type, workflow_run_id, workflow_step, workflow_state_id, actor_idnostring / intStructural attribution for Orkestia runs
exception_class, exception_message, error_code, handled, retryablenomixedImproves grouping
kubernetes / kubernetes_*nomixedCollector enrichment
audiencenoconfiguration | code | internalProducer may only downgrade

A project is not created in the UI first. It is the project string on the first stored event.

Optional: Kafka

Same log JSON can be produced to topic lumen.logs. Metrics have no Kafka consumer — HTTP only. Alert rules with channel=ticket publish lumen.alert.fired for the workflow side to open a ticket.

Fingerprint (error groups)

Grouping is not a second ingest. On write:

  1. Ingest rules run (drop / sample) — dropped lines never group.
  2. A 64-char lowercase hex SHA-256 is computed synchronously.
  3. A background processor attaches the log to an error group and evaluates alert rules.

Levels that group (default): ERROR, WARNING, CRITICAL. Other levels are stored but skipped for groups.

Hash, in order:

  1. Matching fingerprint rule → sha256(fingerprint_key).
  2. Else extract error_type (structured or traceback) and location (last non-library frame, line stripped); normalize message (UUIDs, timestamps, volatile headers, long ints, URLs collapsed).
    • traceback + error_type → sha256(error_type|message_norm)
    • traceback + location → sha256(location|message_norm)
    • else (weak) → sha256(project|level|message_norm)

A new log is searchable immediately. The group (and any alert) appears after the processor. Severity is derived automatically unless you PATCH it (severity_source=manual).

Metrics

POST /api/metrics/ingest
POST /api/metrics/ingest/batch

Required: metric_name, project, value (number).

curl -X POST https://lumen-api.orkestia.dev/api/metrics/ingest \
  -H "X-Api-Key: lumk_REPLACE_ME" \
  -H "Content-Type: application/json" \
  -d '{
    "metric_name": "checkout.latency_ms",
    "project": "billing-api",
    "service_name": "checkout",
    "value": 187.4,
    "metric_type": "gauge",
    "attrs": { "route": "/charge" }
  }'
FieldReqNotes
metric_nameyes
projectyes
valueyesfloat
service_namenodefault ""
timestampnoISO-8601, default now
metric_typenodefault gauge
resource_attrs, attrsnoDimensions (attrs.key / resource_attrs.key in query filter)
count, sum_valuenoHistogram-style points
trace_id, span_idnoUUIDs

Batch 201: { "inserted": N, "items": [ { "id": "<uuid>" }, … ] }.

Pulse

Separate path. Do not POST product events to /api/logs/ingest. Do not put lumk_ in a browser.

POST /api/product/ingest
POST /api/product/ingest/batch     { "events": [ … ] }  or  { "items": [ … ] }

Required: event_name, project. Optional: signal_kind (must be "product"), event_id, occurred_at, anonymous_id, user_id, session_id, properties, context. organization_uuid in the body is ignored.

Caps (events / min · / month): Free 1,200 / 1M; Pro 10,000 / 25M; Enterprise 60,000 / 250M. Browser keys: batch max 50, plus allowed_origins / allowed_projects. Codes include PULSE_NOT_PROVISIONED, PULSE_ORIGIN_FORBIDDEN, PULSE_PROJECT_FORBIDDEN, PULSE_PII_PROPERTY_FORBIDDEN, PULSE_RATE_LIMITED, 413 PULSE_BATCH_TOO_LARGE.

Traces (registration)

Logs correlate if they carry UUID trace_id / span_id. You can also open a trace with an org session (not an ingest key):

POST /api/traces                    { "root_project": "billing-api", "trace_id"?: uuid }
POST /api/traces/{trace_id}/spans   { "project", "operation_name", "span_id"?, "parent_span_id"?, "status"?: "ok" }
PATCH /api/traces/{trace_id}        close: status, ended_at

GET /api/traces/{trace_id} merges spans, nested logs, and referenced error groups into timeline. Without UUIDs you still get error groups; you do not get a span tree.

Python SDK

pip install orkestia-lumen-sdk
# pip install "orkestia-lumen-sdk[flask]"   # or [fastapi]
import orkestia_lumen_sdk as lumen

lumen.init(
    base_url="https://lumen-api.orkestia.dev",
    api_key="lumk_…",
    project="billing-api",
    source="python-sdk",
    environment="production",
    release="v2.4.1",
    delivery_mode="background",  # or "sync"
)

lumen.capture_exception(exc, context={"job_id": "job_123"})
lumen.capture_message("ready", level="INFO")
lumen.set_tag("region", "us-east-1")
lumen.set_trace_context(trace_id="…", span_id="…")
lumen.flush()

init also reads LUMEN_URL, LUMEN_API_KEY, LUMEN_PROJECT. Delivery is POST /api/logs/ingest/batch. Client redacts obvious secrets and truncates message/traceback. capture_exception fills exception_* / root_*. CaptureResult is bool-compatible (delivered, dropped, status_code, server_id). Retries: 408, 429, 500, 502, 503, 504.

Also: LumenLoggingHandler, Flask / FastAPI / Kafka integrations. Workflow control-plane calls stay on the Node / Python workflow SDKs — different host and token.

Next

Query API

Filters, pagination, rules JSON.

Collector

How cluster lines map onto this schema.

MCP

Same reads and triage as tools.