Cost & Performance Optimization

Orkestia gives you two execution styles that sit on the same engine: AI reasoning (DGI designs and adapts flows on the fly) and deterministic compositions (a compiled DAG runs the same way every time). Cost and latency live almost entirely in the choice between them, in how much runner capacity you keep warm, and in how you fan work out asynchronously. This page is the practical playbook: where the money and the milliseconds go, the levers you have, and the tradeoffs each lever carries.

The three cost centers

Most Orkestia spend traces back to three places. Optimize them in roughly this order — the first usually has the biggest leverage.

Lever 1 — Compile AI-designed flows to deterministic compositions

This is the highest-leverage move in the platform. DGI is excellent at designing a workflow from intent, but you should not pay an LLM to re-derive that design on every execution. The Virtual Workflow Engine exists precisely to capture a design once and turn it into config the deterministic engine can run forever — with no model in the hot path.

How the compile step removes LLM cost

When DGI (or any producer — a UI, an API client, an agent) settles on a plan, it expresses it as a VirtualWorkflow: typed layers of steps with explicit input mappings. The virtual engine then:

1. Discovers the live catalog of installed workflow types (no static list — it reads what the installed libraries actually register).
2. Validates structure (acyclic DAG, unique step IDs), references (every {provider}.{service}.{operation} exists), and compatibility (every input mapping type-checks).
3. Compiles to a plain JSON DAG matching the engine's DAGWorkflow shape.

The real workflow engine consumes that JSON exactly as it would any hand-authored DAG. There is no runtime marker and no model call — at execution time it is pure deterministic orchestration.

The economic shape is simple: reasoning cost is amortized across every subsequent run. A flow designed once and run ten thousand times pays the LLM bill once.

When to keep DGI in the loop anyway

Compilation is the right default for anything repeated, but not everything should be frozen. Keep reasoning live when the shape of the work genuinely changes per request.

The deeper treatment of when each style fires — and how they interoperate — is in Hybrid Execution Model.

Lever 2 — Right-size runner warm pools and capacity

Runners (and the broader execution environments they model) are where idle cost accumulates. A runner that is warm but unused costs you provider minutes for zero work; a cold start costs you pickup latency. Tuning the warm pool is the act of choosing where on that curve you want to sit.

The core tradeoff

 cost  ◄────────────────────────────────► latency
 high warm pool          balanced          scale-to-zero
 (instant pickup,        (some warm,        (cheapest,
  pay for idle)          spill cold)        cold-start delay)

Orkestia scales runner groups with a reconciliation loop that continuously converges the pool toward the group's configured min/max — your primary cost dial. GitHub workflow_job events are a best-effort nudge that triggers an immediate reconcile (queued may scale up, completed may scale down); because the loop, not the webhook, is authoritative, scaling stays correct and bounded even if an event is missed.

{
  "scale": { "min": 0, "max": 10, "idle_timeout_s": 300 }
}

Reap dead capacity automatically

Idle isn't the only waste — orphaned capacity is worse, because it costs money and serves nothing. Runner groups run periodic health checks: unhealthy runners are deregistered from GitHub and the compute reaped so dead runners don't sit on the bill or starve the queue. The reconciler also sweeps zombie registrations left behind by a failed teardown. You get this for free, but it's worth watching the orphaned-runner rate (it should trend to zero) and reacting if it doesn't. See Runner Management and drift detection & self-healing.

Lever 3 — Use Kafka async for throughput, not just durability

Long-running steps — cloud provisioning, drains, AWS waiters, anything that blocks on an external system — must not occupy a synchronous request. Orkestia's canonical async path puts these on the Kafka workflow.transition bus, consumed by the workflow Kafka consumer. This is both a reliability property (work survives pod restarts) and a performance property: it decouples submission rate from execution rate, so a burst of work fans out across consumers instead of serializing behind a request thread.

Practical consequences for cost and speed:

• Parallelism is the default. Independent workflow groups execute fully in parallel; only operations on the same serialized entity (e.g. the same runner group, guarded by a PG advisory lock) are ordered. Structure work so unrelated items don't share a lock key.
• No long-poll waiting threads. Waiting on a cloud operation happens via transitions, not a held connection, so you don't pay for idle compute holding a socket.
• Backpressure is visible. Consumer lag is your throughput signal — if it grows, scale consumers rather than runners.

Lever 4 — Use Lumen to find the slow and expensive steps

You can't optimize what you can't see. Lumen is Orkestia's workflow-native observability layer, and it attributes every signal to structural context — workflow, actor, state, trace, span, project, environment, release. That attribution is exactly what lets you answer "which step is costing me" instead of guessing.

The signals that matter for cost and performance work:

A typical optimization loop:

1. Open the Lumen dashboard, sort by latency or error volume.
2. Drill into the slow trace; identify the dominant span.
3. Decide the lever — is it an LLM call that should be compiled away, a step that should run in a parallel layer, or a flaky step generating retries?
4. Apply the change; watch the same span's p95 across the next releases to confirm.

Beware ingest rules that hide the signal

Lumen's ingest rules can drop or sample logs to control observability cost. That's a legitimate lever — but over-aggressive sampling can discard the very spans you need to find a slow step, and bad fingerprint rules can mask a retry storm by mis-grouping it. Tune ingest rules with the same care as the workflows they observe.

Putting it together — an optimization checklist

Tradeoffs at a glance

Related reading

• Hybrid Execution Model — how AI reasoning and deterministic compositions interoperate
• Drift Detection & Self-Healing — keeping runner capacity correct and reaped
• Concepts: Workflows, Runners, Lumen, DGI
• Operations: Deployment Models and Security & Compliance
• Reference: Workflow Types Registry and the external Orkestia capability catalog