Telemetry

import { createServer } from 'experimental-a2/server'
import { otel } from 'experimental-a2/otel'
import { orders } from '@/contracts'

export const ordersServer = createServer({
  contract: orders,
  telemetry: otel(),
  // ...store, scheduler, and consumers
})

With any OpenTelemetry setup registered (on Vercel, @vercel/otel), one trace shows the request, a2.append, the inline a2.drain, an a2.event span per consumer run, and the appends those runs make. Failures mark their a2.event span, and a2.event.outcome = dead_lettered is the attribute to alert on. Agent servers add a2.ai.* spans.

experimental-a2/otel

otel(options?: { tracer?: Tracer; tracerName?: string }): A2Telemetry

OpenTelemetry instrumentation for a server (@opentelemetry/api is a peer dependency). Pass it as telemetry. Spans nest through the active context, so the happy path lands in one trace. A2Telemetry is a one-method interface: anything with span(name, attributes, fn) works, and otel() is the adapter A2 ships.

The span catalogue

This catalogue is a contract: renames and additions are breaking for dashboards and are called out in the package's CHANGELOG.md. All spans carry a2.contract and a2.session_id. Core spans describe the durable runtime; a2.ai.* spans describe the AI runtime's consumers.

Durations include nested work: a2.event contains its AI run and later commit, AI runs call Core, and a2.state includes reducer code. Acknowledgements overlap control processing, claims can cover several events, and snapshots run in the background, so summing spans, or subtracting summed children, does not yield exclusive Core or AI overhead. Model and application I/O is included; its child spans have the detail.

Attributes arrive in two waves: at start (passed to span()) and mid-span (via handle.setAttribute, for outcomes and counts known only after the work ran). A recorder must merge both; the ones you alert on are all mid-span.

• Consumer lag: group a2.event spans by a2.event.consumer. a2.event.lag is how many events the session had appended after the one it claimed, and a2.event.age_ms how old that event was when its run started. A consumer that keeps up keeps both near zero; a rising a2.event.lag means it is falling behind its log.
• Drain outcomes: settled means nothing actionable is left; busy means live per-event claims or a failed event's retry hold remain; stalled means a caught failure in this drain needs a later retry. failed appears only on scheduler deliveries: the drain threw after arming its successor watchdog, the span records the error, and that successor retries the session.
• A failing consumer run marks its a2.event span with the exception and error status. Alert on a2.event.outcome = dead_lettered. A dead-lettered event blocks only later events in its lane, until session.redrive() resolves it; other lanes stay eligible.
• superseded means a stale completion or failure did not change the log.