LiveKit integration — instrumentation, egress recording, and webhooks

Parlot stays OpenTelemetry–first: agent telemetry uses OTLP and standard / Parlot semantic attributes. Session audio is stored in Cloudflare R2 via LiveKit Room Composite Egress; Parlot confirms uploads with a signed egress_ended webhook.

Webhooks are recording-only. Session transcript, turns, waterfall, usage, and close outcome come from OTLP (agent configure()). If LiveKit never reaches Parlot with egress_ended, the session still looks complete — only audio_available / duration confirmation is delayed. Opening GET /v1/sessions/:id can still flip audio on via R2 HEAD reconcile when the object already exists in R2. You do not need project-level LiveKit webhooks for telemetry.

---

1. Agent instrumentation (developer path)

Minimal setup

1. Install parlot-instrumentation-livekit from the SDK repo.
2. At the top of your agent entry module, call configure() once, then connect in your entrypoint as usual:

from parlot.instrumentation.livekit import configure

configure()

from livekit.agents import AgentSession, JobContext, WorkerOptions, cli


async def entrypoint(ctx: JobContext):
    await ctx.connect()

    session = AgentSession(...)
    await session.start(agent=..., room=ctx.room)

configure() builds a TracerProvider with an OTLP exporter, registers it with livekit.agents.telemetry, fetches GET /v1/telemetry/bootstrap (when PARLOT_API_KEY is set), patches JobContext.connect to refresh room metadata and start egress when recording is enabled, and auto-installs AgentSession event hooks (semantic/commit layer) plus span processing (pipeline/waterfall layer).

Session bootstrap runs on LiveKit agent_state_changed when the agent transitions initializing → listening (the Starting phase of AgentSession lifecycle). That mints one Parlot session.id, starts a parlot.session span for ingestion, and stamps platform.ref.* keys. Room metadata refresh and egress run after bootstrap (or immediately on JobContext.connect if bootstrap already completed). The session closes via the AgentSession close event.

Canonical agent identity

Parlot stamps framework-agnostic session attributes on parlot.session:

Attribute	LiveKit source
session.agent_id	WorkerOptions.agent_name / job agent_name, overridable via configure(agent_id="…")
session.agent_framework	"livekit"
session.agent_framework_raw_id	LiveKit job id (lk.job_id)
session.agent_chain	Deployment id plus runtime routing (hotel-receptionist → Orchestrator → cancel_task)
gen_ai.agent.version	Agent deployment version — see SDK README agent version

The platform registers deployments in Postgres on first ingest and uses session.agent_id for the Agents portfolio and session filters. Version history appears on the Agents portfolio and workspace when gen_ai.agent.version is stamped.

Environment variables (agent)

Variable	Role
PARLOT_ENDPOINT	Parlot ingest base URL (OTLP, bootstrap, upload-grant). Exporter uses /v1/traces. Required unless you pass endpoint= into configure().
PARLOT_API_KEY	Required for recording / bootstrap. Org-scoped Bearer token minted in Parlot Settings → API Keys.
PARLOT_CAPTURE_CONTENT	Optional. false / 0 / no to reduce prompt/response capture.
PARLOT_DIAGNOSTICS	Optional. Default on. Set off / 0 / false / no to disable SDK self-diagnostics (export/handler failures). Metadata only; drops under sustained failure (no retry storm).
LIVEKIT_URL, LIVEKIT_API_KEY, LIVEKIT_API_SECRET	Standard LiveKit Agents credentials (also used to start egress).

Recording policy (precedence: job metadata → configure(record=…) → Settings → Recording via bootstrap):

• UI: Parlot → Settings → Recording — per-agent toggles for known deployments, plus globs (* or receptionist*,cal-*) for agents not yet ingested
• Code: configure(record=True), configure(record=False), or configure(record=["my-agent*"])
• Dispatch: job metadata { "record": true } or { "record": false }

Errors in the session waterfall

Failed LLM and tool work is surfaced in spans_agent.error_flag at ingest. LiveKit stamps OpenTelemetry exception.type / exception.message on spans such as llm_request_run when a provider call fails; tool failures use lk.function_tool.is_error. The collector maps those signals (and OTLP span status ERROR) into error_flag and error_type on each operational span row.

Turn-level has_error in session detail is derived at query time: any child spans_agent row under that turn’s trace_id with error_flag = true. You do not need a separate turn attribute — fix provider credentials or tool logic locally; Parlot will show the failed span in the waterfall once OTLP reaches the collector.

---

2. Enabling session recording

Recording uses Room Composite Egress (OGG, audio-only) to R2. Flow:

1. Agent calls POST /v1/recordings/upload-grant → JWT-minted temporary R2 credentials (scoped to {org}/sessions/{sessionId}/) + canonical r2://…/audio.ogg URI.
2. Agent starts egress (OGG → R2). When a LiveKit webhook signing key is present on bootstrap, the SDK also sends WebhookConfig (URL + signing key).
3. OTLP exports session.recording.audio_uri, session.recording.anchor_wall_ms, session.recording.egress_id with audio_available=false until confirmation. On egress failure, session.recording.webhook_error is stamped instead.
4. When webhooks are configured, LiveKit POSTs egress_ended to Parlot → audio_available=true + duration. Without webhooks, opening GET /v1/sessions/:id can still flip audio on via R2 HEAD reconcile.

LiveKit integration in Parlot (recommended for faster confirmation)

Org LiveKit API key/secret in Settings → Integrations are for signed egress webhooks (faster audio_available confirmation). Recordings can still start and upload to R2 without them; confirmation in the UI may take longer until you open the session (lazy R2 HEAD reconcile). Session telemetry over OTLP does not depend on this integration.

1. LiveKit Cloud → Settings → Keys → create an API key with roomRecord (needed on the agent to start egress via LIVEKIT_*).
2. Parlot → Settings → API Keys — mint a key for the same org; set it as PARLOT_API_KEY on the agent.
3. Enable recording in Parlot → Settings → Recording (toggle the agent, or set globs / * for new agents), or call configure(record=True) / place { "record": true } in job metadata for unnamed dispatches.
4. Optional — faster confirmation: Parlot → Settings → Integrations → LiveKit (PUT /v1/settings/integrations with name: "livekit"):
   • LiveKit API key — used in WebhookConfig.signing_key when starting egress with webhooks
   • LiveKit API secret — used server-side only to verify egress webhook signatures (encrypted at rest; never sent to the worker)

No project-level webhook URL in LiveKit Cloud — when the signing key is present, the SDK sends WebhookConfig on each egress request. Not required for session telemetry (OTLP).

---

3. If recording stays “processing”

Symptom	Likely cause
No egress started	Settings → Recording / configure(record=…) / job { "record": true }; agent LIVEKIT_*; R2 on collector (503 r2_not_configured)
Slow audio confirmation (no webhook)	Expected without LiveKit integration — open the session to trigger R2 HEAD reconcile, or save LiveKit API key/secret
audio_available=false until opened	Webhook delivery failed or signature mismatch; opening the session may still reconcile via R2 HEAD
No URI in session	Egress failed — check agent logs; grant expired; Settings → Recording / configure(record=…) / metadata
Delayed confirmation	Parlot R2 HEAD reconcile on GET /v1/sessions/:id may flip audio_available without webhook

---

4. Conversation contract translation (LiveKit)

The conversation attribute contract is framework-agnostic. The LiveKit adapter (parlot-instrumentation-livekit) reads LiveKit Agents OTel spans in-process, stamps Parlot attributes from parlot.core.attrs, emits handoff spans as parlot.agent.handoff, and strips all lk.* keys before OTLP export (SanitizeVendorAttrsSpanExporter).

Span name → gen_ai.operation.name

LiveKit span (vendor)	gen_ai.operation.name	Notes
user_turn	speech_to_text	Maps transcript → turn.user_text
agent_turn	agent_response	Maps response + latencies → turn.* attrs + metrics
llm_node / llm_request_run	chat	Instructions → agent.instructions_excerpt; tools → agent.tool.names
function_tool	execute_tool	agent.tool.name, agent.tool.is_error, tool payloads
parlot.agent.handoff	agent_handoff	Emitted from conversation_item_added (agent_handoff)
tts_node	text_to_speech	TTFB → turn.tts_ttfb_ms metric
eou_detection	end_of_utterance_detection	voice.eou.language → session.languages
amd	classify_contact	voice.amd.category → session.amd

Span name → agent_role (waterfall / timeline)

LiveKit span (vendor)	agent_role	Notes
user_turn / eou_detection	stt	User transcript on turn.user_text
agent_turn / drain_agent_activity	pipeline	Agent turn boundary; text-console input may carry turn.user_text
llm_node / llm_request_run	llm
tts_node / tts_request_run	tts
function_tool	tool
parlot.agent.handoff	handoff	Also written to session handoffs
amd	amd

When a span carries explicit agent.role, the collector prefers that value over the LiveKit default mapping.

Span name → agent.stage (timeline labels)

Each operational span carries agent.stage (SDK on export; collector derives from span name when missing at ingest). UI labels combine role + stage — e.g. llm + run → LLM · API call. Multiple spans per role per turn is normal LiveKit nesting (llm_node → llm_request → llm_request_run). The table below covers the LiveKit mapping.

LiveKit span	agent.stage	UI label
llm_node	node	LLM · inference
llm_request	request	LLM · request
llm_request_run	run	LLM · API call
tts_node	node	TTS · synthesis
tts_request_run	run	TTS · API call
agent_turn	turn	Turn
user_turn	turn	STT · utterance

In events mode, llm_node does not stamp turn.agent_text from chat context (prior-turn assistant text may be stale); trust agent_turn for authoritative agent utterance.

Vendor attribute → Parlot attribute (export)

LiveKit attribute (read in processor)	Parlot attribute (OTLP)
lk.instructions / lk.chat_ctx	agent.instructions_excerpt; on llm_node, last role=user message from chat context → turn.user_text (excludes function_call items)
lk.function_tools	agent.tool.names (JSON array)
lk.function_tool.name	agent.tool.name
lk.function_tool.is_error	agent.tool.is_error
lk.user_transcript / lk.user_input	turn.user_text
lk.response.text	turn.agent_text
lk.e2e_latency	turn.e2e_latency_s
lk.response.ttft	turn.llm_ttft_s
lk.transcript_confidence	voice.stt.confidence
lk.transcription_delay	turn.transcription_delay_s
lk.end_of_turn_delay	turn.eou_delay_s
lk.participant_identity	participant.channel_identity, session.user_id
lk.eou.language	voice.eou.language
lk.amd.category	voice.amd.category

Turn metrics (LiveKit source → Parlot histogram)

Parlot metric	Source span	LiveKit attribute
turn.e2e_latency_ms	agent_turn	lk.e2e_latency
turn.llm_ttft_ms	agent_turn	lk.response.ttft
turn.tts_ttfb_ms	tts_node	lk.response.ttfb
turn.transcription_delay_ms	agent_turn	lk.transcription_delay
turn.eou_delay_ms	agent_turn	lk.end_of_turn_delay

LiveKit-native lk.agents.turn.* and lk.agents.usage.* metrics may still land in otel_metrics_raw but are not used by the Parlot pipeline.

Session close (parlot.session.close)

LiveKit CloseReason	session.close_reason
participant_disconnected	participant_disconnected
user_initiated	user_initiated
error	error
task_completed	task_completed
job_shutdown	job_shutdown

Primary emission: AgentSession.on("close") via event hooks.

AgentSession events (semantic / commit layer)

When configure() patches AgentSession, Parlot subscribes to LiveKit events for turn boundaries and session aggregates. Operational waterfall detail stays on OTel spans.

LiveKit event	Parlot output
agent_state_changed (initializing → listening)	Session bootstrap: parlot.session, ContextVar scope, deferred room refresh + egress
conversation_item_added (user/assistant message)	parlot.turn root trace per committed message; deduped by item id
conversation_item_added (agent_handoff)	parlot.agent.handoff span + session handoff count / agent chain
user_input_transcribed (final)	Pending speaker_id / language for next user turn (livekit_stt_event diarization). Set STT language: "multi" (or equivalent auto-detect mode) so UserInputTranscribedEvent.language is populated and stamped on turn.language.
function_tools_executed	session.tool_call_count (batch count; individual tool spans still export)
session_usage_updated	Authoritative session.total_*_tokens (overrides span accumulation)
error (non-recoverable)	session.close_error on close when no explicit close error
close	parlot.session.close with normalized session.close_reason

Span-based turn emission (user_turn / agent_turn → parlot.turn) is disabled once event hooks install (turn_source=events). Span enrichment for latency, confidence, and pipeline attrs continues unchanged when LiveKit OTEL is enabled. Semantic session lifecycle (bootstrap, turns, close, tokens) does not depend on the job_entrypoint span.

Export allowlist (LiveKit)

Spans exported to Parlot (after local processing): parlot.session, parlot.turn, parlot.session.close, parlot.agent.handoff, and operational names user_turn, agent_turn, llm_node, llm_request, llm_request_run, tts_node, tts_request_run, function_tool, eou_detection, drain_agent_activity, amd.

---

5. Human escalation

Human takeover (SIP bridge, warm transfer, supervisor barge-in) is not the same as AI sub-agent routing. Internal AI handoffs use normal sub-agents and do not mark a session escalated. Human escalation must be signaled explicitly.

Recommended contract

When a live rep takes over, register them as a human_rep participant. Parlot marks the session as escalated and renders human_rep as a distinct participant in the interaction graph.

Signal	When to use
record_human_rep(participant_id)	Explicit call when you know the rep identity (SIP leg joined, supervisor connected)
human_escalation() context manager	Wrap a dial/transfer so the next participant who joins is stamped automatically
configure(auto_escalate_sip=True)	Zero-code path for standard SIP warm transfers
turn.participant_role=human_rep	Ground truth from speech (set automatically when the rep is registered and speaks)

agent.transfer.reason=human_escalation on handoff spans is an optional analytics signal for why a transfer happened. It is not used for escalation detection today.

Explicit call

from parlot.instrumentation.livekit import record_human_rep

record_human_rep("support_rep_jane", label="Jane")

This stamps the rep on the live session (works even if the rep never speaks) and registers the participant so future turns use turn.participant_role=human_rep.

LiveKit: SIP warm transfer (auto-detect)

from parlot.instrumentation.livekit import configure

configure(auto_escalate_sip=True)

Or match participant metadata:

configure(
    escalation_metadata_match={
        "type": "agent_transfer",
        "is_human": "true",
    }
)

LiveKit: context manager around a dial

from parlot.instrumentation.livekit import human_escalation

async def transfer_to_support_desk(session, sip_uri):
    with human_escalation(label="L2 Support Queue"):
        await session.dial(sip_uri)

Role vocabulary

turn.participant_role	Meaning
user	Caller / end user
agent	AI worker
human_rep	Live human who took over the session

Participant roles use turn.participant_role with the values above.

---

6. Custom metadata and external references

Use these helpers to attach your own data to a live Parlot session. Import them from parlot.instrumentation.livekit (same package as configure()).

Custom session metadata

set_session_metadata stamps key/value pairs under the reserved session.metadata.* namespace. Values appear on the session in Parlot (Overview → Custom metadata).

from parlot.instrumentation.livekit import set_session_metadata

set_session_metadata(order_id="12345", crm_ticket="TKT-9")

Notes:

• Call after the session has started (agent_state_changed → listening). Values set with no active session are ignored.
• Keys are normalized to session.metadata.<name> (do not invent a different prefix).
• Values are stored as strings. Prefer short identifiers and labels; avoid transcripts or PII that should stay out of session attributes.
• Turn-level custom metadata is not supported yet.

For a single key:

from parlot.instrumentation.livekit import set_session_attribute

set_session_attribute("retry_count", 2)

External references (searchable IDs)

add_platform_ref attaches a searchable external ID to the session (for example a CRM ticket or your own call ID). Paste that value into Parlot session search / resolve to open the matching session.

from parlot.instrumentation.livekit import add_platform_ref

add_platform_ref("crm_ticket", "TKT-9")

LiveKit room / job IDs are stamped automatically as platform.ref.* during session bootstrap. Use add_platform_ref for your own identifiers. Optional framework= defaults to "custom":

add_platform_ref("order_id", "ORD-100", framework="shopify")

---

7. LiveKit observability (OTEL): enabled vs disabled

This section describes what Parlot stores and surfaces when LiveKit Agents pipeline tracing is active vs when it is not. It is not about whether the agent calls parlot.configure() — Parlot instrumentation must stay on for any ingest.

Terminology

Setting	Meaning
parlot.configure() on	Required. Parlot registers a TracerProvider with livekit.agents.telemetry, exports OTLP to Parlot, and installs AgentSession event hooks.
LiveKit pipeline OTEL on	LiveKit emits internal spans (user_turn, agent_turn, llm_node, tts_node, function_tool, eou_detection, amd, …) into the active tracer. With Parlot configured, those spans are enriched and exported.
LiveKit pipeline OTEL off	LiveKit does not emit pipeline spans (or tracing is a noop). Parlot still exports event-sourced spans (parlot.session, parlot.turn, parlot.agent.handoff, parlot.session.close) and any spans Parlot creates itself.

Session bootstrap and close no longer depend on job_entrypoint. Bootstrap runs on agent_state_changed when the agent transitions initializing → listening (AgentSession lifecycle); close runs on the AgentSession close event (events reference).

Empirical comparison (same agent, two local sessions)

These sessions exercised the hotel-receptionist agent against the same local stack; only LiveKit pipeline OTEL differed:

	OTEL on 019e801e…	OTEL off 019e8020…
Duration	~54s, 9 turns	~73s, 10 turns
sessions row	Yes — voice, livekit, closed (tool_failure)	Yes — same shape
session_turns	9 rows (user + agent roles)	10 rows
session_agents waterfall	48 spans (llm, tts, tool, stt, pipeline, handoff)	83 spans (same roles; longer call → more rows)
Transcript / redacted previews	14 span_content_redacted rows	23 rows
Handoffs	4 (session_handoffs)	4
Session aggregates	tokens, turn count, tool count, agent chain, topology, intent	Same fields populated
External refs	room/job/platform refs	Same
Turn sentiment (close worker)	4 user turns scored	5 user turns scored
turn.e2e_latency_ms metric	Present (11 points)	Absent
turn.e2e_latency_s on spans	1 span	0
voice.stt.confidence	1 stt span	2 stt spans
media_segment_* on turns	All 0	All 0
Recording	audio_available=false (egress quota)	Same

Both sessions received a full semantic session record and a non-empty waterfall. The clearest OTEL-on-only signal in this pair was end-to-end turn latency (turn.e2e_latency_s / turn.e2e_latency_ms), which is sourced from lk.e2e_latency on LiveKit agent_turn spans. Neither session populated non-zero turn.llm_ttft_ms, turn.tts_ttfb_ms, turn.transcription_delay_ms, or turn.eou_delay_ms (metrics rows existed but summed to zero).

Coverage matrix

Capability	OTEL off (events + Parlot)	OTEL on (+ LiveKit pipeline spans)
Session resolve, id, channel, framework, duration	Yes	Yes
Platform external refs (room_sid, job_id, …)	Yes	Yes
Turn timeline (who said what, turn index, trace-per-turn)	Yes — conversation_item_added → parlot.turn	Yes
Session close reason / close error	Yes — close / error events	Yes
Token totals	Yes — session_usage_updated	Yes (+ per-span usage on LLM rows when spans fire)
Tool call count	Yes — function_tools_executed	Yes
Handoffs (count, chain, session_handoffs)	Yes — agent_handoff items + spans	Yes
Intent / topology / agent chain at close	Yes — derived server-side	Yes — richer when LLM spans carry agent.instructions_excerpt
Per-turn latency histograms	Partial — from ChatMessage.metrics on committed items when LiveKit attaches them (per-turn latency)	Full — agent_turn / tts_node / user_turn lk.* attrs → turn.* metrics
Waterfall: LLM / TTS / tool rows	Only if pipeline spans still emit*	Yes — llm_node, tts_node, function_tool, …
STT confidence (voice.stt.confidence)	Unreliable — needs user_turn span	Yes — on stt role rows
EOU / AMD (eou_detection, amd spans)	No	Yes — when enabled in the agent
Provider errors on spans (error_flag, exception.*)	No failed-provider rows	Yes — failed llm_request_run / function_tool
Per-tool I/O previews	Batch count only from events	Yes — tool.input/output.payload_preview on tool spans
Audio ↔ turn alignment (media_segment_*, speech_wall_*)	No today (events path does not stamp timing)	Yes — from user_turn / agent_turn span timestamps
Turn has_error in UI	Stays false (derived from child session_agents.error_flag)	Yes when a child span errors
Fleet interaction graph / per-agent latency edges	Degraded — few operational edges	Yes — needs LLM/tool span graph
Session recording (R2 egress)	Yes — independent of pipeline OTEL	Yes

*When parlot.configure() registers the tracer provider, LiveKit may still emit pipeline spans even if LiveKit Cloud observability is toggled off elsewhere. Treat “OTEL off” as no pipeline spans reaching Parlot, not merely disabling a cloud dashboard.

What Parlot can promise without LiveKit pipeline OTEL

With parlot.configure() only (events + Parlot OTLP, no LiveKit pipeline spans):

• Episode record — paste session id → read the call: turns, roles, text, close outcome, coarse usage, handoffs, intent label.
• Session list / search — metadata, end state, token totals, participant count.
• Evals on transcript — session-close workers can still score user turns from stored text.
• Recording — egress to R2 when PARLOT_API_KEY and recording policy are set; org LiveKit integration is optional (faster webhook confirmation; otherwise R2 reconcile).

Not available (or materially degraded) without pipeline spans:

• Expanded turn waterfall (LLM/TTS/tool timing tree) and span detail panel I/O.
• Turn-level latency charts beyond what ChatMessage.metrics provides on committed messages.
• STT quality, EOU, and AMD pipeline signals.
• Accurate has_error per turn and provider-level failure attribution.
• Audio segment bars synced to turns (media_segment_* stays at 0).
• Fleet graph and per-agent operational analytics.

Recommendation

• Debugging / SRE / latency work: keep LiveKit pipeline OTEL active and parlot.configure() on. The waterfall, error attribution, and latency histograms depend on it.
• Transcript-only / compliance / eval workflows: parlot.configure() + AgentSession events are sufficient for the semantic session record; do not expect a populated debugger waterfall.
• Never disable all tracing if you still call parlot.configure() — Parlot replaces the provider. “Turn off OTEL” should mean disabling LiveKit’s pipeline span emission or not routing it to Parlot, not removing Parlot instrumentation.

---

8. Mental model

Two layers: LiveKit events commit conversational semantics (turns, handoffs, usage, close). LiveKit OTel spans carry the pipeline waterfall (LLM, TTS, tools, AMD, EOU). The adapter maps both to Parlot OTLP keys; the collector stays framework-agnostic.

Source	Configured by	Carries
OTLP	Agent (PARLOT_*, configure())	Traces, refs, optimistic session.recording.audio_uri, anchor, session.recording.egress_id, or session.recording.webhook_error
Upload grant	Agent → collector	Scoped temp R2 creds (session_token) for LiveKit egress
Egress webhook	LiveKit → collector	Confirms upload, duration, audio_available=true
R2 reconcile	Session API (lazy)	Safety net when webhook is delayed

For platform env (R2_ACCOUNT_ID, R2_CONTENT_BUCKET, R2_ACCESS_KEY_ID, R2_SECRET_ACCESS_KEY on collector + API, PARLOT_PUBLIC_INGEST_URL for webhooks, DATABASE_URL, integration secrets), see the platform local E2E runbook. R2 key scheme: {org_id}/sessions/{session_id}/audio.ogg.