LiveKit Agents

Read this if you run a voice agent on LiveKit Agents and want the speech it synthesises to carry a watermark and an evidence entry. You will wrap the agent's tts_node with one call. Skip if you mark stored recordings (Audio), or your agent runs at 8 kHz (streaming marks nothing below 16 kHz; "8 kHz bots" below). A keyed deployment does reach callers over SIP ("Survival" below). Time: 15 minutes plus the AudioSeal download. Needs: Python 3.12, pip install laidline-livekit "laidline[audio]", a laidline.yaml from laidline init, TTS at 16 kHz or more.

In short

• mark_audio_frames(marker, frames, ctx=...) marks every synthesised rtc.AudioFrame and writes one evidence entry per call.
• It needs an in-process Marker; entries are streaming entries and carry no Content Credentials manifest.
• Frames come out regrouped into 0.5 s blocks and keep their userdata.
• Below 16 kHz the streaming mark does not survive, so such frames pass through unmarked.
• Unkeyed, the streaming mark does not survive an 8 kHz phone leg. Keyed (watermark.payload_keys; what laidline init sets up), the stream marker's payload reads through G.711, G.726, MP3 and Opus from 12 kbit/s when the audio stays on its 20 ms grid and runs 8 s or more (Audio).
• Read a WebRTC recording with the hardened decode; the plain detector scored 0.26 to 0.28 on it.

laidline-livekit wraps an agent's tts_node: mark_audio_frames(marker, frames, ctx=...) marks every synthesized rtc.AudioFrame with the streaming AudioSeal mark and writes one evidence entry per call. The package page on PyPI (laidline-livekit) repeats the installation, the tts_node snippet and the frame rules.

# (CUDA torch makes a 5.7 GB environment; the AudioSeal mark runs on the CPU, see Install for the CPU build)
pip install laidline-livekit "laidline[audio]" "livekit-agents[openai]>=1.8,<2" livekit

from laidline import Marker, MarkContext
from livekit.agents import Agent
from laidline_livekit import mark_audio_frames

marker = Marker.from_config("laidline.yaml")

class MarkedAgent(Agent):
    async def tts_node(self, text, model_settings):
        frames = Agent.default.tts_node(self, text, model_settings)
        async for frame in mark_audio_frames(marker, frames, ctx=MarkContext(request_id="call-1")):
            yield frame

The wiring above (the package page's return mark_audio_frames(...) form) was run in a LiveKit room on 2026-10-21: an agent, a subscriber that records the room, and a local livekit-server --dev.

What you need

• The weights before the first session: laidline models fetch --audio --no-video --no-image (AudioSeal, 187 MB). Without it the first marked utterance waits for the download.
• TTS at 16 kHz or more ("8 kHz bots" below). OpenAI TTS gives 24 kHz.
• About 10 s of speech for the payload (the floor is 8.0 s on the 20 ms grid; a barge-in costs 0.5 s). A shorter recording or a 5 s slice shows the mark's presence only.
• The agent's config at detect time: set LAIDLINE_CONFIG to the agent's laidline.yaml, then laidline detect --json out.wav. The config is looked up at --config, LAIDLINE_CONFIG, ./laidline.yaml and /etc/laidline/config.yaml, never under LAIDLINE_HOME.
• The hardened decode for WebRTC recordings. After the WebRTC leg the plain detector scored 0.26 to 0.28 against its 0.5 threshold; only the hardened decode found the mark. watermark.audio.decode: auto chose it on the run's detecting host, which has a CUDA GPU; on a CPU-only host it chooses the fast read, so set LAIDLINE_WATERMARK__AUDIO__DECODE=hardened there (slow on a CPU, ../config.md). For a 10 s slice of the recording, LAIDLINE_WATERMARK__AUDIO__PAYLOAD__HARDENED_READ=on was needed as well to read the payload.
• For a local trial, livekit/livekit-server:v1.13.10 --dev --node-ip 127.0.0.1 with ports 7880, 7881 and 7882/udp.

Known limits

• OpenAI's gpt-4o-mini-tts stalled during the run: no first byte within 70 s in 6 of 6 polls, while tts-1 answered in 0.87 to 2.12 s, so the run used OPENAI_TTS_MODEL=tts-1. The OpenAI plugin's read timeout is fixed at 30 s, so a stalled request times out and is retried, and the session falls silent meanwhile. The mark does not depend on the TTS model.
• The agent's job process peaked at 1,068.6 MB, over LiveKit's 1,000 MB memory warning (advisory); expect that warning in the log.

The wrapper needs an in-process Marker; a MarkClient is refused, because the streaming path has no HTTP form. It uses the streaming session of audio.md, so the entries have watermark.streaming: true and no C2PA manifest (manifest_skipped:streaming). Frames come out regrouped into 0.5 s blocks (same rate and channels; per-frame timing at 20 ms granularity is not kept) and keep their userdata (LiveKit's timed transcripts), and the entry is written when the iterator ends, including an interrupted turn. The embedder is the streaming card (watermark.audio.stream_kind: block, the only kind: an experimental context kind was measured and removed: 22.4 % identity). To detect the streamed mark elsewhere, install laidline[audio] there too.

8 kHz bots get no watermark from streaming. Below 16 kHz the streaming mark does not survive (detection 0.0 at 8 kHz), so frames at such rates pass through unchanged and the entry says watermark: null with the warning watermark_skipped:rate_below_16k. Run the TTS at 16 kHz or more and resample for the phone leg afterwards, or mark stored recordings in batch mode.

Survival. The streaming mark survives an offline Opus transcode at 24 and 48 kbps (100 percent on 500 LibriSpeech clips). One real WebRTC session on loopback was run: the subscriber's 20 s recording (RED/Opus, decoded at 48 kHz) read the exact payload. The plain detector scored 0.26 on it, below its 0.5 threshold, so the payload came from the hardened decode (see "What you need"); a 5 s slice gave presence only. LiveKit rooms carry audio as Opus over WebRTC.

Phone legs depend on the mode. Unkeyed, the mark does not survive a simulated phone leg (G.711 codec simulation): 0 percent after G.711 mu-law and after an 8 kHz round trip, the codec path of LiveKit SIP calls to the PSTN. Keyed (watermark.payload_keys), mark_audio_frames marks through the keyed stream marker (Marker.audio_stream) and the 70-bit payload reads back through the simulated phone path. With the shipped default (payload.phase_search off) a 10 s utterance reads 100 of 100 at 16 kHz and 100 of 100 at 24 kHz, and 10 s through simulated G.711 and through Opus at 24 kbit/s read 100 of 100 each (100 per cell; ffmpeg codecs, not carrier paths). Three conditions apply. The payload needs 8 s of delivered audio (the marker holds back 0.5 s, so 8 s fed delivers 7.5 s and reads presence only). The audio has to stay on its 20 ms grid: a jitter buffer or a codec that delays it off the grid keeps the presence and by default loses the payload (6 of 100 at 16 kHz); with payload.phase_search: on (ships off) that cell reads 194 of 200 and G.722 192 of 200. Below 16 kHz streaming marks nothing. Numbers and method: Audio and ../benchmarks.md, "Voice calls".

A real call with the batch mark: 99 percent (100 Twilio number-to-number calls recorded at the callee, G.711 8 kHz, carrier routing unconfirmed; ../benchmarks.md). For an unkeyed agent, or an utterance under 8 s, provenance on a SIP call comes from the evidence entry and from C2PA on stored recordings, not from the streaming watermark. The streaming mark's embed SNR is mean 17.4 dB (5th percentile 8.7 dB), audible on some voices. Method for the other rows: ../benchmarks.md, "Audio robustness".

What a call costs. mark_audio_frames meters the seconds of audio it yields, per call. The entry's duration is metered at 0.05 credits a second: 3 credits a minute of bot speech, 15 for a 5-minute call with the bot speaking throughout, 180 an hour. The caller, hold and the gaps between turns never pass through the marker, so they count nothing; silence inside the TTS output counts, speech cut off by an interrupted turn counts only what left, and a session below 16 kHz counts 0. At monthly prices a 5-minute call costs $0.15 on Lite, $0.05 on Starter, $0.03 on Growth and under $0.01 on Scale; the free allowance of 1,200 credits is 400 minutes. See Licensing.

Dependency note. livekit-agents 1.8 caps OpenTelemetry below 1.45; pip resolves that for you, and it only matters if another package in the same environment needs a newer OpenTelemetry.

What was proven, and how

Unit tests against the real livekit-agents 1.8.3 package (livekit 1.1.18) with the real AudioSeal weights: synthetic rtc.AudioFrames from an async generator, covering marking, userdata kept (and carried from empty frames), stereo at 48 kHz, early aclose, no frames, argument checks and a close cancelled while queued.

Live, on 2026-10-21: livekit-agents 1.8.8, livekit 1.1.20 and livekit-server 1.13.10 (--dev, loopback), with laidline[audio], and OpenAI tts-1 (the gpt-4o-mini-tts endpoint stalled during the run). An AgentSession spoke one 17.3 s utterance through the wrapped tts_node into room laidline-proof. A second participant recorded 20 s, and laidline detect read payload 2d35f25bf1c61e8c37, the agent's evidence entry, as confirmed_shared_key, grade D: the config is a trial issuer with the shared key.

Troubleshooting

Reference

• Integrations: what each integration was proven against, and how.
• Audio: the streaming session this wrapper uses.
• Benchmarks: the audio robustness figures and method.
• Config: the watermark.audio section, including the decode setting.
• Install: extras and the CPU and CUDA torch indexes.