vLLM: text watermark in the sampling loop

Read this if you serve a model with vLLM (or run one with transformers) and need the text it produces to carry a watermark you can detect later. You will install the Laidline logits processor into vLLM, front it with laidline serve, and score text with /detect. Skip if your model sits behind a vendor API you do not run: the watermark has to act while the model samples, so there is nothing to mark afterwards. Time: an hour, most of it the vLLM install. Needs: Python 3.12, an NVIDIA GPU, a vLLM nightly build, laidline[text] and laidline-vllm in vLLM's environment.

In short

• Text can only be marked while the model samples; laidline-vllm does that inside vLLM with a key only you hold.
• SynthID is used for sampled traffic and KGW for greedy traffic; the sidecar asks for it per request.
• It needs vLLM's Model Runner V2 interface, which only nightly builds have; the Tested table names the build.
• Each marked completion gets a text evidence entry (token counts, SHA-256 of the completion, never the text).
• /detect scores text with the same key; paraphrase, translation and heavy editing remove the signal.

Text is the one modality Laidline cannot mark after the fact: the watermark has to bias token choices while the model samples. laidline-vllm is a vLLM logits processor that does this with a key only your deployment holds, SynthID-style for sampled traffic and KGW for greedy traffic. The Laidline sidecar in front of vLLM asks for the watermark on each request and writes the evidence entry; /detect scores text later.

client --> laidline serve (proxy)  --vllm_xargs.laidline-->  vllm serve + laidline-vllm (marks while sampling)
                |  <--  completion + x-laidline-watermarked / x-laidline-scheme  --|
                +--> text evidence entry (token counts, SHA-256 of the completion; never the text)

Proven live on an RTX 5080 (16 GB) against a real vLLM server: 9 of 9 checks, five against vLLM directly and four through the proxy (see "What it proved").

Which vLLM

The plugin uses vLLM's Model Runner V2 (MRV2) logits-processor interface, vllm.v1.worker.gpu.sample.logits_processor. No release has it as of the tested date below: not 0.19.1, and not 0.30.0. It is on vLLM's main branch and in the nightlies.

To reproduce the tested build, install that commit from vLLM's per-commit wheel index (pip install "vllm==0.30.1rc1.dev86+g5b3f280a6" --pre --extra-index-url https://wheels.vllm.ai/5b3f280a6df68c4f8264014380249ba530173ed6), or take the latest nightly from https://wheels.vllm.ai/nightly. Nightly wheels are CUDA 13 builds, so torch must be the cu130 build of the version vLLM resolved: add --extra-index-url https://download.pytorch.org/whl/cu130 for torch, torchvision and torchaudio only. Pin VLLM_USE_V2_MODEL_RUNNER=1: MRV2 is the default on these builds, but vLLM falls back to the V1 runner, which cannot load this processor, when a model or flag needs a feature MRV2 lacks. The pin makes that fail loudly.

Install

Into vLLM's own environment (not the sidecar's):

# with vLLM's Python
pip install "laidline[text]" laidline-vllm

torch and transformers already come with vLLM and satisfy the text extra; the extra also brings the C2PA library that import laidline loads.

• Entry point. laidline-vllm registers vllm.logits_processors: laidline (laidline_vllm.logits_processor:LaidlineTextWatermark). vLLM loads it for every server in that environment; no flag is needed. Only requests that opt in are marked.
• Config. LAIDLINE_CONFIG names a Laidline config; the processor reads watermark.text from it (the whole file must validate). Use the same file, or at least the same watermark.text.key_seed and key_id, as the sidecar and the detector. Without a config file, or with watermark.text.enabled: false, the processor logs a warning and marks nothing; an invalid file stops the server.
• Ack middleware. --middleware laidline_vllm.middleware.ack_header adds x-laidline-watermarked: 1 and x-laidline-scheme: <watermark.text.scheme> to every response. Use the dotted path: vLLM splits it on the last ., so module:function fails at startup.

LAIDLINE_CONFIG=/etc/laidline/config.yaml VLLM_USE_V2_MODEL_RUNNER=1 VLLM_USE_FLASHINFER_SAMPLER=0 \
  vllm serve Qwen/Qwen2.5-0.5B-Instruct --host 127.0.0.1 --middleware laidline_vllm.middleware.ack_header

VLLM_USE_FLASHINFER_SAMPLER=0 is needed unless the CUDA toolkit (nvcc) is installed: the FlashInfer sampler compiles at warmup and the engine dies about two minutes in. vllm serve listens on all interfaces by default, so --host 127.0.0.1 keeps the raw vLLM endpoint off the network beside the proxy (laidline serve binds 127.0.0.1 by default).

The server log confirms the processor: LaidlineTextWatermark active: scheme=synthid key_id=k1 fingerprint=bd58623f ngram_len=5 depth=30. The fingerprint identifies your key without revealing it; the value shown is the one for the public development seed, and your own seed gives another.

The ack header means "the middleware is installed and its config has watermark.text.enabled: true"; it reports the config's scheme on every response, also on a KGW request and on a SynthID request that was declined as greedy, and cannot see whether the engine loaded the logits processor. Without a usable config, or with text disabled, it reports x-laidline-scheme: unknown, which the proxy treats as no ack; the proxy applies its own greedy check (temperature below 1e-5, vLLM's threshold).

Sidecar. The sidecar lives in its own environment: pip install "laidline[image,server,text]", then laidline init ..., then add upstream: {base_url: http://127.0.0.1:8000} to laidline.yaml (init writes none).

Also plan for:

• Disk. vLLM's environment is about 11 GB.
• Image weights on the sidecar. Its start-up self-test marks an image, so even a text-only sidecar downloads about 1.5 GB of image weights on its first start; fetch them ahead with laidline models fetch.

Asking for the watermark: vllm_xargs.laidline

A request opts in with vllm_xargs.laidline, an object with optional key_id and scheme (synthid or kgw). Over HTTP the object must be JSON-encoded, because vLLM's OpenAI server accepts only strings, numbers and lists as vllm_xargs values and answers 400 to a nested object:

{"model": "Qwen/Qwen2.5-0.5B-Instruct", "messages": [{"role": "user", "content": "Write a story."}],
 "max_tokens": 400, "temperature": 1.0,
 "vllm_xargs": {"laidline": "{\"key_id\": \"k1\", \"scheme\": \"synthid\"}"}}

The Python LLM API also takes a plain dict in SamplingParams.extra_args["laidline"]. An unknown scheme, unknown keys or a value that is not a JSON object is rejected with 400 at admission. A per-request key_id is honoured (the processor keeps cores for up to 16 key ids per server).

Behind the proxy, clients send nothing extra: with watermark.text.enabled: true the proxy injects {"key_id": "<watermark.text.key_id>", "scheme": "<watermark.text.scheme>"} JSON-encoded, replaces any client key_id, keeps a client scheme, and for streams sets stream_options.include_usage so it can count tokens. See OpenAI proxy.

The detector scores only watermark.text.scheme: KGW completions detect only with a config whose scheme is kgw (keep a second config for them).

Greedy decoding, speculative decoding, memory

• SynthID does nothing at temperature 0 (it cannot move the argmax), so the processor declines SynthID for greedy requests (temperature below 1e-5) and the proxy records them unmarked with reason greedy_decoding. Use scheme: kgw for greedy traffic: KGW biases a keyed green list and changes greedy output.
• Speculative decoding is not supported. Draft rows are left unmarked.
• Memory. SynthID's per-step work is [rows, vocab, depth] wide. It runs 16 rows at a time; at a 152k vocabulary and depth 30 that is about 140 MB per row. Measured: 16 concurrent marked requests raised GPU memory from 11,397 MiB to a peak of 13,629 MiB (+2.2 GiB), and torch's caching allocator keeps it. Leave that headroom outside --gpu-memory-utilization.
• Device independence. The SynthID sampling table and the KGW green lists are drawn on CPU and moved to the GPU, so text marked on any GPU verifies on the CPU detector. KGW is keyed on len(tokenizer) (151,665 for Qwen2.5), not the padded logits width.

Startup time and the development seed

On the tested machine, startup took 109 s cold and 28 s warm with the model already downloaded on a local disk; expect several minutes when the model or the environment sits on a slow or network-mounted drive. The server is ready when GET /health answers 200 and the log shows the LaidlineTextWatermark active line above.

Use your own watermark.text.key_seed for real traffic. A public development seed is fine for trying the plugin and never for traffic you want to attribute: generate yours with python -c "import secrets; print(secrets.token_hex(16))" and keep it secret.

What it proved

Run on 2026-09-24 with a fixed seed per prompt and 400 tokens per completion (an earlier development key_seed):

The two KGW rows (z and p) were measured before the detector began scoring each distinct n-gram once, so they count every repeat of an n-gram.

A repeat campaign on 2026-10-19 on the same GPU with a vLLM nightly marked 20 of 20 completions through the sidecar and detected 30 of 30 marked texts.

Plus an unknown scheme rejected with 400, and through the proxy: a non-stream completion injected, marked (status: ok, upstream_ack: true) and scored p < 1e-3; a stream relayed with its usage chunk and the same token count in evidence; a greedy request recorded unmarked / greedy_decoding.

transformers

The same watermark without vLLM, for models you run with transformers (laidline-transformers):

pip install laidline-transformers "laidline[text]"

Plain pip install gives whatever torch your index serves. For a GPU, add your driver's CUDA index for torch, for example --extra-index-url https://download.pytorch.org/whl/cu130, and move the model to the device (AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16).to("cuda")). generate_marked builds the watermark on model.device, so a CUDA model works.

from transformers import AutoModelForCausalLM, AutoTokenizer

from laidline import MarkContext, Marker
from laidline_transformers import generate_marked

model_id = "Qwen/Qwen2.5-0.5B-Instruct"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)  # CPU is fine for a 0.5B model, just slow
marker = Marker.from_config("laidline.yaml")  # watermark.text.enabled: true
# in-process detect accepts any tokenizer id while detect_models is empty; once the list is set, model_id must be in it
text, record = generate_marked(
    model, tokenizer, "Write a long story about a lighthouse keeper.", marker, MarkContext(),
    max_new_tokens=400, temperature=1.0,
)
print(record.status, record.reason, record.tokens)  # ok None 400
print(marker.detect(text, model=model_id).verdict)  # marked_by_this_system

generate_marked samples by default (do_sample=True) and writes the same text evidence entry as the proxy.

Run on 2026-10-21 with the install line above in a fresh environment: Qwen2.5-0.5B-Instruct on an RTX 5080 (bf16) marked 3 of 3 completions of 340 to 400 tokens, and all three were detected at grade B (p < 1e-127). Two unmarked controls with the same sampling were not detected (p 0.04 and 0.13). Peak GPU memory was 1.2 GB, and generation ran at 8 to 17 tokens/s on a shared GPU, which is not a throughput claim.

Detection

curl -X POST localhost:8080/detect -H 'content-type: application/json' \
  -d '{"text": "<the completion>", "model": "Qwen/Qwen2.5-0.5B-Instruct"}'

model is the tokenizer id. Over HTTP it must be listed exactly in watermark.text.detect_models; an empty list disables text detection over HTTP (text_detect_models_empty), an unlisted id is model_not_allowed, so anonymous callers cannot trigger Hub downloads or probe local paths. Limits: text longer than max_detect_chars (100,000) is a 413; a model over 256 characters is a 422; only the first max_detect_tokens (8,192) tokens are scored, and for KGW at most kgw.max_detect_tokens (1,024), with a caveat saying so. In-process, marker.detect(text, model=...) is looser: any id while detect_models is empty, and always a local tokenizer directory.

The detector is training-free: the mean g-value (SynthID, centred on the key's own sampling-table mean rather than 0.5) or the green-token share (KGW) as a one-sided z-test. KGW scores each distinct (context, token) n-gram once, so repeated n-grams (the loops greedy decoding falls into) add no evidence. Counting every repeat would let unmarked greedy loops reach z 6.3. KGW's p-value is the exact binomial upper tail of the green count. Present means p < detect_p_value (1e-3). Grades: B over at least min_tokens (200) tokens (for KGW, 200 tokens covered by distinct n-grams, so a repeating text needs more) and with p < grade_b_p_value (1e-5, about 1.9 times the tokens of the 1e-3 gate for the same per-token signal); D below either; never A (Grading). A short or heavily edited marked text that was B at 1e-3 can be D now; the text robustness bench has not re-measured that cost. Paraphrase, translation and heavy editing remove the signal.

The first text detection in a process imports transformers and loads the tokenizer, which took about 54 s on a slow network-mounted filesystem and 2.4 s on a local disk with the tokenizer cached (later calls 10 to 22 ms). The server does not warm this up itself: send one /detect text request after start (or before marking the instance ready) so the first real caller does not wait.

vLLM native watermarking

vLLM 0.30.0 ships its own generation watermark: --watermark-config '{"key": ..., "algorithm": "gumbel"}' (Gumbel-max sampling with a Philox PRF; also dual_key_gumbel, which supports speculative decoding). It is configured per server, in the engine, with no plugin. Laidline does not use it; use the laidline-vllm processor on a nightly.

Limits

• The plugin needs a vLLM build with the Model Runner V2 interface; no release has it yet (see "Which vLLM").
• SynthID does nothing at temperature 0 and speculative decoding is unmarked ("Greedy decoding, speculative decoding, memory").
• A text detection of grade B is the ceiling: grade A is never given to text (Grading), and grade B is a statistical claim (p < 1e-5 per text), not the registry-confirmed grade B of an image or video.
• Paraphrase, translation and heavy editing remove the signal; figures: Robustness.

Troubleshooting

Reference

• Integrations: what each integration was proven against, and how.
• OpenAI proxy: how the sidecar injects the request and counts tokens.
• Config: the watermark.text section.
• Evidence: the text evidence entry.
• Grading: how a detected text is graded.
• Install: extras and the CPU and CUDA torch indexes.