Robustness baseline

Read this if you integrate or evaluate Laidline's watermarks and need to know which edits a mark survives, why the others break it, and what a decode-side fix could recover. You will get the measured survival rates per medium, the mechanism behind each failure and the headroom before anything has to be re-embedded. Skip if you want the full result tables (Benchmarks) or how a result becomes a grade (Grading). Time: 20 minutes for sections 1 to 3; the rest is reference. Needs: nothing; pip install laidline-bench reruns the suites.

In short

• A mark survives compression, resizing, mild crops, tone edits and codecs; it does not survive large crops, rotation, screenshots, tempo or pitch changes, heavy video compression or a paraphrase.
• Every failure is traced to a mechanism (a whole-frame resize, a 20 ms bit grid, an equal-weight average, a global z-test), so several gaps are decode-side and can be closed without re-marking existing assets (section 5 separates them from the ones that need a new embedding).
• Rates come with Wilson 95 percent intervals; read small samples (music n=30, video n=43) as coarse.
• Oracle rows are not product numbers: they estimate what a perfect resync could recover.
• Only benign edits are in scope; adversarial removal is not.

What each Laidline watermark survives today, why the failures happen, and how much a decode-side fix could recover before anything is re-embedded. The numbers come from the suites in laidline-bench (laidline bench --help). The full per-medium tables are in Benchmarks. Benign edits only: adversarial removal is out of scope.

New images are marked with PixelSeal by default (watermark.image.scheme: pixelseal); the image numbers below are TrustMark's, the scheme PixelSeal replaced. What the PixelSeal scheme survives with the product's decoder, its unmarked-photo test, the analytic false-accept bound, the real-screenshot measurement and its timings are in Grading, "PixelSeal image scheme" (module laidline_bench.pixelseal_product).

1. How to read the numbers
2. How each pipeline works
3. Cross-media matrix
4. Root causes
5. Headroom and the hardening items
6. Corrections and caveats recorded

1. How to read the numbers

The runs

• Every run: RTX 5080, torch 2.14.0+cu130, TF32 off for matmul and cuDNN, cuDNN deterministic, cuDNN benchmark off (the machine block of each result; the product's default, watermark.cuda_precision: strict), with two exceptions: the video keyframe scheme ran on the CPU, and the phantom run's machine block does not record the TF32 and cuDNN flags (the bench turns TF32 off on cuda).
• The per-item records and the decoders' soft outputs (*.arrays.npz) are kept with the original runs and are not part of the package; with them soft voting, windows, offsets and thresholds can be evaluated offline without a re-run, and only a new embedding needs a new run. The packages in compliance/data/ ship the rates, the sweeps and "Where it breaks", not the per-item data.

Rates, intervals and small samples

Every rate comes with its Wilson 95% interval. With small n the interval matters more than the rate:

• Music n=30: a single clip is 3.3% of the corpus. Read music rows as coarse.
• Video n=43 (41 for keyframes): one clip moves a rate from 100.0% to 97.7%, and 0 of 43 unmarked clips bounds the false-positive rate only below 8.2%. n=43 is the robustness sample; the false-positive evidence is the 2026-10 null campaign's 4,854 unmarked clips (upper bound 0.079%).
• Text sweeps n=100 and image sweeps n=98: a sweep point locates a break, not a precise rate.

Operating points and false positives

Audio columns: "detected" is the detector score at or above the threshold; "exact" is detected with all 16 issuer bits right, which is what detect needs to name the issuer. Image and video rates are exact decodes; text rates are p < 0.001 unless the column says grade B.

Oracle rows

An oracle row applies the true inverse of an attack before decoding: the true rotation angle, the true crop position on a mid-grey canvas of the original size, the photo's true box on a screenshot, a trim rounded up to a whole 20 ms audio block, a video crop padded back at its true offset. They are not product numbers. They estimate what a perfect decode-side resync could recover, so an oracle far above its attack says "search at decode time", and an oracle near its attack says "the mark itself is gone; change the embedding".

• The crop oracles (image centre and corner crop, video crop) are not guaranteed ceilings. The grey canvas blanks the area the crop lost, while the plain crop keeps its content at a new scale (TrustMark and VideoSeal resize the whole frame), so an oracle can fall below its attack. The image bench flags such points; in this run there were none (an earlier small pilot had one, at corner crop 80). Treat crop oracles as estimates.
• Text has no oracle rows.

Sweeps and "Where it breaks"

Each sweep reruns one attack family over a parameter ladder on a seeded subsample, with the same metrics as the main rows. Its "Where it breaks" row reads:

Margins, per medium:

Hints: mostly near suggests soft voting or a stronger ECC; mostly mid or far with an oracle of at least 80% suggests a resync search; mostly far with a low or no oracle suggests an embedding change. Text uses "more evidence" (near) and "windowed scoring" (embed) instead.

2. How each pipeline works

Paths are inside the installed laidline package unless they start with trustmark/ or audioseal/ (the installed trustmark 0.9.2 and audioseal 0.2.0). Line numbers refer to the source that was measured and can shift between releases.

Image: TrustMark 0.9.2

A recovered crop would still be fingerprinted whole, so it would likely land at grade C (the pHash distance of crops is not measured; in keyed mode that C is on the full tier only, the public tier shows D or nothing): hardening a crop decode also means changing confirmation.

Audio: AudioSeal 0.2.0

Where the mark lives, from the filters (speech, 100 clips): the batch bits need the low band (high-pass 300 Hz leaves 88.0% exact, 500 Hz 1.0%, while detection stays at 100.0%) and survive a 4 kHz low-pass (99.0%). The streaming bits need the band above 6 kHz (low-pass 8 kHz leaves 99.0% exact, 6 kHz 0%). Streaming detection survives the bench's gentle two-pole filters (lowpass_4k and bandpass_phone: 100.0% detected, 0% exact) but not the brick-wall 8 kHz round trip or G.711 (0% detected), which is why a phone leg removes the unkeyed streaming mark. A keyed stream's payload reads through simulated G.711 (Audio).

Audio: the keyed P70 payload

In keyed mode the 16 bare bits above become audioseal-p70-r1 (Payload keys, "Audio"):

• 70 bits under a BCH code (m = 7, t = 9);
• spread over 1.0 s cells of an 8 s cycle;
• read per 1.0 s segment on 20 ms grids and decoded per word (Chase-II p = 8);
• one tag check per call.

That fixes two of the 16-bit failures in "Root causes" below. The whole-clip mean is replaced by per-segment reads with error correction. The grid problem is handled two ways: the default read (CPU) covers files whose start is intact on the 20 ms grid, and deep scanning (GPU, off by default) searches the phase for trims, offsets and excerpts. Under the hardened decode a span read also reads a mark inside a longer recording (measured at 9 s in 30 s, 12 and 24 s in 120 s) at the phase search's own coarse peaks (98, 93 through Opus 24 kbit/s, and 98 of 100 on fresh units; Grading). On 2 CPU threads it adds 0.02 s to a 120 s detect that reads at the first peak (the call is about 67 s, mostly the hardened decode) and about 2.9 s when all three phases are read. Measured on 200 fresh clips (Benchmarks, "Keyed audio payload"):

There were 0 wrong payloads and 0 own-accepts on 1,591 null clips and 1.2 million decoder-level nulls. Lines not met:

• Third-party label on own marks (closed by the rule shipped, below): under the two-grid rule that ran, one own mark was labelled third-party at Opus 12 kbit/s under deep scanning, and the same one-word mechanism showed on two more own marks in the forced reads. The rule now shipped needs the foreign head on three or more grids (chosen after that data; 0 of 31 such calls, no power). The powered measurement (3,000 of Laidline's own marks under foreign keys, each read scored as if no own payload had been chosen) failed its line (7 or more labels fail): the three-grid rule labelled 13 of 3,000 (the two-grid rule 80, R6' (two grids and two word positions) 21). The own payload was chosen on all 3,000 calls, so the product showed none of the 13 labels. The rule now shipped also needs each counted word to score at least 0.97, a floor chosen after that data and tested once on a fresh 3,000 own marks: 0 of 3,000 labelled (rate shown below 1 per 1,000), 2,999 of 3,000 own heads and 150 of 150 foreign marks kept; under Opus 12 kbit/s the product labelled 129 of 150 foreign marks, reported as measured (Payload keys, "Audio"; Benchmarks, "Third-party rule S97 and its hold-out"). Measured at full float32 with deterministic cuDNN, the shipped CUDA precision.
• Timing: in the quiet re-time the default 3-grid reader met every CPU time line. Its 30 s miss in the first run was load. The 5-grid reader, which is not the default, missed at 12 s.
• Timing (report only): one file-registry cell of the keyed audio timing check is unexplained and indicative. The measured channel stays within the bound stated in Payload keys residual 10 on every backend measured; the mitigations are the equal-work construction and the registry recommendation (the file registry is the development backend; production uses Postgres or S3).
• Stream against file: the stream path read 3 more than the file path at Opus 12 kbit/s. The line failed by its letter; it was accepted after the data.
• The 89-fresh-clip check of Opus 12 kbit/s on whole files is UNPROVEN at that size.
• GPU against CPU, as shipped (rules pre-registered): Laidline runs its models on a GPU at full float32 with deterministic cuDNN (watermark.cuda_precision: strict, the default; Configuration). With decode: hardened and the keyed constants on both, the same 350 saved clips were read by the product on the GPU (the precision set by the product itself, as shipped under watermark.device: auto) and on the CPU. The clips are deliberately hard: short marks inside longer clips, codec cuts and unmarked clips that came near a threshold; they are not population rates. The line, 0 units differing in a decision (presence on either tier, payload, the chosen payload, grade, status, tag, third-party label, deep-scan stage and payload read), passed: 0 of 350.
  • The path that reported presence differed on 1 unit, a 0.5 s mark inside a 30 s clip: both devices showed it present, the CPU on the windowed path and the GPU on the located path. The two devices read it at the same phase, but they disagree on the clip's last seconds, where the mark is not: AudioSeal's detector carries a recurrent (LSTM) layer through the clip, and the tiny arithmetic differences between the CPU's and the GPU's float32 kernels grow along it: on this clip, from about 1e-9 in a frame's score in the first second to a few hundredths from 18 s on, and up to 0.76 in the last 5 s. Scores that drift across 0.5 there change the shares of windows that hold no mark: the best 1 s window on both devices lies there, at 0.81 of its samples on the CPU (over the 0.70 threshold: windowed) and 0.64 on the GPU (under it, so the GPU reported the 0.5 s window on the mark itself: located), and the CPU's best 0.5 s window, which centres the cell statistic, moved there too.
  • The cell statistic agreed within 1e-6 wherever both devices searched the same phase and cell span (226 of 256 compared). The phase or the span moved on the other 30, and 28 of those differed by more than 1e-4 (26 unmarked clips, 2 excerpts); none changed the cell gate's admission.
  • Before this setting, the GPU ran torch's default reduced-precision (TF32) convolutions. The same 350 clips measured that way: 9 differed, 4 of them in presence (3 short excerpts; 1 unmarked 30 s clip shown present on the GPU's full tier) and 1 on the public tier, 0 in payload, tag, grade B or third-party label. The decode compares shares of samples scored above 0.5, and a gate statistic, with fixed thresholds; where a share or the statistic sits near its threshold, a small arithmetic change decides it the other way.
  • Every published GPU figure for keyed audio detection (the cell gate's floor and hold-out counts, the excerpt rows, the third-party hold-out) was measured at full float32 with deterministic cuDNN, which is now what the product runs. The GPU records of this line are identical, field for field, to a full-precision run made with the bench's own switch (350 of 350), and the CPU records to that run's CPU records (350 of 350).
  • Full float32 does not make the two devices bit-equal on long clips (the recurrent layer above): GPU and CPU agree on decisions on these 350 clips, not on every statistic.

Audio: voice calls (the 2026-10 matrix)

The keyed buffered stream, simulated phone paths, read with the shipped defaults of 2026-10-27 (rows and method in Benchmarks: Voice calls; the rows a voice bot can rely on in Audio). What limits the payload, in the order it was found:

• Length. Presence reads from 1.5 s delivered; the payload from 8.0 s delivered on the 20 ms grid (a stream fed 10 s delivers 9.5 s, since 0.5 s is held back at close). Claim 10 s.
• Alignment. The default read counts 320-sample blocks from the first sample. An excerpt starting 1 to 319 samples off the grid read 6 of 100 (16 kHz) and 4 of 100 (24 kHz) against 100 of 100 on the grid; a pure delay of 2 to 480 samples that is not a whole block read 0 of 50. Presence survives (77 to 81 %). This is what fails G.722 (22-sample delay) and AMR-WB (95 samples), which read 50 of 50 and 48 of 50 once the delay is removed. The keyless phase search added after the run (Payload keys, "Audio") targets it; it is off by default (payload.phase_search) and was measured in the v2 matrix (Audio, "Voice calls").
• Codecs that destroy the mark. Opus at 8 kbit/s (4 of 50 even aligned), Speex (0 of 50) and AMR-NB (28 of 50 aligned). The mark's residual sits under 4 kHz, 23.5 dB under the speech; bandwidth is not the cause.
• Noise. White noise loses half the payloads at about 22 dB SNR; babble reads 94 % at 20 dB. Packet loss up to 10 % (repeat or zero fill) costs nothing measurable (100 of 100 per cell).
• Rooms. At 0 dB direct-to-reverb ratio the payload is usually lost (3 to 13 % at RT60 0.1 to 0.5 s) and presence stays at 83 to 94 %; +6 dB reads 67 % and +12 dB 100 % at RT60 0.3 s. Reverb followed by G.711 reads 3 %.
• Wrong payloads, not met. The line "0 wrong payloads" failed on 6 of 6,200 marked units (reverb, white noise at 20 dB, G.722), each with a failed tag and a payload of another issuer. No tier now shows a payload whose tag failed.

Video: VideoSeal 1.0 and the keyframe stopgap

Text: statistical mark at generation time

3. Cross-media matrix

Headline rates from the main rows. Audio cells read "detected / exact" at 0.39 when the two differ. A second value in a cell is the other mode or scheme named in the column header, where it differs. n/a: not applicable or not measured.

The KGW greedy arm (a small model decoding greedily) detects 95.0% at identity but reaches grade B on 1.0%: see section 4.

4. Root causes

Each failure traced to its mechanism, with the sweep that supports it. Rates are exact decodes (audio: issuer exact) unless marked "detected".

Image: the whole-frame resize and the hard BCH

B and C tolerate rotation better than Q (at 5°: B 40.8%, C 28.7%; at 4°: B 81.6%, C 55.2%) and C tolerates the 75% crop better (74.7%). Horizontal flip survives on every variant (Q 100.0%); vertical flip is not a bench row, and a development probe found it does not decode.

Audio: the 20 ms bit grid and whole-clip averaging

Music (n=30) repeats the batch pattern with a weaker issuer read: 83.3% exact with no change, 23.3% to 40.0% after Opus 10 to 24 kbps (mostly near: 22 near of 22 failing at 24 kbps), and no clip exact after a 1% tempo change.

Video: the equal-weight average and the BCH radius

VideoSeal margin counts (near, mid, far) can differ from the exact-decode rates by 1 to 2 clips: the 248-bit margin also counts the 28 zero-padding bits, which the decoder restores before BCH. The same holds for the VideoSeal margins in section 5.

VideoSeal splices hold (25% of the frames marked: 95.3%), so partial edits are not the weakness; compression and framing are. The keyframe scheme is more robust to compression and scale (crf 28 78.0%, scale 0.5 90.2%) but fails every edit that moves its grid.

Text: the global z-test and scheme-only scoring

5. Headroom and the hardening items

Image: the targets (variant Q)

Every extra decode attempt is another chance of a phantom. Each search is paired with fingerprint confirmation and a phantom re-run; the saved unmarked soft outputs (500 originals in the image run, 5,000 photos in the phantom run, 100 logits per decoder each) let a search's false-positive cost be estimated offline before it ships.

The hardening items, mapped to the evidence

Decode-side items change only detect: cheap, and they help assets that are already marked. Embedding items need a new mark: existing assets keep today's robustness, and every bench reruns.

Proposed targets where none is set

Proposed targets, each grounded in the curve it names. Every one keeps its medium's false-positive target: audio below 1.0% at the operating point (not met for unkeyed music, 7.25%, in the 2026-10 null campaign), video 0 mis-decodes and 0 unmarked decodes, text the binomial guard on 0.1% for presence and a grade B gate at 1e-5.

No target is proposed for paraphrase (49.5% to 57.0% on the sampled arms, 30.5% for greedy KGW), for greedy KGW grade B (1.0%) or for heavy video compression (crf 35 14.0%): each needs a scheme or embedding change beyond the current scheme.

Paraphrase is a designed limit, not an open item. A text watermark lives in the choice of tokens; a full rewrite replaces those choices, so no token-level scheme survives it, and Laidline makes no claim that it does. This sits inside the product's design rule for every medium: to strip a Laidline mark, an attacker has to destroy and rebuild the media rather than copy it, which makes the result their own work in substance. The measured rates above (about half the marks survive a same-model paraphrase, nearly all survive translation) are published as the limit. Text hardening is complete with the windowed scan and the KGW cap lift.

Audio presence on the public tier is limited to the fast and normalised reads: the public detect tier shows an AudioSeal presence only when decode: fast or the hardened decode's fast or normalised path read it, or when it is confirmed. Before the fine-read floor (below), the hardened decode flagged unmarked read speech on 0.68 % of single utterances, 5.33 % of 30 s joins and 14.17 % of 120 s joins (0 of 1,247 other clips, an earlier bench), against 0.16 %, 0.67 % and 0.42 % for what the public tier shows. In the 2026-10 null campaign the public tier showed 0.18 %, 0.08 % and 0.24 % of 5,000 unmarked single utterances, 30 s joins and 120 s joins, and 7.25 % of 4,979 music clips. Fallback-only presence stays on the full tier, with a caveat stating the rates (Grading, "Audio presence on the public tier").

The hardened audio fallback is gated by the fine-read floor (Grading, "Hardened audio decode"): a batch-card read found by the phase search or the windowed read counts only if its whole-clip or its 10 s fine-pass message confidence reaches 0.305. With the floor, the full tier flags unmarked read speech on 0.40 % of single utterances (Wilson 0.26 to 0.60 %), 0.67 % of 30 s joins (0.18 to 2.40 %) and 1.25 % of 120 s joins (0.43 to 3.61 %), and 0 of 1,247 other clips (an earlier bench; in the 2026-10 null campaign the full tier flagged 0.78 %, 0.66 % and 1.48 % of 5,000 single utterances, 30 s joins and 120 s joins and 654 of 4,979 music clips, 293 of them by the fallback paths alone). What the floor costs on marked audio: 1 of 3,192 issuer-exact reads on 37 ordinary rows. Its limit, published as measured: a mark shorter than the 10 s fine-pass excerpt inside long unmarked audio is protected only as far as it was measured. That is 29 of 30 fresh 3 s marks inside 30 s (one issuer-exact read lost) and 30 of 30 fresh 12 s marks inside 120 s.

6. Corrections and caveats recorded

Reference

• Benchmarks: the full result tables, latency, validators and image sizes, and how to rerun a suite.
• Grading: how a decode becomes a grade, per signal, and the per-signal limits.
• Keys: keyed payloads for image, audio and video, and their residuals.
• Configuration: watermark.* and detect.* settings quoted above.