AI Watermarking Alters Model Safety Behavior, Research Finds

Watermarking text generated by large language models can change how those models behave, including which tools they invoke and whether they follow their own safety guardrails, according to new research from Lasso Security.

The finding lands at a moment when watermarking is moving from research to regulation. A new European Union law requires AI platforms to watermark the content they generate. Anthropic recently disclosed that its future Claude models will use SynthID-Text, an approach Google created and released as open source.

SynthID-Text works by using a secret key that subtly changes the process a model uses for choosing the next word in a sentence. A top next word choice might be "cloudy," but the key might change it to "overcast." Anyone who knows the key can determine whether a text was generated by the platform using it. The mechanism is designed to be imperceptible to readers while making machine-generated text detectable — the core requirement behind the EU's push for AI content provenance.

The research shows that this tampering with word selection carries side effects beyond detectability. Watermarking can change the tools a model calls when it powers an agent and the likelihood that it adheres to safety training it has received. The risk grows under adversarial prompts, where an attacker tries to make a model carry out a harmful action such as revealing a password or other sensitive information. Instructions a model would normally refuse will, in some cases, be performed once watermarking is deployed.

"As compared to the same models without watermarking, it is definitely going to change their behavior, especially when we place it under adversarial conditions, or we make these models call tools when they're powering an agent," Andrea Siposova, an AI security researcher at Lasso Security, told Ars Technica. "Watermarking is made to not be perceptible to a reader, but we know that when we are changing anything about what the model is generating, it is going to cause some tradeoffs, it's going to show up somewhere."

The stakes extend beyond a single vendor. Anthropic's adoption of Google's open-source watermarking scheme signals that SynthID-Text, or approaches like it, could become a de facto standard across the industry as companies race to comply with the EU law. If watermarking measurably degrades refusal behavior and tool-use reliability, every platform shipping it inherits a new attack surface at the agent layer.

That surface matters because agents increasingly operate with access to tools, APIs, and sensitive data. A shift in word selection that nudges a model past a guardrail it was trained to respect is not a theoretical concern in that context — it is a direct path to executing harmful instructions the model would otherwise decline.

The research underscores the need for developers to thoroughly test how their LLMs and agents behave when watermarking is in place before rolling it out broadly. As compliance deadlines approach and more platforms adopt provenance schemes, security testing under watermarking is likely to become a standard part of the model release checklist rather than an afterthought.