  LLMs now exhibit human-like skills in various fields, leading to worries
about misuse. Thus, detecting generated text is crucial. However, passive
detection methods are stuck in domain specificity and limited adversarial
robustness. To achieve reliable detection, a watermark-based method was
proposed for white-box LLMs, allowing them to embed watermarks during text
generation. The method involves randomly dividing the model vocabulary to
obtain a special list and adjusting the probability distribution to promote the
selection of words in the list. A detection algorithm aware of the list can
identify the watermarked text. However, this method is not applicable in many
real-world scenarios where only black-box language models are available. For
instance, third-parties that develop API-based vertical applications cannot
watermark text themselves because API providers only supply generated text and
withhold probability distributions to shield their commercial interests. To
allow third-parties to autonomously inject watermarks into generated text, we
develop a watermarking framework for black-box language model usage scenarios.
Specifically, we first define a binary encoding function to compute a random
binary encoding corresponding to a word. The encodings computed for
non-watermarked text conform to a Bernoulli distribution, wherein the
probability of a word representing bit-1 being approximately 0.5. To inject a
watermark, we alter the distribution by selectively replacing words
representing bit-0 with context-based synonyms that represent bit-1. A
statistical test is then used to identify the watermark. Experiments
demonstrate the effectiveness of our method on both Chinese and English
datasets. Furthermore, results under re-translation, polishing, word deletion,
and synonym substitution attacks reveal that it is arduous to remove the
watermark without compromising the original semantics.
