SLIM: Stealthy Low-Coverage Black-Box Watermarking via Latent-Space Confusion Zones

This repository contains the code and running instructions for the ACL paper "SLIM: Stealthy Low-Coverage Black-Box Watermarking via Latent-Space Confusion Zones". The project studies a stealthy black-box watermarking pipeline that identifies low-coverage candidate texts, constructs watermarked training data, trains a target language model, and verifies the watermark through t-statistic-based analysis.

Setup

1. Create a Python environment

python -m venv .venv
source .venv/bin/activate

On Windows PowerShell:

python -m venv .venv
.venv\Scripts\Activate.ps1

2. Install dependencies

Install the Python packages required by the scripts:

pip install torch transformers datasets huggingface_hub openai pandas matplotlib scipy sentence-transformers scikit-learn evaluate bert-score

If you prefer to use requirements.txt, you can use:

pip install -r requirements.txt

3. Configure API access

This project requires both Hugging Face and OpenAI access.

Set your Hugging Face token:

export HF_TOKEN=your_huggingface_token

On Windows PowerShell:

$env:HF_TOKEN="your_huggingface_token"

Set your OpenAI API key:

export OPENAI_API_KEY=your_openai_api_key

On Windows PowerShell:

$env:OPENAI_API_KEY="your_openai_api_key"

4. Hardware note

CandidatePrepare.py, ModelFineTune.py, and ModelPreTrain.py load large language models. A CUDA GPU with enough VRAM is strongly recommended for practical runs.

Repository Workflow

The full workflow is:

1. Prepare candidate sequences.
2. Generate the watermarked dataset.
3. Fine-tune or pre-train a model on the watermarked dataset.
4. Compute the t-statistic for verification.

Step 1: Prepare Candidate Sequences

Run:

python CandidatePrepare.py

Default behavior:

• Loads the base selection model and source dataset.
• Computes perplexity-based filtering.
• Saves the selected candidate sequence to Final_Candidate.jsonl.

Example with explicit arguments:

python CandidatePrepare.py \
  --seed 42 \
  --model-name EleutherAI/pythia-12b-deduped \
  --ds-name gfissore/arxiv-abstracts-2021 \
  --dynamic-threshold 20 \
  --min-tokens 180 \
  --save-path Final_Candidate.jsonl

Step 2: Generate the Watermarked Dataset

Run:

python WatermarkDataset.py

Default behavior:

• Reads Final_Candidate.jsonl.
• Splits the selected candidate into prefix and continuation.
• Uses the OpenAI API to generate paraphrased prefixes and new continuations.
• Writes the final watermarked training data to Watermark_dataset_final.csv.

Example with explicit arguments:

python WatermarkDataset.py \
  --seed 42 \
  --ds-name gfissore/arxiv-abstracts-2021 \
  --split-ratio 0.2 \
  --ver-num 16 \
  --input-file Final_Candidate.jsonl \
  --out-csv Watermark_dataset_final.csv

Step 3: Train on the Watermarked Dataset

You can choose either fine-tuning or pre-training.

Option A: Fine-tuning

python ModelFineTune.py

Example:

python ModelFineTune.py \
  --seed 42 \
  --model-name google/gemma-3-4b-pt \
  --csv-path Watermark_dataset_final.csv \
  --ver-num 16 \
  --max-len 1024 \
  --min-tokens 50 \
  --output-dir ./gemma-3-4b-pt_WM_FT

Option B: Pre-training

python ModelPreTrain.py

Example:

python ModelPreTrain.py \
  --seed 42 \
  --model-name EleutherAI/pythia-1.4b \
  --csv-path Watermark_dataset_final.csv \
  --ver-num 16 \
  --max-len 512 \
  --min-tokens 50 \
  --output-dir ./pythia-1.4b_WM_PT

Step 4: Compute the T-Statistic for Verification

Run:

python TStatistixCalculation.py

Default behavior:

• Reads Final_Candidate.jsonl.
• Generates continuations from the verification model.
• Computes BERTScore-based statistics and reports the t-statistic.

Example:

python TStatistixCalculation.py \
  --inp-file Final_Candidate.jsonl \
  --n-words 3 \
  --model-dir ./gemma-3-4b-pt_WM_FT

Notes

• The scripts are written as standalone Python CLI programs.
• Default argument values are already defined inside each file.