Universal Adversarial Watermarking for Text Image Protection via Template-based Underpainting

Universal Adversarial Watermarking (UAW) is a unified framework that simultaneously embeds adversarial perturbations and robust watermarks into text images. It provides dual protection by resisting OCR-based text extraction while ensuring reliable traceability through watermark retrieval.

Project Structure

UAW
├── AllConfig/                 # Configuration files and model checkpoints
├── AllData/
├── CSR/
├── EAST/
├── EasyOCR/
├── model_CRAFT/
├── model_DBnet/
├── model_PAN/
├── results/                   # Log files
├── TCM/
├── TextBoxesPP/
├── Tools/
├── UAWUP/                     # Our code
├── UDUP/
├── watermarklab/
├── README.md
└── watermarklab

Environment Setup

Follow the steps below to configure and run the project:

1. Create and activate a conda virtual environment:

   conda create -n uaw python=3.8
   conda activate uaw

2. Install project dependencies: Use the following command to install the required packages:

   pip install -r requirements.txt --ignore-installed

Project Workflow

Please ensure that all file paths are modified as needed before running the steps below.

Stage 1: Bit Templates Generation

python UAWUP/train_uawup.py  # One-time offline cost

Stage 2: Watermark Modulation

python UAWUP/eval_watermark/embed_dir.py

Stage 3: Watermark Extraction

python UAWUP/eval_watermark/extract_dir.py

Stage 4: Evaluate Adversarial Properties

python UAWUP/eval_adversarial/eval_dir.py

Experimental Code

IV.C. Quantitative Comparison in Text Image Protection

Experiment for CSR

python CSR/csr_watermark_test_batch.py --input-dir results/eval_watermark/ti_a×b/ti --output-dir results/eval_watermark/ti_a×b/ti_csr

Experiment for UDUP

python UAWUP/eval_watermark/embed_dir.py --use_wm False --root_eval results/AllData_results/results_udup/size=30_step=3_eps=120_lambdaw=0.1 --source_dir results/eval_watermark/ti_a×b/ti --save_dir results/eval_watermark/ti_a×b/ti_udup

python UAWUP/eval_adversarial/eval_dir.py --img_dir results/eval_watermark/ti_a×b/ti_udup --gt_img_dir results/eval_watermark/ti_a×b/ti --save_root results/eval_adversarial/ti_a×b/ti_udup

Experiment for FAWA

python UAWUP/fawa_pytorch.py --output_dir results/eval_adversarial/FAWA/attacked --wm_text TDSC

python UAWUP/eval_adversarial/eval_dir.py --img_dir results/eval_watermark/ti_a×b/ti_udup --gt_img_dir results/eval_watermark/ti_a×b/ti --save_root results/eval_adversarial/ti_a×b/ti_udup

Experiment for UAW(Ours)

python UAWUP/eval_watermark/embed_dir.py --use_wm True --patch_dir results/AllData_results/results_uawup_eps100/size=30_step=3_eps=100_lambdaw=0.01_lambday=1e-05 --source_dir results/eval_watermark/ti_a×b/ti --save_dir results/eval_watermark/ti_a×b/awti

python UAWUP/eval_adversarial/eval_dir.py --img_dir results/eval_watermark/ti_a×b/awti --gt_img_dir results/eval_watermark/ti_a×b/ti --save_root results/eval_adversarial/ti_a×b/awti

python UAWUP/eval_watermark/extract_dir.py --patch_dir results/AllData_results/results_uawup_eps100/size=30_step=3_eps=100_lambdaw=0.01_lambday=1e-05 --awti_dir results/eval_watermark/ti_a×b/awti

IV.D. Universality Evaluation of the Proposed Method

Character Size

python UAWUP/eval_watermark/embed_fontsize.py
python UAWUP/eval_watermark/extract_fontsize.py
python UAWUP/eval_adversarial/eval_character_size.py

Character Color

python UAWUP/eval_watermark/embed_color.py
python UAWUP/eval_watermark/extract_color.py
python UAWUP/eval_adversarial/eval_color.py

Language

python UAWUP/eval_watermark/embed_dir.py --patch_dir results/AllData_results/results_uawup_eps100/size=30_step=3_eps=100_lambdaw=0.01_lambday=1e-05 --source_dir AllData/web_page --save_dir results/eval_watermark/real/web_page

python UAWUP/eval_watermark/extract_dir.py --awti_dir results/eval_watermark/real/web_page --patch_dir results/AllData_results/results_uawup_eps100/size=30_step=3_eps=100_lambdaw=0.01_lambday=1e-05

python UAWUP/eval_adversarial/eval_dir.py --img_dir results/eval_watermark/real/web_page --gt_img_dir AllData/web_page --save_root results/eval_adversarial/real/web_page

Complexity of Background

python UAWUP/eval_watermark/embed_dir.py --patch_dir results/AllData_results/results_uawup_eps100/size=30_step=3_eps=100_lambdaw=0.01_lambday=1e-05 --patch_iter 27 --source_dir results/eval_watermark/real/bg_overlay --save_dir results/eval_watermark/real/bg_overlay

python UAWUP/eval_watermark/extract_dir.py --awti_dir results/eval_watermark/real/bg_overlay  --patch_dir results/AllData_results/results_uawup_eps100/size=30_step=3_eps=100_lambdaw=0.01_lambday=1e-05

python UAWUP/eval_adversarial/eval_dir.py --img_dir results/eval_watermark/real/bg_overlay --gt_img_dir results/eval_watermark/real/bg_overlay --save_root results/eval_adversarial/real/bg_overlay

I.V. Robustness Evaluation of the Proposed Method

Adversarial Robustness & Watermarking Robustness

python UAWUP/eval_watermark/embed_dir.py --patch_dir results/AllData_results/results_uawup_eps100/size=30_step=3_eps=100_lambdaw=0.01_lambday=1e-05 --source_dir results/eval_watermark/ti_a×b/ti --save_dir results/eval_watermark/noise

python UAWUP/test/noise_batch.py
python UAWUP/eval_adversarial/eval_distortion.py
python UAWUP/eval_watermark/extract_noise.py

IV.F. Evaluation on Real-world Applications

Privacy Protection for Sensitive Certificate Images

python UAWUP/eval_watermark/embed_dir.py --patch_dir results/AllData_results/results_uawup_eps100/size=30_step=3_eps=100_lambdaw=0.01_lambday=1e-05 --source_dir AllData/certificate --save_dir results/eval_watermark/real/certificate

python UAWUP/eval_watermark/extract_dir.py --awti_dir results/eval_watermark/real/certificate  --patch_dir results/AllData_results/results_uawup_eps100/size=30_step=3_eps=100_lambdaw=0.01_lambday=1e-05

python UAWUP/eval_adversarial/eval_dir.py --img_dir results/eval_watermark/real/certificate --gt_img_dir results/eval_watermark/real/certificate --save_root results/eval_adversarial/real/certificate

Transferability to Unknown OCR Systems

python UAWUP/eval_watermark/embed_dir.py --patch_dir results/AllData_results/results_uawup_eps100/size=30_step=3_eps=100_lambdaw=0.01_lambday=1e-05 --source_dir results/eval_watermark/ti_a×b/ti --save_dir results/eval_watermark/ti_a×b/awti

python UAWUP/eval_adversarial/eval_transfer.py

Robustness Tests under Shooting Scenarios

python UAWUP/eval_watermark/embed_dir.py --patch_dir results/AllData_results/gray/s30_eps100_step3_w0.01_x0 --patch_iter 41 --source_dir results/eval_watermark/ti_a×b/ti --save_dir results/eval_watermark/gray/awti

python UAWUP/eval_watermark/extract_dir.py  --awti_dir results/eval_watermark/gray/print_shoot/100cm  --patch_dir results/AllData_results/gray/s30_eps100_step3_w0.01_x0 --patch_iter 41

python UAWUP/eval_adversarial/eval_dir.py --img_dir results/eval_watermark/gray/print_shoot/100cm --gt_img_dir results/eval_watermark/ti_a×b/ti --save_root results/eval_adversarial/gray/print_shoot/100cm

Time Consumption for Real-time Requirements

python UAWUP/train_uawup.py  # One-time offline cost
python UAWUP/eval_watermark/embed_dir.py
python UAWUP/eval_watermark/extract_dir.py

IV.G. Discussion on the Hyperparameter Settings

MUI and Size of Bit Template for Adversarial Effectiveness & MUI and Size of Bit Template for Watermark Reliability & Hyperparameter $\lambda_m$ & Ablation Study of Loss Functions

python UAWUP/train_uawup.py  # One-time offline cost
python UAWUP/eval_watermark/embed_size_it.py
python UAWUP/eval_watermark/extract_size_it.py
python UAWUP/eval_adversarial/eval_size_eps.py

Impact of Adversarial and Watermarking Interaction

python UAWUP/eval_watermark/embed_random.py
python UAWUP/eval_watermark/extract_dir.py --patch_dir results/eval_watermark/size_it/awu --awti_dir results/eval_watermark/size_it/awti_random
python UAWUP/eval_adversarial/eval_random.py

Watermark Matrix Size

python UAWUP/eval_watermark/embed_dir.py --patch_dir results/AllData_results/results_uawup_eps100/size=30_step=3_eps=100_lambdaw=0.01_lambday=1e-05 --source_dir results/eval_watermark/ti_a×b/ti --save_dir results/eval_watermark/ti_a×b/awti_9×9 --wm_h 9 --wm_w 9
python UAWUP/eval_watermark/extract_dir.py --patch_dir results/AllData_results/results_uawup_eps100/size=30_step=3_eps=100_lambdaw=0.01_lambday=1e-05 --awti_dir results/eval_watermark/ti_a×b/awti_9×9
python UAWUP/eval_adversarial/eval_dir.py --img_dir results/eval_watermark/ti_a×b/awti_9×9 --gt_img_dir results/eval_watermark/ti_a×b/ti --save_root results/eval_adversarial/ti_a×b/awti_9×9

References:

1. UDUP
2. FAWA
3. CRAFT
4. DBNet
5. EasyOCR
6. EAST
7. TextBoxes++
8. TCM
9. watermarklab