ICL-Evader

Official repository for the paper ICL-Evader: Zero-Query Black-Box Evasion Attacks on In-Context Learning and Their Defenses (WWW 2026).

Project Overview

This repository includes both notebook-based and modular CLI-based pipelines.

• Notebook workflow:
  • blackbox_adv_examples_against_ICL_with_LLMs_defense.ipynb is the original end-to-end experiment notebook.
  • data_preprocessing.ipynb handles data preprocessing and train/test preparation.
• Modular workflow:
  • src/icl_evader/ contains reusable modules for attacks, defenses, ICL, metrics, and experiments.
  • scripts/run_modular.py runs baseline, attack, and defense experiment suites.
  • scripts/run_joint_defense.py hardens a single ICL prompt with the joint defense strategy.

Data Layout

• original_data/: raw and intermediate task data.
• final_data/: processed datasets (typically train.csv and test.csv).
• configs/: runnable templates for baseline, attack, defense, and joint defense hardening.

Installation

Python version: 3.12

Option A: uv (recommended)

# install uv if needed: https://docs.astral.sh/uv/getting-started/installation/
uv venv --python 3.12
source .venv/bin/activate
uv pip install -r requirements.txt

Option B: venv + pip

python3.12 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Reproducibility Pipeline (Modular)

1) Prepare configs

Use and customize:

• configs/baseline.example.json
• configs/attack.example.json
• configs/defense.example.json

Recommended setup:

• model_path points to your local model checkpoint.
• task_data_dirs points to processed datasets under final_data/.
• output_dir points to your experiment output location.

2) Run baseline (clean ICL)

python scripts/run_modular.py baseline --config configs/baseline.example.json

3) Run attack experiments

python scripts/run_modular.py attack --config configs/attack.example.json

4) Run defense experiments

python scripts/run_modular.py defense --config configs/defense.example.json

5) Outputs

Each run writes results under your configured output_dir, including:

• per-run raw outputs (*_outputs.pkl)
• confidence tables (*_confidence.csv)
• cumulative run log (results_all.json)

Joint Defense CLI Usage

This section is independent from modular reproducibility runs.

Setup LLM config (API key + base URL + model)

Use this flow when you want the CLI to infer missing prompt attributes from an OpenAI-compatible API.

1. Export your API key

export OPENAI_API_KEY="<your_api_key>"

2. Configure endpoint and model in configs/joint_defense.llm.example.yaml

# OpenAI default (change to your provider if needed)
endpoint: "https://api.openai.com/v1"
model: "gpt-4o-mini"
api_key_env: "OPENAI_API_KEY"
timeout: 30

Notes:

• endpoint can be either a full .../chat/completions URL or a base URL (for example https://api.openai.com/v1).
• For non-OpenAI providers, set endpoint to your provider's OpenAI-compatible base URL.
• Keep secrets in environment variables; avoid committing literal keys.

CLI examples

1. Joint defense with explicit defense config only

mkdir -p defense_test/outputs

python scripts/run_joint_defense.py \
  --prompt-file defense_test/prompts/toxic_prompt.txt \
  --config defense_test/configs/toxic_config.yaml \
  --output defense_test/outputs/toxic_hardened.txt \
  --output-meta defense_test/outputs/toxic_meta.json

2. Joint defense with LLM-assisted attribute extraction

mkdir -p defense_test/outputs

python scripts/run_joint_defense.py \
  --prompt-file defense_test/prompts/toxic_prompt.txt \
  --llm-config configs/joint_defense.llm.example.yaml \
  --output defense_test/outputs/toxic_hardened.txt \
  --output-meta defense_test/outputs/toxic_meta.json

Equivalent via modular entrypoint:

python scripts/run_modular.py joint-defense \
  --prompt-file defense_test/prompts/toxic_prompt.txt \
  --config defense_test/configs/toxic_config.yaml \
  --output defense_test/outputs/toxic_hardened.txt

If required attributes are missing and no LLM config is provided, the script exits with explicit missing-field guidance.

Quick visual test for hardened prompts

Run sample prompts/configs and print hardened prompts directly in terminal:

bash defense_test/run_render_examples.sh

Generated files are written to defense_test/outputs/.

More Details

For deeper modular documentation, see MODULAR_README.md.