[ICML 2026] Principled RL for Flow Matching Emerges from the Chunk-level Policy Optimization

Yifu Luo^*, Haoyuan Sun^*, Xinhao Hu^*, Penghui Du^*, Keyu Fan, Bo Li, Sinan Du, Xu Wan, Zhiyu Chen, Bo Xia, Yongzhe Chang, Changqian Yu, Kun Gai, Tiantian Zhang, Xueqian Wang

Tsinghua University   |   Kolors Team, Kuaishou Technology   |   Zhejiang University

---

We propose Group Chunking Policy Optimization (GCPO), the first chunk-level reinforcement learning approach for post-training flow matching.

---

Updates:

• 22-07-2026: We released GCPO code and our paper. Glad to share that our paper is accepted by ICML 2026.

---

GCPO Environment

The environment configuration of GCPO is almost the same as the baseline Dance-GRPO.

To set up the environment, first download the base model FLUX to data/flux, then run:

cd GCPO
bash env_setup.sh

GCPO Training

First, preprocess the embeddings:

cd GCPO
bash scripts/preprocess/preprocess_flux_rl_embeddings.sh

Second, the GCPO training can be reproduced by:

bash scripts\finetune\finetune_hpsv3_chunk.sh
bash scripts\finetune\finetune_clip_chunk.sh
bash scripts\finetune\finetune_pickscore_chunk.sh

As the comparision, the GRPO baseline can be reproduced by:

bash scripts\finetune\finetune_hpsv3.sh
bash scripts\finetune\finetune_clip.sh
bash scripts\finetune\finetune_pickscore.sh

GCPO Evaluation

We provide an evaluation sample in the test directory. First replace the checkpoint path and the output path (as well as the possible different prompt path) in the script with your owns, and run:

cd GCPO
bash test/sample_test.sh

This evaluation saves the generations. Second, please refer to HPSv3, ImageReward, PickScore, GenEval, and DPG for obtaining the benckmark scores.