LatentFace

A Generative Framework for Self-Supervised Facial Representation Learning

Environment

We build the environment with conda on Ubuntu 20.04.1. You can use the following command to build the environment.

conda create -n latentface python=3.10
pip install torch==1.13.1+cu117 torchvision==0.14.1+cu117 torchaudio==0.13.1 --extra-index-url https://download.pytorch.org/whl/cu117
pip install seaborn==0.13.0 pandas==1.5.3 scikit-learn==1.3.0 pytorch_lightning==1.9.0 diffusers==0.14.0 scikit-image==0.20.0
pip install opencv-python tqdm tensorboard PyYAML

Datasets

Please download the dataset from
AffectNet, RAF-DB, and LFW/SLLFW. Check your dataset if it match the following format and edit the path in the identity_test.py and configs/train_expr_*.yml files.

AffectNet
├── Manually_Annotated_file_lists
│   ├── training.csv
│   └── validation.csv
└── Manually_Annotated_Images

RAF-DB
├── aligned
└── list_patition_label.txt

Aff-Wild
├── annotations
│   └── EXPR_Set
│       ├── Train_Set
│       └── Validation_Set
└── cropped_aligned

LFW/SLLFW
├── lfw_crop
├── pair_SLLFW.txt
└── pairs.txt

Pretrained Model and Checkpoints

Please download the pretrained model and the downstream checkpoints from here.

Test Performance

AffectNet

python datasets/affectnet_pickle.py --downsample
python expr_test.py --checkpoint checkpoints/affectnet.ckpt --config configs/train_expr_affectnet.yml --gpus 1

AffectNet Pose 30 Subset

python datasets/affectnet_pickle.py --select_path datasets/pose_30_affectnet_list.txt
python expr_test.py --checkpoint checkpoints/affectnet.ckpt --config configs/train_expr_affectnet.yml --gpus 1

AffectNet Pose 45 Subset

python datasets/affectnet_pickle.py --select_path datasets/pose_45_affectnet_list.txt
python expr_test.py --checkpoint checkpoints/affectnet.ckpt --config configs/train_expr_affectnet.yml --gpus 1

RAF-DB

python expr_test.py --checkpoint checkpoints/rafdb.ckpt --config configs/train_expr_rafdb.yml --gpus 1

Aff-Wild2

python expr_test.py --checkpoint checkpoints/affwild.ckpt --config configs/train_expr_affwild.yml --gpus 1

LFW

python identity_test.py --dataset LFW

SLLFW

python identity_test.py --dataset SLLFW

Finetuning

AffectNet

python datasets/affectnet_pickle.py --downsample
python expr_test.py --train --config configs/train_expr_affectnet.yml --max_epochs 20 --gpus 1

RAF-DB

python expr_test.py --train --config configs/train_expr_rafdb.yml --max_epochs 20 --gpus 1

Aff-Wild2

python expr_test.py --train --config configs/train_expr_affwild.yml --max_epochs 20 --gpus 1

Citation

If you find the codes are helpful, please consider citing the following paper.

@misc{he2024generativeframeworkselfsupervisedfacial,
      title={A Generative Framework for Self-Supervised Facial Representation Learning}, 
      author={Ruian He and Zhen Xing and Weimin Tan and Bo Yan},
      year={2024},
      eprint={2309.08273},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2309.08273}, 
}