Innvariant-Representations

Library containing popular Disentanglement Metrics, including our proposed method called EDI.

Implemented

• Datasets: cars3d, chairs3d, dSprites, shapes3d, CelebA, MNIST.

• Metrics: FactorVAE Metric, SAP, MIG, MIG_sup, DCIMIG, DCI, Modularity, EDI, smoothness.

• Models: AE (Autoencoder), VAE (Variational Autoencoder), BetaVAE, FactorVAE, BetaTCVAE, DIPVAE, InfoVAE.

Install

python3 -m venv .venv
source .venv/bin/activate

Install the requirements.

pip install torch torchvision numpy scipy scikit-learn xgboost pyyaml python-box pillow pytest h5py wandb tqdm pandas matplot

or install with the recommended version.

pip install requirements.txt

Usage

Prepare Datasets

1. Walk into the project folder, and create a folder name "data"

   cd measuring_meaningful_latent_space
   mkdir data
   

2. download the dataset from their official providing website, place them in the data folder and unzip.

Reproducing experiments

• Calibration Test

  python -m experiments.test_boundaries

• Simulation Tests

  python experiments.test_non_linearity run

To plot the results --plot should be used in place of --run. Other flags control parameters like metrics to test (eg. --metrics)

• Training Real models (use the scripts provided in the experiments folder) : you can run the following command for training . You can also specify the dataset, model, model specific parameters, and seed.

  python -m experiments.run --dataset dSprites --model-name beta_vae --beta 2 --seed 0

• Evaluating (use the scripts provided in the experiments folder) : to eval, you only need to add --eval command option comparing with training. Noted that it is required to run training first, since evaluation rely on the model saved in output path to evaluate.

  python -m experiments.run --dataset dSprites --model-name beta_vae --beta 2 --seed 0 --eval

Run your own experiments

• You can also run your own scripts using the library in disentangling folder.

Experiments Running Logs

The logs of run experiments are on WandB. Simply log into your wandb and reports will automatically be generated in your account.