Unable to reproduce the results (lowest FID) of Horse2Zebra and Map2Sat

[guyleaf]

Hi, thanks for making this great work.
The result is very impressive.

I'm trying to reproduce the result of Horse2Zebra and Map2Sat, but I cannot get similar FID values.
Is it normal?

Thanks.

Paper result

Horse2Zebra

The lowest FID is 38.86050602209259.

{'test_155': {'fake_1': 67.28442119755371,
              'fake_2': 48.42551658251689,
              'fake_3': 42.348939774854806,
              'fake_4': 40.1248390131058,
              'fake_5': 38.86050602209259},
 'test_105': {'fake_1': 76.17878896021992,
              'fake_2': 54.2598798947314,
              'fake_3': 46.26538334563116,
              'fake_4': 42.20495550810186,
              'fake_5': 40.45703713549316},
 'test_190': {'fake_1': 57.709741354714,
              'fake_2': 47.057026217368126,
              'fake_3': 45.39016298982065,
              'fake_4': 41.18903930884557,
              'fake_5': 40.91145373765819},
 'test_145': {'fake_1': 66.41517974716504,
              'fake_2': 58.00154659998391,
              'fake_3': 52.02280410833965,
              'fake_4': 47.01486833837396,
              'fake_5': 44.17161471652101},
 'test_220': {'fake_1': 75.31590592747565,
              'fake_2': 56.2749939729235,
              'fake_3': 49.92795643094819,
              'fake_4': 46.362888518867095,
              'fake_5': 45.78734516698335}}

Map2Sat

The lowest FID is 50.47061052521997.

{PosixPath('results/maps_SB/test_375'): {'fake_1': 62.51872202588612,                                                                                                                                                                                                                            
                                         'fake_2': 53.268906354620356,                                                                                                                                                                                                                           
                                         'fake_3': 50.47061052521997,                                                                                                                                                                                                                            
                                         'fake_4': 51.00056969740342,                                                                                                                                                                                                                            
                                         'fake_5': 54.53886076520848},                                                                                                                                                                                                                           
 PosixPath('results/maps_SB/test_380'): {'fake_1': 63.75456582610627,                                                                                                                                                                                                                            
                                         'fake_2': 54.06152690467053,                                                                                                                                                                                                                            
                                         'fake_3': 51.608944166887056,
                                         'fake_4': 52.236401623229256,
                                         'fake_5': 55.30514406652054},
 PosixPath('results/maps_SB/test_315'): {'fake_1': 61.31653824103762,
                                         'fake_2': 53.38316725431366,
                                         'fake_3': 51.89277393992245,
                                         'fake_4': 54.22393842199415,
                                         'fake_5': 59.4284568657705},
 PosixPath('results/maps_SB/test_400'): {'fake_1': 64.5273366808837,
                                         'fake_2': 54.21571474249083,
                                         'fake_3': 51.989683097455384,
                                         'fake_4': 53.5245091661194,
                                         'fake_5': 56.79759882286055},
 PosixPath('results/maps_SB/test_370'): {'fake_1': 62.95248898270029,
                                         'fake_2': 54.47948824315364,
                                         'fake_3': 52.69446956424861,
                                         'fake_4': 55.165577782972775,
                                         'fake_5': 60.10190538034851}}

Commands

Horse2Zebra

Train Command

python train.py --dataroot ./datasets/horse2zebra --name h2z_SB --mode sb --lambda_SB 1.0 --lambda_NCE 1.0 --gpu_ids 0 --display_env h2z_SB

Test command

#!/usr/bin/env bash
set -e

for epoch in {5..400..5}
do
    python test.py \
    --dataroot ./datasets/horse2zebra \
    --name h2z_SB_2 \
    --checkpoints_dir ./checkpoints \
    --mode sb \
    --eval \
    --phase test \
    --num_test 1000 \
    --epoch "$epoch" \
    --gpu_ids 0
done

Map2Sat

Train Command

python train.py --dataroot ./datasets/maps --name maps_SB --mode sb --lambda_SB 1.0 --lambda_NCE 1.0 --gpu_ids 1 --direction BtoA --display_env maps_SB

Test command

#!/usr/bin/env bash
set -e

for epoch in {5..400..5}
do
    python test.py \
    --dataroot ./datasets/maps \
    --name maps_SB \
    --checkpoints_dir ./checkpoints \
    --mode sb \
    --eval \
    --phase test \
    --direction BtoA \
    --num_test 2000 \
    --epoch "$epoch" \
    --gpu_ids 1
done

[cyclomon]

Hi, thank you for your interest on our work
The training dynamics can differ due to the initial seed & environment.
If the outputs keep showing different aspect, feel free to comment.