LAFUFU: Latent Acoustic Features for Ultra-Fast Utterance Restoration

This repository contains the official implementation for the paper:

• Łazarz Radosław, Wosik Mateusz, Pudo Mikołaj, Krywalska Urszula, Cieślak Adam, "LAFUFU: Latent Acoustic Features for Ultra-Fast Utterance Restoration"

Training

Training is done by executing train.py. This script can be used to train both the autoencoder and the score model.

Training the autoencoder can be performed with the following command:

python train.py --n_fft 1534 --hop_length 384 --spec_factor 0.065 --spec_abs_exponent 0.667 --sigma-min 0.1 --sigma-max 1.0 --theta 2.0 --backbone latent_ncsnpp --latent --training_mode ae --ae_z_channels 2 --ae_ch 64 --ae_ch_mult 1 2 --ae_embed_dim 2 --ae_regloss_weight 0.1 --base_dir <your_data_dir>

Training the score model can be performed with the following command:

python train.py --n_fft 1534 --hop_length 384 --spec_factor 0.065 --spec_abs_exponent 0.667 --sigma-min 0.1 --sigma-max 1.0 --theta 2.0 --backbone latent_ncsnpp --latent --training_mode diff --ae_z_channels 2 --ae_resolution 192 --ae_ch 64 --ae_ch_mult 1 2 --ae_embed_dim 2 --nf 128 --ch_mult 1 1 2 2 2 --loss_type mae --base_dir <your_data_dir> --ae_ckpt <path_to_your_ae_ckpt>

Evaluation

To evaluate on a test set, run

python enhancement.py --test_dir <your_test_dir> --enhanced_dir <your_enhanced_dir> --ckpt <path_to_model_checkpoint>

Citations / References

We kindly ask you to cite our paper as follows:

@article{lafufu2025,
    title={LAFUFU: Latent Acoustic Features for Ultra-Fast Utterance Restoration},
    author={Rados{\l}aw {\L}azarz and Mateusz Wosik and Miko{\l}aj Pudo and Urszula Krywalska and Adam Cieślak},
    year={2025},
    url={https://github.com/SamsungLabs/LAFUFU}
}