Implementation CRPS decoder

[clessig]

Description

Small cleanup of #2578 by @saibr

Usage:

decoder_ens_latent_perturbation : 
  num_members: 2    # 0 / 1 = disabled; >1 = number of ensemble members
  sigma_init: 0.2  # initial noise standard deviation
  sigma_learnable: True  # true = nn.Parameter; false = fixed buffer

Issue Number

Closes #2622

Checklist before asking for review

• I have performed a self-review of my code
• My changes comply with basic sanity checks:
  • I have fixed formatting issues with ./scripts/actions.sh lint
  • I have run unit tests with ./scripts/actions.sh unit-test
  • I have documented my code and I have updated the docstrings.
  • I have added unit tests, if relevant
• I have tried my changes with data and code:
  • I have run the integration tests with ./scripts/actions.sh integration-test
  • (bigger changes) I have run a full training and I have written in the comment the run_id(s): launch-slurm.py --time 60
  • (bigger changes and experiments) I have shared a hegdedoc in the github issue with all the configurations and runs for this experiments
• I have informed and aligned with people impacted by my change:
  • for config changes: the MatterMost channels and/or a design doc
  • for changes of dependencies: the MatterMost software development channel

[clessig]

@shmh40 : this is a slight cleanup of #2578 . Can you please have a look.

[shmh40]

decoder_ens_latent_perturbation : 
  num_members: 2    **# 0 / 1 = disabled;** >1 = number of ensemble members
  sigma_init: 0.2  # initial noise standard deviation
  sigma_learnable: True  # true = nn.Parameter; false = fixed buffer

1 is not actually disabled right? I think this is what you want, but just checking.

[shmh40]

This latent_perturbation_num_members no longer exists. It has been replaced by decoder_ens_latent_perturbation.num_members

[shmh40]

So here we initialise this as 0, hence sigma is 1? And then it only takes a sensible value in reset_parameters? Is reset_parameters always called? What happens when we continue or finetune a run? Why don't we just read sigma_init here?

[shmh40]

Since this is nn.Parameter, is this also subject to weight decay on all parameters? Do we want this?

[shmh40]

Please see comments.