Add new schedulers

[Erlemar]

It is always good to have more options to choose. So it would be a good idea to add more schedulers. The steps are the following:

• in conf/scheduler add a config for a new scheduler
• if this scheduler requires some other library, update requirements
• run tests to check that everything works

Example: https://github.com/Erlemar/pytorch_tempest/blob/master/conf/scheduler/cyclic.yaml

# @package _group_
class_name: torch.optim.lr_scheduler.CyclicLR
step: step
params:
  base_lr: ${training.lr}
  max_lr: 0.1

• # @package _group_ - default necessary line for hydra
• class_name - full name/path to the object
• params: parameters, which are overriden. If scheduler has more parameters than defined in config, then default values will be used.

There are 3 possible cases of adding a scheduler:

• default pytorch scheduler. Simply add config for it.
• schedulerfrom another library. Add this library to requirements, define config with class_name based on the library. For example cyclicLR.CyclicCosAnnealingLR
• schedulerfrom custom class. Add class to src/scheduler and add config with full path to the class starting with src