[Feature] Add probability gate to variable masking configs#1339
Open
yyexela wants to merge 2 commits into
Open
[Feature] Add probability gate to variable masking configs#1339yyexela wants to merge 2 commits into
yyexela wants to merge 2 commits into
Conversation
Add a probability field to UniformMaskingConfig so a uniform scheme leaves all channels intact on a fraction of steps. With probability 1 - probability the scheme fires no draw and drops nothing; at probability=1.0 (default) the gate short-circuits so no gate draw is taken, keeping the RNG stream identical to configs that omit the field. No gate is offered on BernoulliMaskingConfig: gating a Bernoulli pool at probability is identical to scaling rate by probability, so it would be redundant.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Variable masking (#1329) drops input channels during training, but a
UniformMaskingConfigscheme fired on every step. This PR adds aprobabilitygate toUniformMaskingConfigso a uniform scheme can leave all channels intact on a fraction of steps.At
probability=1.0(default) the gate always fires the draw, preserving current behavior. At0.5the scheme drops nothing on half the steps. The gate short-circuits atprobability=1.0so no gate draw is taken, keeping the RNG stream (and existing regression baselines) identical for configs that omit the field.No
probabilitygate is offered onBernoulliMaskingConfig: gating a Bernoulli pool atprobabilityis identical to scalingratebyprobability, so it would be redundant.Changes:
fme.core.var_masking.UniformMaskingConfig/UniformMaskingGenerator: addprobability: float = 1.0field (gate), validated in[0, 1]via_validate_probability, threaded throughbuild()._validate_probabilityrejectsboolexplicitly, sinceboolis anintsubclass that would otherwise coerce to0.0/1.0.fme.core.var_masking.BernoulliMaskingConfig: no gate (redundant withrate).Tests added
If dependencies changed, "deps only" image rebuilt and "latest_deps_only_image.txt" file updated