Extend Prognostic Prescriber to Coupled Modeling #1030
Conversation
…ere only ACE for overriding prognostic variables during eval/inference
… logic and align atmosphere evaluator
elynnwu
left a comment
There was a problem hiding this comment.
If I understand correctly, this only works for coupled eval, not inference. But this feature works for both ace eval/inference. Consider adding ocean_stepper_override / atmosphere_stepper_override to InferenceConfig and threading them into its loaders,, so the feature is available in coupled.inference also. I think it could be useful if we have crashes on a long running job where we don't necessary have target data to compare with.
…ing variables and validating overrider at top level
…m/ace into feature/coupled-override-perscriber
There was a problem hiding this comment.
Correctness of this PR seems to be good.
That said, I think we can get rid of all of the private attribute accesses (step_selector._step_config_instance, the stepper._config patching in sync_coupled_stepper_runtime_stepper_configs / load_coupled_stepper) and config name recomputations with a bit of straightforward refactoring suggested by Claude Fable 5:
- Add a public
prescribed_prognostic_namesgetter toStepConfigABC(and expose throughStepSelector/StepperConfig/Stepper). The ABC already has the mutatorreplace_prescribed_prognostic_namesand pairs replace_ocean withget_ocean— this is just the missing getter half.multi_calldelegates towrapped_steplike its other properties,separate_radiationreturns[]to match its no-op replace. That deletes_get_prescribed_prognostic_names_from_stepand itsisinstancerecursion.- Make
ocean_forcing_window_names/atmosphere_forcing_window_namescomputed@propertys instead of__post_init__-cached lists. The set ops are cheap, and therefresh_*methods (and all their call sites) disappear — mutations are reflected automatically.- Establish the config-aliasing invariant in
CoupledStepper.__init__/from_stateby setting the nestedComponentConfig.stepperfrom the publicocean.config/atmosphere.configproperties, instead of patching it post-hoc insync_coupled_stepper_runtime_stepper_configs. Thenload_coupled_stepperis a plainfrom_stateagain.
Regarding this comment if you look at this commit I did this exact thing, the problem is it touches a lot of code and seems pretty bloaty so in the end of removed it. Can you take a look at this commit to see if you think its worth it? The other 2 suggestions are worth it IMO but I am on the fence about the first one. |
There was a problem hiding this comment.
See the output from running the /pre-review skill in a clean session below. To address, I suggest:
- Item 1: Please disallow
oceanandderived_forcingsoverrides onCoupledStepperin this PR. If we need them we can figure out how to enable them in a follow-up PR. See https://github.com/ai2cm/ace/pull/1030/changes#r3562194140. - Item 2: Please add the suggested "validation error when
prescribed_prognostic_namesintersects the ocean-to-atmosphere-supplied names". - Item 3: See https://github.com/ai2cm/ace/pull/1030/changes#r3562082953
- Item 4: As stated.
- Item 5: As stated.
- Item 6: I'd suggest adding
CoupledStepperConfig.apply_component_overrides()in this PR, but could be done as a follow-up.
Pre-review agent: I did a pass over the full diff ahead of review (merged latest
mainlocally; the new tests infme/coupled/test_stepper.py,test_evaluator.py, andtest_inference.pyall pass on the merged tree). Overall the PR is in good shape — the on-demand forcing-window properties plus the config aliasing inCoupledStepper.__init__is a clean design, and the standalone-checkpoint validation fails loudly rather than silently ignoring overrides. A few things worth attention before/during review:
Stale cached name sets for
ocean=/derived_forcings=overrides.CoupledStepperConfig.__post_init__caches_ocean_forcing_exogenous_names,_atmosphere_forcing_exogenous_names,_shared_forcing_exogenous_names, and_atmosphere_ocean_configfrom the component configs, and also runs cross-component validation there. The new override path (apply_coupled_stepper_config_inference_overrides/apply_stepper_overrideon loaded component steppers) mutates the nested stepper configs after__post_init__has run. That's fine forprescribed_prognostic_names(the new window-name properties compute those on demand), but aStepperOverrideConfig(ocean=...)orderived_forcings=...override could change the atmosphere's ocean config or input names while the cached exogenous sets and_atmosphere_ocean_configstay stale — silently. Suggest either (a) rejectingocean/derived_forcingsoverrides in the coupled path for now (likemulti_callalready is), or (b) recomputing/revalidating after mutation.Prescribed atmosphere names can be silently clobbered by ocean-supplied fields. In
_get_atmosphere_forcings,forcing_data.update(forcings_from_ocean)runs after prescribed prognostics are pulled fromatmos_data. If someone prescribes a name that the coupler also supplies from the ocean (e.g. the surface temperature name), the prescription is silently overwritten. A validation error whenprescribed_prognostic_namesintersects the ocean-to-atmosphere-supplied names would make this loud.Private cross-module import.
evaluator.pyimports_apply_stepper_overridesfromfme.ace.stepper.single_module. Along with my earlier inline comment about folding_apply_stepper_overridesintoapply_stepper_override: one way to resolve both is to makeapply_stepper_overridethe single public entry point acceptingStepper | StepperConfig, with themulti_call-on-config rejection built in, and drop the private import.PR description is stale. It says the PR "added
refresh_{ocean,atmosphere}_forcing_window_namesto recompute after inference-time overrides," but the final code has no refresh methods — window names are now on-demand properties. Since the description becomes the squash commit message, please update it to describe the final state.Minor cleanups:
from unittest.mock import MagicMockappears inside three test function bodies — lift to module top.- A couple of comments/docstrings narrate the refactor history rather than the code ("replaces the previous post-hoc config-sync helper" in
CoupledStepper.__init__; "the branch order is preserved so behavior matches the previous per-caller implementations" in_apply_stepper_overrides). These won't mean anything after merge — trim to what the code needs.- Stray added blank line in
CoupledStepperConfig.__post_init__.(Non-blocking design note.)
CoupledStepper.__init__reaching in withconfig.ocean.stepper = ocean.config, and the evaluator free functions mutatingcoupled_config.ocean.stepper, both treat another config's fields as public. A sanctioned method onCoupledStepperConfig(e.g.apply_component_overrides(ocean_override, atmosphere_override)) would keep that traversal inside the owning class. Fine to defer.One intended behavior change worth stating in the description:
get_forcing_window_data_requirementsnow includes prescribed prognostic names, so any checkpoint whose component steppers already carryprescribed_prognostic_nameswill now require those variables in the forcing dataset (a loud data-loading failure rather than a silent skip — which is the point of the PR, just worth calling out).— 🤖 Drafted by Claude (claude-fable-5) via Claude Code
| _apply_stepper_overrides( | ||
| stepper, override_config, replace_multi_call=stepper.replace_multi_call | ||
| ) |
There was a problem hiding this comment.
Nit: You could just include the contents of _apply_stepper_overrides in the body of apply_stepper_override. I think the diff would also be a bit smaller if you edited load_stepper above and put apply_stepper_override below it.
| ) | ||
|
|
||
|
|
||
| def apply_stepper_override_to_nested_stepper_config( |
There was a problem hiding this comment.
Given that this method isn't typed in a way that is specific to fme.coupled, I think it should live in fme/ace/stepper/single_module.py (and maybe just call it apply_stepper_override_to_stepper_config).
This would also avoid the import of the private _apply_stepper_overrides method.
If you remove it, please remove mention of CoupledStepperConfig or "coupled" from the docstring, since fme.ace shouldn't refer back to fme.coupled.
| def _reject_multi_call(multi_call: MultiCallConfig | None) -> None: | ||
| raise ValueError( | ||
| "StepperOverrideConfig.multi_call cannot be applied when loading " | ||
| "CoupledStepperConfig without constructing a Stepper; use load_stepper " | ||
| "with a full checkpoint instead." | ||
| ) |
There was a problem hiding this comment.
I would just handle the StepperOverrideConfig validation to disallow replace_multi_call, replace_ocean, and replace_derived_forcings as a private method in this module that you can apply to ocean_override and atmosphere_override inside of apply_coupled_stepper_config_inference_overrides().
| checkpoint_path: The path to the serialized checkpoint. | ||
| override_config: Configuration options to override (optional). | ||
| def _apply_stepper_overrides( | ||
| target: "Stepper | StepperConfig", |
There was a problem hiding this comment.
Nit: call this stepper_or_config rather than target.
Coupled models could not use the prognostic prescriber available in uncoupled ACE, so ocean/atmosphere prognostic variables (e.g. SST,
thetao_18) could not be overwritten from forcing data during evaluation/inference. This PR extends the prescriber to coupled modeling and lets it be enabled at inference time via overrides, so checkpoints trained without prescribed names can still prescribe them.Changes:
fme.coupled.stepper.CoupledStepperConfig: include each component'sprescribed_prognostic_namesin the ocean and atmosphere forcing windows and forcing-window data requirements, so prescribed variables are loaded and supplied as next-step inputs; addedrefresh_{ocean,atmosphere}_forcing_window_namesto recompute after inference-time overrides.fme.coupled.inference.evaluator: thread optional ocean/atmosphereStepperOverrideConfigthrough the coupled load paths andInferenceEvaluatorConfig, applying overrides to both loaded steppers and the nestedStepperConfigused for forcing-window requirements.fme.ace.stepper.single_module: extractapply_stepper_overridefromload_stepperand consolidate override dispatch into a shared helper reused for both aStepperand aStepperConfig.Tests added