Add a diagnostic data-provider interface (isolate OMEGA specifics)#107
Open
joglekara wants to merge 1 commit into
Open
Add a diagnostic data-provider interface (isolate OMEGA specifics)#107joglekara wants to merge 1 commit into
joglekara wants to merge 1 commit into
Conversation
Realizes the primary goal from #105 (per @almilder): isolate the OMEGA-specific diagnostic specifics (raw file loading, calibration, background, lineouts) behind a minimal interface so other users can plug in their own diagnostic (e.g. NIF OTS) or supply preprocessed data -- without touching the jax/AD model code. The whole pipeline downstream of data loading already consumes only a plain `(all_data, sa, all_axes)` tuple. This turns that existing seam into an explicit, registerable contract: - `tsadar/data/__init__.py`: `register_data_provider(name)` + `get_data_provider(config)` registry. A provider is any `config -> (all_data, sa, all_axes)` callable. Selected by `config["data"]["provider"]`, defaulting to "omega". - `tsadar/data/omega.py`: the OMEGA loader (moved out of `fitter`) as the reference provider, self-registered as "omega". - `fitter.fit` now resolves data via `get_data_provider(config)(config)`. `load_data_for_fitting` is kept as a thin back-compat shim. Behavior-preserving: configs without a `provider` key resolve to "omega" and follow the exact previous code path. External diagnostics register with `tsadar.data.register_data_provider` and require no jax/AD knowledge. Toward #105. Stacked on the coupling-break PR (#106). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Introduces a diagnostic data-provider interface — the primary goal @almilder called out in #105: isolate the OMEGA-specific diagnostic specifics behind a minimal interface so others can plug in their own diagnostic (e.g. NIF OTS) or supply preprocessed data without navigating the jax/AD model code.
How
The pipeline already had the right seam: everything downstream of data loading (
LossFunction,ThomsonScatteringDiagnostic, the optimization loops, postprocessing) consumes only a plain(all_data, sa, all_axes)tuple. All OMEGA-specificity sits upstream of it. This PR makes that seam an explicit, registerable contract.tsadar/data/__init__.py—register_data_provider(name)decorator +get_data_provider(config). A provider is anyconfig -> (all_data, sa, all_axes)callable, selected byconfig["data"]["provider"](default"omega").tsadar/data/omega.py— the OMEGA loader (moved out offitter) as the reference provider, self-registered as"omega".fitter.fitresolves data viaget_data_provider(config)(config);load_data_for_fittingstays as a thin back-compat shim.Adding a new diagnostic now takes zero model-code changes:
then
config["data"]["provider"] = "nif_ots".Behavior preservation
Configs without a
providerkey resolve to"omega"and follow the exact previous code path (incl. the multiplexed/rotated angular special case). Verified statically: registry resolves default→omega, empty-config→omega, unknown→clearKeyError; all intra-tsadarimports resolve; everything compiles. (Local jax is too old to run the suite —importsmoke test + suite should be confirmed by a reviewer with a working env.)Open question for @almilder
Supplying preprocessed data is now supported by registering a trivial provider that returns your in-memory
(all_data, sa, all_axes). If you'd rather have a built-in"preprocessed"provider that reads from a file, what format do you want —.npz,xarray/netCDF, or just an in-memory dict passed programmatically? I left that out rather than guess the format.Sequencing / not in scope
mainautomatically once it lands.calc_vs_data), and the Option A install extras (note:tsadar/__init__.pyeagerly importsrunner→fitter→optax/mlflow, so a core-only install needs__init__import-hygiene, not justpyprojectchanges).Toward #105.
🤖 Generated with Claude Code