Multi-instance timeseries capability for standalone CUPiD#397
Multi-instance timeseries capability for standalone CUPiD#397mollymwieringa wants to merge 7 commits into
Conversation
mnlevy1981
left a comment
There was a problem hiding this comment.
I'll wait for you to mark this as "ready for review" before going through it in detail, but one thing that jumped out is a way to avoid needing to add ninst to every configuration file.
I didn't really look through the new config.yml file, but I noticed it is pointing to output in your scratch space -- I'll make sure you have access to /glade/campaign/cesm/development/cross-wg/diagnostic_framework/CESM_output_for_testing/ and then you can copy the files you need over (right now it looks like we're testing on 21 years of ensemble output, maybe we could reduce that to just a few years to save disk space?)
Ensures that when there is no "ninst" value in the CUPiD config file (which would be the default for every other application), the multi-instance capability defaults to the assumption of a single case
|
@mnlevy1981 Just a note that the test case I set up definitely shouldn't be processing 21 years-- I think the 3 simulations are each only a month long. I'll double-check, but let me know if that's still too much storage! |
…se for multi-instance timeseries; also adjusting the configuration so that a baseline case is not required to generate timeseries for the main case within the cesm workflow.
bring main changes into multi-instance
|
@mnlevy1981 Just noting that this is ready for review! I've tested in the CESM workflow, with a couple small changes to the script that sets up the config.yml to get NINST from the CESM case and allow for timeseries production without a baseline case. |
Description of changes:
These changes take first steps toward issue #396 by enabling the run_timeseries.py tool to find model history files produced by a multi-instance-enabled case of CESM (e.g. cice_000X.h instead of cice.h). It involves changes to run_timeseries.py and adding a multi-instance example with its own config file to standalone CUPiD.
These changes will help enable efficient postprocessing of many history files for multi-instances cases into more storage-efficient timeseries files, reducing the amount of overhead required by multi-instance simulations. This benefit is particularly relevant to data assimilation experiments, which use multi-instance simulations to run large ensembles that output history files at high temporal frequencies.
Outstanding work on this project during draft PR stage:
All PRs Checklist:
pre-commitchecks passed (#8 in Adding Notebooks Guide)?