Skip to content

Stress metrics extension#124

Draft
abdelrahman-ayad wants to merge 93 commits into
mainfrom
aa/metrics_extension
Draft

Stress metrics extension#124
abdelrahman-ayad wants to merge 93 commits into
mainfrom
aa/metrics_extension

Conversation

@abdelrahman-ayad

@abdelrahman-ayad abdelrahman-ayad commented Jun 18, 2026

Copy link
Copy Markdown
Collaborator

Summary

Technical details

Implementation notes

  • All new metrics are computed in calc_ra_metrics() after reading PRAS outputs. Duration and Depth derive from the hourly EUE timeseries via two helper functions:
    • calc_max_duration(dfeue_agg) — length of the longest consecutive run of hours with EUE above threshold
    • calc_peak_eue(dfeue_agg, dfload_agg, norm='peak') — peak hourly EUE normalized by peak load
  • LOLD and LOLE use the PRAS _LOLE hourly columns (same source as LOLH):
    • LOLD (calc_lold) — a day is counted as an event-day if at least one hour has LOLE > 0
    • LOLE (calc_lole) — number of loss-of-load events, where an event is a contiguous run of hours with LOLE > 0
  • Note: GSw_PRM_StressThresholdOutageMagnitude (a raw-MW magnitude threshold) was dropped in favor of keeping only the normalized Depth metric, since an absolute-MW threshold isn't comparable across transgrps of very different sizes (same reasoning as the existing EUE-vs-NEUE tradeoff). GSw_PRM_StressThresholdEUE was also removed as a standalone switch; its role is now split between Duration and Depth.
  • Also fixed: update_prm() (used when GSw_PRM_UpdateMethod is 1, 2, or 3) still assumed the pre-multi-metric shape of the failed-criteria data and would crash (KeyError) as soon as any threshold failed. It's now updated to correctly select NEUE-based failures from the new multi-metric structure — since the PRM update math is inherently NEUE/ppm-based — and to no-op (leave the PRM unchanged) when NEUE didn't fail even though another metric did. Added a companion check in runreeds.check_compatibility() that raises a clear error if GSw_PRM_UpdateMethod is 1-3 but NEUE isn't included in GSw_PRM_StressThresholdMetrics.

New stress metric switches

Metric Switch Definition RA dimension PRAS input Example
Duration GSw_PRM_StressThresholdDuration Expected (average) length of the longest consecutive sequence of shortfall hours per year [h] Duration EUE transgrp_12: threshold at 12 hours, per transgrp
Depth GSw_PRM_StressThresholdDepth Expected (average) peak single-hour shortfall normalized by peak load [MW EUE / MW peak demand] Magnitude EUE + load transgrp_0.1: threshold at 0.1 (10% of peak load), per transgrp
LOLD GSw_PRM_StressThresholdLOLD Expected (average) count of event-days per year; a day is counted if at least one hour experiences a shortfall [event-days/year] Frequency LOLE transgrp_0.1: threshold at 0.1 event-days/year, per transgrp
LOLE GSw_PRM_StressThresholdLOLE Expected (average) count of loss-of-load events per year, where an event is a contiguous run of shortfall hours [events/year] Frequency LOLE transgrp_0.1: threshold at 0.1 events/year, per transgrp

Validation, testing, and comparison report(s)

  • Pacific (LOLD, LOLE, Duration, Depth)
  • Pacific (combined: NEUE/LOLH/LOLD/LOLE/Duration/Depth)

Checklist for author

Details to double-check

  • Included comparison reports for appropriate test cases
  • Documentation updated if necessary
  • Code formatting standardized
  • Reusable functions used where possible instead of copy/pasted code

General information to guide review

  • Zero impact on results of default case
  • No large data file(s) added/modified
  • No substantive impact on runtime for full-US reference case
  • No substantive impact on folder size for full-US reference case
  • No change to process flow (runreeds.py, reeds/core/solve/solve.py)
  • No change to code organization
  • No change to package requirements (environment.yml or Project.toml)

Did you use LLM tools (chatbot or copilot) in the preparation of this PR? If so, describe how

  • I used Claude Code to implement the metric calculations, review the branch/PR, and fix a bug in update_prm() uncovered during that review (see Implementation notes).

patrickbrown4 and others added 23 commits June 23, 2026 17:22
…t_eue_events(); silence map_outagerate_new_stressperiods() if empty
if (
(int(sw.GSw_PRM_UpdateMethod) == 0)
or (len(new_stressperiods_write) and (int(sw.GSw_PRM_UpdateMethod) == 3))
or not any(failed[c].name == 'NEUE' for c in failed)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We still want to allow GSw_PRM_UpdateMethod = 1 (fixed PRM increment controlled by GSw_PRM_UpdateFraction) to work regardless of the RA metric chosen, so I think it'd be best to:

  1. Drop the conditional here (done on pb/multimetric)
  2. Add a check to runreeds.check_compatibility() that raises an error if int(sw.GSw_PRM_UpdateMethod) in [2, 3] and 'neue' not in sw.GSw_PRM_StressThresholdMetrics.lower().split('/')

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends the Resource Adequacy (RA) multi-metrics framework by adding new “stress metrics” (Duration, Depth, LOLD, LOLE), updates the stress-period iteration logic to use per-metric threshold switches, and updates downstream plotting/postprocessing to consume the new ra_metrics_*.csv outputs.

Changes:

  • Add new RA metric computations and outputs (ra_metrics_*.csv, eue_events_*.csv) and update stress period selection to support multiple metric types.
  • Update switch compatibility/validation and switch definitions (GSw_PRM_StressThresholdMetrics + per-metric threshold switches).
  • Update plotting and postprocessing to read the new RA metric outputs and remove the legacy NEUE-only pipeline.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
runreeds.py Validates metric selection and enforces NEUE requirement for PRM update methods 1–3.
reeds/resource_adequacy/stress_periods.py Implements new RA metrics, stress-period selection by metric, and PRM-update behavior for multi-metric failures.
reeds/resource_adequacy/run_pras.jl Updates PRAS logging to report LOLE/EUE/NEUE in a standardized way.
reeds/resource_adequacy/ra_calcs.py Updates debugging defaults and diagnostic plotting invocation flow.
reeds/resource_adequacy/diagnostic_plots.py Adds reading/plotting support for newly generated new_stress_periods.csv.
reeds/reedsplots.py Updates RA plotting to use ra_metrics_*.csv and adds EUE event plotting.
postprocessing/single_case_plots.py Switches single-case plotting to RA-metrics plots and adds EUE event plots.
postprocessing/compare_cases.py Switches comparison report ingestion from neue_*.csv to ra_metrics_*.csv.
postprocessing/bokehpivot/reports/templates/reeds2/standard_report_expanded.py Removes legacy NEUE entry from the expanded standard report.
postprocessing/bokehpivot/reeds2.py Removes the legacy “Resource adequacy” (neue.csv) dataset registration.
docs/source/figs/docs/ra-flowcharts.png Updates RA flowchart diagram (LFS pointer changed).
cases.csv Replaces legacy stress threshold switch with per-metric switches and documents new metrics.
cases_test.csv Adds/updates a test case configuration for multi-metric stress thresholds.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +1044 to +1046
for outage_type in vmax:
savename = f'map-outage_{outage_type}_{aggfunc}-{t}i{iteration}.png'
plt.close()
Comment thread reeds/reedsplots.py
Comment on lines +4352 to +4353
neue = reeds.io.read_output(case, f'ra_metrics_{year}i{_iteration}.csv')
neue = neue.loc[neue.metric==metric].set_index(['level','region']).NEUE
Comment thread reeds/reedsplots.py
Comment on lines 4238 to 4242
year2iteration = (
pd.DataFrame([
os.path.basename(i).strip('neue_.csv').split('i')
for i in sorted(glob(os.path.join(case, 'outputs', 'neue_*.csv')))
os.path.basename(i).strip('ra_metrics.csv').split('i')
for i in sorted(glob(os.path.join(case, 'outputs', 'ra_metrics_*.csv')))
], columns=['year','iteration']).astype(int)
Comment thread reeds/reedsplots.py
Comment on lines 6756 to 6760
year2iteration = (
pd.DataFrame([
os.path.basename(i).strip('neue_.csv').split('i')
for i in sorted(glob(os.path.join(case, 'outputs', 'neue_*.csv')))
os.path.basename(i).strip('ra_metrics.csv').split('i')
for i in sorted(glob(os.path.join(case, 'outputs', 'ra_metrics_*.csv')))
], columns=['year','iteration']).astype(int)
Comment on lines +158 to +165
match norm:
case 'peak':
peak_eue = dfeue_agg.max() / dfload_agg.max()
case 'hourly':
peak_eue = (dfeue_agg / dfload_agg).max()
case 'absolute':
peak_eue = dfeue_agg.max()
return peak_eue
@patrickbrown4 patrickbrown4 mentioned this pull request Jul 7, 2026
13 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants