Update main to reflect CEPM workflow - #27
Merged
Merged
Conversation
Added instructions for adding GAMS to user PATH variable in PowerShell.
Fix typo in GAMS installation path description.
- Normalize current health damages output columns into the schema expected by bokehpivot report processing. - Preserve CSV headers when a report source opts into downstream schema normalization. - Fix Vizit report JSON generation for NumPy values and escaped replacement strings. - Replace six.moves urllib import with stdlib urllib.parse. - Verified by py_compile and regenerating the Pacific expanded report with health sections rendering.
- Initialize an empty offshore wind capacity-factor frame when offshore wind processing is disabled. - Keep downstream concatenation inputs consistent with the onshore wind index. - Avoid referencing df_windofs before assignment in no-offshore-wind cases.
- Add AGENT.md with repo orientation, workflow notes, and agent-facing guidance. - Add TESTING.md with pytest usage and report-log interpretation notes. - Add REPORT_OUTPUT_LOGIC.md documenting expanded report outputs, dependencies, and toggle-driven failure modes. - Add cases_categorized.csv to classify case inputs and call out pipeline toggles.
Update AGENT.md and TESTING.md to expand testing and report-debugging guidance. Added explicit references to README, docs, REPORT_OUTPUT_LOGIC.md and other key docs; instructions to inspect runs/<case>/outputs/reeds-report/report.log and HTML when tests/test_outputs.py passes; a py_compile suggestion for quick syntax checks of bokehpivot modules; and a reproducible command to rebuild reports manually. Notes also clarify checking runs/<case>/inputs_case/switches.csv for effective switches, common switch-driven report gaps (e.g. cap_iter with timetype=seq, land_use_total when land_use_analysis=0), OpRes failure diagnostics (rep/opres_periods.csv, opRes_supply_h.csv, Sw_OpRes and lstfiles), and that health_damages_caused_r.csv uses the air-quality schema which bokehpivot normalizes for legacy displays.
These are three files meant to be read by the user to make sense of the repo. We decided that the Sharepoint 'Workflow' folder might be a better home for these.
Add a comprehensive AGENT.md as the primary orientation guide for agents working on ReEDS. The document summarizes repository structure, environment/setup (Python/Julia/GAMS), build and run commands, testing workflows, architecture and run flow, etc.
Merge dev-TF into MVP: AGENT.md and minor readme changes
Rename AGENT.md to AGENTS.md to correct filename pluralization and improve consistency. No content changes were made to the file.
Revert changes to health report rendering and offshore wind recf inputs
Add bootstrap_reeds.ps1 and document its usage in README. The script validates GAMS and Julia (requires Julia 1.12.1), sets ReEDS CONDA-style env vars for the session, ensures Python is pinned to 3.11, runs `uv sync --extra dev` and `julia --project=. instantiate.jl` (unless bypassed), and forwards arguments to `runbatch.py`. README includes a new "Optional PowerShell bootstrap command" section with examples and notes about the `-y/--bypass` option.
Update README and bootstrap_reeds.ps1 to add a --skip-setup alias for the bypass behavior and clarify docs. The PowerShell script now uses a -y switch ($y) to represent bypass mode (avoiding collisions with runbatch options), recognizes both --bypass and --skip-setup tokens, removes them from forwarded args, and adjusts logic to check $y when skipping uv sync and Julia instantiate. Minor wording updates were made to the README to document the new alias.
Add PowerShell bootstrap script and updates README
Added some clarifying comments to bootstrap, trimmed some whitespace from readme
Pull Main Changes from REEDs into Dev
Jeff updating local mvp branch with upstream mvp branch.
Added a note indicating that the README needs review due to recent changes in the upstream ReEDS base.
This PR mostly contains updates from the NLR repo, but with a few of our own env improvements along with it. Specifically, a transition to uv from conda and some additional scripts to help with LLM understanding, running the model more easily, and installing the required packages for our specific azure VM context.
Add `UV_MAMBA_GUIDE.md` documenting how to manually keep `environment.yml` and `pyproject.toml`/`uv.lock` aligned. The guide covers field-by-field mapping, add/remove dependency workflow, lockfile regeneration, and known non-1:1 cases (non-Python packages, pin semantics, git-based deps). It also flags current repo caveats, including stale path references from the recent restructure and that `environment.yml` is currently missing on this branch.
Updates repository guidance after the upstream restructure by replacing stale `runbatch.py` and old path references with current `runreeds.py`/`reeds/...` locations across AGENTS and README content. Moves CEPM-specific docs into `CEPM/` and updates references accordingly. Renames the PowerShell bootstrap script to `bootstrap_CEPM.ps1`, updates its usage/help text, and adds a native-command wrapper so stderr noise from tools like `uv`/`julia`/`gams` does not cause false failures under Windows PowerShell with `$ErrorActionPreference='Stop'`.
Add CEPM/check_env_sync.py, a stdlib-only checker that compares environment.yml (mamba/conda) against pyproject.toml (uv) and reports packages present in only one file or pinned to incompatible versions, ignoring an allowlist of intentional exceptions documented in CEPM/UV_MAMBA_GUIDE.md. Wire it into bootstrap_CEPM.ps1 as a new non-fatal step (7) that warns on unexpected drift but never aborts setup, and document the checker in the guide. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Refresh documentation + add uv-mamba translation
Add an explicit fallback in `reeds/input_processing/recf.py` to create an empty offshore-wind dataframe (aligned to onshore wind index) when `GSw_OfsWind` is disabled. This keeps downstream dataframe concatenation shape-consistent and avoids errors when offshore wind inputs are not generated.
Step 6 now runs a fast offline `Pkg.instantiate` (no registry update) as a check that also cheaply self-heals, and only falls back to the full `julia --project=. instantiate.jl` (which updates the registry) when the project can't be satisfied. Avoids the slow registry update on every run. Re-add the best-effort ntfy.sh notification (topic rmi-cepm-run-batch-finished) after runreeds.py returns, wrapped in try/catch so an offline/failed notification never affects the run. Update the README step list to match (Julia instantiate wording, the environment.yml/pyproject drift check, and the ntfy notification). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
The reeds2pras package was nested under reeds/resource_adequacy/ in the restructure, but its README still pointed `cd` at the old top-level ReEDS/reeds2pras/test location. Update both occurrences to the current ReEDS/reeds/resource_adequacy/reeds2pras/test path. Closes #21 Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Updated notification logic to send a message before and after launching runreeds.py.
Clarified notification timing for ntfy.sh in README.
Renames `bootstrap_CEPM.ps1` to `run_cepm.ps1` and updates docs/references in `README.md`, `AGENTS.md`, and `CEPM/UV_MAMBA_GUIDE.md` to match. The script now supports bootstrap-only `-q/--quiet` (disable ntfy notifications) and `-u/--user` (custom username in ntfy messages), strips long-form tokens before forwarding args to `runreeds.py`, and sends best-effort start/finish notifications with a short timeout.
…into fix/pre-ff-fixes
Add notification for failed ReEDS run using ntfy.sh.
Add NTFY message notification on runreeds failure.
Fix/pre ff fixes: Catch errors with offshore wind, catch some reference errors in reeds2pras readme, update bootstrap_cepm.ps1 and attendant documentation
I think we decided to track this somewhere else
We're not using this any more -- makes sense to delete
ty-fi
marked this pull request as ready for review
July 9, 2026 12:53
github-actions
Bot
added
docs
release
model_changes
data_changes
hourlize
dependencies
labels
samhuestis
approved these changes
Jul 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This is a somewhat comprehensive change to 'main' to reflect the CEPM workflow. It does several things:
The only change to the way the model works is the recf.py fix -- the rest is documentation, switching to UV, making sure tests can complete, and adding the run_CEPM.ps1 wrap-around script.