Nexus Wishlist

[jtkrogel]

Nexus bugs/docs/features with high benefit to fix/write/implement. The goal is to maximize the impact of expended developer time.

Bugs

Added by @jtkrogel

• "queue size" bug. Batch machines often have limits on the queue size (number of jobs that can be submitted to the queue at one time). Nexus users can set a queue size limit to match this behavior. The feature works fine if the Nexus script is run without interruption. However when rerunning post-interruption, Nexus (often/always?) stops submitting new jobs.
• "bundle" bug. When using the bundle function (a simulation object wrapper around other simulation objects), sometimes the bundle simulation will not complete properly. When this happens (or sometimes without?), large swaths of other simulations in the workflow DAG will cease functioning. Might also interact with interrupts.

Documentation

Added by @jtkrogel

• Documentation for all user-facing generate_* functions, both in the manual and as docstrings. Also settings, job, and ppset. This is likely the single most beneficial documentation task.
  • Nexus: Document Structure, Crystal, and some generate functions. #6018
• Many new user examples illustrating various features, input styles, and code interfaces. Organized in the examples directory, with README's, and described in the manual.
• Documentation and examples for structure manipulation operations. Docstrings for the useful functions, examples of combined use to manipulate structures (file read/write, tiling, finding the "best" supercell, bounding box for molecules, molecular composition, distance/vector/neighbor tables, adding/removing layers, element labeling, creation and use of mask arrays for atomic selection and modification, etc).
  • Nexus: Add tests and docstrings for some Structure functions. #5971
• How to add a new machine
• How to add a new simulation code
• Examples for simulation input read/manipulate/write
• Examples for pseudopotential read/manipulate/write, including file format conversions
• Examples for gaussian basisset read/manipulate/write, including file format conversions
• Documentation and examples for other file I/O classes
• Docstrings for other major functions, classes, and member functions. Classes: Settings, Job, Supercomputer (derived), Simulation (derived), ProjectManager

New Features

Added by @jtkrogel

• Dynamic workflows. Removes the DAG workflow constraint. Use native python code to create arbitrary workflows. Enables construction of true QMC algorithms built upon individual calculations (converge to target errorbar, pick the best Jastrow among independent runs, timesteps extrap, finite size extrap, SHDMC, etc). Also enables construction of recovery upon failure workflows.
• New codes, e.g. ORCA
• Slurm machine class. Enables job management on arbitrary Slurm machines. Obviates the need for many users to add custom classes for each machine they have access to.
• Slurm job class. Similarly provides generic support for all flags/keys/etc relating to Slurm jobs.

Added by @brockdyer03

• Update pseudopotential.py user interface to be more object-oriented.

Other Developer Tasks

Added by @jtkrogel

• Bring supported QE input parameters up to date
• Bring supported VASP input parameters up to date
• Hunt for (will require digging!) and add other QMCPACK input parameters
• Hardening status checks for completed runs. Carefully delineate between failure and success based on thorough checks. Also needed to enable dynamic workflows, which demand higher accuracy on this task (supports essential logic based on run failure/success).
• Improvements to analyzer classes. Capture wider array of simulation outputs/products. Reduce nesting to simplify user access to these quantities.

Added by @brockdyer03

• Rework logging, error handling, and warning code to use Python's built in libraries logging and warnings.
  • Nexus: Improve stack trace printing in errors. #5985
  • Nexus: Replace old warning system with Python's warnings library #6007
• Update physical_system.py to use a more minimal and obvious interface.
  • Nexus: Rework physical_system.py #6010
• Refactor fileio.py to make accidental opening without closing less likely.
• Potentially rework Nexus to use pathlib.Path everywhere. (requires lengthy discussion and testing, may never happen)

Automation Support (High value: reduces precious human time)

Added by @jtkrogel

• Automatic pseudopotential download and use (e.g. ccECP, SSSP)
• Automatic k-point grid selection
• Automatic selection of supercell sequences
• Automatic pw energy cutoff selection
• Automatic meshfactor and hybrid rep setup
• Automatic job design (essential for HPC machines). Node count, walker samples, walltime. Obeying machine-specific node/walltime limits. Compute resource constraints ("don't burn more than X node hours").
• Models to estimate input samples required to produce target energy errorbar
• Models to estimate runtime, etc, perhaps accomplished by exploratory short/small run for calibration. Supports automation of job design.
• Automatic detection of anomalous events (e.g. population explosions, other energy instabilities)
• Automatic recovery upon failure (e.g. for failed scf convergence, wavefunction optimization, dmc pop explosions, etc)
• Algorithms: timestep extrapolation, finite size extrapolation, equation of state, high quality density estimation, U/exact exchange scans, etc. See dynamic workflows above.

[prckent]

Would you like other people to add to this list?

[jtkrogel]

Sure. Trying to list roughly in order of importance/priority. List order up for discussion also.

[jtkrogel]

Adding separate headings for "New Features" and "Other Developer Tasks" now.

[brockdyer03]

Added some of my own wishes, updated formatting to specify who added what.

[prckent]

Putting an intentionally short wishlist, although I do have more wishes. After the recent focus on Nexus internals, I think the focus should move to updates that either solve known problems experienced by users or that give a big user visible return for small effort. i.e. Will actually end up in the release notes. I strongly suggest that the bigger wishlist above be viewed through this lens and the high ROI for limited effort items be tackled first.

The following are roughly in order of priority. Apart from the VMC/DMC continuation, I think all are quick and high value.

1. Document how to add a machine. We have had abandoned PRs because of this and it is the first thing many people will have to do to use Nexus at any scale. The supported machines should also be documented, but the next wish may achieve that automatically.

2. Implement a strategy for more effective code documentation & manual generation, e.g. get the docstrings automagically included in the manual so that we can document the user accessible routines just once in the source. Many large Python projects have solutions to this. Once this is done we can decide where to focus expanding the docs. e.g. The generation routines need to be described in the manual, but many already have good docstrings. (Perhaps Nexus: New documentation theme and structure #5978 already does this?).

3. Add a universal "ws" machine that finds out the number of cores available. This is an actual need both for science runs, for testing, and a common issue with our tutorials. (An LLM can trivially write the “give me the number of actual physical cores, ignoring any hyperthreading” function for Linux, BSD, MacOS.) Nexus: Add generic workstation 'ws' machine that automatically detects number of available cpu cores #6003

4. Fix the job error handling problems seen on ws runs, e.g. Nexus: unavailable pyscf in h2o_ae_hf_qmc.py not caught. Runs forever. #5989 . This might be a very small quick fix because it used to work. Presumably some error code is no longer caught or the output parsing needs updating or some Python API changed behavior.

5. Add a method to continue DMC runs in subsequent jobs, following the recipe in the QMCPACK manual. Document the capability in the NEXUS manual.

6. Add examples of using the analyzer classes and document their use. These are almost completely undocumented and therefore “don’t exist” to a casual user. The QE cutoff scan result handling is one of the rare examples. Most important: add some QMC results analysis with qmca and graph (matplotlib) generation, or at least writing of suitable datafiles for graphing. This will help the workflow become the single source of truth for science projects, with separate analysis scripts less likely to be needed.

7. Improve the readability and usability of the —status_only output. e.g. Make passed/failed status very easy to see.

8. Document approaches to running on the head node of a shared machine. e.g. Noting the name of the login node + using tmux. (This is the only viable way I have to practically run on any HPC system, since I don’t have a permanently connected machine). No doubt there are other workarounds in use.

9. Make serial jobs count against the cores available in “wsN” runs where this is a hazard. e.g. If doing a scan over 100s of structures you can have 100s of serial jobs spin up.

[brockdyer03]

2. Implement a strategy for more effective code documentation & manual generation, e.g. get the docstrings automagically included in the manual so that we can document the user accessible routines just once in the source. Many large Python projects have solutions to this. Once this is done we can decide where to focus expanding the docs. e.g. The generation routines need to be described in the manual, but many already have good docstrings. (Perhaps Nexus: Overhaul documentation #5978 already does this?).

Indeed #5978 does do that.

3. Add a universal "ws" machine that finds out the number of cores available. This is an actual need both for science runs, for testing, and a common issue with our tutorials. (An LLM can trivially write the “give me the number of actual physical cores, ignoring any hyperthreading” function for Linux, BSD, MacOS.)

No LLM needed, just some basic knowledge of operating systems. I'll open a PR in the next few days.

[prckent]

Noting that the "ws" machine was handled in #6003