Remote pull: stage temp files in per-user /tmp; GUI shot deletion#70
Open
sguizzo wants to merge 2 commits into
Open
Remote pull: stage temp files in per-user /tmp; GUI shot deletion#70sguizzo wants to merge 2 commits into
sguizzo wants to merge 2 commits into
Conversation
The remote toksearch pull staged its ~80 MB compact .h5 under ~/magnetics_fetch/out/ before copying it home; across runs this blew the GA home quota (OSError [Errno 122] Disk quota exceeded on the cluster-side h5py.File create). Stage the transient file in node-local /tmp instead (it is written, rsync'd back, and never reused), keeping only the small package sync under remote_dir, and delete the staged copy after copy-back so /tmp doesn't accumulate. Make the stage dir per-user (/tmp/magnetics_out_<user>) so concurrent pulls on the shared login node don't collide: the username is put in the top-level /tmp entry (each user owns their own) rather than a subdir under a shared parent. The name comes from the known GA username, falling back to `id -un` on the cluster (works when $USER/$LOGNAME are unset in a non-login ssh shell). Stage dir base is overridable via run_remote(out_stage=...). Co-Authored-By: Claude Opus 4.8 <[email protected]>
Add data-management controls to the shot list: an × on each fetched
shot deletes it, and a "Clear all" in the section header deletes every
fetched shot. Both remove the underlying HDF5 file(s) on the backend,
not just the GUI row.
Backend: h5source.delete_shot() removes EVERY file for a shot (matched
on the stored `shot` attr, so multi-file shots — different windows/fmax
or a bench artifact — all go), delete_all_shots() clears them all; both
refresh the file index. New DELETE /api/machines/{shot} and DELETE
/api/machines routes also drop the shot's cached STFT/QS state; the
per-shot route 404s when nothing matches (e.g. a mock machine).
Frontend: store.removeMachine/clearMachines call the routes, re-list,
and fix the selection; the × and Clear-all only render for real shots
against a live backend (mock demo machines have nothing to delete), each
behind a confirm() since the data loss is irreversible.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two related data-layer/GUI changes for the DIII-D remote fetch path.
1. Stage the remote pull's temp
.h5in per-user/tmp(off the home quota)The
remote(cluster/toksearch) backend ran the pull on omega and wrote itscompact
.h5under~/magnetics_fetch/out/before rsync'ing it home. That fileis ~80 MB and accumulated across runs, so it eventually blew the GA home disk
quota — the pull died with
OSError [Errno 122] Disk quota exceededon thecluster-side
h5py.Filecreate.The staged file is purely transient (written → copied back → never reused), so:
.h5in node-local
/tmp/magnetics_out_<user>/instead of the home dir. Only thesmall package sync still lives under
remote_dir(~/magnetics_fetch)./tmpdoesn'taccumulate either.
_<username>suffix, a top-level entry in sticky-bit/tmp) so concurrent pulls from different users on the shared login node don'tcollide. The username comes from the known GA
username, falling back toid -unon the cluster (works even when$USER/$LOGNAMEare unset in anon-login ssh command shell). Base dir overridable via
run_remote(out_stage=...).Verified live: a real pull staged to
/tmp/magnetics_out_<user>/shot_<n>.h5,copied home, and cleaned up — quota error gone.
2. GUI: delete a shot (× per row) + clear-all, removing its data
Adds data-management controls to the left-rail shot list:
section header deletes every fetched shot;
Backend:
h5source.delete_shot()removes every file for a shot (matched onthe stored
shotattr, so multi-file shots all go),delete_all_shots()clearsthem all; both refresh the file index. New
DELETE /api/machines/{shot}andDELETE /api/machinesroutes also drop the shot's cached STFT/QS state; theper-shot route 404s when nothing matches (e.g. a mock machine).
Frontend:
store.removeMachine/clearMachinescall the routes, re-list, andfix the selection. The × and Clear-all only render for real shots against a live
backend (mock demo machines have nothing to delete), each behind a
confirm()since the data loss is irreversible.
Testing
tests/test_delete.py(5 tests) coversdelete_shot/delete_all_shotsandthe DELETE routes against an isolated temp data dir.
ty check src/magnetics,ruff check,ruff format --checkall clean.🤖 Generated with Claude Code