V1.2.3 UI optimization and new features#214
Open
guillaume-byte wants to merge 22 commits into
Open
Conversation
… two functions; and so adapt UI (#213)
* Fix release note generation from PR commits only * Add __getitem__ to ValueProxy for parameters wrap
…211) Document the Weights Studio per-image bounding-box render caps (GT and PRED capped independently) in the configuration env-var reference and the weights_studio deployment .env examples. Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
* Add save btn and fix UI text issue of the current version * add details of export fct to readme
…re toggles The branch sync left two identical `message SaveCheckpointOperation` definitions, which makes protoc reject the file. Remove the redundant first copy (keeping the documented "manual save now" one) and regenerate the Python descriptor. The grpc stub is unchanged (no rpc was duplicated). Docs: add the Weights Studio feature toggles (ENABLE_PLOTS, ENABLE_DATA_EXPLORATION, ENABLE_HYPERPARAMETERS_OPTIMIZATION, ENABLE_AGENT) to configuration.rst and the AGENTS.md frontend env-var table. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
- GetHistogram RPC: bin one column server-side into <=512 typed bins (min/max/avg/count + per-(origin,discarded) sub-bars) instead of the client pulling every row and binning in the browser. Bit-identical to the client binning; ~116x smaller payload, ~50ms warm. Adds the proto messages + RPC, regenerated pb2/pb2_grpc, DataService.GetHistogram, and servicer delegation. - ApplyDataQuery: skip the forced full-view rebuild for SORT-ONLY operations (a sort just re-orders the existing snapshot). Global sort ~7.5s -> ~0.5s. - _slowUpdateInternals: run the view rebuild on a background thread for reader-triggered (non-force) refreshes, so grid/histogram reads never block on the multi-second collapse+combine. Reader p95 ~3000ms -> ~130ms. Filters/ resets still refresh inline (need fresh data). - ws-classification example: loosen eval (100->500) / checkpoint (25->250) cadence and use a bigger eval batch (16->128) so eval stops dominating wall-clock. Co-authored-by: Alexandru Rotaru <[email protected]> Co-authored-by: Claude Opus 4.8 <[email protected]> Co-authored-by: Guillaume <[email protected]>
…ode in root log dir
…ithub.com/GrayboxTech/weightslab into v1.2.3---UI-Optimization-and-new-features
* Server-side histogram binning + grid/sort perf fixes - GetHistogram RPC: bin one column server-side into <=512 typed bins (min/max/avg/count + per-(origin,discarded) sub-bars) instead of the client pulling every row and binning in the browser. Bit-identical to the client binning; ~116x smaller payload, ~50ms warm. Adds the proto messages + RPC, regenerated pb2/pb2_grpc, DataService.GetHistogram, and servicer delegation. - ApplyDataQuery: skip the forced full-view rebuild for SORT-ONLY operations (a sort just re-orders the existing snapshot). Global sort ~7.5s -> ~0.5s. - _slowUpdateInternals: run the view rebuild on a background thread for reader-triggered (non-force) refreshes, so grid/histogram reads never block on the multi-second collapse+combine. Reader p95 ~3000ms -> ~130ms. Filters/ resets still refresh inline (need fresh data). - ws-classification example: loosen eval (100->500) / checkpoint (25->250) cadence and use a bigger eval batch (16->128) so eval stops dominating wall-clock. Co-Authored-By: Claude Opus 4.8 <[email protected]> * Fix proto files and is/is not ValueProxy from ledgers to ==/!= --------- Co-authored-by: Alexandru Rotaru <[email protected]> Co-authored-by: Claude Opus 4.8 <[email protected]> Co-authored-by: Guillaume <[email protected]>
Strip colorful emoji from console.log, print statements, logger calls, docstrings, and comments across the entire backend codebase. Plain text conveys the same meaning without unicode-rendering concerns. Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
weightslab logdir <root_log_dir> [--no-ui] [--certs] [--grpc-port PORT] Wires the existing load_experiment_for_explore / serve / keep_serving infrastructure into the CLI so a downloaded log dir can be explored without running the original training script. By default also launches the Weights Studio Docker UI stack; pass --no-ui to skip (useful when the UI is already running from a prior 'weightslab ui launch'). Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Both classification and segmentation examples now: - Export signal history + data grid to root_log_dir every 100 steps (configurable via write_export_ratio in config.yaml) - Also export once at end of training for a final snapshot This lets an offline explore session (weightslab logdir) immediately pick up the latest signal/dataframe state without waiting for a checkpoint. Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
…crash Adds a multi-task MNIST example (ws-multitask) demonstrating: - Shared CNN backbone with classification and localization heads - Two separately-tracked WeightsLab losses (cross-entropy + smooth-L1) - Detection-format targets (tight bounding box per digit) for UI bbox overlay - Per-sample accuracy signal for data grid inspection Fixes GuardContext.__enter__ crashing with "Proxy target not set" when no model is registered: get_model() returns a Proxy(None) placeholder; now checks the Proxy _obj target before assigning self.model so unregistered contexts (e.g., Lightning-only tests) work gracefully. Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
…dger) Previous fix only prevented assigning an unresolved Proxy but left self.model pointing to a stale proxy from a prior call. Now always derives self.model from the current ledger state on each __enter__: if get_model() returns a Proxy with no target (no model registered), self.model is set to None rather than falling back to a potentially-stale earlier value. Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Adds _is_ui_running() which checks each _STACK_CONTAINERS entry via docker inspect. logdir_explore uses it to skip ui_launch when the Weights Studio stack is already up, avoiding a redundant Docker bring-up. Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Removes all code added for the offline explore / logdir feature: - backend/explore_mode.py (deleted) - tests/integrations/test_explore_mode.py (deleted) - examples/PyTorch/ws-multitask/ (deleted) - load_experiment_for_explore / _rehydrate_dataframe_from_disk from src.py - logdir_explore / _is_ui_running / logdir subparser from ui_docker_bridge.py - explore mode guards from experiment_service.py and model_service.py - explore mode unit tests from test_trainer_services_unit.py Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
guillaume-byte
force-pushed
the
v1.2.3---UI-Optimization-and-new-features
branch
from
e51b974 to
9831f81
Compare
* Fix dataframe issue with index * feat: add __discard_by_tag__ handler in EditDataSample New special stat_name that discards all samples carrying a given tag across the whole dataset without requiring the frontend to enumerate sample IDs. Uses the existing upsert_df path — same as per-sample discard — grouped by origin for correctness. Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * fix: metadata clone naming (@hash format) + delete no longer blocks gRPC thread - build_metadata_copy_column_names: first clone of original field uses {source}@{hash} (no _{n}); conflicts get _{n} suffix; cloning an already-cloned field reuses the same @hash, only increments _{n} - is_copy_metadata_column_name: updated regex to match new .+@.+ format - __delete_metadata__ handler: replace blocking _slowUpdateInternals(force=True) with non-blocking background kick — view is already up-to-date from manual drop, no need to stall the gRPC response for 5-10s Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * fix: always serialize/deserialize prediction and target as lists in H5 Write path: serialize_value now handles numpy arrays of ALL ndim (not just <= 1). A 2D array was falling through to str() which produced numpy repr strings (space-separated, no commas) that aren't valid JSON. Read path: deserialize_value for MODEL_INOUT_LIST columns now: - maps "nan"/"none"/"" strings to np.nan instead of returning them as-is - removes the single-element unwrap (a [0.5] list stays [0.5], not 0.5) - adds a fallback regex pass to recover old data stored as numpy repr Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * Fix ultralytics BB rendering on UI side and weightslab examples --------- Co-authored-by: Claude Sonnet 4.6 <[email protected]>
* Add anonymous usage telemetry (import + ui launch) Fires a non-blocking async ping to telemetry.graybx.com/v1/ping: - On first daily import (once per 24h, gated to main process) - On every `weightslab ui launch` Payload: random UUID, event name, package version, Python version, OS, CI flag, client timezone. No PII collected. Raw IP is used server-side for geolocation (country/city) and never stored. Opt-out: WL_NO_TELEMETRY=1. CI environments are auto-detected and skipped via well-known env vars (GITHUB_ACTIONS, CI, GITLAB_CI, etc.). Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * Telemetry: one ping per process + per 24h for imports Add in-process guard (_import_pinged_this_process) so multiple `import weightslab` calls within the same Python process only fire one ping, regardless of module reloads or test reruns. The 24h disk-based cooldown still applies on top. Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * Move telemetry module to weightslab/utils/telemetry.py Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * Fix daemon thread + add WL_TELEMETRY_DEBUG mode - Remove daemon=True so short-lived processes (python -c "import weightslab") wait up to 3 s for the HTTP ping before exiting - Add WL_TELEMETRY_DEBUG=1 to print ping success/failure instead of silently swallowing Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * Fix telemetry ping_import called before __version__ is defined Move the telemetry hook to after the __version__ block so the variable exists when ping_import() is called. Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * fix var * change dest point * Add telemetry disclosure notice + README section - First-run notice printed to stderr once (flag stored in ~/.weightslab/telemetry_noticed) - Notice explains what is/isn't collected and how to opt out - Skipped in CI and after the first time it has been shown - README telemetry section added with opt-out instructions Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * Remove telemetry section from README Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * add debug mode for telemetry * fix var Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * Format telemetry notice between #### borders Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * Remove telemetry notice + route all output through logger.debug Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * fix code quality --------- Co-authored-by: Claude Sonnet 4.6 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Next release includes:
And more to come!