Skip to content

community_issues.md

Latest commit

 

History

History
394 lines (339 loc) · 26.9 KB

community_issues.md

File metadata and controls

394 lines (339 loc) · 26.9 KB
Raw

Sparsify GitHub Issues Catalog (37 Actionable Task Templates)

This document contains 37 pre-formatted GitHub issues ready to be copy-pasted into your repository. They are organized by component and categorized by difficulty level (including good first issue tags) to attract and guide contributors.

🎨 Web UI & Front-end

1. [Web UI] Add custom dark/light theme toggle

  • Difficulty: Easy (good first issue)
  • Recommended Labels: webui, enhancement
  • Files involved: site/
  • Description: Currently, the Web UI defaults to dark mode. We should add a persistent theme toggle button in the sidebar that allows users to switch between light and dark mode, storing the preference in localStorage.
  • Steps to implement:
    1. Add a theme toggle button or icon (sun/moon) to the sidebar HTML/JS.
    2. Write CSS classes for light mode theme variables (background colors, text colors, borders).
    3. Implement a JS helper to toggle the .light-mode class on the <body> element and save it to localStorage.

2. [Web UI] Draw real-time cache hit rate charts with SVG/CSS

  • Difficulty: Medium
  • Recommended Labels: webui, enhancement
  • Files involved: site/
  • Description: Telemetry reports hit rates numerically under responses. It would be highly visually appealing to draw a live, smooth line graph or radial progress indicator using SVG to display caching performance.
  • Steps to implement:
    1. Create a container in the chat UI metadata panel for the cache chart.
    2. Use standard canvas or SVG to draw/update data points dynamically as telemetry JSON payload arrives via SSE.
    3. Add smooth CSS transitions to updates.

3. [Web UI] Make chat interface mobile-responsive

  • Difficulty: Easy (good first issue)
  • Recommended Labels: webui, bug
  • Files involved: site/
  • Description: When viewing the Web UI on mobile devices or narrow browser windows, the sidebar overlaps with the chat panel or gets truncated.
  • Steps to implement:
    1. Add CSS media queries for viewport widths under 768px.
    2. Hide the sidebar on narrow screens and implement a hamburger menu toggle button.
    3. Adjust padding and text sizes of speech bubbles for phone dimensions.

4. [Web UI] Visual model manager inside sidebar

  • Difficulty: Medium
  • Recommended Labels: webui, enhancement
  • Files involved: site/
  • Description: Users currently pull and remove models via CLI or the web UI select box. We should add a dedicated "Models" tab/view in the sidebar that lists all models in our catalog with status tags (Ready / Not Installed), and click-to-download/remove buttons.
  • Steps to implement:
    1. Create a visual model list view in HTML/CSS.
    2. Add REST endpoints for /api/models/pull and /api/models/remove in server.py.
    3. Connect frontend buttons to trigger fetches and display download progress bars.

5. [Web UI] Export chats to Markdown or PDF

  • Difficulty: Easy
  • Recommended Labels: webui, enhancement
  • Files involved: site/
  • Description: Add an option in the Web UI to download active chat transcripts as formatted Markdown (.md) or text files.
  • Steps to implement:
    1. Create an "Export Chat" dropdown button near the chat header.
    2. Parse the active chat history DOM/state into markdown text.
    3. Generate a dynamic download file link using client-side JavaScript.

📟 Terminal UI (TUI)

6. [TUI] Prevent terminal crash on window resize

  • Difficulty: Medium
  • Recommended Labels: tui, bug
  • Files involved: src/sparsify/cli.py
  • Description: Resizing the terminal window during an active sparsify run chat session can sometimes cause layout exceptions in prompt_toolkit.
  • Steps to implement:
    1. Hook into window resize events inside the prompt_toolkit generation loop.
    2. Recalculate terminal rows/columns and trigger a clean UI redraw.
    3. Handle exceptions caused by shrinking the window past UI boundary limits.

7. [TUI] Add mouse scroll wheel support in terminal

  • Difficulty: Easy (good first issue)
  • Recommended Labels: tui, enhancement
  • Files involved: src/sparsify/cli.py
  • Description: Mouse scrolling is currently disabled in the full-screen terminal TUI. Enabling mouse scroll support would allow users to look back at past chat history.
  • Steps to implement:
    1. Enable mouse support in prompt_toolkit config parameters.
    2. Bind mouse scroll events to scroll the text window viewport.

8. [TUI] TUI keyboard shortcuts help modal

  • Difficulty: Easy
  • Recommended Labels: tui, enhancement
  • Files involved: src/sparsify/cli.py
  • Description: Users don't know the keys to exit, clear, or manage terminal chat. Pressing ? or Ctrl+H should trigger a clean help pop-up listing all TUI shortcuts.
  • Steps to implement:
    1. Listen for help shortcuts inside TUI input handlers.
    2. Render a simple modal box centered in the console listing key combinations.

9. [TUI] Code block syntax highlighting in bubble replies

  • Difficulty: Medium
  • Recommended Labels: tui, enhancement
  • Files involved: src/sparsify/cli.py
  • Description: While Markdown is parsed, code blocks (e.g. ```python) lack syntax coloring inside the terminal speech bubbles.
  • Steps to implement:
    1. Integrate pygments or rich.syntax styling helper.
    2. Detect code fences inside token reply streams and style code blocks on the fly before drawing.

10. [TUI] Long words overflow wrapping bug in narrow terminals

  • Difficulty: Easy (good first issue)
  • Recommended Labels: tui, bug
  • Files involved: src/sparsify/cli.py
  • Description: If a generated response contains long links or uninterrupted lines (like file paths), the speech bubble breaks bounds and shifts terminal lines.
  • Steps to implement:
    1. Add custom word wrapping or truncation boundaries based on window size.
    2. Wrap long paths cleanly inside bubbles using text-wrap rules.

🗄️ Database & Storage

11. [Database] Export profiling history to CSV/JSON format

  • Difficulty: Easy
  • Recommended Labels: database, enhancement
  • Files involved: src/sparsify/storage/database.py
  • Description: Add a database query command to dump historical runs, telemetry, and memory profile records into standard CSV or JSON files.
  • Steps to implement:
    1. Implement a new CLI handler: sparsify history --export <csv/json>.
    2. Fetch all runs from the database and serialize fields cleanly.

12. [Database] Automated database indexing and size cleanup

  • Difficulty: Medium
  • Recommended Labels: database, maintenance
  • Files involved: src/sparsify/storage/database.py
  • Description: Telemetry history accumulates quickly, bloating the database file size. We need automated purging or size limit warnings.
  • Steps to implement:
    1. Add a setting for telemetry retention limits (e.g. 30 days).
    2. Write a cleanup query that runs once daily or at CLI startup.
    3. Run VACUUM commands periodically to keep database files compact.

13. [Database] Save chat session histories locally

  • Difficulty: Hard
  • Recommended Labels: database, enhancement
  • Files involved: src/sparsify/storage/database.py
  • Description: Currently, closing the server or terminal wipes the active chat context. We should save conversation history in SQLite to load them back on restart.
  • Steps to implement:
    1. Create a conversations database schema: id, session_title, model, created_at.
    2. Save message pairs dynamically as they are processed.
    3. Render past sessions in the Web UI left sidebar.

🧠 Core Paging & Performance (PyTorch / MLX)

14. [Core] Multi-threaded safetensors expert loader

  • Difficulty: Hard
  • Recommended Labels: core, performance
  • Files involved: src/sparsify/paging_torch/store.py
  • Description: Loading experts sequentially introduces block latencies during model surgery. We should load experts using multiple thread workers.
  • Steps to implement:
    1. Implement concurrent ranges reading via Python concurrent.futures.
    2. Overlap the retrieval of multiple routed experts in a single routing step.
    3. Measure speed differences across standard SSD hardware.

15. [Core] Predictive prefetching of experts

  • Difficulty: Hard
  • Recommended Labels: core, performance
  • Files involved: src/sparsify/paging_torch/surgery.py
  • Description: The model routing decision is calculated one block layer ahead. We can use this prediction gap to prefetch the experts for layer N+1 while computing layer N.
  • Steps to implement:
    1. Hook into the forward pass calculation logic.
    2. Spawn a background thread worker to fetch the next block's experts asynchronously.
    3. Measure reduction in forward pass delays.

16. [Core] Reuse memory-mapped file handles

  • Difficulty: Medium
  • Recommended Labels: core, bug
  • Files involved: src/sparsify/paging_torch/store.py
  • Description: Opening file handles for every expert read block creates high system descriptor activity. We should pool and reuse file descriptors.
  • Steps to implement:
    1. Keep a cache of open file handles for target safetensors shards.
    2. Close handles cleanly during server shut down or model unload.

17. [Core] Support GGUF layer offloading alongside PyTorch paging

  • Difficulty: Hard
  • Recommended Labels: core, performance
  • Files involved: src/sparsify/backends/
  • Description: Explore hybrid paging of GGUF formats directly from memory mapping, allowing CPU layers to remain un-allocated.
  • Steps to implement:
    1. Analyze memory mappings for GGUF layouts.
    2. Add custom GGUF tensor paging hook handlers.

18. [Core] CPU Matrix Kernel optimizations (Intel/AMD)

  • Difficulty: Medium
  • Recommended Labels: core, performance
  • Files involved: src/sparsify/backends/pytorch_backend.py
  • Description: When PyTorch runs on standard CPU backends, inference can be slow. We should investigate using mkl or Intel-optimized CPU runtimes.
  • Steps to implement:
    1. Detect Intel CPU extensions.
    2. Set thread configurations: torch.set_num_threads() based on physical CPU cores.

🛠️ Installer & Deployment

19. [Installer] Detect and configure CUDA automatically on Linux/Windows

  • Difficulty: Medium
  • Recommended Labels: installer, enhancement
  • Files involved: install.sh
  • Description: Non-macOS systems need PyTorch with CUDA support. The script should verify CUDA version and install the matching pip wheel version.
  • Steps to implement:
    1. Check for nvcc or nvidia-smi version outputs.
    2. Run pip install targeting the specific PyTorch CUDA wheel URL (e.g. cu121/cu124).

20. [Installer] Docker container setup for server mode

  • Difficulty: Easy (good first issue)
  • Recommended Labels: installer, enhancement
  • Files involved: Dockerfile
  • Description: Provide a Dockerfile and docker-compose script to deploy the Sparsify server inside a containerized host.
  • Steps to implement:
    1. Write a clean Multi-stage Dockerfile setup.
    2. Mount the local model folder to prevent redundant downloads.

🔌 API & Integration

21. [API] Add OpenAI-compatible /v1/embeddings endpoint

  • Difficulty: Medium
  • Recommended Labels: api, enhancement
  • Files involved: src/sparsify/runtime/server.py
  • Description: Allow external vector search software (like Chroma or Qdrant) to connect directly to Sparsify.
  • Steps to implement:
    1. Add /v1/embeddings POST route.
    2. Feed tokens to the model backbone, extract the hidden state, and return the vector JSON.

22. [API] Add basic auth rate-limiting middleware

  • Difficulty: Medium
  • Recommended Labels: api, security
  • Files involved: src/sparsify/runtime/server.py
  • Description: If the API is exposed on local networks, we need authentication.
  • Steps to implement:
    1. Add an optional API Key setting SPARSIFY_API_KEY.
    2. Write request checking middleware that blocks unauthorized calls.

📄 Documentation

23. [Docs] Add a detailed guide on setting up Sparsify on WSL2

  • Difficulty: Easy (good first issue)
  • Recommended Labels: documentation
  • Files involved: docs/
  • Description: Guide Windows developers on setting up GPU-passthrough CUDA drivers inside WSL2.
  • Steps to implement:
    1. Create docs/WSL2_SETUP.md.
    2. Outline step-by-step commands for WSL setup.

24. [Docs] Translate project documentation

  • Difficulty: Easy
  • Recommended Labels: documentation
  • Files involved: docs/
  • Description: Translate README and instructions into popular languages (Spanish, Chinese, Japanese) to grow global adoption.
  • Steps to implement:
    1. Create language-specific docs files.
    2. Link translations in the main README.

25. [Docs] Update contribution guide and templates

  • Difficulty: Easy (good first issue)
  • Recommended Labels: documentation
  • Files involved: CONTRIBUTING.md
  • Description: Prepare coding style checks, tests setup instruction, and pull request checklist template.
  • Steps to implement:
    1. Update contribution documentation with development environment check instructions.
    2. Add standard templates under .github/PULL_REQUEST_TEMPLATE.md.

🖼️ Multimodal — Image & Video Generation

Sparsify's memory story extends beyond MoE LLMs, but the mechanism changes per architecture: dense diffusion transformers fire every weight on every denoising step (per-step weight paging can only thrash), so the wins are stage residency (text encoder → DiT → VAE never run concurrently), 4-bit quantization at load, and — for timestep-MoE video models like Wan2.2 A14B — one expert swap per generation, the best possible SSD-paging access pattern.

26. [Image] Verify FLUX.1-schnell end-to-end and flip it to tested: True

  • Difficulty: Easy (good first issue)
  • Recommended Labels: multimodal, verification
  • Files involved: src/sparsify/backends/image_backend.py, src/sparsify/runtime/model_registry.py
  • Description: The mflux path is wired but honestly flagged tested: False. Run sparsify pull flux:schnell and sparsify imagine flux:schnell "a lighthouse at dusk" on Apple Silicon, record peak memory / duration / output quality, pin the working mflux version range in pyproject.toml, and flip the catalog flag with your measurements in the PR.
  • Steps to implement:
    1. Accept the FLUX license on HuggingFace and pull the model.
    2. Run generation at 1024x1024 / 4 steps; capture peak_memory_gb from the CLI output.
    3. Fix any mflux API drift in MFluxImageBackend (the ModelConfig/Config import surface moves between versions).
    4. Update tested and add measurements to docs/measurements/.

27. [Image] Stage-residency paging for diffusion pipelines (text encoder → DiT → VAE)

  • Difficulty: Hard
  • Recommended Labels: multimodal, core-paging, memory
  • Files involved: src/sparsify/backends/image_backend.py
  • Description: A diffusion pipeline's stages never run concurrently: the text encoder (T5-XXL alone is ~9.5 GB fp16) runs once, then the DiT loops, then the VAE decodes. Loading stages on demand and evicting them after use makes peak memory the largest stage instead of the sum — FLUX in roughly 8 GB. This is Sparsify's core moto applied at stage granularity.
  • Steps to implement:
    1. Encode the prompt, then free the text encoder before instantiating the DiT.
    2. Free DiT weights before VAE decode.
    3. Report per-stage peak memory in ImageGenerationResult.metadata.
    4. Cache the encoded prompt so repeated generations with the same prompt skip the text-encoder load entirely.

28. [Video] Timestep-expert paging for Wan2.2 A14B — one 14B expert resident at a time

  • Difficulty: Hard
  • Recommended Labels: multimodal, core-paging, flagship
  • Files involved: src/sparsify/backends/, src/sparsify/runtime/model_registry.py
  • Description: Wan2.2 T2V A14B is a two-expert timestep-MoE: a high-noise expert denoises early steps, a low-noise expert finishes. Only one expert is ever active, and the switch happens once per generation — the ideal SSD-paging access pattern (contrast with per-token LLM expert paging). Target: a 27B-parameter video model in a 14B-model memory footprint. The catalog already flags this entry timestep_moe: True.
  • Steps to implement:
    1. Load only the high-noise expert at start (mmap from the Sparsify models dir).
    2. At the timestep boundary, free it and page in the low-noise expert.
    3. Add tiled VAE decode (see issue 30) — video latents make activations the second memory cliff.
    4. Record swap latency and peak RSS; compare against loading both experts.

29. [Video] Wire a diffusers-based video engine (Wan 5B / LTX-Video / HunyuanVideo)

  • Difficulty: Hard
  • Recommended Labels: multimodal, enhancement
  • Files involved: src/sparsify/backends/image_backend.py, src/sparsify/runtime/server.py
  • Description: Video entries are cataloged pull-only today and the server returns an honest 501. Wire a PyTorch/diffusers engine (MPS on Apple Silicon, CUDA elsewhere) for one small model first — wan:2.1-1.3b or ltx:video — behind the existing engine: "diffusers" dispatch in get_image_backend().
  • Steps to implement:
    1. Implement DiffusersVideoBackend with load_model/generate_video/unload_model.
    2. Add a /v1/videos/generations endpoint returning MP4 bytes (base64) with duration/steps/peak-memory telemetry.
    3. Use enable_model_cpu_offload() as the baseline memory strategy; document measured peak RSS.

30. [Video] Tiled / temporal-chunked VAE decode to cap activation memory

  • Difficulty: Medium
  • Recommended Labels: multimodal, memory
  • Files involved: src/sparsify/backends/
  • Description: For video models, weights are only half the memory problem — decoding a 5-second 720p latent in one VAE pass allocates tens of GB of activations. Decode in spatial tiles and temporal chunks with overlap blending so activation memory stays bounded regardless of clip length.
  • Steps to implement:
    1. Split the latent into overlapping tiles/chunks sized by a memory budget.
    2. Decode sequentially, blend overlaps, stitch the output.
    3. Expose the budget as a config knob defaulting to a safe fraction of system RAM.

31. [Image] Wire SD 3.5 via DiffusionKit (MLX) behind the diffusionkit engine tag

  • Difficulty: Medium
  • Recommended Labels: multimodal, enhancement
  • Files involved: src/sparsify/backends/image_backend.py
  • Description: sd:3.5-medium / sd:3.5-large are cataloged with engine: "diffusionkit" but raise ImageEngineNotWired. Implement the backend with 4-bit/8-bit quantization options and register it in get_image_backend().
  • Steps to implement:
    1. Add a DiffusionKitImageBackend mirroring MFluxImageBackend's interface.
    2. Handle the gated-repo error path with the same actionable message pull prints.
    3. Add an optional dependency group image-sd in pyproject.toml.

32. [Image] Autoregressive image generation (Janus-Pro) through the LLM path

  • Difficulty: Hard
  • Recommended Labels: multimodal, research
  • Files involved: src/sparsify/backends/, src/sparsify/runtime/
  • Description: Janus-Pro generates images as discrete tokens through a normal LLM decoder — which means future MoE variants of this family would ride Sparsify's existing expert pager unchanged. Wire generation (prompt → image tokens → VQ decode → PNG) via transformers/MPS or an MLX port, and document which pieces the pager already covers.
  • Steps to implement:
    1. Implement token-based generation with the model's official generation recipe (CFG weight, parallel token count).
    2. Decode image tokens with the model's VQ detokenizer.
    3. Benchmark tokens/sec and peak memory; note expert-paging applicability in the PR.

33. [Web UI] Image generation tab with inline preview

  • Difficulty: Medium
  • Recommended Labels: webui, multimodal, enhancement
  • Files involved: site/, src/sparsify/runtime/webui.py
  • Description: The server now exposes POST /v1/images/generations (b64_json + telemetry). Add an "Imagine" view: prompt box, size/steps/seed controls, generated image rendered inline with a download button, and the measured duration_ms / peak_memory_gb shown under the result — memory honesty is the product.
  • Steps to implement:
    1. Add the tab and form; POST to the endpoint; render the base64 PNG.
    2. Show a step-progress placeholder while waiting (the request blocks until done).
    3. Display returned telemetry under the image, styled like chat telemetry.

34. [API] Progress streaming for long generations (step N of M over SSE)

  • Difficulty: Medium
  • Recommended Labels: api, multimodal, enhancement
  • Files involved: src/sparsify/runtime/server.py, src/sparsify/backends/image_backend.py
  • Description: A FLUX generation takes tens of seconds and a video generation minutes; the images endpoint currently blocks silently. Add "stream": true support that emits SSE events per denoising step ({"step": 7, "total": 25}), optionally with a low-res latent preview every K steps, then the final b64 payload.
  • Steps to implement:
    1. Add a per-step callback hook to the image backend.
    2. Emit SSE chunks in the images handler mirroring the chat streaming format.
    3. Consume the events in the Web UI progress bar (pairs with issue 33).

35. [CLI] sparsify imagine interactive picker + gallery folder

  • Difficulty: Easy (good first issue)
  • Recommended Labels: cli, multimodal, good first issue
  • Files involved: src/sparsify/cli.py
  • Description: sparsify imagine currently requires a model argument and writes sparsify-image.png in the CWD. With no arguments it should list pulled image models to pick from, default output into ~/.sparsify/gallery/<timestamp>-<slug>.png, and print the path.
  • Steps to implement:
    1. Reuse the _pick_model_interactively pattern filtered to modality != "text".
    2. Create the gallery dir on first use; slugify the first ~6 prompt words.
    3. Add --open to reveal the result in Finder/xdg-open.

36. [Core] Quantized on-disk cache for image models (skip re-quantizing every load)

  • Difficulty: Medium
  • Recommended Labels: multimodal, performance, storage
  • Files involved: src/sparsify/backends/image_backend.py, src/sparsify/runtime/model_registry.py
  • Description: mflux quantizes bf16 weights to 4-bit on every load — minutes of wasted work and a 33.7 GB read for FLUX each time. Save the quantized model to the models dir on first load (mflux supports saving), register it, and load the 4-bit artifact directly afterwards (~9 GB read). Also protects the SSD from repeated large reads, consistent with the project's SSD-wear stance.
  • Steps to implement:
    1. On first load, save the quantized pipeline under <models_dir>/<repo>--4bit/.
    2. Prefer the quantized artifact in load_model when present.
    3. Add sparsify inspect awareness so both artifacts show with sizes.

37. [Docs] "Generative models beyond text" architecture guide

  • Difficulty: Easy (good first issue)
  • Recommended Labels: documentation, multimodal
  • Files involved: docs/
  • Description: Document why the LLM expert pager doesn't apply to dense DiTs, what does apply (stage residency, quantize-at-load, timestep-expert paging, tiled VAE), the honest support matrix (wired / pull-only / planned), and how to add a new engine behind get_image_backend().
  • Steps to implement:
    1. Write docs/multimodal.md covering the mechanisms above with the catalog table.
    2. Link it from README and the models table footer.