Web demo for HFI-UNet, a Hessian-guided U-Net that segments nuclei in H&E histology images. Pick a sample or upload your own patch, watch the 9-channel input tensor get built live, and inspect the watershed-based instance map.
Live demo · HFI-UNet · Report an issue
Kakera is the public demo for HFI-UNet, a research project on nucleus instance segmentation in H&E-stained histology images. The model itself is a plain U-Net with a single binary semantic head trained with Dice loss. What makes it different is the input: instead of the usual 3-channel RGB, the network sees a 9-channel tensor. Three channels are the raw H&E image. The remaining six come from a Hessian-guided feature builder that runs before the forward pass:
| Index | Name | What it captures |
|---|---|---|
| 3 | Grayscale | Inverted blue-weighted grayscale, so nuclei appear bright |
| 4 | Local Maxima | Binary map of nucleus centroid candidates |
| 5 | Saddle Points | Scale-invariant saddle strength, used as a touching-boundary cue |
| 6 | det(H) Character | Blob vs saddle topology, pivoted at 0.5 |
| 7 | SDF Approximation | Normalized mean curvature, blob interior → 1 |
| 8 | Eigenvalue Ratio | λ₂ / λ₁, a blobiness measure |
These channels are built scale-invariantly so the same physical curvature gives the same channel value across patches. The goal is to hand the network the geometric cues it would otherwise need to learn from raw pixels.
The Space is built around one configuration of HFI-UNet (baseline5_rgb_plus_hessian from the experiment matrix). It exists so the math is not stuck behind a paragraph in a paper. You can see the channels, flip them off one by one, and watch the segmentation degrade in real time.
- 9-channel input visualization. After preprocessing, every channel is rendered as its own panel so you can see what the model actually receives, not just an abstract description of it.
- Three model outputs. The probability heatmap, the binary mask at threshold 0.5, and the watershed-based instance map appear side by side.
- Overlay with a live opacity slider. The contour layer is sent as a separate transparent PNG, so the slider re-composites the canvas client-side without re-running inference.
- Sample gallery. Ten CryoNuSeg organ patches ship with the app: adrenal gland, larynx, lymph nodes, mediastinum, pancreas, pleura, skin, testes, thymus, thyroid gland. You can also drop in your own H&E patch (TIFF, PNG, or JPEG, up to 10 MB).
- Channel ablation. Nine checkboxes zero out the corresponding channel before the forward pass. This is the most useful feature for understanding what the geometric channels actually do. Disabling Saddle Points, for example, removes the touching-boundary cue, and you can watch instance merges appear in the output.
The Channels view exposes all nine input planes after preprocessing. Disabled channels are dimmed and tagged.
The Segmentation view stacks the overlay canvas above three thumbnail panels (probability, binary mask, instances). The slider on the right re-blends the contour layer client-side.
- Backend. FastAPI on Uvicorn, Python 3.11. CPU-bound inference is offloaded to a threadpool so the event loop stays responsive while a forward pass runs.
- Model. PyTorch CPU build.
UNetBaseis vendored verbatim from the HFI-UNet research repo intohfi_unet/. The trained checkpoint lives atlumicero/Kakera-modelon the Hub and is downloaded at startup withhuggingface_hub.hf_hub_download. The state dict is loaded withstrict=Trueafter stripping themodel.prefix that the training-time wrapper added. - Image and feature math. NumPy, SciPy, scikit-image, OpenCV (headless). No matplotlib, no Pillow, no staintools. Stain normalization uses a per-channel histogram match against a fixed reference image.
- Frontend. Vanilla HTML, CSS, and one ES2020 module. No bundler, no framework, no client-side dependency.
- Container. A single-stage Dockerfile based on
python:3.11-slim. CPU only. The app listens on port 7860 to satisfy the Hugging Face Spaces health probe. - Auto-deploy. Every push to
mainon GitHub force-pushes the working tree to the Space via.github/workflows/hf-space-sync.yml. The workflow needs a singleHF_TOKENsecret with write access to the Space.
.
├── app/ FastAPI app and inference engine
│ ├── main.py Routes, startup hook, threadpool offload
│ ├── inference.py Pipeline orchestration (decode → tile → run → render)
│ ├── postprocess.py Distance-transform watershed wrapper
│ ├── samples.py Sample manifest + thumbnail cache
│ ├── tiling.py 512 to 4×256 tile and stitch
│ └── visualize.py PNG renderers (palette, channel coloring, overlay)
├── hfi_unet/ Vendored subset of the HFI-UNet research repo
│ ├── data/preprocessing.py
│ ├── features/ channel_builder, eigenvalues, hessian, morse_points, sdf_approximation
│ ├── inference/watershed_postprocess.py
│ └── models/ blocks.py, unet_base.py
├── samples/ 10 CryoNuSeg organ tiles + Macenko reference + manifest.json
├── static/ index.html, style.css, app.js
├── .github/workflows/ hf-space-sync.yml
├── Dockerfile
├── requirements.txt
└── README.md This file. Also serves as the HF Space card.
The hfi_unet/ directory mirrors the research repo's src/ so the demo's import paths match the training paths. The only edit applied during vendoring is a global from src. → from hfi_unet. rewrite.
The image at samples/reference.tif is the Macenko reference image (the first sorted training image from the CryoNuSeg split). It is loaded once at startup and reused by every stain normalization call.
- The CryoNuSeg dataset by Mahbod, Schaefer, Bancher, Löw, Dorffner, Ecker, and Ellinger (2021). The 10 organ patches and the Macenko reference image are drawn from this dataset and inherit its CC BY-NC-SA 4.0 license.
- The HFI-UNet research code that defines the U-Net, the six Hessian-derived feature channels, and the watershed post-processing path. The files in
hfi_unet/are copied directly from that work; the demo only adds glue. - Hugging Face for hosting the model checkpoint and the Space.
The demo code in this repository is released under the MIT License.
The model checkpoint at lumicero/Kakera-model and the sample images derived from CryoNuSeg inherit the CC BY-NC-SA 4.0 license of the upstream dataset. Research and educational use is fine. For commercial use, contact the dataset and model authors first.