v0.7.0: speaker consolidation + no_repeat_ngram_size + E2E tests

.github/workflows/ci.yml

@@ -0,0 +1,48 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+    branches: [main]
+
+jobs:
+  lint:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+          cache: pip
+      - run: pip install ruff
+      - run: ruff check app/ --ignore E501
+      - run: ruff format --check app/
+
+  test:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+          cache: pip
+      - name: Install lightweight test dependencies (no GPU packages)
+        run: pip install pytest pytest-cov fastapi httpx numpy aiofiles starlette python-multipart
+      - name: Run tests
+        run: |
+          pytest tests/test_security.py tests/test_voiceprint_db.py tests/test_job_service.py \
+            -v --tb=short --no-header
+
+  security-scan:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+      - name: Install pip-audit
+        run: pip install pip-audit
+      - name: Run pip-audit
+        run: pip-audit -r requirements.txt --ignore-vuln PYSEC-2022-42969 || true
+        # || true: 不阻断构建，但在 Summary 中显示漏洞报告

.github/workflows/release.yml

@@ -3,6 +3,9 @@ name: Build and publish Docker image
 on:
   release:
     types: [published]
+  push:
+    tags:
+      - 'v*'
   workflow_dispatch:
 
 jobs:
@@ -26,14 +29,24 @@ jobs:
           username: ${{ github.actor }}
           password: ${{ secrets.GITHUB_TOKEN }}
 
+      - name: Log in to Docker Hub
+        uses: docker/login-action@v3
+        with:
+          username: ${{ secrets.DOCKERHUB_USERNAME }}
+          password: ${{ secrets.DOCKERHUB_TOKEN }}
+
       - name: Compute image tags
         id: tags
         run: |
-          IMAGE="ghcr.io/${{ github.repository }}"
-          TAGS="$IMAGE:latest"
-          if [ "${{ github.event_name }}" = "release" ]; then
-            VERSION="${GITHUB_REF#refs/tags/}"
-            TAGS="$TAGS,$IMAGE:$VERSION"
+          GHCR_IMAGE="ghcr.io/${{ github.repository }}"
+          DOCKERHUB_IMAGE="${{ secrets.DOCKERHUB_USERNAME }}/voscript"
+          TAGS="$GHCR_IMAGE:latest,$DOCKERHUB_IMAGE:latest"
+          # Tag-triggered builds (release or push-tag) get the version tag too.
+          # Strip leading "v" for Docker Hub convention (0.7.0), keep raw ref for GHCR.
+          if [ "${{ github.event_name }}" = "release" ] || [ "${{ github.event_name }}" = "push" ]; then
+            RAW_VERSION="${GITHUB_REF#refs/tags/}"
+            STRIPPED_VERSION="${RAW_VERSION#v}"
+            TAGS="$TAGS,$GHCR_IMAGE:$RAW_VERSION,$DOCKERHUB_IMAGE:$STRIPPED_VERSION"
           fi
           echo "tags=$TAGS" >> "$GITHUB_OUTPUT"
 

.gitignore

@@ -32,3 +32,7 @@ logs/
 node_modules/
 tmp/
 .claude/
+
+# Code review intermediate artifacts
+.full-review/
+.full-review-archive/

CONTRIBUTING.md

@@ -0,0 +1,89 @@
+# 贡献指南
+
+感谢你考虑向 voscript 贡献！
+
+## 开始之前
+
+- 提 issue 之前先搜一下是否已有相同问题
+- 大的改动建议先开 issue 讨论设计方向
+
+## 开发环境
+
+```bash
+# 安装依赖（需要 Python 3.11+）
+pip install fastapi uvicorn pytest pytest-asyncio aiofiles httpx starlette python-multipart
+
+# 运行测试
+pytest tests/ -v
+
+# 启动本地服务（需要 GPU + 模型文件）
+cd app && uvicorn main:app --reload --port 8780
+```
+
+## 提 PR 须知
+
+1. **Fork → 新建分支 → PR**，分支命名建议：`feat/xxx`、`fix/xxx`、`docs/xxx`
+2. **测试**：新功能必须附带对应测试（`tests/` 目录），所有测试必须通过
+3. **文档**：影响 API 或行为的改动需同步更新 `doc/` 中的相关文档
+4. **提交信息**：用英文小写动词开头，例如 `fix: handle zero-length audio` / `feat: add speaker rename API`
+
+## 代码风格
+
+- Python：PEP 8，函数命名 `snake_case`，类型注解尽量完整
+- 不要引入 print 调试语句，用 `logger.debug()`
+- 安全敏感代码（鉴权、文件路径）改动务必在 PR 描述中说明
+
+## 报告安全问题
+
+**不要开公开 issue**。请直接发 email 或通过 GitHub Security Advisory 私下报告。
+详见 [SECURITY.md](./SECURITY.md)。
+
+## 行为准则
+
+友善、建设性地交流。维护者有权关闭不遵守基本礼仪的讨论。
+
+---
+
+# Contributing (English)
+
+Thank you for contributing to voscript!
+
+## Before You Start
+
+- Search existing issues before filing a new one
+- For large changes, open an issue first to discuss design direction
+
+## Development Setup
+
+```bash
+# Requires Python 3.11+
+pip install fastapi uvicorn pytest pytest-asyncio aiofiles httpx starlette python-multipart
+
+# Run tests
+pytest tests/ -v
+
+# Start local server (requires GPU + model files)
+cd app && uvicorn main:app --reload --port 8780
+```
+
+## Pull Request Guidelines
+
+1. **Fork → branch → PR** — suggested branch names: `feat/xxx`, `fix/xxx`, `docs/xxx`
+2. **Tests**: new features must include tests; all tests must pass
+3. **Docs**: changes to API behavior must update the relevant files in `doc/`
+4. **Commit messages**: lowercase verb prefix, e.g. `fix: handle zero-length audio`
+
+## Code Style
+
+- Python: PEP 8, `snake_case` for functions, type annotations where practical
+- Use `logger.debug()` instead of `print`
+- Security-sensitive changes (auth, file paths) must be explained in the PR description
+
+## Reporting Security Issues
+
+**Do not open a public issue.** Use GitHub Security Advisory or email privately.
+See [SECURITY.md](./SECURITY.md).
+
+## Code of Conduct
+
+Be kind and constructive. Maintainers may close discussions that violate basic courtesy norms.

README.en.md

@@ -1,93 +1,92 @@
-# VoScript
+<div align="center">
+
+# 🎙️ VoScript
 
 [简体中文](./README.md) | **English**
 
-Self-hosted GPU transcription service — **meeting transcripts that remember
-their speakers.** A small HTTP API that turns audio into timestamped text
-labelled with speaker names, and that auto-recognizes returning voices
-across recordings.
+<a href="https://github.com/MapleEve/voscript/actions/workflows/ci.yml">
+  <img src="https://img.shields.io/github/actions/workflow/status/MapleEve/voscript/ci.yml?branch=main&style=for-the-badge" alt="CI" />
+</a>
+<img src="https://img.shields.io/badge/License-Apache%202.0-blue.svg?style=for-the-badge" alt="License: Apache 2.0" />
+<img src="https://img.shields.io/badge/Docker-ready-blue?style=for-the-badge&logo=docker" alt="Docker ready" />
 
-```
-Audio  ──►  faster-whisper large-v3              (transcription)
-        ──►  pyannote 3.1                         (speaker diarization)
-        ──►  DeepFilterNet / noisereduce (optional denoising)
-        ──►  WeSpeaker ResNet34                   (speaker embeddings)
-        ──►  VoiceprintDB                         (cosine match vs. enrolled speakers)
-        ──►  timestamped text with identified speaker names
-```
+**Meeting recordings → transcripts with real speaker names. Self-hosted, GPU-powered, remembers every voice.**
 
-What sets it apart from a plain whisper wrapper: the **persistent
-voiceprint library**. Enroll a speaker once, and from then on every
-recording they appear in gets their real name automatically.
+[Quickstart](./doc/quickstart.en.md) · [API Reference](./doc/api.en.md) · [Security](./doc/security.en.md) · [Benchmarks](./doc/benchmarks.en.md) · [Changelog](./doc/changelog.en.md)
 
-> Example consumer: [OpenPlaud(Maple)](https://github.com/MapleEve/openplaud)
-> uses voscript as the backend for meeting recordings. voscript itself is
-> just an HTTP service — any client that can POST multipart audio works.
+</div>
 
-## Documentation
+---
 
-All detailed docs live in [`doc/`](./doc/). Chinese is the default, every
-page has an English counterpart:
+You have a meeting recording with six people. You want to know who said what. Whisper gives you a wall of text. pyannote can split it into "Speaker A / Speaker B / Speaker C" — but it doesn't know who anyone is. You still have to label every recording by hand.
 
-| Topic | 中文 | English |
-| --- | --- | --- |
-| Quickstart | [quickstart.zh.md](./doc/quickstart.zh.md) | [quickstart.en.md](./doc/quickstart.en.md) |
-| API reference | [api.zh.md](./doc/api.zh.md) | [api.en.md](./doc/api.en.md) |
-| **Install guide for AI agents** | [ai-install.zh.md](./doc/ai-install.zh.md) | [ai-install.en.md](./doc/ai-install.en.md) |
-| **Usage guide for AI agents** | [ai-usage.zh.md](./doc/ai-usage.zh.md) | [ai-usage.en.md](./doc/ai-usage.en.md) |
-| Security policy | [security.zh.md](./doc/security.zh.md) | [security.en.md](./doc/security.en.md) |
-| Benchmarks (real-audio wall clock + resource usage) | [benchmarks.zh.md](./doc/benchmarks.zh.md) | [benchmarks.en.md](./doc/benchmarks.en.md) |
-| Changelog | [changelog.zh.md](./doc/changelog.zh.md) | [changelog.en.md](./doc/changelog.en.md) |
-
-First-time deployers: start with the [Quickstart](./doc/quickstart.en.md).
-AI agents integrating the API: read the [AI usage guide](./doc/ai-usage.en.md).
-AI agents deploying the service for a user: read the
-[AI install guide](./doc/ai-install.en.md).
-
-## Features
+VoScript fixes that: **enroll a voice once, and it gets automatically identified in every future recording**. Not "Speaker 2" — "Maple".
 
-- **Async job pipeline**: `queued → converting → denoising (optional) → transcribing → identifying → completed`
-- **Chinese + multilingual transcription** (WhisperX + faster-whisper large-v3, **word-level timestamps** via forced alignment; omit `language` to auto-detect — Mandarin audio outputs Simplified Chinese)
-- **Speaker diarization** (pyannote 3.1) + **WeSpeaker ResNet34** embeddings
-- **Adaptive voiceprint threshold**: `VOICEPRINT_THRESHOLD` (default 0.75) is the base; the actual threshold relaxes per-speaker based on intra-cluster std of enrolled embeddings — fixed −0.05 for 1 sample, `min(3×std, 0.10)` for 2+, floor at 0.60. Lifted recall from 50% to 70% on 10 real recordings with zero false positives
-- **Optional denoising with SNR gate**: `DENOISE_MODEL` (`none` | `deepfilternet` | `noisereduce`); `DENOISE_SNR_THRESHOLD` (default 10.0 dB) — audio above this SNR is considered clean and skipped automatically, preventing DeepFilterNet from degrading already-clean recordings
-- **AS-norm voiceprint scoring**: at startup, automatically builds an impostor cohort from existing transcription embeddings and applies Adaptive Score Normalization — eliminates speaker-dependent baseline bias, ~15–30% relative EER improvement
-- **Persistent voiceprints**: enroll once, auto-match across future recordings. sqlite + sqlite-vec under the hood — top-k nearest-neighbour search scales to thousands of speakers
-- **File hash deduplication**: submitting the same file twice returns the existing result immediately, skipping Whisper GPU inference
-- **Stable HTTP contract**: `/api/transcribe`, `/api/jobs/{id}`, `/api/voiceprints*`, etc. — any HTTP client works
-- **Container runs as non-root**; all `/api/*` routes accept optional Bearer / `X-API-Key` auth (constant-time compare); uploads capped by `MAX_UPLOAD_BYTES`; voiceprint DB is concurrency-safe with atomic writes — full hardening list in [`doc/security.en.md`](./doc/security.en.md)
-- Minimal built-in web UI at `/` for manual testing
+```
+Audio  ──►  faster-whisper large-v3     transcription + word-level timestamps
+       ──►  pyannote 3.1                speaker diarization
+       ──►  WeSpeaker ResNet34           speaker embeddings
+       ──►  VoiceprintDB (AS-norm)       match against enrolled voices
+       ──►  timestamped transcript with real speaker names
+```
 
 ## 30-second start
 
-```bash
-git clone https://github.com/MapleEve/voscript.git
-cd voscript
-
-cp .env.example .env
-# edit .env — at minimum set HF_TOKEN and API_KEY
+> **Security**: set a strong `API_KEY` in `.env` before exposing this on any network. Without it, anyone can delete your voiceprint library or trigger GPU jobs.
 
+```bash
+git clone https://github.com/MapleEve/voscript.git && cd voscript
+cp .env.example .env        # at minimum: HF_TOKEN and API_KEY
 docker compose up -d --build
 curl -sf http://localhost:8780/healthz
 ```
 
-Full steps + troubleshooting in [`doc/quickstart.en.md`](./doc/quickstart.en.md).
+Full setup + troubleshooting → [`doc/quickstart.en.md`](./doc/quickstart.en.md)
+
+## Features
+
+- **Persistent voiceprint library** — enroll once, auto-match across all future recordings. sqlite + sqlite-vec under the hood, top-k nearest-neighbour search, scales to thousands of speakers
+- **AS-norm scoring** — builds an impostor cohort from existing transcription embeddings at startup; eliminates speaker-dependent baseline bias, ~15–30% relative EER improvement
+- **Adaptive threshold** — each speaker's match threshold relaxes dynamically based on enrollment variance; lifted recall from 50% to 70% on 10 real recordings with zero false positives
+- **Speaker cluster consolidation** — when diarization splits one person into multiple clusters, they're automatically merged to a single label
+- **Word-level timestamps** — WhisperX forced alignment, every word precisely timed
+- **Optional denoising with SNR gate** — DeepFilterNet / noisereduce; audio above the SNR threshold is treated as clean and skipped automatically (prevents degrading already-clean recordings)
+- **File hash deduplication** — submitting the same file twice returns the existing result immediately, no GPU re-run
+- **Job persistence** — completed transcriptions remain accessible after restart
+- **Ngram dedup** — `no_repeat_ngram_size` parameter suppresses repetitive filler words in the transcript
+- **Plain HTTP contract** — any client that can send multipart/form-data works, no framework lock-in
+
+Security: path traversal protection, non-root container, upload size cap, constant-time auth, atomic writes — full list in [`doc/security.en.md`](./doc/security.en.md)
+
+## Integration
+
+It's a plain HTTP service. Two config values and you're done:
+
+- **Transcription base URL**: `http://<host>:8780`
+- **API key**: the `API_KEY` you set in `.env`
+
+[BetterAINote](https://github.com/MapleEve/openplaud) connects this way. Any other client works the same. Full API contract → [`doc/api.en.md`](./doc/api.en.md)
 
-## How to integrate
+## Documentation
+
+| Topic | 中文 | English |
+| --- | --- | --- |
+| Quickstart | [quickstart.zh.md](./doc/quickstart.zh.md) | [quickstart.en.md](./doc/quickstart.en.md) |
+| API reference | [api.zh.md](./doc/api.zh.md) | [api.en.md](./doc/api.en.md) |
+| Install guide for AI agents | [ai-install.zh.md](./doc/ai-install.zh.md) | [ai-install.en.md](./doc/ai-install.en.md) |
+| Usage guide for AI agents | [ai-usage.zh.md](./doc/ai-usage.zh.md) | [ai-usage.en.md](./doc/ai-usage.en.md) |
+| Security policy | [security.zh.md](./doc/security.zh.md) | [security.en.md](./doc/security.en.md) |
+| Benchmarks | [benchmarks.zh.md](./doc/benchmarks.zh.md) | [benchmarks.en.md](./doc/benchmarks.en.md) |
+| Changelog | [changelog.zh.md](./doc/changelog.zh.md) | [changelog.en.md](./doc/changelog.en.md) |
 
-voscript is a plain HTTP service — no specific client is required. Anything
-that can send `multipart/form-data` works (curl, axios, requests, browser
-uploads, …).
+## Contributing
 
-A typical integration — OpenPlaud(Maple), under Settings → Transcription:
+PRs welcome — read [CONTRIBUTING.md](./CONTRIBUTING.md) first.
 
-- **Private transcription base URL**: `http://<host>:8780`
-- **Private transcription API key**: the same `API_KEY` as in `.env`
+## Star History
 
-After that its worker routes every recording through this service. If
-you're writing your own client, the full contract + error table lives in
-[`doc/api.en.md`](./doc/api.en.md).
+[![Star History Chart](https://api.star-history.com/svg?repos=MapleEve/voscript&type=date)](https://www.star-history.com/#MapleEve/voscript&type=date)
 
 ## License
 
-MIT — see [LICENSE](./LICENSE).
+Apache 2.0 — [LICENSE](./LICENSE)