v0.7.0: speaker consolidation + no_repeat_ngram_size + E2E tests

[MapleEve]

Summary

• Bug fix: speaker cluster consolidation — multiple diarization clusters matching the same enrolled speaker are now merged to a single canonical label before segment assembly
• Feature: no_repeat_ngram_size API parameter (default 0/off; effective when ≥ 3; suppresses n-gram repetitions in Whisper output)
• Tests: 43 new E2E tests (84 total, 78 pass, 6 expected skips)
• Docs: MIT → Apache 2.0 badge/text fix; no_repeat_ngram_size added to API reference; v0.7.0 changelog; version aligned to 0.7.0

Changes

Bug fix

job_service.py: after building speaker_map, groups clusters by matched_id and remaps non-canonical labels to the highest-similarity match for the same enrolled speaker. Affects segment assembly and unique_speakers.

Feature — no_repeat_ngram_size

• POST /api/transcribe gains optional no_repeat_ngram_size: int (default 0)
• Values < 3 stored as 0 (no-op); values ≥ 3 forwarded to faster-whisper
• Non-integer form values return HTTP 422 (explicit int() validation added)
• Recorded in params object of completed job results

Deployment

docker-compose.yml adds ./app:/app bind mount — code changes deploy via rsync without image rebuild.

Test plan

• python -m pytest tests/e2e/test_api_core.py → 78 passed, 6 skipped, 0 failed
• Path traversal: server returns 404/422 (correct — client resolves ../ before sending)
• no_repeat_ngram_size=banana → 422
• Voiceprint enroll → list → GET → delete chain passes
• Export formats (txt/srt/vtt/json) pass

[github-actions]

👍 @MapleEve

Thank you for raising your pull request and contributing to voscript.
Please make sure you have followed our contributing guidelines. We will review it as soon as possible.
If you encounter any problems, please feel free to connect with us.
非常感谢您提出拉取请求并为 voscript 做出贡献，请确保您已经遵循了我们的贡献指南，我们会尽快审查它。
如果您遇到任何问题，请随时与我们联系。

[Copilot]

Pull request overview

This PR bumps the project to v0.7.0 and extends the transcription service with speaker-cluster consolidation, a no_repeat_ngram_size request parameter (forwarded to faster-whisper), and a significantly expanded test/documentation set alongside internal service/router refactors.

Changes:

• Add speaker consolidation during segment assembly and persist no_repeat_ngram_size in job params.
• Refactor the FastAPI app into config/, api/routers/, and services/ with job status persistence and new security hardening helpers.
• Add unit/security tests plus a large E2E suite, and update docs/CI/configuration accordingly.

Reviewed changes

Copilot reviewed 33 out of 37 changed files in this pull request and generated 10 comments.

Show a summary per file

File	Description
tests/test_voiceprint_db.py	New VoiceprintDB core/adaptive-threshold/AS-norm regression tests.
tests/test_security.py	New security regression tests (auth headers, traversal, allow_pickle=False).
tests/test_job_service.py	New tests for job persistence, orphan recovery, and LRU behavior.
tests/e2e/test_api_core.py	New comprehensive E2E API contract tests (live-server based).
tests/e2e/bench_overlap.py	New bench script to compute overlap stats from diarization segments.
tests/conftest.py	Prefer real fastapi/voiceprint_db when available; improve TestClient fixture isolation.
docker-compose.yml	Add container healthcheck for /healthz.
doc/security.zh.md	Update hardening list and deployment guidance (ZH).
doc/security.en.md	Update hardening list and deployment guidance (EN).
doc/quickstart.zh.md	Document new env vars (ZH).
doc/quickstart.en.md	Document new env vars (EN).
doc/changelog.zh.md	Add v0.7.0 changelog entry (ZH).
doc/changelog.en.md	Add v0.7.0 changelog entry (EN).
doc/benchmarks.zh.md	Note OSD deprecation context (ZH).
doc/api.zh.md	Document new API params/behavior and expanded notes (ZH).
doc/api.en.md	Document new API params/behavior and expanded notes (EN).
doc/ai-usage.zh.md	Update AI usage guidance for adaptive thresholds and cohort lifecycle (ZH).
app/voiceprint_db.py	Add name-dedup enrollment, zero-vector defense, vec serialization fix, cohort build robustness, BLAS cosine scan.
app/static/index.html	Add CSP, move key storage to sessionStorage/memory, replace inline handlers with delegation, improve polling/network handling.
app/services/job_service.py	New job worker + status persistence + LRU job registry + speaker consolidation + periodic cohort rebuild.
app/services/audio_service.py	New ffmpeg/denoise helpers + hash index + traversal/log-filename sanitizers.
app/services/init.py	Package marker for services module.
app/pipeline.py	Add no_repeat_ngram_size forwarding and segment-level audio loading for embeddings.
app/main.py	Refactor into lifespan startup, middleware, and router composition; version set to 0.7.0.
app/config.py	Centralize environment-variable configuration.
app/api/routers/voiceprints.py	New voiceprint CRUD/enroll/rebuild endpoints with validation + allow_pickle=False.
app/api/routers/transcriptions.py	New transcription/job/export endpoints + new param validation + persistence fallback.
app/api/routers/health.py	New /healthz and / (static UI) routes.
app/api/routers/init.py	Package marker for routers module.
app/api/deps.py	Shared deps: API key verification + access to app.state db/pipeline.
app/api/init.py	Package marker for api module.
app/Dockerfile	Add notes about pinning base image digest; remove redundant chown step.
README.md	Update badges/license text and reorganize sections (ZH/primary).
README.en.md	Update badges/license text and reorganize sections (EN).
CONTRIBUTING.md	Add contribution guidelines and dev/test commands.
.gitignore	Ignore review artifact directories.
.github/workflows/ci.yml	Add lint/test/security-scan workflow.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Unused import: JSONResponse is imported but never referenced. This will fail the CI ruff check app/ step (F401). Remove it or use it.

Suggested change

from fastapi.responses import JSONResponse

[Copilot]

This script hard-codes a user-specific default BENCH_DIR under /Users/..., and also defaults to an external VOSCRIPT_URL. For a repo utility script, it’s more robust to default to a path relative to the repo (or require BENCH_DIR/VOSCRIPT_URL to be explicitly set and otherwise exit with a clear message).

[Copilot]

PR description mentions adding an ./app:/app bind mount for rsync-based deploys, but this change only adds a healthcheck. Either add the bind mount here or update the PR description/deployment notes to match what’s actually shipped.

[Copilot]

The contributor docs recommend running pytest tests/ -v, but tests/e2e/ will be collected by default and currently attempts to reach a live server (with external defaults). Either adjust the recommended command to exclude E2E (e.g. pytest tests/test_*.py) or document how to opt into E2E explicitly (markers/env var).

[Copilot]

safe_tr_dir() uses a string startswith() check on resolved paths. This can be bypassed in edge cases (e.g., a resolved path under /data/transcriptions_evil still starts with /data/transcriptions). Prefer path.is_relative_to(TRANSCRIPTIONS_DIR.resolve()) (Py3.9+) or compare os.path.commonpath / add a path separator to the prefix check.

Suggested change

	path = (TRANSCRIPTIONS_DIR / tr_id).resolve()
	if not str(path).startswith(str(TRANSCRIPTIONS_DIR.resolve())):
	base_dir = TRANSCRIPTIONS_DIR.resolve()
	path = (base_dir / tr_id).resolve()
	if not path.is_relative_to(base_dir):

[Copilot]

Unused imports: Depends and Path are imported but never used. This will fail the CI ruff check app/ step (F401). Remove them or use them.

Suggested change

	from pathlib import Path, PurePosixPath
	from threading import Thread
	from typing import Annotated
	from fastapi import APIRouter, Depends, File, Form, HTTPException
	from pathlib import PurePosixPath
	from threading import Thread
	from typing import Annotated
	from fastapi import APIRouter, File, Form, HTTPException

[Copilot]

Unused import: Depends is imported but never used. This will fail the CI ruff check app/ step (F401). Remove it or apply Depends(...) where intended.

Suggested change

	from fastapi import APIRouter, Depends, Form, HTTPException, Request
	from fastapi import APIRouter, Form, HTTPException, Request

[Copilot]

These E2E tests default to an external server (https://nas.esazx.com:8780) and a default API key. Since pytest tests/ will collect tests/e2e/* by default, contributors (and some CI environments) may unintentionally make outbound requests. Consider requiring an explicit opt-in (e.g., skip unless VOSCRIPT_URL is set, or unless RUN_E2E=1, and default to localhost / no default key).

[Copilot]

pip-audit ... || true forces this step to succeed even when vulnerabilities are found, which makes the security-scan job non-enforcing and can hide regressions. If the intent is non-blocking reporting, consider using GitHub Actions’ continue-on-error: true (keeps the failure visible) or uploading the audit output as an artifact; otherwise drop || true to gate on new vulnerabilities.

Suggested change

	run: pip-audit -r requirements.txt --ignore-vuln PYSEC-2022-42969 || true
	# || true: 不阻断构建，但在 Summary 中显示漏洞报告
	continue-on-error: true
	run: pip-audit -r requirements.txt --ignore-vuln PYSEC-2022-42969
	# 非阻断执行，但保留步骤失败状态以便在 Actions 中可见

[Copilot]

_LRUJobsDict can evict an in-progress job while its background thread is still updating jobs[job_id]["status"]. Because __getitem__/__contains__ don’t update recency (and status updates mutate the stored dict without calling __setitem__), a long-running/older job can be popped and the worker will start raising KeyError. Consider making reads move the key to the end (true LRU on access) and/or ensuring active jobs can’t be evicted (e.g., touch via __setitem__ on each update or keep non-terminal jobs in an unbounded map).

Suggested change

	return self._d[key]
	def __contains__(self, key):
	with self._lock:
	return key in self._d
	def get(self, key, default=None):
	with self._lock:
	return self._d.get(key, default)
	value = self._d[key]
	self._d.move_to_end(key)
	return value
	def __contains__(self, key):
	with self._lock:
	if key in self._d:
	self._d.move_to_end(key)
	return True
	return False
	def get(self, key, default=None):
	with self._lock:
	if key in self._d:
	self._d.move_to_end(key)
	return self._d[key]
	return default

[github-actions]

❤️ Great PR @MapleEve ❤️

The growth of project is inseparable from user feedback and contribution, thanks for your contribution!
项目的成长离不开用户反馈和贡献，感谢您的贡献!