Skip to content

QuyDu/Xavier

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

218 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

XAVIER

v5.86 | Released: July 3, 2026 — Face enrollment hardening + Easy Auth v2 fixes + deployment lock-in

Azure AI Services platform for camera-driven threat intelligence: weapon detection, school lockdown orchestration, crimes-against-children case packaging (with NCMEC CyberTipline staging XML), FBI law-enforcement workflows, and maritime deep-search. Built on Computer Vision, Face API, Speech Services, Translator, Content Safety, Document Intelligence, Azure AI Search (RAG), and Azure OpenAI (gpt-5.1).


What's live (v5.46)

  • CSRF token persistence (v5.44): WTF_CSRF_TIME_LIMIT=None — tokens are still bound to session/SECRET_KEY but no longer expire after 1 hour. Long-running admin pages (FBI sync, batch face processing, camera dashboards) no longer fail with bad_request after a tab sits open.
  • Hardened deploy retry (v5.44): infra/deploy-app.ps1 now polls Kudu for completion before any retry and stops the site + wipes the corrupted Oryx tarball before retrying. Eliminates concurrent extraction races that previously crashed the container with panic: An error occurred when trying to extract tarball.
  • Role-aware route gating (v5.43): any authenticated user can view cameras + run face identification; only admins can add/edit/delete cameras, NVRs, persons, or trigger training.
  • Ozolio JPEG fast-path (v5.35): webcam aggregator pages backed by the Ozolio JS player are auto-resolved to the underlying relay.ozolio.com/pub.api?cmd=snap&oid=<EMBED_ID> JPEG snapshot endpoint — no headless browser, no HLS stack, just image/jpeg.
  • public_stream Enhance/Compare (v5.35): the AI Enhance / Compare "Camera" dropdown uses the unified CameraService.get_snapshot helper, so any public_stream camera (Ozolio, direct HLS/MJPEG/JPEG) can be picked as the source image.
  • Public Stream Cameras (v5.34): add any open webcam URL (HLS .m3u8, MJPEG, RTSP, or direct JPEG/PNG snapshot) as a camera — no credentials required. Full capture / process / enhance and auto-detect (objects, faces, weapons, FBI match) work the same as private RTSP cameras.
  • School Safety: multi-camera sweep + auto-lockdown + persistent background watcher with self-healing watchdog + case redeliver.
  • Crimes Against Children: CSAM-gated case packaging, evidence vault bundles, NCMEC Report.xml staging artifact, opt-in NCMEC Member API submit, case redeliver.
  • Maritime Deep-Search: multi-step pipeline, signed escalation webhook, persistent maritime auto-watch, report redeliver.
  • FBI Federal Submission: signed federal-format webhook, person sync, dossier export, redeliver.
  • Cross-scenario Timeline: aggregated school + CAC + maritime + FBI + cost events.
  • Watcher Settings UI: Settings → Operations tab — live status + Start/Stop/Acknowledge buttons for both watchers (admins only).
  • Hardening: in-process per-principal token-bucket rate limiter on scan endpoints (6/min × burst 8); HTTP 429 + retry_after_sec.
  • CI: GitHub Actions runs the full pytest suite on every push/PR (.github/workflows/test.yml).
  • Demo seed: infra/seed-demo.ps1 produces five fixed-id Cosmos demo documents gated by DEMO_SEED_ENABLED.

Deployment target

Provisions a self-contained stack in any Azure or Azure Government subscription. Resource names are derived from -SiteName plus a deterministic per-deployment hash, so each install is isolated.

Value
Cloud AzureCloud or AzureUSGovernment
Subscription Whichever subscription az login is currently scoped to
Resource Group rg-<SiteName>
Region Default eastus (Commercial) / usgovarizona (Gov); override with -Location
Web App app-<SiteName>-<hash>
URL https://app-<SiteName>-<hash>.azurewebsites.{net,us}
Key Vault kv-<SiteName>-<hash>
Cosmos DB cosmos-<SiteName>-<hash> (partition /file_type)
Azure OpenAI oai-<SiteName>-<hash> — model auto-selected by infra/discover.ps1 from regional availability (preference: gpt-5.1gpt-4.1gpt-4ogpt-35-turbo) plus text-embedding-3-large
Gunicorn --workers=1 --threads=8 --worker-class=gthread --timeout=300 (required — many in-memory singletons)

Deploying

# Full deploy (~10-20 min, 3x auto-retry — Oryx CDN flakes attempt 1-2 normally)
pwsh -NoProfile -File infra/deploy.ps1 -SiteName <your-site> -AzureGov -UpdateSite -NoPrompts

# Verify live version (resolve the URL from your deploy output, or read it from the app)
$webApp = az webapp list -g rg-<your-site> --query "[0].defaultHostName" -o tsv
(iwr "https://$webApp/api/version" -UseBasicParsing).Content

SiteName is the short base name (e.g. myapp), NOT the full webapp name. The Bicep template appends a deterministic hash so the full webapp name is app-<SiteName>-<hash>.

Optional flags:

  • -PreferModel <name> — try a specific OpenAI model first (e.g. gpt-4.1)
  • -PinModel (with -UpdateSite) — keep the currently-deployed model; print a diff but skip the swap

Tests

cd webapp
python -m pytest tests/ -q

Current suite: 155 tests passing. CI runs the same command on push / PR / manual dispatch.


Configuration flags (App Settings)

Setting Default Purpose
THREAT_ACTIONS_ENABLED off Master switch for webhook dispatch (lockdown, LE, CAC, maritime). Without this, webhooks are built + signed + audited but NEVER posted (dry-run).
NOTIFY_SMS_ENABLED false Master switch for ACS lockdown SMS blasts. Off → messages are built but NEVER sent (no network). Requires ACS_CONNECTION_STRING (Key Vault) + ACS_SMS_FROM_NUMBER.
ACS_SMS_FROM_NUMBER (empty) Verified ACS sender phone number (E.164, e.g. +18335551234) for lockdown SMS.
SCHOOL_AUTO_WATCH_ENABLED false Auto-start school watcher on pod boot
SCHOOL_AUTO_WATCH_INTERVAL_SEC 60 (15..3600) Tick rate
SCHOOL_AUTO_WATCH_COOLDOWN_SEC 600 (60..86400) After-lockdown cooldown
MARITIME_AUTO_WATCH_ENABLED false Auto-start maritime watcher
MARITIME_AUTO_WATCH_INTERVAL_SEC 300 (60..3600) Tick rate
MARITIME_AUTO_WATCH_COOLDOWN_SEC 1800 (60..86400) After-escalation cooldown
DEMO_SEED_ENABLED false Allow /ai/demo-seed/* to mutate Cosmos
XAVIER_DISABLE_CSRF unset Test / dev only

Key Vault secrets

  • threat-webhook-signing-secret
  • school-lockdown-webhook-url
  • law-enforcement-webhook-url
  • child-protection-webhook-url
  • maritime-alert-webhook-url
  • ncmec-member-api-url (optional, v5.33 — enables /ai/cac/cases/<id>/ncmec/submit)
  • ncmec-member-api-key (optional, v5.33)

Top-level endpoints

Path Auth Purpose
/api/version, /api/health public Liveness + version
/realtime/api/lockdown session + CSRF Manual lockdown initiation (button label: INITIATE LOCKDOWN)
/ai/threat/scan session Unified threat scan
/ai/school/sweep session (rate-limited) School multi-camera sweep
/ai/school/cases/<id>/redeliver admin Replay school lockdown webhook
/ai/school/watch/{status|start|stop|acknowledge} status public, mutations admin School auto-watch
/ai/maritime/analyze session (rate-limited) Maritime deep-search
/ai/maritime/reports/<id>/redeliver admin Replay maritime webhook (v5.33)
/ai/maritime/watch/{status|start|stop|acknowledge} status public, mutations admin Maritime auto-watch
/ai/cac/scan session (rate-limited) CAC scan (bundles NCMEC XML in evidence package)
/ai/cac/cases/<id>/redeliver admin Replay CAC webhook
/ai/cac/cases/<id>/ncmec/submit admin Opt-in NCMEC Member API submit (v5.33, staging by default)
/ai/timeline, /ai/timeline/events admin Cross-scenario timeline
/ai/demo-seed/{status|run|clear} admin + flag Demo dataset seed/clear
/ai/fbi/*, /evidence/* admin Federal submission + evidence vault
/settings → Operations tab admin Watcher controls UI (v5.33)
/cameras/api/add admin Add camera (now accepts manufacturer="public_stream" + stream_url for open webcams — v5.34)

Cosmos file_type values

fbi_submission, maritime_intel_report, child_protection_case, school_safety_case, evidence_package, school_watch_state, maritime_watch_state, plus the existing analysis_result, cost_event, audit_log, chat_session, etc.


Release history

Ver Title
v5.22 Maritime Intel (vision prompt + auto-escalation)
v5.23 CAC Evidence Packager
v5.24 FBI Federal Submission
v5.25 Maritime Deep-Search
v5.26 Crimes Against Children
v5.27 School Safety Orchestration
v5.29 School Auto-Watch
v5.30 Cross-scenario Timeline + service_metadata
v5.31 Watcher Persistence + Demo Seed
v5.32 Hardening + Coverage + Docs (rate-limit, maritime auto-watch, NCMEC XML, CI, lockdown copy)
v5.33 Backlog Cleanup (maritime redeliver, NCMEC Member API opt-in, watcher settings UI)
v5.34 Public Stream Cameras (open webcam URLs with full capture/process/enhance + auto-detect)
v5.35 Open Webcam Aggregators (Ozolio JPEG fast-path + public_stream Enhance/Compare)
v5.43 Role-aware route gating (admin allowlist + viewer role)
v5.44 Hardened deploy retry + disable CSRF token expiry
v5.45 Face identification speedup: instant name rendering, persistent HTTP sessions, list_persons cache, 8x parallel FBI enrollment
v5.46 Security hardening: SQL injection guard, localhost bypass gated, bounded face cache (LRU), secrets removed from os.environ, CSRF disable blocked in prod, OpenAI session reuse, Retry-After capped, camera field allowlist, X-Request-ID validation

Architecture gotchas

  • Single Gunicorn worker required — JobQueue, alert_bus, watchers, and rate-limit buckets are in-memory.
  • CSRF blocks unauthenticated POSTs; exempt via csrf_ext.exempt(view_fn) in webapp/app/__init__.py.
  • Front Door 230-sec timeout drives the async-job design; long scans return a job-id.
  • JobQueue status: succeeded (NOT completed).
  • Threat orchestrator webhook gate: THREAT_ACTIONS_ENABLED=true REQUIRED before any webhook leaves the pod.
  • Rate limiter: per-pod, per-principal token bucket. Returns HTTP 429 + retry_after_sec.
  • NCMEC XML is staging="true" and is bundled into CAC evidence by default. The /ai/cac/cases/<id>/ncmec/submit Member API call is opt-in — without ncmec-member-api-url+ncmec-member-api-key in Key Vault it short-circuits with status="not_configured" and performs no network I/O.
  • Watcher state docs use their file_type as partition key with a fixed id so upserts replace in place.
  • Demo seed is gated by DEMO_SEED_ENABLED=true — never seeds without it.

Repo layout

azure-discovery/   # picks best-available services/models per region
infra/             # Bicep + PowerShell deploy / discover / teardown
webapp/            # Flask app
  app/
    routes/        # blueprints (ai, api, realtime, cameras, results, …)
    services/      # singletons: openai_svc, vision, face, cosmos, school_watch,
                   # maritime_watch, child_protection, school_safety, ncmec,
                   # rate_limit, threat_orchestrator, evidence_packager, …
    templates/     # Bootstrap + Jinja
    static/        # css/js/icons
  tests/           # pytest suite (155 tests, runs in CI)
.github/workflows/ # CI (pytest)

Priority scenarios (all live)

  1. School weapon detection + lockdown orchestration — v5.27 / 29 / 31 / 32
  2. Crimes Against Children — v5.26 / 32 / 33
  3. FBI / law-enforcement workflows — v5.24
  4. Maritime detection — v5.25 / 32 / 33

Security / secrets handling

  • No secrets are stored in this repo. All credentials (Cosmos keys, OpenAI keys, AAD client secret, webhook signing secret, NCMEC API key, FLASK_SECRET_KEY) live in Azure Key Vault and are surfaced to the App Service via Key Vault references in App Settings.
  • .env is git-ignored. Only webapp/.env.example is tracked \u2014 it documents variable names with empty values for local dev.
  • Build artifacts (*.zip) are git-ignored. Deployment zips produced by az webapp up contain only repo source; they are not committed. If Microsoft Purview / DLP flags them in OneDrive, it is a false positive on placeholder strings (e.g. RTSP URL builder password=\"{password}\" in services/camera.py, or env-var names in .env.example). Delete the local zip; it will be regenerated on next deploy.\n- Recommended: keep this workspace OUT of OneDrive sync. The git remote on GitHub is the canonical store; OneDrive sync is redundant and triggers periodic DLP scans on transient build artifacts.\n\n---\n\n## License\n\nInternal Microsoft project. All rights reserved.

About

Azure AI Services platform for camera-driven threat intelligence: weapon detection + school lockdown orchestration, crimes-against-children case packaging (NCMEC), FBI law-enforcement workflows, and maritime deep-search. Flask + Azure Government.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors