Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
name: tests

on:
pull_request:
push:
branches: [main]

jobs:
scrubber-tests:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Install deps
run: pip install pytest httpx presidio-analyzer presidio-anonymizer
- name: Download spaCy model
run: python -m spacy download en_core_web_md
- name: Run scrubber tests (incl. golden leak canary)
run: pytest tests/ -q
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -174,7 +174,9 @@ path still runs.
- `profile/` -- `config.yaml` (the full incognito profile), `SOUL.md` (the Ghost identity), `.env.example`, `pii_denylist.example.txt`, `uncensored_prefill.json`
- `privacy/`
- `scrubbing_proxy.py` -- the PII/secret scrubber + local model-catalog endpoint; forwards cleaned requests to og-veil
- `rotating_proxy.py` -- Webshare rotation + blocklist · `gen_searxng_settings.py`
- `scrub_patterns.py` -- single source of the secret + PII regexes (shared by both scrub paths)
- `presidio_scrub.py` -- NER PII detection (Presidio + spaCy) with reversible placeholders + stream de-anon
- `rotating_proxy.py` -- Webshare rotation (opt-in IP-masking)
- `ensure_scrubber_route.py` -- self-heals the engine's hosted route back to the scrubber after a token refresh
- _(the OHTTP/HPKE/registry/verification + Supabase auth that used to live here now comes from the `opengradient-veil` package -- run `og-veil`)_
- `scripts/` -- `fork-engine.sh` (copy + relocate venv + isolate skills) and `debrand.py` (scrub visible strings + the two ASCII-art logos)
Expand Down
3 changes: 1 addition & 2 deletions bin/ghost
Original file line number Diff line number Diff line change
Expand Up @@ -69,10 +69,9 @@ elif [ "$HC" = "200" ]; then
else
STATUS="👻 ghost · ⚠️ scrubbing bridge DOWN (bridge=$HC) -- hosted unreachable, $FB_DOWN"
fi
if scutil --dns 2>/dev/null | grep -q "103.86.9"; then VPN="✅ NordVPN"; else VPN="⚠️ no NordVPN"; fi
[ -n "$PASS_PATHS" ] && STATUS="$STATUS · 🗂️ path-aware (real paths visible to hosted model)"
[ -f "$NOSCRUB_MARK" ] && STATUS="$STATUS · 🔓 PII redaction OFF (secrets still scrubbed)"
echo "$STATUS · $VPN" >&2
echo "$STATUS" >&2

HB="__ENG__/venv/bin/hermes"
if [ -n "$PASS_PATHS" ]; then
Expand Down
9 changes: 8 additions & 1 deletion install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,13 @@
# GHOST_CHAT_APP_URL= override the website used for `ghost-login` (default chat.opengradient.ai)
set -euo pipefail

# macOS only: the privacy stack runs as launchd LaunchAgents and uses BSD tooling. Fail fast
# with a clear message rather than part-installing on Linux/WSL and erroring confusingly later.
if [ "$(uname -s)" != "Darwin" ]; then
echo "!! ghost's installer currently supports macOS only (it uses launchd). Detected: $(uname -s)." >&2
exit 1
fi

REPO="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ENGINE_HOME="${ENGINE_HOME:-$HOME/.hermes}" # where the Hermes engine installs (official installer default)
GHOST_HOME="${GHOST_HOME:-$HOME/.ghost}" # ghost's ISOLATED state (profiles, privacy, auth)
Expand Down Expand Up @@ -132,7 +139,7 @@ fi

# ---------- 3. privacy stack (PII scrubber + og-veil always; rotating proxy only with GHOST_PROXY) ----------
say "Privacy stack (PII/secret scrubber -> og-veil${USE_PROXY:+ + rotating proxy})"
mkdir -p "$PRIV/searxng"
mkdir -p "$PRIV"
cp "$REPO"/privacy/*.py "$PRIV/"
# Enable the NER PII scrubber when Presidio + the spaCy model are present; else leave it off
# (the bridge falls back to the regex scrubber). Toggle anytime: touch/rm $PRIV/.presidio.
Expand Down
52 changes: 0 additions & 52 deletions privacy/gen_searxng_settings.py

This file was deleted.

Loading