English | 한국어 | 中文 | 日本語 | Español
Fork notice — This is a fork of fivetaku/insane-search maintained by @miter37. It includes the upstream codebase plus PR #8 (v0.10.0, awaiting upstream review): 9 backward-compatible upgrades — JS shell detection, bot-wall classification, transient-status retry, body size cap, multi-strategy content extraction (trafilatura → JSON-LD →
__NEXT_DATA__→ og:description → crude), PDF extraction, Playwright render-merge, cookie banner removal, heavy-resource blocking. See CHANGELOG.md for the full list. Usefivetaku/insane-searchif you want the upstream release; use this fork for the additional extraction / recovery features.
Impossible is nothing. If it's public, insane-search gets in.
A resilient public-page reader for Claude Code. No API keys, no proxy setup.
/plugin marketplace add https://github.com/fivetaku/gptaku_plugins.git
/plugin install insane-search@gptaku-plugins
/reload-pluginsNo commands to learn. Ask Claude Code normally — insane-search kicks in when a fetch gets blocked.
Just ask normally — insane-search kicks in when a fetch gets blocked:
"Find what people are saying about Claude Code on Reddit and summarize the top threads." "Search X for posts about insane-search." "Summarize this YouTube video."
Expected: Claude reaches each site's public route — Reddit's feed, X via oEmbed, YouTube captions — with no login and no API key, and returns usable text, where the same request returns "I can't access that" without the plugin.
X · Reddit · YouTube · Hacker News · Naver · Coupang · LinkedIn · Medium · Substack · arXiv · GitHub · Stack Overflow · Bluesky · Mastodon — plus any site with a public page, feed, or /rss. Full platform list & methods → PLATFORMS.md.
- It escalates, never pre-judges — public API readers → syndication gateways → TLS impersonation → a real headless browser, trying each route until one works.
- It looks human — builds a full browser identity (real TLS fingerprint, cookie warming, referer chain), not just a swapped User-Agent.
- It finds hidden APIs — watches the real browser's network traffic and reuses the site's own internal JSON.
- Zero setup — auto-installs what it needs (
curl_cffi,yt-dlp, …) on first use. No API keys, no signup.
| When you hit… | Claude Code alone | + insane-search |
|---|---|---|
A 403 / WAF-blocked page |
✖ gives up | ✓ escalates until one route gets through |
| Platform content (X, Reddit, HN) | ✖ often blocked or empty | ✓ public API readers + syndication |
| An anti-bot / CAPTCHA challenge | ✖ stops | ✓ TLS impersonation, then a real headless browser |
| A media page (YouTube, 1,800+ sites) | ✖ no transcript | ✓ yt-dlp metadata + captions |
Missing tools (curl_cffi, yt-dlp) |
— | ✓ auto-installs on first use |
| API keys / signup | — | ✓ none |
| A login wall or paywall | ✖ | ✖ stops here, and says so (see Boundaries) |
The differentiating row is the one thing the default fetcher can't do: it keeps trying public routes until one works.
| Without it | With insane-search |
|---|---|
Ordinary fetch hits 403 and Claude says it can't read the page |
The plugin picks a public-access route and returns usable text |
| You manually try mirrors, archives, mobile URLs | Fallbacks escalate automatically, public-only |
| A blocked page is a dead end | Metadata and structured data still yield titles, prices, summaries |
Phase 0→3 adaptive scheduler — details
Each phase runs only if the previous one fails or detects a blocking signal:
- Phase 0 — special public endpoints (platform APIs, feeds) it can't discover generically
- Phase 1 — lightweight probes: public API readers, syndication gateways, mobile /
.json//rssURL variants - Phase 2 — TLS impersonation (curl_cffi: safari → chrome → firefox) with a full browser identity
- Phase 3 — a real headless browser, which also surfaces the public JSON APIs a site uses internally
- Exit — a login or paywall is detected: it reports "authentication required" rather than pretending
Every response is also scanned for OGP / JSON-LD, so even partial pages yield titles, summaries, and prices.
insane-search is a reader for public content, not a way around authentication.
- Reaches what's available through public pages, public APIs, feeds, archives, and a browser's public responses.
- Stops at logins and paywalls — it reports
authentication requiredinstead of trying to defeat them. - Never logs in as you and never stores or transmits credentials.
- All routes use no-auth public endpoints and standard, documented techniques.
MIT
Part of gptaku-plugins — Claude Code plugins that break through the walls everything else stops at.