RepoHunter is an automation tool engineered to discover trending GitHub repositories on auto-pilot. It seamlessly detects the host operating system, deploys a headless stealth browser to bypass bot protection, and outputs both an interactive multi-language terminal dashboard and a structured markdown report (daily_trends.md) tailored for technical recruiters, talent scouts, and tech marketers.
- Visits
github.com/trendingdeploying an anti-fingerprinting stealth browser. - Dynamically extracts repository names, descriptions, languages, and daily star velocity.
- Renders an interactive multi-language CLI dashboard via terminal streams.
- Compiles an enterprise-ready Markdown report optimized for content curation and outreach.
RepoHunter shifts the paradigm from standard web scraping into a strategic tech sourcing and content curation workflow:
- Talent Acquisition: Instantly locate high-performing open-source developers by monitoring trending repositories on a daily cadence.
- Content Engineering: Generate raw, metrics-driven insights to feed technical marketing channels and developer outreach campaigns effortlessly.
- Market Intelligence: Identify shifting tech stacks and emerging programming languages before mapping technical screening benchmarks.
- Robust Multiplatform Architecture: Built around
pathlib.Pathto provide deterministic, absolute path resolution across Windows, macOS, and Linux without fragile string manipulations. - Rigorous Binary Validation: Safely checks binary context and execution flags on Unix systems via execution bit validation (
os.X_OK) before execution loops begin. - Stealth Automation Layer: Embedded with the Obscura binary context engine to significantly minimize scraping blocks and fingerprint tracking.
- Streamlined User Interface: Renders dynamic telemetry data, progress countdown bars, and data grid grids cleanly using Rich.
- Flexible Tracking Policies: Configurable localized scan intervals supporting English, Spanish, and Portuguese execution flows.
- Resilient Data Capture: Armed with sequential fallback CSS selectors, structural cascades, and structural regex matching rules inside BeautifulSoup.
- High Schema Tolerance: Built to remain fully operational even if GitHub mutates its layout, shifts class indicators, or adjusts production DOM nodes.
- Predictive Matching: Dynamically falls back to broader article elements when specific markup wrappers fail to resolve.
- Graceful Failures: Intercepts external process runtime exceptions (
subprocesstimeouts or failure codes) and traps OS errors cleanly. - Actionable Telemetry: Eliminates unhandled application crashes by routing raw validation errors into human-readable diagnostic reports.
- Input/Output Protection: Wraps filesystem write boundaries securely to protect historical
daily_trends.mdfiles from corruption during disk errors.
- Python 3.8+
- BeautifulSoup4 (Hardened DOM parsing layer)
- Rich (Enhanced terminal UI grid rendering)
- Obscura (Rust-powered headless browser backend for anti-fingerprinting automation)
# Clone the repository
git clone [https://github.com/AxthonyV/RepoHunter.git](https://github.com/AxthonyV/RepoHunter.git)
cd RepoHunter
# Provision project dependencies
pip install -r requirements.txt
Note for macOS/Linux environments: Ensure binaries possess the required execution permissions:
chmod +x obscura-linux/obscura
chmod +x obscura-macos/obscura
- Boot up the control interface:
python repo_hunter.py
- Select your tracking policy inside the interactive CLI manager:
- Fast: Executes diagnostic scans every 10 minutes.
- Normal: Standard data-collection cadence every 1 hour.
- Slow: Long-term archival scans every 12 hours.
- Custom: Explicitly pass execution boundaries in seconds.
- Once: Performs a singular validation loop and exits gracefully.
- Inspect the programmatic output file:
cat daily_trends.md
- Open a terminal instance within the project's root directory.
- Execute
python repo_hunter.py. - Select
Onceto validate runtime paths, execution parameters, and network streams. - Review
daily_trends.mdto confirm the successful mapping of structural metrics.
The daily_trends.md engine generates production-ready markdown maps tracking:
- Repository identification namespaces.
- Direct hyperlinked routing references.
- Structural schema descriptions.
- Primary programming language tokens.
- Dynamic localized velocity spikes (Stars accumulated today).
Contributions that scale parsing patterns, include webhook dispatch integration (Slack, Discord, X), or extend reporting metrics are highly welcome.
- Fork the codebase.
- Branch out into a discrete feature context (
git checkout -b feature/AmazingFeature). - Commit localized logic changes safely (
git commit -m 'feat: add AmazingFeature context'). - Push structural branch changes up (
git push origin feature/AmazingFeature). - Open a formal Pull Request.
This automation relies heavily on Obscura for robust headless processing. Massive credits to the developers of Obscura for deploying an exceptional Rust-based scraper layer.
Distributed under the MIT License. See LICENSE for details.
AxthonyV
- GitHub: @AxthonyV
If you find this software architecture valuable, consider adding a repository star ⭐ to support the development.