Chirp RadioReference Scraper 🚀

This project uses a small index file, radioref.csv, to map RadioReference CTID pages (county/city titles) to their numeric CTID IDs. This file is required for ZIP-to-CTID mapping in the GUI and for accurate RadioReference lookups.

If radioref.csv is missing or outdated, please see the Troubleshooting section below.

📚 Documentation: See docs/README_ENHANCEMENTS.md for complete feature documentation, and docs/CHANGELOG.md for version history and updates.

Technical Business Profile 🏢

ChirpScrape is a technical-business utility for radio data operations with a local-first workflow, packaging support, and practical automation for field and office teams.

• 🔐 Reliability-focused data collection and export workflows
• ⚙️ Automation-friendly CLI and GUI runtime
• 🖥️ Cross-platform packaging support for deployment
• 📊 CSV-first outputs for downstream operational pipelines

Quick Start ✅

1. One-step runtime (recommended)

   Run the ChirpScrape launcher to create/repair the virtual environment, install dependencies, and launch the app in GUI mode by default (no switches):

   Linux / macOS:

   ./ChirpScrape
   # or: python3 ChirpScrape

   Windows (PowerShell):

   python ChirpScrape

   Windows (cmd.exe):

   python ChirpScrape

2. Manual setup (alternative)

   If you prefer to create the venv yourself, follow the platform-specific steps below.

   Linux / macOS (bash/zsh):

   python3 -m venv .venv
   source .venv/bin/activate
   pip install -r requirements.txt
   python3 chirp_scraper.py --gui

   Windows (PowerShell):

   python -m venv .venv
   .\.venv\Scripts\Activate.ps1
   pip install -r requirements.txt
   python chirp_scraper.py --gui

   Windows (cmd.exe):

   python -m venv .venv
   .\.venv\Scripts\activate.bat
   pip install -r requirements.txt
   python chirp_scraper.py --gui

3. Check output:

   • Output files are generated in the project directory by default (e.g., chirp_output.csv).
   • Sample output in this repository is stored at outputs/chirp_output.csv.

One Runtime File

Primary runtime command (GUI default, no switches):

./ChirpScrape

Runtime launchers are kept in the main project directory for easy discovery:

• ChirpScrape
• bootstrap.py

Use bootstrap.py as the single runtime entrypoint for install/run/test/security:

python3 bootstrap.py run        # install deps, then run app
python3 bootstrap.py install    # install deps only
python3 bootstrap.py test       # syntax + unittest smoke tests
python3 bootstrap.py security   # lightweight static security scan
python3 bootstrap.py package    # build one-time distributable for this OS
python3 bootstrap.py all        # install + security + test + run

Legacy flags still work (--install-only, --test, --security-check, --gui).

Build App, DMG, EXE (One-Time Packaging)

Packaging is platform-native. Build on the target OS:

Linux Onefile Binary

./scripts/build_linux_onefile.sh

Output:

• dist/ChirpScrape

macOS App + DMG

./scripts/build_macos_app_dmg.sh

Outputs:

• dist/ChirpScrape.app
• dist/ChirpScrape.dmg

Windows EXE

Run in PowerShell:

./scripts/build_windows_exe.ps1

Output:

• dist/ChirpScrape.exe

Notes:

• .app/.dmg must be built on macOS.
• .exe must be built on Windows.
• A Linux build was generated in this workspace and verified with --help at dist/ChirpScrape.

Donations 🙏

Developing and maintaining open source software takes significant time and resources. Your support helps cover development, testing, and hosting costs. Every contribution makes a difference!

🎁 Donation Portal: A professional donation page appears on first launch. You can dismiss it and access it anytime via Help → Contact → Donations in the app menu.

Thank you for considering a donation!

Why Donate? 💡

• Open source software fosters innovation and collaboration.
• Supports learning and skill development for programmers.
• Provides cost-effective solutions for everyone.
• Drives technological advancement and builds strong communities.

Choose Your Donation Method

PayPal

paypal.me/Dr1zztD

Cash App

$teerRight

---

Troubleshooting 🛠️

Python Environment & Dependency Issues 🐍

If you encounter numpy/pandas import errors:

ChirpScrape now supports Python 3.13 with properly compatible dependencies:

• numpy 2.4.2+ - Required for Python 3.13 compatibility
• pandas 3.0.0+ - Updated for latest numpy and Python versions

Solution: If you already have an old venv, start fresh:

# Remove old environment
rm -rf .venv

# Create new environment and install dependencies
python3 -m venv .venv
source .venv/bin/activate  # (or .\.venv\Scripts\Activate.ps1 on Windows)
pip install -r requirements.txt

The bootstrap.py script handles this automatically, so using it is recommended for a clean installation.

RadioReference Index File (radioref.csv) 📂

If you see errors or missing data related to RadioReference lookups, or if radioref.csv is missing or outdated, you need to (re)generate the index file. Use the helper script below:

Generate or Update radioref.csv

Run this command to crawl RadioReference and build or refresh the index:

./.venv/bin/python make_radioref_list.py --start-id 1 --max-id 3000 --append

Notes:

• The crawl can take a long time; use --delay to be polite and --stop-after-missing to stop after many consecutive misses.
• chirp_rr_zip_scraper.py will still run without radioref.csv, but ZIP lookups that depend on the index may show "(no ctid)" and fall back to ZIP-level RadioReference pages.

See README.txt for additional project notes.

Help → Firmware submenu: New firmware unlock resources (Baofeng unlock guides and firmware links) are available in the app under the Help → Firmware menu. See the in-app Help for step-by-step links and resources.

Advertising & Media: Advertising copy and media (including the Facebook advert) are available in the Advert/ folder. See Advert/Facebook.md for Facebook-specific ad content.

Legal ⚖️

This software is provided as-is. When scraping websites, ensure you follow the target site's terms of service and robots.txt. The author is not responsible for misuse.

Chat

• WhatsApp: https://wa.me/13127235816

Remote SSH run (CHIRP_Script local Linux)

Use this script when connecting to a remote host via SSH to run the local project path.

./scripts/rssh-run.sh user@remotehost /home/user/code/Chirp_Scrape

This will:

• enter the remote project directory
• ensure gradlew is executable
• run ./gradlew --no-daemon --stacktrace --info clean build

neofetch is optional (Linux convenience):

ssh user@remotehost 'neofetch --ascii_distro Ubuntu'