Skip to content

Refactor: dedicated downloader subsystem #63

Description

@RBhupi

Area

Acquisition — data download or queuing

Motivation

Developer experience — improves development or testing

Problem

The current NEXRAD pipeline relies on the external nexradaws package for Level II data acquisition. While it works well, it introduces following limitations:

  • not available through conda-forge
  • largely unmaintained
  • feature requests and bug fixes cannot be addressed upstream
  • downloader implementation is tightly coupled with the pipeline
  • difficult to extend to realtime monitoring or alternative data sources

Proposal

Introduce a new top-level package

adapt/
    downloaders/
        base.py
        registry.py
        nexrad/
            downloader.py
            aws.py
            watcher.py
            models.py
            filters.py

Downloaders become reusable infrastructure components rather than scientific modules.

Proposed solution

The first implementation will be a lightweight native NEXRAD AWS downloader built directly on boto3.

Goals

  • remove dependency on nexradaws
  • direct boto3 implementation
  • support archive and realtime workflows

Future CLI

adapt run-nexrad \
    --only download \
    --start 2026-05-01T00:00 \
    --end 2026-05-01T06:00 \
    --radar KLOT

which downloads data without executing the scientific pipeline.

Alternatives considered

No response

Additional context

No response

Checklist

  • I searched existing issues and this has not been requested before
  • This is one focused request, not multiple features bundled together

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions