Skip to content

dafuct/rss-aggregator

Repository files navigation

ReadStack — a reader for tech feeds

Live at readstack.dev.

ReadStack aggregates a configurable list of RSS/Atom feeds into one distraction-free reader. It does three things:

  1. Reads feeds in-app — a clean serif reader pane renders each article's server-sanitized HTML (images kept), so you rarely leave for the original site.
  2. Remembers what you read — optional accounts add save-for-later, read tracking, and an "unread only" view.
  3. Surfaces what's trending — a daily Launch Radar leaderboard fuses GitHub Trending, Hacker News, and Product Hunt into one ranked board.

Spring Boot REST API + React frontend, in one monorepo:

backend/    Spring Boot 4 (Java 25, Gradle) — fetches, parses, sanitizes, caches, serves /api;
                                                accounts + saved/read state + Launch Radar on SQLite
frontend/   React 19 + TypeScript (Vite)    — sepia "ReadStack Reader" (top strip / reading column / Up Next rail)

Articles are read in-app: the reader renders the article's sanitized content (with images) where the feed provides it, and falls back to a "Read original" link for stub-only sources (e.g. Hacker News). Feeds are organized into categories you pick from the top strip; signing in is optional — anonymous users can read everything.

Requirements

  • JDK 25 (the Gradle toolchain resolves it automatically if installed)
  • Node.js 20+ (the Docker build uses Node 24)
  • Docker 25+ with Compose v2 (only for the containerized run mode)
  • (optional) A free Product Hunt API token to enable the Product Hunt source in Launch Radar

Running with Docker

docker compose up --build

Open http://localhost:3000. nginx serves the built SPA and proxies /api to the backend container; the backend API is also published directly on http://localhost:8080 (loopback only). Right after startup the list can be briefly empty — the backend reports healthy before the first feed refresh finishes. Override host ports if they're taken:

BACKEND_PORT=8085 FRONTEND_PORT=3001 docker compose up --build

Accounts, saved/read state, and Launch Radar snapshots live in a SQLite database on a persisted named volume (rss-data), so they survive image rebuilds. A few environment knobs (see docker-compose.yml):

  • RSS_DB_PATH — SQLite file location (defaults to /data/rss.db in the container).
  • SESSION_COOKIE_SECURE — set true in production behind TLS; left false so a local HTTP run keeps its session cookie.
  • PRODUCT_HUNT_TOKEN — optional; blank means the Product Hunt source no-ops and Radar builds from GitHub + Hacker News only.

The images are multi-stage builds (Temurin 25 JDK→JRE alpine; Node 24 → unprivileged nginx), both run as non-root, and the frontend waits for the backend's health check before starting.

Running locally (dev mode)

Backend (port 8080):

./gradlew bootRun

If 8080 is taken on your machine: ./gradlew bootRun --args='--server.port=8085'. bootRun's working directory is backend/, so point RSS_DB_PATH at an absolute path whose directory exists, e.g. RSS_DB_PATH=/absolute/path/to/rss/data/rss.db ./gradlew bootRun (defaults to ./data/rss.db relative to backend/).

Frontend (port 5173):

cd frontend
npm install
npm run dev

Open http://localhost:5173. Articles appear once the backend finishes its first feed refresh (a few seconds after startup; watch for Refreshed cache: N articles in the backend log).

Backend tests:

./gradlew test

How it works

Feed reader

  • On startup and every 10 minutes (news.refresh-interval), a @Scheduled job fetches all feeds concurrently on virtual threads — one task per feed via Executors.newVirtualThreadPerTaskExecutor(). Each feed has its own timeout (news.fetch-timeout); the whole refresh is also bounded by a wall-clock deadline (news.refresh-timeout) so one hung feed can't freeze the loop. A slow or failing feed is logged with Skipping feed … and skipped without affecting the others.
  • A single shared java.net.http.HttpClient is used for all fetches, with a custom User-Agent (news.user-agent — some feeds reject the default Java agent) and Redirect.NORMAL so redirecting links (hnrss, Google News) resolve.
  • Each feed is parsed with ROME and mapped to a common Article record (id, title, url, source, category, author, publishedAt, summary, content). publishedAt falls back to the entry's updated date. The full article HTML is sanitized with jsoup (scripts, iframes, and event handlers stripped; images kept) for safe in-app rendering; the short summary is the plain-text excerpt shown on cards. Stub-only entries (link posts with no real body) get null content and are shown as "Read original" links.
  • A stable id (truncated SHA-256 of the URL) makes each article shareable at /article/{id} with no database lookup needed.
  • Articles are merged, deduplicated by canonical URL (a UrlNormalizer strips tracking params and syndication noise so the same story from different feeds collapses; the newest-dated duplicate wins) and sorted newest first (articles without a date go last), then stored in an in-memory cache (with an id → article index). User requests always serve from the cache — they never trigger live fetches.

Accounts, saved & read state

  • Optional email + password accounts: BCrypt-hashed passwords, an HTTP-only session cookie (SameSite=Lax), CSRF protection via a readable XSRF-TOKEN cookie the SPA echoes back, and a 30-day remember-me persistent token so logins survive deploy restarts.
  • Signed-in users get save for later, mark as read, and an "unread only" filter in the Up Next rail. Anonymous users can read everything; these features render as sign-in prompts.
  • Persistence is SQLite via Spring Data JPA. Schema is created by an idempotent schema.sql on every startup (CREATE TABLE IF NOT EXISTS) — Flyway 11 bundled with Spring Boot 4 has no SQLite support, so Hibernate never touches DDL and future schema changes are additive edits to that file.

Launch Radar

  • A daily leaderboard of trending developer tools and projects at /radar, fused from three sources — GitHub Trending, Hacker News, and Product Hunt — into one ranked board. Items found on more than one source are boosted (radar.boost-per-extra-source, capped at radar.boost-cap) and ranked by a momentum score.
  • Each morning (radar.snapshot-cron, 06:00 UTC by default) the build freezes an immutable snapshot for that date. Items seen within the last radar.dedup-lookback-days (default 30) are suppressed so the board doesn't repeat day over day. The frontend exposes today, this week, and an archive of past days.

REST API

News (public)

Endpoint Description
GET /api/news?limit=50&offset=0&q=<keyword>&source=<name>&category=<name>&unread=<bool> Page of article cards (no heavy content body; includes a readable flag), newest first. q filters case-insensitively across title + summary; source/category filter by feed/category name; unread=true (signed-in only) hides already-read articles; limit defaults to 50 (capped at 200), offset for paging. All parameters optional.
GET /api/news/{id} A single full article including sanitized content, or 404 if it has left the cache.
GET /api/feeds The category tree: categories, each with its sources and live article counts.

Accounts & personalization (session cookie)

Endpoint Description
POST /api/auth/register Create an account { email, password, displayName }.
POST /api/auth/login Sign in { email, password, rememberMe }; sets the session cookie.
POST /api/auth/logout End the session (authenticated).
GET /api/auth/me The current user, or null if anonymous.
GET /api/me/saved · POST/DELETE /api/me/saved/{id} List, save, or unsave articles (authenticated).
GET /api/me/saved/ids Just the saved article ids, for marking cards in the UI.
POST /api/me/read/{id} Mark an article read (drives the "unread only" filter).

Launch Radar (public)

Endpoint Description
GET /api/radar/today Today's leaderboard.
GET /api/radar/week Items trending across the last week (with daysAppeared).
GET /api/radar/archive The list of available snapshot days.
GET /api/radar/day/{date} One past day's frozen board, or 404.

Proxy and CORS

Three mechanisms let the frontend talk to the backend, depending on run mode:

  • Vite dev proxy (frontend/vite.config.ts): requests to /api/* on localhost:5173 are forwarded to http://localhost:8080, so the app uses same-origin relative URLs and needs no CORS in the normal dev setup. Override the target with VITE_API_PROXY_TARGET=http://localhost:8085 npm run dev if your backend runs on a different port.
  • nginx proxy in Docker (frontend/nginx.conf): in the compose stack, nginx serves the SPA and proxies /api/* to the backend service over the compose network — the browser stays same-origin, so CORS is not involved at all.
  • CORS (WebConfiguration.java, wired into Spring Security via http.cors()): the backend additionally allows credentialed requests on /api/** from the configured origins (news.allowed-originshttp://localhost:5173, http://localhost:3000, and https://readstack.dev), so calling the API directly from a frontend origin — bypassing the proxies — also works.

Adding or removing feeds

Feeds are grouped into categories under news.categories in backend/src/main/resources/application.yml:

news:
  categories:
    - name: "Java & JVM"
      feeds:
        - { name: "Spring Blog", url: "https://spring.io/blog.atom" }
        - { name: "Your New Feed", url: "https://example.com/rss.xml" }   # add
    - name: "Your New Category"      # add a whole category like this
      feeds:
        - { name: "Some Source", url: "https://example.com/feed" }

Each feed has a display name (shown as the source in the rail and on cards) and a url. Restart the backend; new feeds and categories are picked up on the next refresh, and the UI populates itself from /api/feeds — no frontend change needed. Other knobs in the same file: refresh-interval, fetch-timeout, refresh-timeout (ISO-8601 durations), user-agent, allowed-origins.

Filtering out paid / stub / non-English articles

There is no reliable universal "is paid" flag in RSS, so the backend uses config-driven filters applied at refresh, so hidden articles never enter the cache:

  • news.paywall-markers — a list of case-insensitive phrases; any article whose title/summary/content contains one is dropped. Defaults target subscription previews (e.g. "this post is for paid subscribers"). Set to [] to disable.
  • news.english-only (default true) — drops non-English articles using a script + function-word heuristic. Set false to keep all languages.
  • news.hide-unreadable (default false) — when true, drops all stub/preview articles (anything without full in-app content, including Hacker News and Medium link cards). Leave false to keep those as "Read original" cards.

Note: feeds like Medium truncate their RSS to a preview, so those articles always appear as "Read original" link cards rather than full in-app reads — that's a feed limitation, not a setting.

Launch Radar has its own block under radar.* in the same file: enabled, snapshot-cron/zone, top-n, dedup-lookback-days, boost-per-extra-source/ boost-cap, and per-source limits under radar.github, radar.hacker-news, and radar.product-hunt (where token reads PRODUCT_HUNT_TOKEN).

Project layout

backend/src/main/java/dev/makar/rss/
  config/      NewsProperties + RadarProperties (binding), HttpClient bean,
               SecurityConfiguration (auth, CORS, CSRF, remember-me), CsrfCookieFilter, Clock
  model/       Article, ArticleCard (list DTO), CategorySources (feed tree)
  service/     FeedFetcher (HTTP + ROME), FeedEntryMapper (jsoup sanitize),
               ArticleId (stable id), UrlNormalizer (canonical dedup key),
               LanguageDetector (english-only filter), ArticleAggregator (dedup + sort),
               NewsService (cache + scheduled refresh + queries + feed tree)
  account/     UserAccount + AccountService (register/login), SavedArticle / ReadArticle
               (+ services & JPA repositories), InstantStringConverter
  radar/       RadarService (snapshot + queries), RadarAggregator (fuse + score),
               GitHubTrendingSource / HackerNewsSource / ProductHuntSource, model/ (board, entry)
  web/         NewsController, AuthController, MeController, RadarController,
               ApiExceptionHandler, CurrentUser, dto/

frontend/src/
  api/         typed fetch client (news, article, feeds, auth, saved, read, radar)
  hooks/       useNews (infinite), useFeeds, useArticle, useAuth, useSaved, useMarkRead,
               useRadar, useDebouncedValue, usePersistentState
  components/  TopStrip, ReadingColumn, UpNextRail, CategoryMenu, AccountMenu,
               LaunchRadar, RadarCard, SourceAvatar
  lib/         timeAgo, avatar, heroImage, readingTime
  types.ts     ArticleCard / Article / CategorySources / radar types mirroring the backend

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages