Skip to content

README.md

Latest commit

 

History

History
304 lines (229 loc) · 12.2 KB

README.md

File metadata and controls

304 lines (229 loc) · 12.2 KB
Raw

ThesisOS

ThesisOS is a local-first Research OS for institutional-style sector, company, macro, news, valuation, thesis, claim, evidence, report, and review workflows.

It combines a deterministic data and analysis engine, a FastAPI backend, a Next.js research desk, and a CLI for repeatable local execution. The system is built to collect public and local financial data, normalize statements, calculate documented metrics, classify events, track thesis and claim changes, and generate reviewable research packets and Markdown reports.

ThesisOS is not an auto-trading system, an order execution system, a position sizing engine, a generic chatbot, an LLM-first investment recommender, a commercial SaaS product, or a system where LLM output is source of truth.

Canonical Names

Item Name
Product ThesisOS
Repository folder thesisos
Python package research_os
CLI research-os
Web title ThesisOS Research Desk

Core Boundaries

Deterministic Source Of Truth

  • Calculations are deterministic Python or SQL functions.
  • LLMs must not perform arithmetic or create source-of-truth records.
  • Every metric needs a documented formula and tests.
  • Important claims must link to source data, calculation output, or evidence IDs.
  • Reports must include data_cutoff, calculation_version, warnings, and sources or evidence IDs.

Missing Data

  • Missing numeric values remain None/null.
  • UI displays missing values as Not available.
  • Missing values are never silently filled with zero.
  • Calculations return None plus warnings when required inputs are missing or denominators are invalid.
  • Readiness and quality reports expose missing inputs.

Internal Ratings And Trading Boundary

ThesisOS may calculate deterministic target prices, bear/base/bull target ranges, implied upside/downside, target revisions, valuation views, and internal research ratings from explicit local inputs and policy.

Allowed deterministic target/rating outputs:

Output Allowed values
Internal research rating buy, hold, sell, not_rated
Valuation view positive, neutral, negative, not_available
Target revision raise, maintain, cut, not_available

These outputs are internal research classifications only. They are not trading instructions, order instructions, position sizing recommendations, or guaranteed investment outcomes. ThesisOS does not place orders, size positions, rebalance portfolios, or automate trading.

Claim status and thesis status are separate from internal ratings. Claim and thesis statuses must not use buy/sell/hold.

LLM/Codex Review Boundary

Optional LLM/Codex review is draft-only and non-authoritative.

  • Disabled by default or prompt-only by default.
  • Used for review/commentary, not source-of-truth mutation.
  • Must not mutate source data, update thesis status/confidence, update claim status/confidence, create evidence links, calculate target prices, generate internal ratings, or produce trading/order/position-sizing instructions.
  • Codex integration must use official Codex CLI/SDK invocation paths only.
  • ThesisOS must not read Codex credentials, inspect ~/.codex, parse OAuth tokens, copy generated API keys, or expose credentials to the frontend.
  • OPENAI_API_KEY is not required for ThesisOS core and is not read by default.

Current Capabilities

This status is based on the repository state. Planned items are listed separately in the roadmap rather than described as complete.

Area Current state
Core app FastAPI backend, Next.js web app, DuckDB local database, repository layer, CLI, Data Center, jobs, and local pipeline orchestration
Sector/company foundation Sector profiles, company metadata, DB-backed sector/company APIs and pages, local-first config fallback
DART/events OpenDART filing-list fetch/cache/ingest, corp-code management, filing APIs/pages, deterministic filing event classification, event APIs/pages
Financial statements Raw DART financial statement ingestion, account mapping, statement standardization, mapping coverage, standardized rows, financial features, margins, FCF, net debt, YoY/QoQ, readiness
Peer/market/valuation Peer comparison engine/API/pages, manual market price CSV ingestion/API/pages, valuation readiness/inputs, PER, PBR, EV/EBITDA, EV/Sales, EV/FCF, valuation API/pages
Target price/internal rating Local target policy, target inputs, deterministic target engine, bear/base/bull ranges, implied upside/downside, internal rating, valuation view, target revision, API/pages, packet/report sections
Packets/reports/pipeline Research packet builder/API/viewer, deterministic Markdown report generator/API/viewer, target/rating report section, full local pipeline CLI/API/Data Center controls
Thesis/claim/evidence Thesis tracking skeleton/API/pages, claim mapping skeleton/API/pages, claim evidence links, claim update logs, deterministic claim evidence review, claim review API/pages
LLM/Codex review Optional provider abstraction, Codex adapter skeleton, prompt builders, mock provider, CLI/API/pages, Data Center controls, disabled/prompt-only by default, no credential reading
Macro Macro series catalog, local macro CSV ingestion, macro observations, macro features, macro readiness, macro API/pages, Data Center macro controls; no live FRED/ECOS/BLS/BEA/KOSIS calls yet
News Local news source catalog, manual/RSS/GDELT-style fixture ingestion, article metadata storage, deterministic entity/sector mapping, and deterministic news event classification skeleton

For more detail, see project/CURRENT_STATUS.md and project/ROADMAP.md.

Data Layers

Layer Purpose
Raw/cache Preserve source responses and local uploads where practical
Standardized Normalize accounts, periods, entities, statement type, and metadata
Feature Store deterministic financial, market, peer, valuation, target, macro, and future news/scenario features
Research Store packets, reports, theses, claims, evidence links, claim reviews, and draft LLM review outputs

Frontend code never reads DuckDB or raw files directly. The web app calls FastAPI only, and FastAPI/CLI call shared service-layer and research_os modules.

Analysis Layers

  • DART filings and filing-derived event classification.
  • Raw and standardized financial statements.
  • Financial features such as YoY, QoQ, margins, FCF, and net debt.
  • Peer comparison and score tables.
  • Manual market price ingestion and valuation multiples.
  • Deterministic target price and internal rating outputs.
  • Macro time-series catalog, observations, features, and readiness.
  • News source catalog, local ingestion, article metadata, deterministic entity/sector mapping, deterministic event classification, and future evidence integration.
  • Future scenario analysis for rate, FX, inflation, demand, liquidity, and sector exposure drivers.

Research And Review Layers

  • Research packets are deterministic JSON-like aggregates for review.
  • Markdown reports are generated from stored packets and local calculation output.
  • Thesis and claim records track research state separately from target ratings.
  • Claim reviews are deterministic evidence checks.
  • LLM/Codex reviews are optional draft notes layered on top of packets, reports, claims, or claim reviews.

API Key Policy

Local deterministic workflows do not require API keys.

Key Required for core? Use
OPENDART_API_KEY No Required only for explicit live OpenDART fetches in backend/CLI flows
FRED_API_KEY No Future optional live macro connector
ECOS_API_KEY No Future optional live macro connector
BLS_API_KEY No Future optional live macro connector
BEA_API_KEY No Future optional live macro connector
KOSIS_API_KEY No Future optional live macro connector
News provider keys No Future optional news connectors
OPENAI_API_KEY No Not required for ThesisOS core; not read by default

No secrets should use NEXT_PUBLIC_*. Frontend code must not access provider, OpenAI, or Codex credentials.

News Roadmap

The recommended free/local-first news strategy is:

  • Official publisher RSS feeds.
  • GDELT DOC API/RSS/JSON.
  • Manual news CSV ingestion.
  • Optional free news API providers later.

Avoid initially:

  • Paywall bypass.
  • Uncontrolled full-text scraping.
  • Storing copyrighted full article bodies unless explicitly allowed.
  • Treating Google/Bing RSS as a core source without caution.
  • Treating LLM summaries as source of truth.

Quickstart

Create and activate a virtual environment:

python3 -m venv .venv
source .venv/bin/activate

Install the project in editable mode with development dependencies:

python -m pip install -e ".[dev]"

Copy the environment template only if you need local settings:

cp .env.example .env

Initialize and seed the local DuckDB metadata:

make db-init-seed

Run the FastAPI backend:

make api

Run the Next.js app:

cd apps/web && npm install
make web

CLI Examples

Run a deterministic local pipeline plan without writing to DuckDB:

research-os pipeline plan --sector-id semiconductor_equipment_kr --as-of-date 2026-05-08

Run the local pipeline in dry-run or apply mode:

research-os pipeline run --sector-id semiconductor_equipment_kr --as-of-date 2026-05-08 --dry-run
research-os pipeline run --sector-id semiconductor_equipment_kr --as-of-date 2026-05-08 --apply
research-os pipeline run --sector-id semiconductor_equipment_kr --as-of-date 2026-05-08 --apply --include-report

Seed local macro catalog and build deterministic macro features:

research-os macro catalog seed
research-os macro observations ingest-csv --csv-path tests/fixtures/macro/macro_series_success.csv
research-os macro features build-all --apply

Preview deterministic target prices and internal ratings:

research-os target-price policy validate
research-os target-price build --sector-id semiconductor_equipment_kr --dry-run

OpenDART Use

Tests and local deterministic workflows use fixtures or cached data. Real OpenDART calls require OPENDART_API_KEY and an explicit live-fetch command.

Example filing-list fetch:

research-os dart filings fetch-list \
  --entity-id wonik_ips \
  --corp-code <PM_VERIFIED_CORP_CODE> \
  --bgn-de 20250101 \
  --end-de 20251231 \
  --allow-network

The frontend must never call OpenDART directly or receive DART API keys.

Validation Commands

Documentation-only changes should still pass the full project validation suite:

git diff --stat
python3 -m pytest -q
make test
make lint
cd apps/web && npm run typecheck
cd apps/web && npm run build

No tests should require external API calls, real Codex calls, OpenAI API access, paid data providers, or private local files.

Roadmap

Horizon Focus
Current Local-first DART, financials, peer, market, valuation, target/rating, reports, thesis/claims, draft LLM review, and macro foundation
Near term Macro driver mapping, sector macro exposure, scenario analysis, macro evidence integration, data quality dashboard, provider settings
News roadmap RSS/GDELT/manual CSV ingestion, article metadata, entity/sector mapping, event classification, evidence links, report sections
Research depth Stronger claim/evidence workflow, evidence inbox, review queues, report comparison, thesis-change history
Operating maturity Better job observability, pipeline readiness gates, reproducible demo datasets, stricter CI and docs release process

See project/ROADMAP.md for the fuller roadmap.

Core Rules

  • Never invent financial numbers.
  • Never use an LLM for arithmetic calculations.
  • All calculations must be deterministic Python or SQL functions.
  • Every metric must have a documented formula and tests.
  • Missing data must be surfaced explicitly and must not be silently filled with zero.
  • Frontend code must call FastAPI only.
  • Reports must include data_cutoff and calculation_version.
  • Internal ratings are deterministic research outputs, not trading instructions.
  • LLM/Codex review is optional, draft-only, and non-authoritative.
  • Secrets, API keys, credentials, local private data, raw data, cache files, and generated reports must not be committed.