GEX Faker API

Mock API server that replays historical GexBot market data with per-API-key sequential playback. Includes REST API, WebSocket streaming, CLI downloader, and scheduled download daemon.

GEXBOT-FAKER-API.mp4

NOTE: This only downloads and replays GexBot market data.

Features

REST API with Swagger UI documentation at /docs
WebSocket streaming (5 hubs, Azure Web PubSub compatible)
Per-API-key playback position tracking
Hot reload data dates without server restart
Sync Broadcast System for external service synchronization
CLI for downloading historical data from GexBot
Daemon for scheduled automatic downloads
Push notifications via ntfy.sh on download completion
Docker deployment ready

Quick Start

Prerequisites

Go 1.24+ - verify with go version
just command runner - verify with just --version
Docker - verify with docker --version
GexBot API key with Quant Subscription (required for downloading data)

1. Clone and Configure Environment

git clone [email protected]:dgnsrekt/gexbot-faker-api.git
cd gexbot-faker-api

# Copy example env and add your GexBot API key
cp gexbot.example.env .env
# Edit .env and set GEXBOT_API_KEY=your_api_key_here

2. Create Download Config

Create a custom config to select your tickers, packages, and categories:

cp configs/default.yaml configs/custom.yaml

Edit configs/custom.yaml to customize:

tickers: Enable/disable tickers (SPX, NDX, SPY, etc.)
packages: Enable/disable data packages (state, classic, orderflow)
categories: Enable/disable specific data types within each package

3. Download Initial Data

# Download last 7 days of market data (adjust number as needed)
just download-lookback 7

Note: The downloader automatically skips weekends and market holidays.

4. Start the API

With Docker (Recommended):

just up        # Start API server and daemon
just logs      # View logs

Or run locally:

just serve-gex-faker              # Build and run server
open http://localhost:8080/docs   # Access API docs

Components

API Server

REST API serving historical GEX data with sequential playback per API key.

Endpoints (see /docs for full reference):

/{ticker}/classic/{aggregation} - Classic GEX chain data
/{ticker}/state/{type} - State GEX profiles and Greeks
/{ticker}/orderflow/orderflow - Orderflow metrics
/available-data/{date} - Discover available data for a date
/download/{date}/{ticker}/links - Get all download links for a date/ticker
/download/{date}/{ticker}/classic/{aggregation} - Download classic data
/download/{date}/{ticker}/state/{type} - Download state data
/download/{date}/{ticker}/orderflow - Download orderflow data
/negotiate - WebSocket connection URLs
/health, /tickers, /available-dates - Server info
/reload-date - Hot reload data for a different date
/reset-cache - Reset playback positions
/seek-to-timestamp - Seek positions to a specific timestamp

Key behavior: Each API key maintains independent playback position. Data advances on each request.

Hot Reload

Switch data dates at runtime without restarting the server:

# Reload to a different date
curl -X POST http://localhost:8080/reload-date \
  -H "Content-Type: application/json" \
  -d '{"date": "2025-12-04"}'

Response:

{
  "status": "success",
  "previous_date": "2025-11-28",
  "new_date": "2025-12-04",
  "loaded_at": "2025-12-27T15:30:00Z",
  "files_loaded": 45
}

Behavior:

Validates the date exists before unloading current data
Pauses WebSocket streaming during reload
Resets all cache positions to 0 for clean playback
Returns 400 for invalid/missing dates, 409 if reload already in progress

Cache Management

Control playback positions for testing and synchronization.

Reset positions:

# Reset all positions for an API key
curl -X POST "http://localhost:8080/reset-cache?key=mykey"

# Reset ALL positions (all keys)
curl -X POST http://localhost:8080/reset-cache

Seek to timestamp:

# Seek all positions for an API key to a specific timestamp
curl -X POST http://localhost:8080/seek-to-timestamp \
  -H "Content-Type: application/json" \
  -d '{"timestamp": 1767191500, "key": "mykey"}'

Response:

{
  "status": "success",
  "message": "Seeked 45 positions to timestamp 1767191500",
  "positions_set": 45,
  "details": [
    {"data_key": "SPX/classic/gex_zero", "index": 98, "timestamp": 1767191500}
  ]
}

Use case: Sync with external services by seeking to a broadcast timestamp from the Sync Broadcast System.

WebSocket Streaming

Real-time data streaming via 5 specialized hubs:

Hub	Data Type
`orderflow`	DEX, GEX, convexity, vanna, charm metrics
`classic`	Classic GEX chain
`state_gex`	State GEX profiles
`state_greeks_zero`	Greeks (0DTE)
`state_greeks_one`	Greeks (1DTE+)

See WEBSOCKET.md for protocol details.

Sync Broadcast System

SSE-based market time broadcast for synchronizing external services with the faker's playback position. External services subscribe to receive position updates and can seek their own data to match.

Endpoint: GET /sync/stream?key={api-key}

Enable via environment:

SYNC_BROADCAST_SYSTEM_ENABLED=true

Example subscription:

curl -N "http://localhost:8080/sync/stream?key=my-api-key"

Response (SSE stream):

event: snapshot
id: 1
data: {"broadcaster_id":"gexbot-faker","data_date":"2025-12-05","cache_mode":"exhaust","timestamp":1766870060074,"sequence":1,"positions":[{"cache_key":"SPX/classic/my-a****","index":42,"data_length":23375,"data_timestamp":1764945003,"exhausted":false}]}

event: batch
id: 2
data: {...positions updated every interval...}

Position fields:

cache_key: Cache key path with masked API key (REST or WebSocket format)
index: Current playback position
data_length: Total records available
data_timestamp: Unix timestamp from the data at current position
exhausted: True if position has reached end (exhaust mode only)

Use case: Other faker-like services with timestamp-indexed data can subscribe and seek their own data to match the broadcaster's market time.

Python WebSocket Client

Use quant-python-sockets to connect to the WebSocket feeds.

To use with faker API, change main.py line 36:

# Original (production):
BASE_URL = "https://api.gexbot.com"

# Change to (faker):
BASE_URL = "http://localhost:8080"

Then run:

export GEXBOT_API_KEY=test123  # Faker accepts any key
python main.py

Downloader CLI

Download historical data from the GexBot API.

# Single date
./bin/gexbot-downloader download 2025-11-14

# Date range
./bin/gexbot-downloader download 2025-11-01 2025-11-14

# Custom tickers/packages
./bin/gexbot-downloader download --tickers SPX,NDX --packages state 2025-11-14

# Preview (dry run)
./bin/gexbot-downloader download --dry-run 2025-11-14

EOD reports are retained as the canonical compressed archive under data/eod/YYYY-MM-DD/TICKER/. JSONL is materialized only when the server needs a date.

# Convert and verify existing JSONL; --prune removes it after verification
./bin/gexbot-downloader eod pack 2026-07-17 --ticker SPY --prune

# Verify every archive or restore a date for replay
./bin/gexbot-downloader eod verify all
./bin/gexbot-downloader eod materialize 2026-07-17

Daemon Service

Automated EOD downloads with market-day awareness. Reports are retried every five minutes; after 8:00 PM ET the daemon falls back to individual downloads.

Variable	Default	Description
`DAEMON_SCHEDULE_HOUR`	17	First EOD attempt (ET)
`DAEMON_SCHEDULE_MINUTE`	5	First EOD attempt minute
`DAEMON_TIMEZONE`	America/New_York	Timezone
`DAEMON_RUN_ON_STARTUP`	true	Check/download on start

Push Notifications (ntfy)

Both the daemon and CLI downloader support push notifications via ntfy.sh when downloads complete or fail.

Variable	Default	Description
`NTFY_ENABLED`	false	Enable push notifications
`NTFY_SERVER`	https://ntfy.sh	ntfy server URL (supports self-hosted)
`NTFY_TOPIC`	(required)	Topic name for notifications
`NTFY_PRIORITY`	default	Priority: min, low, default, high, urgent
`NTFY_TAGS`	package	Comma-separated emoji tags
`NTFY_TOKEN`	(optional)	Access token for private topics

Quick setup:

# In .env
NTFY_ENABLED=true
NTFY_TOPIC=my-gexbot-downloads

Subscribe to notifications at https://ntfy.sh/my-gexbot-downloads or use the ntfy app.

Configuration

Server Environment Variables

Variable	Default	Description
`PORT`	8080	HTTP server port
`DATA_DIR`	./data	Data directory path
`DATA_DATE`	latest	Date to load (YYYY-MM-DD or "latest")
`DATA_MODE`	memory	`memory` (fast) or `stream` (low RAM)
`CACHE_MODE`	exhaust	`exhaust` (404 at end) or `rotation` (loop)
`WS_ENABLED`	true	Enable WebSocket streaming
`WS_STREAM_INTERVAL`	1s	Broadcast interval
`WS_GROUP_PREFIX`	blue	Prefix for WebSocket group names
`SYNC_BROADCAST_SYSTEM_ENABLED`	false	Enable SSE sync broadcast endpoint
`SYNC_BROADCAST_SYSTEM_ID`	hostname	Broadcaster identifier
`SYNC_BROADCAST_SYSTEM_INTERVAL`	1s	Position broadcast interval

Downloader Configuration

Create configs/default.yaml or set GEXBOT_API_KEY:

api:
  api_key: "${GEXBOT_API_KEY}"
  timeout_sec: 300
  retry_count: 3

download:
  workers: 3
  rate_per_second: 2
  resume_enabled: true

output:
  directory: "data"
  auto_convert_to_jsonl: true

Data Reference

Packages and Categories

Package	Categories
state	gex_full, gex_zero, gex_one, delta_zero, gamma_zero, delta_one, gamma_one, vanna_zero, vanna_one, charm_zero, charm_one
classic	gex_full, gex_zero, gex_one
orderflow	orderflow

Tickers

Indexes: SPX, NDX, RUT, VIX ETFs: SPY, QQQ, IWM Futures: ES_SPX, NQ_NDX

Data Directory Structure

data/
├── eod/
│   └── 2025-11-14/
│       └── SPX/
│           ├── eod_report_SPX_2025-11-14.zip
│           └── eod_report_SPX_2025-11-14.zip.manifest.json
└── 2025-11-14/                 # materialized replay cache
    └── SPX/
        ├── classic/gex_zero.jsonl
        ├── state/delta_zero.jsonl
        └── orderflow/orderflow.jsonl

The ZIP is the canonical archive. Its members use GEXBot's TICKER/package/category/date_ticker_package_category.json.gz layout and contain JSON arrays. JSONL remains uncompressed because stream playback needs record offsets that gzip cannot provide efficiently.

Development

# Build
just build                          # Downloader
just build-gex-faker                # Server

# Code generation
just generate-gex-faker-api-spec    # OpenAPI → Go
just generate-protos                # Protobuf → Go

# Test and lint
just test
just lint

# Docker
just up                             # Start containers
just down                           # Stop containers
just logs                           # View logs

License

See LICENSE file.

Contact Information

Telegram = Twitter = Tradingview = Discord = @dgnsrekt

Email = [email protected]

Note: It may take me a few days to reply.

Name		Name	Last commit message	Last commit date
Latest commit History 93 Commits
api		api
cmd		cmd
compatibility		compatibility
configs		configs
docs		docs
graphify-out		graphify-out
internal		internal
proto		proto
tools		tools
.dockerignore		.dockerignore
.gitignore		.gitignore
.golangci.yml		.golangci.yml
.pre-commit-config.yaml		.pre-commit-config.yaml
CLAUDE.md		CLAUDE.md
DRY_VIOLATIONS_REPORT.md		DRY_VIOLATIONS_REPORT.md
Dockerfile		Dockerfile
LICENSE		LICENSE
README.md		README.md
WEBSOCKET.md		WEBSOCKET.md
docker-compose.yml		docker-compose.yml
gexbot.example.env		gexbot.example.env
go.mod		go.mod
go.sum		go.sum
justfile		justfile

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

GEX Faker API

Features

Quick Start

Prerequisites

1. Clone and Configure Environment

2. Create Download Config

3. Download Initial Data

4. Start the API

Components

API Server

Hot Reload

Cache Management

WebSocket Streaming

Sync Broadcast System

Python WebSocket Client

Downloader CLI

Daemon Service

Push Notifications (ntfy)

Configuration

Server Environment Variables

Downloader Configuration

Data Reference

Packages and Categories

Tickers

Data Directory Structure

Development

License

Contact Information

About

Uh oh!

Releases

Packages

Uh oh!

Contributors

Uh oh!

Languages

Folders and files

Latest commit

History

Repository files navigation

GEX Faker API

Features

Quick Start

Prerequisites

1. Clone and Configure Environment

2. Create Download Config

3. Download Initial Data

4. Start the API

Components

API Server

Hot Reload

Cache Management

WebSocket Streaming

Sync Broadcast System

Python WebSocket Client

Downloader CLI

Daemon Service

Push Notifications (ntfy)

Configuration

Server Environment Variables

Downloader Configuration

Data Reference

Packages and Categories

Tickers

Data Directory Structure

Development

License

Contact Information

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Packages