Polyweather is a weather data pipeline + AI-based trading bot that combines:
- Weather data ingestion — METAR (WSSS Changi Airport), PWS (ISINGA249), global NWP (ACCESS, ARPEGE, ECMWF, GEM, GFS, ICON), and NWP Ensemble (GFS & ECMWF IFS via Open-Meteo)
- Temperature prediction — NWP Bias Corrector v2.0 (Stacked Ensemble: XGBoost + LightGBM + Ridge meta-learner, residual error correction of global NWP models) + Two-Harmonic (LightGBM, daily Tmax)
- AI Trading Agent — LLM Decision Service based on Xiaomi MiMo analyzing weather and Polymarket conditions with real-time trigger support, anomaly detection, and portfolio monitoring
- Trading Bot — Order execution to Polymarket CLOB API (dry-run / testnet / live) with Implied Price Refresher
- Terminal dashboard — real-time Textual-based
- Automated backup to Google Drive via rclone
- System Architecture
- Requirements
- First Time Setup
- Mandatory Configuration Before Deploy
- Running the Pipeline
- Step-by-Step Deploy Flow
- All Environment Variables Reference
- Auto-Redeem Winnings
- Automated Database Backup
- Docker Compose Mode
- Utility & Monitoring
┌─────────────────────────────────────────────────────────────────────────┐
│ POLYWEATHER │
├─────────────────────────────┬───────────────────────────────────────────┤
│ DATA LAYER │ AI REASONING LAYER │
│ │ │
│ METAR scraper ──┐ │ ┌── decision_prompt.md (template) │
│ PWS scraper ──┼──▶ PostgreSQL ◀── Two-Harmonic Model (LightGBM) │
│ NWP scraper ──┘ │ Redis │ NWP Bias Corrector (XGBoost) │
│ │ │ │ │
│ market_discovery ───┼───────┼─────────────────┤ │
│ poly-websocket ────┘ trader:market:* ▼ │
│ LLM Decision Service │
│ (Xiaomi MiMo v2.5-pro agent) │
│ │ │
│ ▼ │
│ decision:recommendation │
│ (Redis channel) │
├─────────────────────────────────────────────┼───────────────────────────┤
│ TRADING LAYER │ │
│ ▼ │
│ clob_client_v2 ◀── decision:recommendation │ │
│ │ Risk Manager (spread, │ │
│ │ liquidity, daily loss, │ │
│ │ confidence checks) │ │
│ ├── dry-run → log only │ │
│ ├── amoy → Polygon Amoy testnet │ │
│ └── market → Polymarket mainnet │ │
│ │ │ │
│ ▼ │ │
│ position_redeemer │ notifier (Discord Bot) │
│ (auto-redeem kemenangan) ────────┼──▶ (Notifikasi & Status) │
└─────────────────────────────────────────────┴───────────────────────────┘
Weather scraper → raw_metar_observations / raw_pws_observations (PostgreSQL Bronze)
↓ (Automatic DB trigger)
metar_clean / pws_clean (PostgreSQL Silver)
↓
nwp_ingestor.py → nwp_forecasts → nwp_daily
↓
model_nwp_bias.py → nwp_bias_predictions / nwp_bias_history
model_twoharmonic.py → forecast_predictions
↓
LLM Decision Service (collect data + send to Xiaomi MiMo)
↓
decision:recommendation (Redis pub/sub)
↓
clob_client_v2.py → Polymarket CLOB API
↓
trader:execution_ack:{decision_id} (Redis, TTL 60s)
↓
LLM Decision Service (receive ACK, close CoT)
↓
llm_decisions (PostgreSQL — audit trail & win/loss tracking)
| Key | Writer | Reader | Content |
|---|---|---|---|
trader:market:info |
market_discovery | llm-decision, anomaly_detector | Event info + sub-market list |
trader:market:detail |
market_discovery | llm-decision, tool_registry | Details per bracket (price, orderbook) |
trader:market:orderbook |
market_discovery | tool_registry (get_orderbook) | Order book per token {token_id: metrics} |
trader:market:prices |
market_discovery | anomaly_detector | Price per token {token_id: price} |
trader:market:all_tokens |
market_discovery | poly-websocket | All active tokens (YES+NO) |
trader:market:live:{token_id} |
poly-websocket | llm-decision (staleness), dashboard | Live snapshot per token |
trader:market:history:{token_id} |
poly-websocket | tool_registry (get_price_history) | Tick history per token (max 200) |
trader:history |
clob_client_v2 | dashboard | All transactions log (max 1000) |
trader:position |
clob_client_v2 | llm-decision (data_collector) | Current open positions |
trader:daily_pnl |
clob_client_v2 (RiskManager) | llm-decision (data_collector) | Daily P&L + risk manager state |
decision:recommendation |
llm-decision (publisher) | clob_client_v2 | Trading recommendation (pub/sub channel) |
trader:execution_ack:{id} |
clob_client_v2 | llm-decision (Phase B ACK) | Order execution confirmation (TTL 60s) |
model:nwp_bias:prediction |
model_nwp_bias | dashboard, llm-decision | Today's NWP bias correction prediction |
model:nwp_bias:status |
model_nwp_bias | dashboard | Model training status |
modeler:nwp_bias:update |
model_nwp_bias | llm-decision (listener) | Prediction update event (pub/sub) |
Polyweather uses a PostgreSQL database schema organized into several layers (Medallion Architecture) to ensure data integrity from raw sources to model outputs.
========================================================================================================================
POLYWEATHER DATABASE FULL SCHEMA (COMPLETE)
========================================================================================================================
[ LAYER 0: METADATA ]
+--------------------------------+ +--------------------------------+
| stations | | pws_sensors |
+--------------------------------+ +--------------------------------+
| station_id (PK) VARCHAR10 | | sensor_id (PK) VARCHAR20 |
| station_name TEXT | | sensor_name TEXT |
| latitude NUMERIC | | latitude NUMERIC |
| longitude NUMERIC | | longitude NUMERIC |
| elevation_m INTEGER | | elevation_m INTEGER |
| timezone VARCHAR50 | | timezone VARCHAR50 |
| is_active BOOLEAN | | interval_minutes INTEGER |
| created_at TIMESTAMPTZ| | is_active BOOLEAN |
| updated_at TIMESTAMPTZ| | created_at TIMESTAMPTZ|
+---------------+----------------+ | updated_at TIMESTAMPTZ|
^ +---------------+----------------+
| ^
| (Foreign Key) | (Foreign Key)
+-------------------+ +----------------+
| |
+-----------------------------------+---+--------------------------------+
| station_sensor_map |
+------------------------------------------------------------------------+
| station_id (PK, FK) VARCHAR10 |
| sensor_id (PK, FK) VARCHAR20 |
| is_primary BOOLEAN |
| created_at TIMESTAMPTZ |
+------------------------------------------------------------------------+
[ LAYER 1: BRONZE (RAW) ] [ LAYER 6: NWP (MODEL CUACA GLOBAL) ]
+--------------------------------+ +--------------------------------+
| raw_metar_observations | | nwp_models |
+--------------------------------+ +--------------------------------+
| station_id (PK, FK) VARCHAR10 | | model_id (PK) VARCHAR20 |
| observation_time (PK) TIMESTAMPTZ| | model_name TEXT |
| local_time TEXT | | provider TEXT |
| raw_text TEXT | | resolution_km NUMERIC |
| report_type VARCHAR20 | | is_active BOOLEAN |
| temp_c TEXT | | notes TEXT |
| dewpoint_c TEXT | | created_at TIMESTAMPTZ|
| pressure_mb TEXT | +---------------+----------------+
| wind_dir TEXT | ^
| wind_speed_kt TEXT | | (Foreign Key)
| wind_gust_kt TEXT | +---------------+----------------+
| wind_dir_var VARCHAR20 | | nwp_forecasts |
| visibility TEXT | +--------------------------------+
| cloud_layers TEXT | | nwp_id (PK) BIGSERIAL |
| wx_string TEXT | | model_id (FK) VARCHAR20 |
| flight_category VARCHAR10 | | station_id (FK) VARCHAR10 |
| auto TEXT | | forecast_time TIMESTAMPTZ|
| recent_weather TEXT | | forecast_time_local TIMESTAMP |
| rvr TEXT | | forecast_date DATE |
| remarks TEXT | | forecast_hour SMALLINT |
| rmk_indicators TEXT | | latitude NUMERIC |
| longitude NUMERIC | | longitude NUMERIC |
| elevation_m TEXT | | temperature_2m NUMERIC |
| source_file TEXT | | relative_humidity_2m NUMERIC |
| ingested_at TIMESTAMPTZ| | ingested_at TIMESTAMPTZ|
+--------------------------------+ +--------------------------------+
+--------------------------------+ +--------------------------------+
| raw_pws_observations | | nwp_data_quality_log |
+--------------------------------+ +--------------------------------+
| sensor_id (PK, FK) VARCHAR20 | | log_id (PK) BIGSERIAL |
| full_time (PK) TIMESTAMPTZ| | station_id (FK) VARCHAR10 |
| data_date DATE | | model_id (FK) VARCHAR20 |
| time_str VARCHAR10 | | forecast_date DATE |
| temp_c TEXT | | flag_type VARCHAR30 |
| heat_index_c TEXT | | reason TEXT |
| dew_point_c TEXT | | created_at TIMESTAMPTZ|
| humidity_pct TEXT | +--------------------------------+
| wind_dir_deg TEXT |
| wind_speed_kmh TEXT |
| gust_kmh TEXT |
| pressure_hpa TEXT |
| rain_rate_mmh TEXT |
| rain_total_mm TEXT |
| solar_wm2 TEXT |
| source_file TEXT |
| ingested_at TIMESTAMPTZ|
+--------------------------------+
[ LAYER 2: SILVER (CLEAN) ] [ LAYER 6: NWP DAILY (FEATURES) ]
+--------------------------------+ +--------------------------------+
| metar_clean | | nwp_daily |
+--------------------------------+ +--------------------------------+
| station_id (PK, FK) VARCHAR10 | | model_id (PK, FK) VARCHAR20 |
| observation_time (PK) TIMESTAMPTZ| | station_id (PK, FK) VARCHAR10 |
| local_time TIMESTAMP | | forecast_date (PK) DATE |
| obs_date DATE | | tmax_nwp NUMERIC |
| raw_text TEXT | | tmin_nwp NUMERIC |
| report_type VARCHAR20 | | tmean_nwp NUMERIC |
| temp_c NUMERIC | | t08_nwp NUMERIC |
| dewpoint_c NUMERIC | | rh08_nwp NUMERIC |
| pressure_mb NUMERIC | | td08_nwp NUMERIC |
| wind_dir_deg NUMERIC | | pressure08_nwp NUMERIC |
| wind_is_vrb BOOLEAN | | cloud_morning_nwp NUMERIC |
| wind_speed_kt NUMERIC | | cape_morning_nwp NUMERIC |
| wind_gust_kt NUMERIC | | precip_morning_nwp NUMERIC |
| wind_dir_var VARCHAR20 | | solar_max_nwp NUMERIC |
| visibility NUMERIC | | solar_mean_nwp NUMERIC |
| cloud_layers TEXT | | wind_max_nwp NUMERIC |
| wx_string TEXT | | wind_mean_nwp NUMERIC |
| flight_category VARCHAR10 | | cape_max_nwp NUMERIC |
| recent_weather TEXT | | vpd_mean_nwp NUMERIC |
| is_speci BOOLEAN | | freezing_level_mean NUMERIC |
| is_auto BOOLEAN | | n_hours SMALLINT |
| is_cor BOOLEAN | | created_at TIMESTAMPTZ|
| has_cb_or_tcu BOOLEAN | +--------------------------------+
| created_at TIMESTAMPTZ|
+--------------------------------+ [ LAYER 6b: NWP ENSEMBLE ]
+--------------------------------+
+--------------------------------+ | nwp_ensemble_forecasts |
| pws_clean | +--------------------------------+
+--------------------------------+ | model_id (PK, FK) VARCHAR20 |
| sensor_id (PK, FK) VARCHAR20 | | member_id (PK) SMALLINT |
| full_time (PK) TIMESTAMPTZ| | station_id (PK, FK) VARCHAR10 |
| obs_date DATE | | forecast_time (PK) TIMESTAMPTZ|
| time_local TIME | | model_run_time (PK) TIMESTAMPTZ|
| temp_c NUMERIC | | lead_time_hours SMALLINT |
| heat_index_c NUMERIC | | temperature_2m NUMERIC |
| dew_point_c NUMERIC | | relative_humidity_2m NUMERIC |
| humidity_pct NUMERIC | | dew_point_2m NUMERIC |
| wind_dir_deg NUMERIC | | apparent_temperature NUMERIC |
| wind_speed_kmh NUMERIC | | precipitation NUMERIC |
| gust_kmh NUMERIC | | rain NUMERIC |
| pressure_hpa NUMERIC | | wind_speed_10m NUMERIC |
| rain_rate_mmh NUMERIC | | wind_gusts_10m NUMERIC |
| rain_total_mm NUMERIC | | pressure_msl NUMERIC |
| solar_wm2 NUMERIC | | ingested_at TIMESTAMPTZ|
| created_at TIMESTAMPTZ| +--------------------------------+
+--------------------------------+
+--------------------------------+
[ LAYER 5: MODEL OUTPUT ] | nwp_ensemble_stats |
+--------------------------------+ +--------------------------------+
| model_runs | | station_id (PK, FK) VARCHAR10 |
+--------------------------------+ | forecast_time (PK) TIMESTAMPTZ|
| run_id (PK) BIGSERIAL | | model_run_time (PK) TIMESTAMPTZ|
| station_id (FK) VARCHAR10 | | temp_mean NUMERIC |
| model_type VARCHAR20 | | temp_std NUMERIC |
| model_name TEXT | | temp_min NUMERIC |
| model_version TEXT | | temp_max NUMERIC |
| training_date_min DATE | | temp_p10 NUMERIC |
| training_date_max DATE | | temp_p25 NUMERIC |
| n_features INTEGER | | temp_p50 NUMERIC |
| n_train_samples INTEGER | | temp_p75 NUMERIC |
| rmse NUMERIC | | temp_p90 NUMERIC |
| mae NUMERIC | | precip_mean NUMERIC |
| feature_list JSONB | | n_members SMALLINT |
| model_params JSONB | | ingested_at TIMESTAMPTZ|
| created_at TIMESTAMPTZ| +--------------------------------+
+---------------+----------------+
^ [ LAYER 5b: NWP BIAS CORRECTOR ]
| (Foreign Key) +--------------------------------+
+---------------+----------------+ | nwp_bias_predictions |
| forecast_predictions | +--------------------------------+
+--------------------------------+ | pred_id (PK) BIGSERIAL |
| pred_id (PK) BIGSERIAL | | station_id (FK) VARCHAR10 |
| run_id (FK) BIGINT | | forecast_date (UQ) DATE |
| station_id (FK) VARCHAR10 | | predicted_at TIMESTAMPTZ|
| forecast_date (UQ) DATE | | nwp_ensemble_tmax NUMERIC |
| predicted_at TIMESTAMPTZ| | nwp_spread_tmax NUMERIC |
| pred_tmax NUMERIC | | residual_pred NUMERIC |
| pred_hour NUMERIC | | corrected_tmax NUMERIC |
| actual_tmax NUMERIC | | confidence_pct NUMERIC |
| actual_hour NUMERIC | | model_mae NUMERIC |
| blend_weight_main NUMERIC | | model_name TEXT |
| blend_weight_fallback NUMERIC | +--------------------------------+
| risk_signal NUMERIC |
| ci_low NUMERIC | +--------------------------------+
| ci_high NUMERIC | | nwp_bias_history |
+--------------------------------+ +--------------------------------+
| bias_id (PK) BIGSERIAL |
+--------------------------------+ | station_id (FK) VARCHAR10 |
| nowcast_dashboard_history | | forecast_date (UQ) DATE |
+--------------------------------+ | nwp_ensemble_tmax NUMERIC |
| history_id (PK) BIGSERIAL | | nwp_spread_tmax NUMERIC |
| station_id (FK) VARCHAR10 | | t08_metar NUMERIC |
| predicted_at TIMESTAMPTZ| | rh08_metar NUMERIC |
| last_metar_time TIMESTAMPTZ| | cloud08_oktas NUMERIC |
| expected_target_time TIMESTAMPTZ| | precip_flag SMALLINT |
| metar_temp_at_prediction NUMERIC| | pressure08_mb NUMERIC |
| pred_temp NUMERIC | | t_pagi_pws NUMERIC |
| actual_metar_time TIMESTAMPTZ| | rh_pagi_pws NUMERIC |
| actual_temp NUMERIC | | rain_6h_pws NUMERIC |
| abs_error NUMERIC | | actual_tmax NUMERIC |
| resolution_status VARCHAR16 | | residual_actual NUMERIC |
| confidence_pct NUMERIC | | residual_pred NUMERIC |
| interval_approx NUMERIC | | corrected_tmax NUMERIC |
| model_name TEXT | | abs_error NUMERIC |
| runtime_score NUMERIC | | created_at TIMESTAMPTZ|
| updated_at TIMESTAMPTZ| +--------------------------------+
+--------------------------------+
+--------------------------------+
[ LAYER 7: LLM DECISION SERVICE ] | model_versions |
+--------------------------------+ +--------------------------------+
| llm_decisions | | version_id (PK) UUID |
+--------------------------------+ | model_name VARCHAR |
| decision_id (PK) UUID | | created_at TIMESTAMPTZ|
| created_at TIMESTAMPTZ| | training_start_date DATE |
| trigger_type VARCHAR30 | | training_end_date DATE |
| token_id VARCHAR255| | feature_list JSONB |
| action VARCHAR10 | | hyperparameters JSONB |
| size_usdc NUMERIC | | metrics JSONB |
| price_limit NUMERIC | | artifact_path VARCHAR |
| confidence INTEGER | | is_active BOOLEAN |
| reasoning_summary TEXT | +--------------------------------+
| pred_tmax_snapshot NUMERIC |
| nwp_ensemble_snapshot NUMERIC | +--------------------------------+
| market_price_snapshot NUMERIC | | nwp_bias_performance |
| p_weather NUMERIC | +--------------------------------+
| p_market NUMERIC | | station_id (FK) VARCHAR10 |
| edge NUMERIC | | forecast_date DATE |
| tmax_estimate_c NUMERIC | | model_version_id (FK) UUID |
| model_confidence VARCHAR | | predicted_tmax NUMERIC |
| outcome VARCHAR10 | | actual_tmax NUMERIC |
| resolved_at TIMESTAMPTZ| | abs_error NUMERIC |
| squared_error NUMERIC |
| within_80pct_interval BOOLEAN |
| crps NUMERIC |
| created_at TIMESTAMPTZ|
+--------------------------------+
| Platform | Minimum |
|---|---|
| OS | Windows 10/11, Ubuntu 22.04+, Debian 12+, or WSL2 |
| Docker | Docker Engine ≥ 24 + Compose plugin |
| Python | 3.12+ with pip |
| rclone | ≥ v1.65 (for Google Drive backup, auto-install via setuplinux.sh) |
| pg_dump | PostgreSQL client tools (auto-install via setuplinux.sh) |
The repository uses dual virtual environments to prevent binary conflicts between OS:
- Windows →
.venv_windows/ - Linux/WSL →
.venv_linux/
The deployment script creates and populates the venv automatically.
| Service | Main Libraries |
|---|---|
services/Scraping |
requests, pandas, metar, pytz, psycopg2-binary |
services/modeler |
numpy, pandas, scikit-learn, lightgbm, xgboost, psycopg2-binary |
services/trader |
redis, websocket-client, py-clob-client |
services/llm-decision |
openai, redis, psycopg2-binary, hypothesis, pytest |
Run once on a new server. Installs Docker, rclone, pg_dump, and setups Google Drive:
chmod +x scripts/linux_auto/setuplinux.sh
bash scripts/linux_auto/setuplinux.shWhat the script does:
| Step | Action |
|---|---|
| 1 | Detects distro (Ubuntu/Debian/Fedora/Arch) |
| 2 | Installs Docker Engine + Compose plugin |
| 3 | Enables Docker service, adds user to docker group |
| 4 | Installs rclone (official installer) |
| 5 | Installs postgresql-client (pg_dump) |
| 6 | Configures rclone → Google Drive (stops when OAuth URL appears) |
When the OAuth URL appears: copy → open in browser → login → authorize → return to terminal → Enter.
Skip options if some are already installed:
bash scripts/linux_auto/setuplinux.sh --skip-docker
bash scripts/linux_auto/setuplinux.sh --skip-rclone
bash scripts/linux_auto/setuplinux.sh --skip-gdriveNo special setup required. Ensure Docker Desktop and Python 3.12+ are installed, then deploy directly. The PowerShell script will create the venv and install dependencies automatically.
This is the most important section. Without this configuration, the trading bot and AI agent will not run.
cp .env.example .envOpen .env with your preferred text editor and fill in all the required variables.
The AI agent (LLM Decision Service) uses Xiaomi MiMo v2.5-Pro as its reasoning engine.
Steps:
- Register at platform.xiaomimimo.com
- Choose plan: PAYG (pay as you go) or TokenPlan (monthly subscription)
- Go to Developer Console → generate a new API Key
- Fill in
.env:
MIMO_API_KEY=sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxx
# Choose Base URL according to account type:
# PAYG (pay as you go):
# MIMO_BASE_URL=https://api.xiaomimimo.com/v1
#
# TokenPlan (monthly subscription — Singapore region):
MIMO_BASE_URL=https://token-plan-sgp.xiaomimimo.com/v1
#
# TokenPlan (monthly subscription — China region):
# MIMO_BASE_URL=https://token-plan-cn.xiaomimimo.com/v1
MIMO_MODEL=mimo-v2.5-proMANDATORY — the
llm-decisionservice will not start withoutMIMO_API_KEY.IMPORTANT: TokenPlan and PAYG use DIFFERENT endpoints. If you subscribe monthly (TokenPlan), you MUST use the
token-plan-sgportoken-plan-cnendpoint, notapi.xiaomimimo.com. Wrong endpoint = API key rejected.
The METAR data scraper requires an API key from CheckWX to fetch aviation weather reports.
Steps:
- Register for a free account at api.checkwx.com
- Once registered, generate an API key from your dashboard.
- Fill in
.env:
CHECKWX_API_KEY=your_checkwx_api_key_hereMANDATORY — the
metar_WXaggregatorscraper requires this key to function properly.
The trading bot requires a Polygon wallet to interact with the Polymarket CLOB API.
Use MetaMask, Rabby, or other EVM wallets:
- Open your wallet → Create a new account
- Give it a name, e.g., "Polyweather Trading"
- Go to Account details → Export Private Key
- Enter your wallet password
- Copy the private key (starts with
0x)
IMPORTANT: Never use a main wallet/hot wallet containing large assets. Use a dedicated wallet with a limited balance according to the established bankroll.
Polymarket V2 (since 2026) uses a Proxy/Deposit Wallet architecture. You MUST create a deposit wallet before the bot can trade:
- Visit the Polymarket website (or testnet.polymarket.com if using Amoy. make sure you have already claimed the Free Testnet Tokens (Faucet) beforehand).
- Connect your new wallet (e.g., via Rabby/MetaMask)
- Polymarket will automatically deploy the Deposit Wallet smart contract (ERC-1967 Proxy) for your wallet (EOA).
- (Highly Recommended for market mode) Make a small deposit/withdraw on the website to ensure your proxy wallet is fully active.
The bot will automatically detect your Deposit Wallet address via API during initialization.
Inside the .env file, the configurations for Mainnet and Amoy Testnet are written side-by-side (they don't overwrite each other). You only need to fill in the private key in the appropriate variable:
# Fill this if you want to trade on Live Mainnet (Real Money)
POLYGON_PRIVATE_KEY=0x_your_mainnet_private_key
# Fill this if you want to trade on Amoy Testnet (Play Money/Testing)
AMOY_PRIVATE_KEY=0x_your_testnet_private_keyNote: All smart contract addresses (like AMOY_USDC_ADDRESS and AMOY_CTF_EXCHANGE) and default Chain IDs are already configured automatically in .env.example and you don't need to change them unless there's a network update.
Polymarket creates new markets every day. market_discovery.py will automatically find today's token IDs. But if you want to set them manually:
# Leave empty initially — market_discovery.py will fill it automatically
POLYMARKET_TOKEN_ID=
POLYMARKET_CONDITION_ID=Token IDs can be found by running:
python services/trader/src/market_discovery.pyThere are three modes available. Default is dry-run (safe, no real orders). You can select the mode by passing the --mode (or -Mode) parameter when running the deploy script.
| Mode | Description | Real Money? |
|---|---|---|
dry-run |
Paper trading. Orders are only logged and saved to Redis. No connection to Polymarket. | ❌ No |
amoy |
Polygon Amoy Testnet. Real transactions on the test blockchain, tokens have no value. | ❌ No |
market |
Live mainnet. Real orders to Polymarket with actual USDC. | ✅ YES |
Suitable for testing AI prediction logic, weather data scraping, and order book simulation without financial risk.
- Environment Configuration:
- Ensure the
MIMO_API_KEYandDATABASE_URLvariables are filled in the.envfile. - Polygon Wallet / Private Key is not required for this mode.
- Ensure the
- Run Pipeline:
- Linux:
bash scripts/deploy.sh --mode dry-run - Windows:
.\scripts\deploy.ps1 -Mode dry-run
- Linux:
- Monitoring:
- The terminal dashboard appears automatically to view simulated open positions and real-time P&L based on actual Polymarket market movements. (The LLM Decision Agent will run automatically in the background).
Suitable for validating technical blockchain workflows (RPC connection, transaction signing with private keys, token authorization, gas fees) safely using free test tokens.
- Setup Wallet & Amoy Network:
- Create a new EVM wallet specifically for testnet (e.g., in MetaMask). IMPORTANT: Do not use your main wallet.
- Add the Amoy network to your wallet if it's not already there:
- Network Name: Polygon Amoy Testnet
- RPC URL:
https://rpc-amoy.polygon.technology - Chain ID:
80002 - Currency Symbol: POL (or MATIC)
- Export the Private Key from that wallet and save it in the
AMOY_PRIVATE_KEYvariable in the.envfile.
2. Get MATIC/POL Testnet (For Gas Fees):
- Visit the Polygon Faucet or Alchemy Amoy Faucet.
- IMPORTANT - Select the Correct Network: On the Faucet page, there will be many blockchain options. Make sure you select Polygon Amoy (or Amoy). Do not select Mumbai (because it's deprecated) or other testnet networks like Sepolia/Goerli.
- Enter your new EOA wallet address, complete the captcha, and click the request button to get free gas tokens (POL/MATIC).
-
Get Testnet USDC (For Trading Capital):
-
On the Amoy network, Polymarket contracts use a specific Mock USDC (
0x9c4e1703476e875070ee25b56a58b008cfb8fa78) as collateral, not the official USDC from Circle Faucet. -
You don't need to use an external web faucet for USDC. Simply mint those tokens directly by running the following utility script (make sure your wallet already has testnet POL/MATIC balance for gas fees):
python services/trader/testing/poly-amoy.py --mint
(This script will call the
mintfunction directly on the Mock USDC smart contract and fill your wallet balance with 1000 USDC for free).[!IMPORTANT] Minting Process Must Be Run Manually: This
--mintstep is not run automatically by the deployment script (deploy.ps1ordeploy.sh) to avoid wasting your wallet's POL gas fees due to repeated minting transactions every time the bot is restarted. Make sure you run it manually just once before starting the deploy script. -
You can also check your wallet balance on the Amoy blockchain using the command:
python services/trader/testing/poly-amoy.py --check-balance
-
-
Authorize / Approve USDC Tokens:
- So that Polymarket smart contracts are allowed to use your USDC balance for trading, run the following command to perform authorization automatically:
python services/trader/testing/poly-amoy.py --approve-usdc
- So that Polymarket smart contracts are allowed to use your USDC balance for trading, run the following command to perform authorization automatically:
-
Run the Full Pipeline:
- Linux:
bash scripts/deploy.sh --mode amoy - Windows:
.\scripts\deploy.ps1 -Mode amoy - (The entire system, including the LLM Decision Agent, will run automatically in the background and is ready to execute real orders on the Testnet).
- Linux:
IMPORTANT: Use only after you are sure the bot and AI are running stably in simulation and testnet modes.
-
Setup Wallet & Env:
- Create a dedicated new trading wallet, get its private key, and enter it into
POLYGON_PRIVATE_KEYin the.envfile. - Make sure you have logged into the Polymarket website to activate the Deposit Wallet (see instructions above).
- Set the Risk Management parameters in
.env(e.g.,TRADER_BANKROLL_USDCaccording to your total balance).
- Create a dedicated new trading wallet, get its private key, and enter it into
-
Deposit Real Assets (pUSD):
- Send your USDC (Polygon network) balance and actual MATIC (minimum ~0.5 MATIC for long-term gas fees) to your trading wallet address.
- Connect your wallet to the Polymarket website and make a deposit. The Polymarket V2 system will automatically convert your USDC into pUSD (Polymarket USD) within your Deposit Wallet. The bot uses pUSD as default collateral.
-
Authorize USDC Tokens:
- When you deposit USDC to Polymarket using its official UI website, the Polymarket system will automatically request your signature to grant permission/approval to use your pUSD balance to the Polymarket smart contracts (CTF Exchange & NegRisk adapters). Therefore, just deposit through the UI website, you do not need to place a small bet manually.
-
Run Pipeline:
- Linux:
bash scripts/deploy.sh --mode market - Windows:
.\scripts\deploy.ps1 -Mode market - If you run them separately/manually, explicitly run the decision AI Agent with:
python services/llm-decision/main.py - (The LLM Decision Agent will process real-time bets using real money).
- Linux:
Note: If you don't provide the --mode argument when running the deploy script, the bot will automatically read the TRADING_MODE variable in .env (or use the default dry-run if empty).
These parameters protect capital from excessive losses. Adjust according to the bankroll you have.
TRADER_BANKROLL_USDC=100 # Total capital in USDC
TRADER_MIN_SINGLE_ORDER_USDC=1 # Minimum per order (default: 1 USDC)
TRADER_MAX_SINGLE_ORDER_USDC=3 # Maximum per order (default: 3 USDC)
TRADER_MAX_POSITION_PCT=5 # Max position per trade as % of bankroll (default: 5%)
TRADER_MAX_DAILY_LOSS_PCT=10 # Daily loss limit as % of bankroll (default: 10%)
TRADER_MAX_SPREAD=0.15 # Maximum order book spread (default: 0.15 = 15 cents)
TRADER_MIN_LIQUIDITY_USDC=50 # Minimum market liquidity (default: 50 USDC, not yet enabled in validator code)Example for a 200 USDC bankroll:
TRADER_BANKROLL_USDC=200
TRADER_MIN_SINGLE_ORDER_USDC=1 # min order stays at 1 USDC
TRADER_MAX_SINGLE_ORDER_USDC=10 # raised to 10 USDC per order (if you want larger orders)
TRADER_MAX_POSITION_PCT=5 # max position per trade 10 USDC (5% × 200)
TRADER_MAX_DAILY_LOSS_PCT=10 # daily loss limit 20 USDC (10% × 200)- Anti-Hallucination (All-in Protection): The bot automatically rejects orders sized > 50% of total bankroll to avoid LLM failure sending all-in bets.
- Minimum Confidence Threshold: The bot only executes recommendations with a minimum confidence level of 30%.
- Risk State Persistence: Accumulation of daily loss (
daily_loss) and number of trades are stored in Redis (trader:daily_pnl) so they don't reset when the bot or trader service is restarted on the same day.
bash scripts/deploy.shAvailable options:
| Flag | Description |
|---|---|
--mode <mode> |
Choose trading mode: market (live with real money), amoy (Polygon testnet), or dry-run (simulation / paper trading). Default: dry-run. |
--skip-migration |
Skip CSV migration to database |
--keep-volume |
Keep existing database volumes (do not wipe) |
--pgcron |
Use PostgreSQL 16 + pg_cron |
--timescale |
Use TimescaleDB + pg_cron |
--nodash |
Do not run dashboard, show worker logs in terminal |
# Run Amoy testnet
bash scripts/deploy.sh --mode amoy
# New VPS, existing data, dry-run mode
bash scripts/deploy.sh --keep-volume --skip-migration --mode dry-run
# Clean deploy + TimescaleDB + Live market
bash scripts/deploy.sh --timescale --mode market
# No dashboard (suitable for headless servers)
bash scripts/deploy.sh --nodash --dry-run.\scripts\deploy.ps1Available options:
| Flag | Description |
|---|---|
-Mode <mode> or double-dash version (--mode <mode>) |
Choose trading mode: market (live with real money), amoy (Polygon testnet), or dry-run (simulation / paper trading). Default: dry-run. |
-SkipMigration or --skip-migration |
Skip CSV migration to database |
-KeepVolume or --keep-volume |
Keep existing database volumes (do not wipe) |
-PgCron or --pgcron |
Use PostgreSQL 16 + pg_cron |
-Timescale or --timescale |
Use TimescaleDB + pg_cron |
-NoDash or --nodash |
Do not run dashboard, show worker logs in terminal |
# Run Amoy testnet
.\scripts\deploy.ps1 --mode amoy
# Run live market
.\scripts\deploy.ps1 --mode market
# Run dry-run without dashboard
.\scripts\deploy.ps1 --mode dry-run --nodash.\scripts\deploy.ps1 -KeepVolume -SkipMigration
.\scripts\deploy.ps1 -NoDash
# GNU style alternatives (double-dash) also supported:
.\scripts\deploy.ps1 --nodash
.\scripts\deploy.ps1 --keep-volume --skip-migrationThe deploy script automatically executes the following sequence:
| Step | Action | Log |
|---|---|---|
| 1 | Clean old containers & volumes (except --keep-volume) |
— |
| 2 | Start PostgreSQL + Redis via Docker Compose | — |
| 3 | Wait for DB & Redis to be healthy | — |
| 4 | Import CSV to database (migrate_data.py & nwp_ingestor.py) |
— |
| 5 | Stop old workers | — |
| 5.5 | Historical Data Sync (blocking) — METAR, PWS, Global NWP, & NWP Ensemble | — |
| 6 | METAR scraper (background) | logs/metar.log |
| 7 | PWS scraper (background) | logs/pws.log |
| 7.5 | Global NWP Model scraper (background) | logs/nwp.log |
| 7.6 | NWP Ensemble Model scraper (background) | logs/ensemble.log |
| 8 | NWP Bias Corrector v2.0 watch mode (background) | logs/modeler.log |
| 8.5 | Two-Harmonic modeler daily daemon (background) | logs/twoharmonic.log |
| 8.6 | DB Backup daemon (background, every Sunday) | logs/db_backup.log |
| 8.7 | Market Discovery — daemon refreshes market every 60 secs | logs/trader_discovery.log |
| 8.7 | Implied Price Refresher — refreshes implied% from Gamma every 5 secs | logs/trader_implied.log |
| 8.7 | WebSocket feed — real-time market data (background) | logs/trader_ws.log |
| 8.7 | Trading bot matching TRADER_MODE (background) |
logs/trader_bot.log |
| 8.8 | Position Redeemer daemon (background, auto-redeems winnings) | logs/redeemer.log |
| 8.9 | LLM Decision Agent (background, AI trading brain, scheduler, stop loss) | logs/llm_decision.log |
| 9 | Terminal dashboard (foreground) or log stream (--nodash) |
— |
Press Ctrl+C to stop — all background workers will be stopped automatically.
This service is the trading "brain" that makes real-time decisions based on market data and prediction model outputs. Starting from this update, the LLM Decision service runs automatically in the background when you run deploy.ps1 or deploy.sh. You can see its log output in the logs/llm_decision.log file or via the stream if using the --nodash mode.
This service has six trigger conditions:
| Trigger | Condition | Explanation |
|---|---|---|
morning |
Every day after 08:30 SGT | Polls forecast_predictions until Two-Harmonic finishes (deadline 10:00 SGT) |
afternoon |
Every day at 15:00 SGT (±60 seconds) | Re-analyzes with today's actual observation data |
anomaly |
Price movement in an active sub-market | Triggered if an active sub-market moves ≥15% (or based on LLM_ANOMALY_THRESHOLD_PCT) within 3 minutes (LLM_ANOMALY_WINDOW_MINUTES) |
twoharmonic_update |
Two-Harmonic prediction update | Triggered real-time when the Two-Harmonic model publishes a new prediction update to the Redis channel modeler:twoharmonic:update |
nwp_bias_update |
NWP Bias prediction update | Triggered real-time when the NWP Bias Corrector model publishes a new prediction update to the Redis channel modeler:nwp_bias:update |
stop_loss_evaluation |
Open position stop loss | Triggered by Portfolio Monitor when open position losses exceed the threshold (LLM_STOP_LOSS_PCT, default: 20%) |
Trigger met
→ Collect data (PostgreSQL + Redis, parallel, 30 seconds timeout per source)
→ Render prompt (decision_prompt.md + collected data)
→ Xiaomi MiMo Loop:
- Agent calls tools (get_live_market_snapshot, get_forecast_accuracy, etc)
- Enforcement: final_decision rejected if mandatory tools are not called
- Agent outputs final_decision: true + JSON decision
→ Phase A: Check price staleness (baseline taken at session start, before first LLM call)
- If price moves >5% since session started → agent gets 1x revision chance
→ Save to PostgreSQL (llm_decisions) — if fails, abort publication
→ Publish to Redis channel decision:recommendation (one message per sub-market)
→ Phase B: Wait for ACK from trader bot (max 15 seconds per decision_id)
- Execution result (filled/rejected/timeout) injected into agent CoT
- Agent knows if order succeeded before session closes
| Tool | Source | Purpose |
|---|---|---|
get_live_market_snapshot() |
Redis (trader:market:live:*) |
Snapshot of all active sub-markets + last 50 ticks |
get_forecast_accuracy(days) |
PostgreSQL | Two-Harmonic & Nowcast MAE for last N days (max 90) |
get_metar_recent(hours) |
PostgreSQL | Raw METAR data (max 72 hours, max 300 rows) |
get_metar_daily_summary(days) |
PostgreSQL | Daily METAR summary (max 90 days) |
get_pws_recent(hours) |
PostgreSQL | Local PWS data (max 72 hours, 5 minute resolution) |
get_pws_daily_summary(days) |
PostgreSQL | Daily PWS summary (max 90 days) |
get_nwp_ensemble_range(date_from, date_to) |
PostgreSQL | 5 NWP models consensus for date range (max 90 days) |
get_nwp_model_detail(model_id, date_from, date_to) |
PostgreSQL | Detail of 1 NWP model (max 30 days) |
get_orderbook(token_id) |
Redis (trader:market:orderbook) |
Detailed order book per token |
| Trigger | Must Call |
|---|---|
morning |
get_live_market_snapshot + get_forecast_accuracy + get_metar_daily_summary + get_nwp_ensemble_range |
afternoon |
get_live_market_snapshot + get_forecast_accuracy + get_metar_recent + get_hourly_nowcast |
anomaly |
get_live_market_snapshot + get_hourly_nowcast |
twoharmonic_update |
get_live_market_snapshot + get_forecast_accuracy + get_nwp_ensemble_range |
nwp_bias_update |
get_live_market_snapshot + get_forecast_accuracy |
stop_loss_evaluation |
get_live_market_snapshot + get_orderbook |
If the agent tries to output final_decision: true without calling all mandatory tools, the decision is rejected and the agent is asked to call the missing tools.
The prompt template can be modified without restarting the service:
services/llm-decision/prompts/decision_prompt.md
The template uses placeholders: {forecast_data}, {nwp_bias_data}, {nwp_data}, {metar_data}, {pws_data}, {market_data}, {position_data}, {decision_history}, {trigger_type}, {history_days}.
Every agent decision is saved to PostgreSQL for audit and backtesting:
| Column | Description |
|---|---|
decision_id |
Unique UUID |
trigger_type |
morning / afternoon / anomaly / twoharmonic_update / nwp_bias_update / stop_loss_evaluation |
action |
BUY_YES / BUY_NO / HOLD |
p_weather |
Agent's weather probability for this bracket |
p_market |
Market YES price during decision |
edge |
p_weather − p_market (decision quality signal) |
tmax_estimate_c |
Agent's specific Tmax estimate |
model_confidence |
Model confidence level (high / moderate / low / insufficient) |
pred_tmax_snapshot |
Two-Harmonic / NWP Bias Corrector Tmax prediction at that time |
nwp_ensemble_snapshot |
NWP Ensemble at that time |
market_price_snapshot |
Market YES price snapshot at that time |
reasoning_summary |
Brief reasoning from the agent |
outcome |
WIN / LOSS / PENDING (auto-updated when market resolves) |
| Variable | Default | Description |
|---|---|---|
POSTGRES_DB |
polyweather |
Database name |
POSTGRES_USER |
postgres |
Username |
POSTGRES_PASSWORD |
123 |
Change in production! |
POSTGRES_HOST |
localhost |
PostgreSQL host |
POSTGRES_PORT |
5433 |
PostgreSQL port |
DATABASE_URL |
(from above variables) | Full connection string |
APP_TIMEZONE |
Asia/Singapore |
Operational timezone |
| Variable | Default | Description |
|---|---|---|
REDIS_HOST |
localhost |
Host |
REDIS_PORT |
6379 |
Port |
REDIS_DB |
0 |
Database number |
| Variable | Default | Description |
|---|---|---|
TRADING_MODE |
dry-run |
dry-run / amoy / market |
POLYGON_PRIVATE_KEY |
(empty) | Mainnet private key |
AMOY_PRIVATE_KEY |
(empty) | Amoy testnet private key |
POLYMARKET_TOKEN_ID |
(empty) | Automatically filled by market_discovery |
TRADER_BANKROLL_USDC |
100 |
Total USDC capital |
TRADER_MIN_SINGLE_ORDER_USDC |
1 |
Min per order (USDC) |
TRADER_MAX_SINGLE_ORDER_USDC |
3 |
Max per order (USDC) |
TRADER_MAX_POSITION_PCT |
5 |
Max position per trade (% bankroll) |
TRADER_MAX_DAILY_LOSS_PCT |
10 |
Daily loss limit (% bankroll) |
TRADER_MAX_SPREAD |
0.15 |
Max order book spread |
TRADER_MIN_LIQUIDITY_USDC |
50 |
Min market liquidity (USDC) (not yet enabled in validator code) |
NOTE: Full OpenAI API Compatibility The LLM Decision agent uses the standard
openaiPython package. It is fully compatible with any LLM provider that supports the OpenAI API standard (e.g., standard OpenAI ChatGPT, DeepSeek, Groq, Together AI, Local Ollama, vLLM). You can easily swap the agent to any other model just by editing theMIMO_API_KEY,MIMO_BASE_URL, andMIMO_MODELvariables below.NOTE: Agent Trading Style (Moderate Risk) By default, the LLM agent is configured with a Moderate Risk trading style. It balances between capitalizing on high-edge opportunities and preserving capital. You can change the agent's behavior, personality, or risk tolerance at any time by modifying its system prompt located at
services/llm-decision/prompts/decision_prompt.md.
| Variable | Default | Description |
|---|---|---|
MIMO_API_KEY |
(empty) | MANDATORY |
MIMO_MODEL |
mimo-v2.5-pro |
Model: mimo-v2.5-pro |
MIMO_BASE_URL |
https://token-plan-sgp.xiaomimimo.com/v1 |
API Base URL (choose based on account type: PAYG or TokenPlan) |
LLM_MORNING_TRIGGER_TIME |
08:30 |
Morning polling time (SGT, HH:MM) |
LLM_AFTERNOON_TRIGGER_TIME |
15:00 |
Afternoon trigger time (SGT, HH:MM) |
LLM_ANOMALY_THRESHOLD_PCT |
15 |
Movement percentage for anomaly trigger (1–100) |
LLM_ANOMALY_WINDOW_MINUTES |
3 |
Anomaly detection time window (1–60 mins) |
LLM_COOLDOWN_MINUTES |
10 |
Cooldown after anomaly (1–60 mins) |
LLM_STOP_LOSS_PCT |
20 |
Open position loss limit before stop loss trigger (1-100) |
LLM_MAX_TOOL_CALLS |
10 |
Max tool calls per agent cycle (1–50) |
LLM_HISTORY_DAYS |
7 |
History days sent to agent (1–90) |
LLM_PRICE_STALE_THRESHOLD |
0.05 |
Price movement fraction for agent revision trigger (default 5%) |
LLM_ACK_TIMEOUT_SECONDS |
15.0 |
Timeout waiting for ACK from trader bot (seconds) |
| Variable | Default | Description |
|---|---|---|
POLYGON_RPC_URL |
https://polygon-rpc.com |
Polygon mainnet RPC |
CTF_ADDRESS |
0x4D97DCd97eC945f40cF65F87097ACe5EA0476045 |
Gnosis CTF contract — do not change |
USDC_MAINNET_ADDRESS |
0x2791Bca1f2de4661ED88A30C99A7a9449Aa84174 |
USDC contract — do not change |
REDEEMER_GAS_LIMIT |
200000 |
redeemPositions transaction gas limit |
| Variable | Default | Description |
|---|---|---|
BACKUP_RCLONE_REMOTE |
poly_remote:polyweather_backups |
rclone remote + destination folder |
BACKUP_KEEP_LOCAL |
7 |
Days to keep local backup before deleting |
BACKUP_KEEP_REMOTE |
12 |
Number of files in Google Drive |
BACKUP_RETENTION_DAYS |
5000 |
DB data days before pruning |
Polyweather is equipped with an integrated Discord bot (services/notifier) that serves dual functions:
- Push Notifications (Automatic): The bot monitors redemption history (
trader:redeem_history) and sends notifications to your Discord channel in real-time whenever a reward or trading position is successfully redeemed. - Slash Commands (Interactive): You can interact with the bot in Discord using the following commands:
/asset: Displays the total bankroll balance (USDC) on Polymarket./position: Displays the list of open trading positions along with their unrealized PnL value live./history: Displays the trading and redeem history, equipped with a dynamicdaysparameter (example:/history days: 31).
Bot Setup:
- Make sure you have created a Bot App in the Discord Developer Portal.
- How to get
DISCORD_BOT_TOKEN:- In the Developer Portal, select your application and click the "Bot" tab in the left menu.
- Click the "Reset Token" button and copy the random text string that appears.
- On the same page, under the Privileged Gateway Intents section, make sure you check the Message Content Intent, then click "Save Changes".
- Enter the copied token into the
DISCORD_BOT_TOKENvariable in the.envfile.
- How to get
DISCORD_CHANNEL_ID:- Open the Discord app, go to User Settings (gear icon) Advanced and enable Developer Mode.
- Open the server where the bot has been invited, right-click on the text channel where notifications should be sent, then click "Copy Channel ID".
- Enter that sequence of numbers into the
DISCORD_CHANNEL_IDvariable in the.envfile.
- The bot runs inside a Docker container and will automatically start when the deployment is executed (
docker-compose up).
After the Polymarket market resolves, position_redeemer.py automatically cashes out winning tokens into USDC via the Gnosis CTF smart contract.
Polymarket uses Gnosis CTF (Conditional Token Framework). Each binary market has two tokens:
- YES token (index set 1) — worth 1 USDC if outcome = YES
- NO token (index set 2) — worth 1 USDC if outcome = NO
The bot can buy YES or NO tokens depending on the agent's analysis. The winning side is automatically redeemed.
The Redeemer runs automatically when the deploy script is executed. To check manually:
python services/trader/src/position_redeemer.py --checkEnsure the wallet always has some MATIC for gas fees (~0.001–0.01 MATIC per redemption).
Runs every Sunday at 00:00 as a background daemon.
Process:
pg_dumpthe entire database →.sql.gzfile- Upload to Google Drive via rclone
- Delete local backups >
BACKUP_KEEP_LOCALdays - Retain only the last
BACKUP_KEEP_REMOTEfiles on Google Drive - Data Retention (Data Pruning): Checks the number of unique data days in the database. If there are tables with data exceeding
BACKUP_RETENTION_DAYSdays (default: 5000 days), the oldest rows will be deleted until exactly 5000 days remain. Note: Data will never be deleted if today's backup has not been successful.
Test backup immediately:
# Linux
.venv_linux/bin/python Database/backup/db_backup.py --run-now
# Windows
.venv_windows\Scripts\python.exe Database/backup/db_backup.py --run-nowCheck next schedule:
python Database/backup/db_backup.py --statusComplete guide for rclone setup is in Database/BACKUP_SETUP.md.