Polyweather

Polyweather is a weather-data and Polymarket trading pipeline with a selectable decision engine. It combines:

Weather data ingestion — METAR (WSSS Changi Airport), PWS (ISINGA249), global NWP (ACCESS, ARPEGE, ECMWF, GEM, GFS, ICON), and NWP Ensemble (GFS & ECMWF IFS via Open-Meteo)
Temperature prediction — NWP Bias Corrector v2.0 (Stacked Ensemble: XGBoost + LightGBM + Ridge meta-learner, residual error correction of global NWP models) + Two-Harmonic (LightGBM, daily Tmax)
Optional AI Trading Agent — LLM Decision Service based on Xiaomi MiMo analyzing weather and Polymarket conditions with real-time trigger support, anomaly detection, and portfolio monitoring
Deterministic Decision Engine — non-LLM branch that converts forecast residuals into an exclusive-bucket probability distribution, applies executable-price/risk checks, and abstains when evidence is insufficient
Trading Bot — Order execution to Polymarket CLOB API (dry-run / testnet / live) with Implied Price Refresher
Terminal dashboard — real-time Textual-based
Automated backup to Google Drive via rclone

System Architecture

The data/model/trader layers are shared. At launch, exactly one recommendation producer is selected: the LLM service or the deterministic service. The diagram below shows the AI branch; the deterministic branch joins the same decision:recommendation queue without calling an LLM.

┌─────────────────────────────────────────────────────────────────────────┐
│                            POLYWEATHER                                  │
├─────────────────────────────┬───────────────────────────────────────────┤
│   DATA LAYER                │   AI REASONING LAYER                      │
│                             │                                           │
│  METAR scraper  ──┐         │  ┌── decision_prompt.md (template)        │
│  PWS scraper    ──┼──▶ PostgreSQL ◀── Two-Harmonic Model (LightGBM)     │
│  NWP scraper    ──┘  │ Redis  │         NWP Bias Corrector (XGBoost)    │
│                      │       │                 │                        │
│  market_discovery ───┼───────┼─────────────────┤                        │
│  poly-websocket  ────┘  trader:market:*        ▼                        │
│                                      LLM Decision Service               │
│                                      (Xiaomi MiMo v2.5-pro agent)       │
│                                             │                           │
│                                             ▼                           │
│                                    decision:recommendation              │
│                                        (Redis channel)                  │
├─────────────────────────────────────────────┼───────────────────────────┤
│   TRADING LAYER                             │                           │
│                                             ▼                           │
│  clob_client_v2 ◀── decision:recommendation │                           │
│       │          Risk Manager (spread,      │                           │
│       │          liquidity, daily loss,     │                           │
│       │          confidence checks)         │                           │
│       ├── dry-run  → log only               │                           │
│       ├── amoy     → Polygon Amoy testnet   │                           │
│       └── market   → Polymarket mainnet     │                           │
│                          │                  │                           │
│                          ▼                  │                           │
│                 position_redeemer           │   notifier (Discord Bot)  │
│            (auto-redeem kemenangan) ────────┼──▶ (Notifikasi & Status)  │
└─────────────────────────────────────────────┴───────────────────────────┘

Complete Data Flow

Weather scraper → raw_metar_observations / raw_pws_observations (PostgreSQL Bronze)
                         ↓ (Automatic DB trigger)
               metar_clean / pws_clean (PostgreSQL Silver)
                         ↓
               nwp_ingestor.py → nwp_forecasts → nwp_daily
                         ↓
               model_nwp_bias.py → nwp_bias_predictions / nwp_bias_history
               model_twoharmonic.py → forecast_predictions
                         ↓
               Selected decision service (AI/MiMo or deterministic rules)
                         ↓
               decision:recommendation (Redis list queue)
                         ↓
               clob_client_v2.py → PostgreSQL order intent
                         ↓
               Polymarket CLOB API (skipped in dry-run)
                         ↓
               trade_orders / trade_fills (PostgreSQL ledger)
                         ↓
               trader:execution_ack:{decision_id} (Redis, TTL 300s)

Redis Keys

Key	Writer	Reader	Content
`trader:market:info`	market_discovery	llm-decision, anomaly_detector	Event info + sub-market list
`trader:market:detail`	market_discovery	llm-decision, tool_registry	Details per bracket (price, orderbook)
`trader:market:orderbook`	market_discovery	tool_registry (get_orderbook)	Order book per token `{token_id: metrics}`
`trader:market:prices`	market_discovery	anomaly_detector	Price per token `{token_id: price}`
`trader:market:all_tokens`	market_discovery	poly-websocket	All active tokens (YES+NO)
`trader:market:live:{token_id}`	poly-websocket	selected decision service, trader, dashboard	Fresh two-sided depth snapshot per token
`trader:market:history:{token_id}`	poly-websocket	tool_registry (get_price_history)	Tick history per token (max 200)
`trader:history`	clob_client_v2	dashboard	All transactions log (max 1000)
`trader:position`	clob_client_v2	llm-decision (data_collector)	Current open positions
`trader:daily_pnl`	clob_client_v2 (RiskManager)	llm-decision (data_collector)	Daily P&L + risk manager state
`decision:recommendation`	selected AI or deterministic producer	clob_client_v2	Durable-style Redis list queue for trading recommendations
`trader:execution_ack:{id}`	clob_client_v2	selected decision service	Order execution confirmation (TTL 300s)
`model:nwp_bias:prediction`	model_nwp_bias	dashboard, llm-decision	Today's NWP bias correction prediction
`model:nwp_bias:status`	model_nwp_bias	dashboard	Model training status
`modeler:nwp_bias:update`	model_nwp_bias	llm-decision (listener)	Prediction update event (pub/sub)

Database Structure (Medallion Architecture)

Polyweather uses a PostgreSQL database schema organized into several layers (Medallion Architecture) to ensure data integrity from raw sources to model outputs.

========================================================================================================================
                                     POLYWEATHER DATABASE FULL SCHEMA (COMPLETE)
========================================================================================================================

    [ LAYER 0: METADATA ]
    +--------------------------------+       +--------------------------------+
    | stations                       |       | pws_sensors                    |
    +--------------------------------+       +--------------------------------+
    | station_id (PK)      VARCHAR10 |       | sensor_id (PK)       VARCHAR20 |
    | station_name         TEXT      |       | sensor_name          TEXT      |
    | latitude             NUMERIC   |       | latitude             NUMERIC   |
    | longitude            NUMERIC   |       | longitude            NUMERIC   |
    | elevation_m          INTEGER   |       | elevation_m          INTEGER   |
    | timezone             VARCHAR50 |       | timezone             VARCHAR50 |
    | is_active            BOOLEAN   |       | interval_minutes     INTEGER   |
    | created_at           TIMESTAMPTZ|      | is_active            BOOLEAN   |
    | updated_at           TIMESTAMPTZ|      | created_at           TIMESTAMPTZ|
    +---------------+----------------+       | updated_at           TIMESTAMPTZ|
                    ^                        +---------------+----------------+
                    |                                        ^
                    | (Foreign Key)                          | (Foreign Key)
                    +-------------------+   +----------------+
                                        |   |
    +-----------------------------------+---+--------------------------------+
    | station_sensor_map                                                     |
    +------------------------------------------------------------------------+
    | station_id (PK, FK)  VARCHAR10                                         |
    | sensor_id  (PK, FK)  VARCHAR20                                         |
    | is_primary           BOOLEAN                                           |
    | created_at           TIMESTAMPTZ                                       |
    +------------------------------------------------------------------------+

    [ LAYER 1: BRONZE (RAW) ]                [ LAYER 6: NWP (MODEL CUACA GLOBAL) ]
    +--------------------------------+       +--------------------------------+
    | raw_metar_observations         |       | nwp_models                     |
    +--------------------------------+       +--------------------------------+
    | station_id (PK, FK)  VARCHAR10 |       | model_id (PK)        VARCHAR20 |
    | observation_time (PK) TIMESTAMPTZ|      | model_name           TEXT      |
    | local_time           TEXT      |       | provider             TEXT      |
    | raw_text             TEXT      |       | resolution_km        NUMERIC   |
    | report_type          VARCHAR20 |       | is_active            BOOLEAN   |
    | temp_c               TEXT      |       | notes                TEXT      |
    | dewpoint_c           TEXT      |       | created_at           TIMESTAMPTZ|
    | pressure_mb          TEXT      |       +---------------+----------------+
    | wind_dir             TEXT      |                       ^
    | wind_speed_kt        TEXT      |                       | (Foreign Key)
    | wind_gust_kt         TEXT      |       +---------------+----------------+
    | wind_dir_var         VARCHAR20 |       | nwp_forecasts                  |
    | visibility           TEXT      |       +--------------------------------+
    | cloud_layers         TEXT      |       | nwp_id (PK)          BIGSERIAL |
    | wx_string            TEXT      |       | model_id (FK)        VARCHAR20 |
    | flight_category      VARCHAR10 |       | station_id (FK)      VARCHAR10 |
    | auto                 TEXT      |       | forecast_time        TIMESTAMPTZ|
    | recent_weather       TEXT      |       | forecast_time_local  TIMESTAMP |
    | rvr                  TEXT      |       | forecast_date        DATE      |
    | remarks              TEXT      |       | forecast_hour        SMALLINT  |
    | rmk_indicators       TEXT      |       | latitude             NUMERIC   |
    | longitude            NUMERIC   |       | longitude            NUMERIC   |
    | elevation_m          TEXT      |       | temperature_2m       NUMERIC   |
    | source_file          TEXT      |       | relative_humidity_2m NUMERIC   |
    | ingested_at          TIMESTAMPTZ|      | ingested_at          TIMESTAMPTZ|
    +--------------------------------+       +--------------------------------+

    +--------------------------------+       +--------------------------------+
    | raw_pws_observations           |       | nwp_data_quality_log           |
    +--------------------------------+       +--------------------------------+
    | sensor_id (PK, FK)   VARCHAR20 |       | log_id (PK)          BIGSERIAL |
    | full_time (PK)       TIMESTAMPTZ|      | station_id (FK)      VARCHAR10 |
    | data_date            DATE      |       | model_id (FK)        VARCHAR20 |
    | time_str             VARCHAR10 |       | forecast_date        DATE      |
    | temp_c               TEXT      |       | flag_type            VARCHAR30 |
    | heat_index_c         TEXT      |       | reason               TEXT      |
    | dew_point_c          TEXT      |       | created_at           TIMESTAMPTZ|
    | humidity_pct         TEXT      |       +--------------------------------+
    | wind_dir_deg         TEXT      |
    | wind_speed_kmh       TEXT      |
    | gust_kmh             TEXT      |
    | pressure_hpa         TEXT      |
    | rain_rate_mmh        TEXT      |
    | rain_total_mm        TEXT      |
    | solar_wm2            TEXT      |
    | source_file          TEXT      |
    | ingested_at          TIMESTAMPTZ|
    +--------------------------------+

    [ LAYER 2: SILVER (CLEAN) ]              [ LAYER 6: NWP DAILY (FEATURES) ]
    +--------------------------------+       +--------------------------------+
    | metar_clean                    |       | nwp_daily                      |
    +--------------------------------+       +--------------------------------+
    | station_id (PK, FK)  VARCHAR10 |       | model_id (PK, FK)    VARCHAR20 |
    | observation_time (PK) TIMESTAMPTZ|      | station_id (PK, FK)  VARCHAR10 |
    | local_time           TIMESTAMP |       | forecast_date (PK)   DATE      |
    | obs_date             DATE      |       | tmax_nwp             NUMERIC   |
    | raw_text             TEXT      |       | tmin_nwp             NUMERIC   |
    | report_type          VARCHAR20 |       | tmean_nwp            NUMERIC   |
    | temp_c               NUMERIC   |       | t08_nwp              NUMERIC   |
    | dewpoint_c           NUMERIC   |       | rh08_nwp             NUMERIC   |
    | pressure_mb          NUMERIC   |       | td08_nwp             NUMERIC   |
    | wind_dir_deg         NUMERIC   |       | pressure08_nwp       NUMERIC   |
    | wind_is_vrb          BOOLEAN   |       | cloud_morning_nwp    NUMERIC   |
    | wind_speed_kt        NUMERIC   |       | cape_morning_nwp     NUMERIC   |
    | wind_gust_kt         NUMERIC   |       | precip_morning_nwp   NUMERIC   |
    | wind_dir_var         VARCHAR20 |       | solar_max_nwp        NUMERIC   |
    | visibility           NUMERIC   |       | solar_mean_nwp       NUMERIC   |
    | cloud_layers         TEXT      |       | wind_max_nwp         NUMERIC   |
    | wx_string            TEXT      |       | wind_mean_nwp        NUMERIC   |
    | flight_category      VARCHAR10 |       | cape_max_nwp         NUMERIC   |
    | recent_weather       TEXT      |       | vpd_mean_nwp         NUMERIC   |
    | is_speci             BOOLEAN   |       | freezing_level_mean  NUMERIC   |
    | is_auto              BOOLEAN   |       | n_hours              SMALLINT  |
    | is_cor               BOOLEAN   |       | created_at           TIMESTAMPTZ|
    | has_cb_or_tcu        BOOLEAN   |       +--------------------------------+
    | created_at           TIMESTAMPTZ|
    +--------------------------------+       [ LAYER 6b: NWP ENSEMBLE ]
                                             +--------------------------------+
    +--------------------------------+       | nwp_ensemble_forecasts         |
    | pws_clean                      |       +--------------------------------+
    +--------------------------------+       | model_id (PK, FK)    VARCHAR20 |
    | sensor_id (PK, FK)   VARCHAR20 |       | member_id (PK)       SMALLINT |
    | full_time (PK)       TIMESTAMPTZ|      | station_id (PK, FK)  VARCHAR10 |
    | obs_date             DATE      |       | forecast_time (PK)   TIMESTAMPTZ|
    | time_local           TIME      |       | model_run_time (PK)  TIMESTAMPTZ|
    | temp_c               NUMERIC   |       | lead_time_hours      SMALLINT  |
    | heat_index_c         NUMERIC   |       | temperature_2m       NUMERIC   |
    | dew_point_c          NUMERIC   |       | relative_humidity_2m NUMERIC   |
    | humidity_pct         NUMERIC   |       | dew_point_2m         NUMERIC   |
    | wind_dir_deg         NUMERIC   |       | apparent_temperature  NUMERIC   |
    | wind_speed_kmh       NUMERIC   |       | precipitation        NUMERIC   |
    | gust_kmh             NUMERIC   |       | rain                 NUMERIC   |
    | pressure_hpa         NUMERIC   |       | wind_speed_10m       NUMERIC   |
    | rain_rate_mmh        NUMERIC   |       | wind_gusts_10m       NUMERIC   |
    | rain_total_mm        NUMERIC   |       | pressure_msl         NUMERIC   |
    | solar_wm2            NUMERIC   |       | ingested_at          TIMESTAMPTZ|
    | created_at           TIMESTAMPTZ|       +--------------------------------+
    +--------------------------------+
                                             +--------------------------------+
    [ LAYER 5: MODEL OUTPUT ]                | nwp_ensemble_stats             |
    +--------------------------------+       +--------------------------------+
    | model_runs                     |       | station_id (PK, FK)  VARCHAR10 |
    +--------------------------------+       | forecast_time (PK)   TIMESTAMPTZ|
    | run_id (PK)          BIGSERIAL |       | model_run_time (PK)  TIMESTAMPTZ|
    | station_id (FK)      VARCHAR10 |       | temp_mean            NUMERIC   |
    | model_type           VARCHAR20 |       | temp_std             NUMERIC   |
    | model_name           TEXT      |       | temp_min             NUMERIC   |
    | model_version        TEXT      |       | temp_max             NUMERIC   |
    | training_date_min    DATE      |       | temp_p10             NUMERIC   |
    | training_date_max    DATE      |       | temp_p25             NUMERIC   |
    | n_features           INTEGER   |       | temp_p50             NUMERIC   |
    | n_train_samples      INTEGER   |       | temp_p75             NUMERIC   |
    | rmse                 NUMERIC   |       | temp_p90             NUMERIC   |
    | mae                  NUMERIC   |       | precip_mean          NUMERIC   |
    | feature_list         JSONB     |       | n_members            SMALLINT  |
    | model_params         JSONB     |       | ingested_at          TIMESTAMPTZ|
    | created_at           TIMESTAMPTZ|      +--------------------------------+
    +---------------+----------------+
                    ^                        [ LAYER 5b: NWP BIAS CORRECTOR ]
                    | (Foreign Key)          +--------------------------------+
    +---------------+----------------+       | nwp_bias_predictions           |
    | forecast_predictions           |       +--------------------------------+
    +--------------------------------+       | pred_id (PK)         BIGSERIAL |
    | pred_id (PK)         BIGSERIAL |       | station_id (FK)      VARCHAR10 |
    | run_id (FK)          BIGINT    |       | forecast_date (UQ)   DATE      |
    | station_id (FK)      VARCHAR10 |       | predicted_at         TIMESTAMPTZ|
    | forecast_date (UQ)   DATE      |       | nwp_ensemble_tmax    NUMERIC   |
    | predicted_at         TIMESTAMPTZ|      | nwp_spread_tmax      NUMERIC   |
    | pred_tmax            NUMERIC   |       | residual_pred        NUMERIC   |
    | pred_hour            NUMERIC   |       | corrected_tmax       NUMERIC   |
    | actual_tmax          NUMERIC   |       | confidence_pct       NUMERIC   |
    | actual_hour          NUMERIC   |       | model_mae            NUMERIC   |
    | blend_weight_main    NUMERIC   |       | model_name           TEXT      |
    | blend_weight_fallback NUMERIC  |       +--------------------------------+
    | risk_signal          NUMERIC   |
    | ci_low               NUMERIC   |       +--------------------------------+
    | ci_high              NUMERIC   |       | nwp_bias_history               |
    +--------------------------------+       +--------------------------------+
                                             | bias_id (PK)         BIGSERIAL |
    +--------------------------------+       | station_id (FK)      VARCHAR10 |
    | nowcast_dashboard_history      |       | forecast_date (UQ)   DATE      |
    +--------------------------------+       | nwp_ensemble_tmax    NUMERIC   |
    | history_id (PK)      BIGSERIAL |       | nwp_spread_tmax      NUMERIC   |
    | station_id (FK)      VARCHAR10 |       | t08_metar            NUMERIC   |
    | predicted_at         TIMESTAMPTZ|      | rh08_metar           NUMERIC   |
    | last_metar_time      TIMESTAMPTZ|      | cloud08_oktas        NUMERIC   |
    | expected_target_time TIMESTAMPTZ|      | precip_flag          SMALLINT  |
    | metar_temp_at_prediction NUMERIC|      | pressure08_mb        NUMERIC   |
    | pred_temp            NUMERIC   |       | t_pagi_pws           NUMERIC   |
    | actual_metar_time    TIMESTAMPTZ|      | rh_pagi_pws          NUMERIC   |
    | actual_temp          NUMERIC   |       | rain_6h_pws          NUMERIC   |
    | abs_error            NUMERIC   |       | actual_tmax          NUMERIC   |
    | resolution_status    VARCHAR16 |       | residual_actual      NUMERIC   |
    | confidence_pct       NUMERIC   |       | residual_pred        NUMERIC   |
    | interval_approx      NUMERIC   |       | corrected_tmax       NUMERIC   |
    | model_name           TEXT      |       | abs_error            NUMERIC   |
    | runtime_score        NUMERIC   |       | created_at           TIMESTAMPTZ|
    | updated_at           TIMESTAMPTZ|      +--------------------------------+
    +--------------------------------+
                                             +--------------------------------+
    [ LAYER 7: LLM DECISION SERVICE ]        | model_versions                 |
    +--------------------------------+       +--------------------------------+
    | llm_decisions                  |       | version_id (PK)      UUID      |
    +--------------------------------+       | model_name           VARCHAR   |
    | decision_id (PK)     UUID      |       | created_at           TIMESTAMPTZ|
    | created_at           TIMESTAMPTZ|      | training_start_date  DATE      |
    | trigger_type         VARCHAR30 |       | training_end_date    DATE      |
    | token_id             VARCHAR255|       | feature_list         JSONB     |
    | action               VARCHAR10 |       | hyperparameters      JSONB     |
    | size_usdc            NUMERIC   |       | metrics              JSONB     |
    | price_limit          NUMERIC   |       | artifact_path        VARCHAR   |
    | confidence           INTEGER   |       | is_active            BOOLEAN   |
    | reasoning_summary    TEXT      |       +--------------------------------+
    | pred_tmax_snapshot   NUMERIC   |
    | nwp_ensemble_snapshot NUMERIC  |       +--------------------------------+
    | market_price_snapshot NUMERIC  |       | nwp_bias_performance           |
    | p_weather            NUMERIC   |       +--------------------------------+
    | p_market             NUMERIC   |       | station_id (FK)      VARCHAR10 |
    | edge                 NUMERIC   |       | forecast_date        DATE      |
    | tmax_estimate_c      NUMERIC   |       | model_version_id (FK) UUID     |
    | model_confidence     VARCHAR   |       | predicted_tmax       NUMERIC   |
    | outcome              VARCHAR10 |       | actual_tmax          NUMERIC   |
    | resolved_at          TIMESTAMPTZ|      | abs_error            NUMERIC   |
                                             | squared_error        NUMERIC   |
                                             | within_80pct_interval BOOLEAN  |
                                             | crps                 NUMERIC   |
                                             | created_at           TIMESTAMPTZ|
                                             +--------------------------------+

System Requirements

Platform	Minimum
OS	Windows 10/11, Ubuntu 22.04+, Debian 12+, or WSL2
Docker	Docker Engine ≥ 24 + Compose plugin
Python	3.12+ with pip
rclone	≥ v1.65 (for Google Drive backup, auto-install via `setuplinux.sh`)
pg_dump	PostgreSQL client tools (auto-install via `setuplinux.sh`)

Python Dependencies (Per Service)

The repository uses dual virtual environments to prevent binary conflicts between OS:

Windows → .venv_windows/
Linux/WSL → .venv_linux/

The deployment script creates and populates the venv automatically.

Service	Main Libraries
`services/Scraping`	`requests`, `pandas`, `metar`, `pytz`, `psycopg2-binary`
`services/modeler`	`numpy`, `pandas`, `scikit-learn`, `lightgbm`, `xgboost`, `psycopg2-binary`
`services/trader`	`redis`, `websocket-client`, `py-clob-client`
`services/llm-decision`	`openai`, `redis`, `psycopg2-binary`, `hypothesis`, `pytest`

PyTorch is not part of the default modeler installation. ConvLSTM remains an experimental, disabled ensemble member; setting NWP_ENABLE_CONVLSTM=true causes either deployment launcher to install services/modeler/requirements-convlstm.txt and load it only on first use.

First Time Setup

Linux / WSL / VPS

Run once on a new server. Installs Docker, rclone, pg_dump, and setups Google Drive:

chmod +x scripts/linux_auto/setuplinux.sh
bash scripts/linux_auto/setuplinux.sh

What the script does:

Step	Action
1	Detects distro (Ubuntu/Debian/Fedora/Arch)
2	Installs Docker Engine + Compose plugin
3	Enables Docker service, adds user to `docker` group
4	Installs rclone (official installer)
5	Installs `postgresql-client` (`pg_dump`)
6	Configures rclone → Google Drive (stops when OAuth URL appears)

When the OAuth URL appears: copy → open in browser → login → authorize → return to terminal → Enter.

Skip options if some are already installed:

bash scripts/linux_auto/setuplinux.sh --skip-docker
bash scripts/linux_auto/setuplinux.sh --skip-rclone
bash scripts/linux_auto/setuplinux.sh --skip-gdrive

Windows

No special setup required. Ensure Docker Desktop and Python 3.12+ are installed, then deploy directly. The PowerShell script will create the venv and install dependencies automatically.

Mandatory Configuration Before Deploy

Configure only the components you select. MIMO_API_KEY is required for the AI branch, but not for deterministic mode.

1. Create .env file

cp .env.example .env

Open .env with your preferred text editor and fill in all the required variables.

2. Setup Xiaomi MiMo API

The AI agent (LLM Decision Service) uses Xiaomi MiMo v2.5-Pro as its reasoning engine.

Steps:

Register at platform.xiaomimimo.com
Choose plan: PAYG (pay as you go) or TokenPlan (monthly subscription)
Go to Developer Console → generate a new API Key
Fill in .env:

MIMO_API_KEY=sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxx

# Choose Base URL according to account type:
# PAYG (pay as you go):
# MIMO_BASE_URL=https://api.xiaomimimo.com/v1
#
# TokenPlan (monthly subscription — Singapore region):
MIMO_BASE_URL=https://token-plan-sgp.xiaomimimo.com/v1
#
# TokenPlan (monthly subscription — China region):
# MIMO_BASE_URL=https://token-plan-cn.xiaomimimo.com/v1

MIMO_MODEL=mimo-v2.5-pro

MANDATORY — the llm-decision service will not start without MIMO_API_KEY.

IMPORTANT: TokenPlan and PAYG use DIFFERENT endpoints. If you subscribe monthly (TokenPlan), you MUST use the token-plan-sgp or token-plan-cn endpoint, not api.xiaomimimo.com. Wrong endpoint = API key rejected.

3. Setup CheckWX API

The METAR data scraper requires an API key from CheckWX to fetch aviation weather reports.

Steps:

Register for a free account at api.checkwx.com
Once registered, generate an API key from your dashboard.
Fill in .env:

CHECKWX_API_KEY=your_checkwx_api_key_here

MANDATORY — the metar_WXaggregator scraper requires this key to function properly.

WUNDERGROUND_API_KEY is optional for the local PWS feature scraper. If it is empty, both launchers skip PWS history/background collection and continue with a warning. This PWS feed is a predictor only; it is not the immutable Weather Underground WSSS settlement-label feed required before live decisions.

Historical METAR synchronization audits continuity over the latest 120 days (METAR_HISTORY_DAYS) and uses the IEM ASOS archive to fill holes left by the recent AviationWeather feed. Gaps longer than 24 hours make the sync fail closed instead of silently training on a discontinuous timeline.

Global NWP synchronization likewise audits every model/date over the latest 92 complete days (NWP_HISTORY_DAYS) instead of trusting the newest timestamp. Missing or incomplete date ranges are repaired separately, and deployment stops if any configured model still has incomplete coverage.

4. Setup Polygon Wallet

The trading bot requires a Polygon wallet to interact with the Polymarket CLOB API.

1. Create a New Wallet (recommended exclusively for trading)

Use MetaMask, Rabby, or other EVM wallets:

Open your wallet → Create a new account
Give it a name, e.g., "Polyweather Trading"
Go to Account details → Export Private Key
Enter your wallet password
Copy the private key (starts with 0x)

IMPORTANT: Never use a main wallet/hot wallet containing large assets. Use a dedicated wallet with a limited balance according to the established bankroll.

2. Activate Deposit Wallet (Polymarket V2)

Polymarket V2 (since 2026) uses a Proxy/Deposit Wallet architecture. You MUST create a deposit wallet before the bot can trade:

Visit the Polymarket website (or testnet.polymarket.com if using Amoy. make sure you have already claimed the Free Testnet Tokens (Faucet) beforehand).
Connect your new wallet (e.g., via Rabby/MetaMask)
Polymarket will automatically deploy the Deposit Wallet smart contract (ERC-1967 Proxy) for your wallet (EOA).
(Highly Recommended for market mode) Make a small deposit/withdraw on the website to ensure your proxy wallet is fully active.

The bot will automatically detect your Deposit Wallet address via API during initialization.

3. Configuration in `.env` file

Inside the .env file, the configurations for Mainnet and Amoy Testnet are written side-by-side (they don't overwrite each other). You only need to fill in the private key in the appropriate variable:

# Fill this if you want to trade on Live Mainnet (Real Money)
POLYGON_PRIVATE_KEY=0x_your_mainnet_private_key

# Fill this if you want to trade on Amoy Testnet (Play Money/Testing)
AMOY_PRIVATE_KEY=0x_your_testnet_private_key

Note: All smart contract addresses (like AMOY_USDC_ADDRESS and AMOY_CTF_EXCHANGE) and default Chain IDs are already configured automatically in .env.example and you don't need to change them unless there's a network update.

4. Market Token ID

Polymarket creates new markets every day. market_discovery.py will automatically find today's token IDs. But if you want to set them manually:

# Leave empty initially — market_discovery.py will fill it automatically
POLYMARKET_TOKEN_ID=
POLYMARKET_CONDITION_ID=

Token IDs can be found by running:

python services/trader/src/market_discovery.py

5. Select Decision Engine and Trading Mode

The launcher runs exactly one decision producer. Pass --decision-mode ai to use the MiMo agent or --decision-mode deterministic to use the rule-based forecast/edge engine without an AI agent. If omitted in an interactive terminal, a 1) AI / 2) Deterministic menu is shown; automation must provide the mode explicitly.

There are three trading backends. Default is dry-run (safe, no real orders). Use --trading-mode on Linux/WSL or -TradingMode on PowerShell.

Mode	Description	Real Money?
`dry-run`	Paper execution. Attempts use live market-data inputs when reachable, persist to PostgreSQL, and mirror to Redis; no order is sent to Polymarket.	❌ No
`amoy`	Polygon Amoy Testnet. Real transactions on the test blockchain, tokens have no value.	❌ No
`market`	Live mainnet. Real orders to Polymarket with actual USDC.	✅ YES

Step-by-Step Guide Based on Selected Mode

A. `dry-run` Mode (Simulation / Paper Trading)

Suitable for testing deterministic or AI decision logic, weather ingestion, and order-book handling without financial risk.

Environment Configuration:
- Ensure DATABASE_URL is configured. MIMO_API_KEY is required only when --decision-mode ai is selected.
- Polygon Wallet / Private Key is not required for this mode.
Run Pipeline:
- Linux: bash scripts/deploy.sh --decision-mode deterministic --trading-mode dry-run
- Windows: .\scripts\deploy.ps1 -DecisionMode deterministic -TradingMode dry-run
Monitoring:
- The terminal dashboard appears automatically to view simulated positions. Only the selected AI or deterministic decision service runs in the background.

B. `amoy` Mode (Polygon Testnet)

Suitable for validating technical blockchain workflows (RPC connection, transaction signing with private keys, token authorization, gas fees) safely using free test tokens.

Setup Wallet & Amoy Network:
- Create a new EVM wallet specifically for testnet (e.g., in MetaMask). IMPORTANT: Do not use your main wallet.
- Add the Amoy network to your wallet if it's not already there:
  - Network Name: Polygon Amoy Testnet
  - RPC URL: https://rpc-amoy.polygon.technology
  - Chain ID: 80002
  - Currency Symbol: POL (or MATIC)
- Export the Private Key from that wallet and save it in the AMOY_PRIVATE_KEY variable in the .env file.

2. Get MATIC/POL Testnet (For Gas Fees):

Visit the Polygon Faucet or Alchemy Amoy Faucet.
IMPORTANT - Select the Correct Network: On the Faucet page, there will be many blockchain options. Make sure you select Polygon Amoy (or Amoy). Do not select Mumbai (because it's deprecated) or other testnet networks like Sepolia/Goerli.
Enter your new EOA wallet address, complete the captcha, and click the request button to get free gas tokens (POL/MATIC).

Get Testnet USDC (For Trading Capital):
- On the Amoy network, Polymarket contracts use a specific Mock USDC (0x9c4e1703476e875070ee25b56a58b008cfb8fa78) as collateral, not the official USDC from Circle Faucet.
- You don't need to use an external web faucet for USDC. Simply mint those tokens directly by running the following utility script (make sure your wallet already has testnet POL/MATIC balance for gas fees):
```
python services/trader/testing/poly-amoy.py --mint
```
  (This script will call the mint function directly on the Mock USDC smart contract and fill your wallet balance with 1000 USDC for free).
  
  [!IMPORTANT] Minting Process Must Be Run Manually: This --mint step is not run automatically by the deployment script (deploy.ps1 or deploy.sh) to avoid wasting your wallet's POL gas fees due to repeated minting transactions every time the bot is restarted. Make sure you run it manually just once before starting the deploy script.
- You can also check your wallet balance on the Amoy blockchain using the command:
```
python services/trader/testing/poly-amoy.py --check-balance
```
Authorize / Approve USDC Tokens:
- So that Polymarket smart contracts are allowed to use your USDC balance for trading, run the following command to perform authorization automatically:
```
python services/trader/testing/poly-amoy.py --approve-usdc
```
Run the Full Pipeline:
- Linux: bash scripts/deploy.sh --decision-mode ai --trading-mode amoy
- Windows: .\scripts\deploy.ps1 -DecisionMode ai -TradingMode amoy
- Replace ai with deterministic to test the non-agent decision path.

C. `market` Mode (Live Mainnet / Real Money)

IMPORTANT: Use only after the selected decision engine, execution ledger, and risk controls have passed the shadow/live-readiness gates described in the audit report.

Setup Wallet & Env:
- Create a dedicated new trading wallet, get its private key, and enter it into POLYGON_PRIVATE_KEY in the .env file.
- Make sure you have logged into the Polymarket website to activate the Deposit Wallet (see instructions above).
- Set the Risk Management parameters in .env (e.g., TRADER_BANKROLL_USDC according to your total balance).
Deposit Real Assets (pUSD):
- Send your USDC (Polygon network) balance and actual MATIC (minimum ~0.5 MATIC for long-term gas fees) to your trading wallet address.
- Connect your wallet to the Polymarket website and make a deposit. The Polymarket V2 system will automatically convert your USDC into pUSD (Polymarket USD) within your Deposit Wallet. The bot uses pUSD as default collateral.
Authorize USDC Tokens:
- When you deposit USDC to Polymarket using its official UI website, the Polymarket system will automatically request your signature to grant permission/approval to use your pUSD balance to the Polymarket smart contracts (CTF Exchange & NegRisk adapters). Therefore, just deposit through the UI website, you do not need to place a small bet manually.
Run Pipeline:
- Linux: bash scripts/deploy.sh --decision-mode deterministic --trading-mode market --allow-live-trading
- Windows: .\scripts\deploy.ps1 -DecisionMode deterministic -TradingMode market -AllowLiveTrading
- For manual startup, run one producer only: python services/llm-decision/main.py for AI, or python services/deterministic-decision/main.py for the non-AI branch.
- The deterministic branch additionally remains locked until DETERMINISTIC_LIVE_ENABLED=true; this flag must only be enabled after all audit gates pass.

Note: live market mode is rejected unless the explicit live-trading acknowledgement is present in the same command. This prevents an old .env from silently starting real-money execution.

6. Risk Management Configuration

These parameters protect capital from excessive losses. orderMinSize is a dynamic number of shares supplied by the venue; the USD guard below does not replace it. Use USD 10 only for the intended extreme-bankroll benchmark.

TRADER_BANKROLL_USDC=100        # Total capital in USDC
TRADER_MIN_SINGLE_ORDER_USDC=1  # Minimum per order (default: 1 USDC)
TRADER_MAX_SINGLE_ORDER_USDC=3   # Maximum per order (default: 3 USDC)
TRADER_MAX_POSITION_PCT=5       # Max position per trade as % of bankroll (default: 5%)
TRADER_MAX_DAILY_LOSS_PCT=10    # Daily loss limit as % of bankroll (default: 10%)
TRADER_MAX_TOTAL_EXPOSURE_PCT=30 # Aggregate unresolved exposure cap
TRADER_MAX_SPREAD=0.15          # Maximum order book spread (default: 0.15 = 15 cents)
TRADER_MIN_LIQUIDITY_USDC=15    # Minimum executable ask-side liquidity

Extreme-bankroll preservation profile (USD 10):

TRADER_BANKROLL_USDC=10
TRADER_MIN_SINGLE_ORDER_USDC=1
TRADER_MAX_SINGLE_ORDER_USDC=1
TRADER_MAX_POSITION_PCT=10
TRADER_MAX_DAILY_LOSS_PCT=10
TRADER_MAX_TOTAL_EXPOSURE_PCT=10
TRADER_MAX_SPREAD=0.08
TRADER_MIN_LIQUIDITY_USDC=4
DETERMINISTIC_KELLY_FRACTION=0.25
DETERMINISTIC_MAX_ENTRY_PCT=10

USD 10 is not an operating recommendation. With the audited five-share venue minimum, an ask of 0.50 costs about USD 2.59 including the declared fee and slippage, or 25.9% of bankroll. The engine must abstain whenever its dynamic minimum-share ticket does not fit the caps; it never rounds an unsafe size up.

The share-aware stress test recommends USD 250 as the smallest robust single-outcome pilot profile at a 2% entry cap across tested asks through 0.99. USD 500 provides the same mechanical coverage at a 1% cap. These are minimum- ticket mechanics, not proof of profitable expectancy; keep market mode disabled until durable exposure/settlement and cross-process locking are complete.

Additional Features & Protections in Code (`clob_client_v2.py`):

All-in Protection: The bot rejects malformed recommendations sized above 50% of total bankroll regardless of whether AI or deterministic mode produced them.
Minimum Confidence Threshold: The bot only executes recommendations with a minimum confidence level of 30%.
Conservative Daily Budget: Until settlement P&L is known, capital deployed that day is treated as worst-case loss.
Risk Cache Persistence: Daily counters and the open-position view survive ordinary restarts through Redis AOF. They remain cache state, not the final accounting ledger; live stays locked until fills, lots, settlements, realized P&L, and equity reconcile durably in PostgreSQL.

Running the Pipeline

Linux / WSL

bash scripts/deploy.sh --decision-mode deterministic --trading-mode dry-run

Available options:

Flag	Description
`--decision-mode ai\|deterministic`	Select the only decision producer that will run. Prompted interactively if omitted.
`--trading-mode dry-run\|amoy\|market`	Select the trader backend. Default: `dry-run`.
`--allow-live-trading`	Required acknowledgement for `market` mode.
`--skip-migration`	Skip CSV migration to database
`--reset-data`	Explicitly delete the active PostgreSQL and Redis volumes after confirmation. Ordinary runs preserve them.
`--db-engine plain\|pgcron\|timescale`	Select one database image.
`--pgcron`	Use PostgreSQL 16 + `pg_cron`
`--timescale`	Use TimescaleDB + `pg_cron`
`--nodash`	Do not run dashboard, show worker logs in terminal

# Run Amoy testnet
bash scripts/deploy.sh --decision-mode ai --trading-mode amoy

# New VPS, existing data, dry-run mode
bash scripts/deploy.sh --decision-mode deterministic --skip-migration --trading-mode dry-run

# Clean deploy + TimescaleDB + Live market
bash scripts/deploy.sh --decision-mode deterministic --db-engine timescale --trading-mode market --allow-live-trading

# No dashboard (suitable for headless servers)
bash scripts/deploy.sh --decision-mode deterministic --trading-mode dry-run --nodash

Windows PowerShell

.\scripts\deploy.ps1 -DecisionMode deterministic -TradingMode dry-run

Available options:

Flag	Description
`-DecisionMode ai\|deterministic`	Select the only decision producer that will run. Prompted interactively if omitted.
`-TradingMode dry-run\|amoy\|market`	Select the trader backend. Default: `dry-run`.
`-AllowLiveTrading`	Required acknowledgement for `market` mode.
`-SkipMigration` or `--skip-migration`	Skip CSV migration to database
`-ResetData` or `--reset-data`	Explicitly delete active PostgreSQL and Redis volumes after confirmation.
`-DatabaseEngine plain\|pgcron\|timescale`	Select one database image.
`-PgCron` or `--pgcron`	Use PostgreSQL 16 + `pg_cron`
`-Timescale` or `--timescale`	Use TimescaleDB + `pg_cron`
`-NoDash` or `--nodash`	Do not run dashboard, show worker logs in terminal

# Run Amoy testnet
.\scripts\deploy.ps1 -DecisionMode ai -TradingMode amoy

# Run live market
.\scripts\deploy.ps1 -DecisionMode deterministic -TradingMode market -AllowLiveTrading

# Run dry-run without dashboard
.\scripts\deploy.ps1 -DecisionMode deterministic -TradingMode dry-run -NoDash

.\scripts\deploy.ps1 -DecisionMode deterministic -SkipMigration
.\scripts\deploy.ps1 -DecisionMode ai -NoDash

# GNU style alternatives (double-dash) also supported:
.\scripts\deploy.ps1 --decision-mode deterministic --nodash
.\scripts\deploy.ps1 --decision-mode deterministic --skip-migration

Step-by-Step Deploy Flow

The deploy script automatically executes the following sequence:

Step	Action	Log
1	Restart old containers; preserve volumes unless an explicit confirmed reset was requested	—
2	Start PostgreSQL + Redis via Docker Compose	—
3	Wait for DB & Redis, then apply the idempotent deterministic-trading migration	—
4	Import CSV to database (`migrate_data.py` & `nwp_ingestor.py`)	—
5	Stop old workers	—
5.5	Historical Data Sync (blocking) — METAR, Global NWP, NWP Ensemble, and PWS only when `WUNDERGROUND_API_KEY` is configured	—
6	METAR scraper (background)	`logs/metar.log`
7	PWS scraper (optional background worker; skipped without its API key)	`logs/pws.log`
7.5	Global NWP Model scraper (background)	`logs/nwp.log`
7.6	NWP Ensemble Model scraper (background)	`logs/ensemble.log`
8	NWP Bias Corrector v2.0 watch mode (background)	`logs/modeler.log`
8.5	Two-Harmonic modeler daily daemon (background)	`logs/twoharmonic.log`
8.6	DB Backup daemon (background, every Sunday)	`logs/db_backup.log`
8.7	Market Discovery — daemon refreshes market every 60 secs	`logs/trader_discovery.log`
8.7	Implied Price Refresher — refreshes implied% from Gamma every 5 secs	`logs/trader_implied.log`
8.7	WebSocket feed — real-time market data (background)	`logs/trader_ws.log`
8.7	Trading bot matching `TRADER_MODE` (background)	`logs/trader_bot.log`
8.8	Position Redeemer monitor (background; proxy-wallet redemption currently fail-closed)	`logs/redeemer.log`
8.9	Selected Decision Engine — AI agent or deterministic service, never both	`logs/llm_decision.log` or `logs/deterministic_decision.log`
9	Terminal dashboard (foreground) or log stream (`--nodash`)	—

On PowerShell, Ctrl+C stops the launcher-managed workers. The Linux launcher uses detached workers; stop them with the process command printed by the launcher.

LLM Decision Service

This service is the optional AI trading brain. It runs only when the launcher is started with DecisionMode=ai; deterministic mode starts services/deterministic-decision/main.py instead. Only one producer is allowed to write recommendations during a run. AI logs are written to logs/llm_decision.log.

The agent is not trusted to invent market semantics. Its output is constrained to the event's mutually exclusive exact Celsius buckets and to BUY_YES/BUY_NO/HOLD; probability fields remain bound to the selected bucket/token. METAR and local PWS are explicitly labeled as proxies. The final Weather Underground WSSS value is the settlement truth, and the trader repeats date, token, side, quote-freshness, liquidity, and risk validation before any execution.

How It Works

This service has six trigger conditions:

Trigger	Condition	Explanation
`morning`	Every day after 08:30 SGT	Polls `forecast_predictions` until Two-Harmonic finishes (deadline 10:00 SGT)
`afternoon`	Every day at 15:00 SGT (±60 seconds)	Re-analyzes with today's actual observation data
`anomaly`	Price movement in an active sub-market	Triggered if an active sub-market moves ≥15% (or based on `LLM_ANOMALY_THRESHOLD_PCT`) within 3 minutes (`LLM_ANOMALY_WINDOW_MINUTES`)
`twoharmonic_update`	Two-Harmonic prediction update	Triggered real-time when the Two-Harmonic model publishes a new prediction update to the Redis channel `modeler:twoharmonic:update`
`nwp_bias_update`	NWP Bias prediction update	Triggered real-time when the NWP Bias Corrector model publishes a new prediction update to the Redis channel `modeler:nwp_bias:update`
`stop_loss_evaluation`	Open position stop loss	Triggered by Portfolio Monitor when open position losses exceed the threshold (`LLM_STOP_LOSS_PCT`, default: 20%)

One Agent Cycle Flow

Trigger met
    → Collect data (PostgreSQL + Redis, parallel, 30 seconds timeout per source)
    → Render prompt (decision_prompt.md + collected data)
    → Xiaomi MiMo Loop:
        - Agent calls tools (get_live_market_snapshot, get_forecast_accuracy, etc)
        - Enforcement: final_decision rejected if mandatory tools are not called
        - Agent outputs final_decision: true + JSON decision
    → Phase A: Check price staleness (baseline taken at session start, before first LLM call)
        - If price moves >5% since session started → agent gets 1x revision chance
    → Save to PostgreSQL (llm_decisions) — if fails, abort publication
    → Publish to Redis channel decision:recommendation (one message per sub-market)
    → Phase B: Wait for ACK from trader bot (max 15 seconds per decision_id)
        - Execution result (filled/rejected/timeout) injected into agent CoT
        - Agent knows if order succeeded before session closes

Available Tools for the Agent

Tool	Source	Purpose
`get_live_market_snapshot()`	Redis (`trader:market:live:*`)	Snapshot of all active sub-markets + last 50 ticks
`get_forecast_accuracy(days)`	PostgreSQL	Forecast performance against METAR-derived proxy labels (max 90 days)
`get_official_settlements(days)`	PostgreSQL	Finalized Weather Underground WSSS labels, when ingested
`get_metar_recent(hours)`	PostgreSQL	Raw METAR data (max 72 hours, max 300 rows)
`get_metar_daily_summary(days)`	PostgreSQL	Daily METAR summary (max 90 days)
`get_pws_recent(hours)`	PostgreSQL	Local PWS data (max 72 hours, 5 minute resolution)
`get_pws_daily_summary(days)`	PostgreSQL	Daily PWS summary (max 90 days)
`get_nwp_ensemble_range(date_from, date_to)`	PostgreSQL	5 NWP models consensus for date range (max 90 days)
`get_nwp_model_detail(model_id, date_from, date_to)`	PostgreSQL	Detail of 1 NWP model (max 30 days)
`get_orderbook(token_id)`	Redis (`trader:market:orderbook`)	Detailed order book per token

Mandatory Tools per Trigger

Trigger	Must Call
`morning`	`get_live_market_snapshot` + `get_forecast_accuracy` + `get_metar_daily_summary` + `get_nwp_ensemble_range`
`afternoon`	`get_live_market_snapshot` + `get_forecast_accuracy` + `get_metar_recent` + `get_hourly_nowcast`
`anomaly`	`get_live_market_snapshot` + `get_hourly_nowcast`
`twoharmonic_update`	`get_live_market_snapshot` + `get_forecast_accuracy` + `get_nwp_ensemble_range`
`nwp_bias_update`	`get_live_market_snapshot` + `get_forecast_accuracy`
`stop_loss_evaluation`	`get_live_market_snapshot` + `get_orderbook`

If the agent tries to output final_decision: true without calling all mandatory tools, the decision is rejected and the agent is asked to call the missing tools.

Prompt Template

The prompt template can be modified without restarting the service:

services/llm-decision/prompts/decision_prompt.md

The template uses placeholders: {forecast_data}, {nwp_bias_data}, {nwp_data}, {metar_data}, {pws_data}, {market_data}, {position_data}, {decision_history}, {trigger_type}, {history_days}.

llm_decisions Table

Every agent decision is saved to PostgreSQL for audit and backtesting:

Column	Description
`decision_id`	Unique UUID
`trigger_type`	morning / afternoon / anomaly / twoharmonic_update / nwp_bias_update / stop_loss_evaluation
`action`	BUY_YES / BUY_NO / HOLD
`p_weather`	Agent's weather probability for this bracket
`p_market`	Market YES price during decision
`edge`	p_weather − p_market (decision quality signal)
`tmax_estimate_c`	Agent's specific Tmax estimate
`model_confidence`	Model confidence level (high / moderate / low / insufficient)
`pred_tmax_snapshot`	Two-Harmonic / NWP Bias Corrector Tmax prediction at that time
`nwp_ensemble_snapshot`	NWP Ensemble at that time
`market_price_snapshot`	Market YES price snapshot at that time
`reasoning_summary`	Brief reasoning from the agent
`outcome`	WIN / LOSS / PENDING (auto-updated when market resolves)

All Environment Variables Reference

Database

Variable	Default	Description
`POSTGRES_DB`	`polyweather`	Database name
`POSTGRES_USER`	`postgres`	Username
`POSTGRES_PASSWORD`	`123`	Change in production!
`POSTGRES_HOST`	`localhost`	PostgreSQL host
`POSTGRES_PORT`	`5433`	PostgreSQL port
`DATABASE_URL`	(from above variables)	Full connection string
`APP_TIMEZONE`	`Asia/Singapore`	Operational timezone

Redis

Variable	Default	Description
`REDIS_HOST`	`localhost`	Host
`REDIS_PORT`	`6379`	Port
`REDIS_DB`	`0`	Database number

Trading

Variable	Default	Description
`TRADING_MODE`	`dry-run`	`dry-run` / `amoy` / `market`
`POLYGON_PRIVATE_KEY`	(empty)	Mainnet private key
`AMOY_PRIVATE_KEY`	(empty)	Amoy testnet private key
`POLYMARKET_TOKEN_ID`	(empty)	Automatically filled by market_discovery
`TRADER_BANKROLL_USDC`	`100`	Total USDC capital
`TRADER_MIN_SINGLE_ORDER_USDC`	`1`	Min per order (USDC)
`TRADER_MAX_SINGLE_ORDER_USDC`	`3`	Max per order (USDC)
`TRADER_MAX_POSITION_PCT`	`5`	Max position per trade (% bankroll)
`TRADER_MAX_DAILY_LOSS_PCT`	`10`	Daily loss limit (% bankroll)
`TRADER_MAX_TOTAL_EXPOSURE_PCT`	`30`	Aggregate unresolved exposure (% bankroll)
`TRADER_MAX_SPREAD`	`0.15`	Max order book spread
`TRADER_MIN_LIQUIDITY_USDC`	`50`	Minimum executable ask-side liquidity (USDC)

LLM Decision Service

NOTE: Full OpenAI API Compatibility The LLM Decision agent uses the standard openai Python package. It is fully compatible with any LLM provider that supports the OpenAI API standard (e.g., standard OpenAI ChatGPT, DeepSeek, Groq, Together AI, Local Ollama, vLLM). You can easily swap the agent to any other model just by editing the MIMO_API_KEY, MIMO_BASE_URL, and MIMO_MODEL variables below.

NOTE: Agent Trading Style (Moderate Risk) By default, the LLM agent is configured with a Moderate Risk trading style. It balances between capitalizing on high-edge opportunities and preserving capital. You can change the agent's behavior, personality, or risk tolerance at any time by modifying its system prompt located at services/llm-decision/prompts/decision_prompt.md.

Variable	Default	Description
`MIMO_API_KEY`	(empty)	MANDATORY
`MIMO_MODEL`	`mimo-v2.5-pro`	Model: `mimo-v2.5-pro`
`MIMO_BASE_URL`	`https://token-plan-sgp.xiaomimimo.com/v1`	API Base URL (choose based on account type: PAYG or TokenPlan)
`LLM_MORNING_TRIGGER_TIME`	`08:30`	Morning polling time (SGT, HH:MM)
`LLM_AFTERNOON_TRIGGER_TIME`	`15:00`	Afternoon trigger time (SGT, HH:MM)
`LLM_ANOMALY_THRESHOLD_PCT`	`15`	Movement percentage for anomaly trigger (1–100)
`LLM_ANOMALY_WINDOW_MINUTES`	`3`	Anomaly detection time window (1–60 mins)
`LLM_COOLDOWN_MINUTES`	`10`	Cooldown after anomaly (1–60 mins)
`LLM_STOP_LOSS_PCT`	`20`	Open position loss limit before stop loss trigger (1-100)
`LLM_MAX_TOOL_CALLS`	`10`	Max tool calls per agent cycle (1–50)
`LLM_HISTORY_DAYS`	`7`	History days sent to agent (1–90)
`LLM_PRICE_STALE_THRESHOLD`	`0.05`	Price movement fraction for agent revision trigger (default 5%)
`LLM_ACK_TIMEOUT_SECONDS`	`15.0`	Timeout waiting for ACK from trader bot (seconds)

Redemption Monitor (CTF Smart Contract)

Variable	Default	Description
`POLYGON_RPC_URL`	`https://polygon-rpc.com`	Polygon mainnet RPC
`CTF_ADDRESS`	`0x4D97DCd97eC945f40cF65F87097ACe5EA0476045`	Gnosis CTF contract — do not change
`USDC_MAINNET_ADDRESS`	`0x2791Bca1f2de4661ED88A30C99A7a9449Aa84174`	USDC contract — do not change
`REDEEMER_GAS_LIMIT`	`200000`	Gas limit for direct-EOA `redeemPositions`; it does not enable proxy-wallet redemption

Google Drive Backup

Variable	Default	Description
`BACKUP_RCLONE_REMOTE`	`poly_remote:polyweather_backups`	rclone remote + destination folder
`BACKUP_KEEP_LOCAL`	`7`	Days to keep local backup before deleting
`BACKUP_KEEP_REMOTE`	`12`	Number of files in Google Drive
`BACKUP_RETENTION_DAYS`	`5000`	DB data days before pruning

Discord Notifier Bot

Polyweather is equipped with an integrated Discord bot (services/notifier) that serves dual functions:

Push Notifications (Conditional): The bot monitors redemption history (trader:redeem_history) and sends notifications when a redemption has actually succeeded. No success event is produced for a blocked proxy-wallet attempt.
Slash Commands (Interactive): You can interact with the bot in Discord using the following commands:
- /asset : Displays the total bankroll balance (USDC) on Polymarket.
- /position : Displays the list of open trading positions along with their unrealized PnL value live.
- /history : Displays the trading and redeem history, equipped with a dynamic days parameter (example: /history days: 31).

Bot Setup:

Make sure you have created a Bot App in the Discord Developer Portal.
How to get DISCORD_BOT_TOKEN:
- In the Developer Portal, select your application and click the "Bot" tab in the left menu.
- Click the "Reset Token" button and copy the random text string that appears.
- On the same page, under the Privileged Gateway Intents section, make sure you check the Message Content Intent, then click "Save Changes".
- Enter the copied token into the DISCORD_BOT_TOKEN variable in the .env file.
How to get DISCORD_CHANNEL_ID:
- Open the Discord app, go to User Settings (gear icon) Advanced and enable Developer Mode.
- Open the server where the bot has been invited, right-click on the text channel where notifications should be sent, then click "Copy Channel ID".
- Enter that sequence of numbers into the DISCORD_CHANNEL_ID variable in the .env file.
The bot runs inside a Docker container and will automatically start when the deployment is executed (docker-compose up).

Redemption Monitor and Current Limitation

position_redeemer.py discovers redeemable positions and can check direct EOA balances. It does not currently provide unattended redemption for the normal deposit/proxy-wallet path (SIGNATURE_TYPE=3). When the token holder is a proxy wallet, the service intentionally returns REDEMPTION BLOCKED instead of submitting a successful-looking EOA transaction that would redeem nothing.

How It Works

Polymarket uses Gnosis CTF (Conditional Token Framework). Each binary market has two tokens:

YES token (index set 1) — worth 1 USDC if outcome = YES
NO token (index set 2) — worth 1 USDC if outcome = NO

Both decision engines can buy YES or NO. Cash-out from a proxy wallet must currently be completed through the official Polymarket interface, or through a separately implemented and audited relayer-wallet batch flow. Treat this as a live-trading blocker, not as an operational auto-redeem feature.

Running

The monitor runs automatically when the deploy script is executed. To perform a read-only one-shot check:

python services/trader/src/position_redeemer.py --check

--check does not submit a transaction. Do not enable an unreviewed relayer implementation merely to bypass the fail-closed proxy-wallet guard.

Automated Database Backup

Runs every Sunday at 00:00 as a background daemon.

Process:

pg_dump the entire database → .sql.gz file
Upload to Google Drive via rclone
Delete local backups > BACKUP_KEEP_LOCAL days
Retain only the last BACKUP_KEEP_REMOTE files on Google Drive
Data Retention (Data Pruning): Checks the number of unique data days in the database. If there are tables with data exceeding BACKUP_RETENTION_DAYS days (default: 5000 days), the oldest rows will be deleted until exactly 5000 days remain. Note: Data will never be deleted if today's backup has not been successful.

Test backup immediately:

# Linux
.venv_linux/bin/python Database/backup/db_backup.py --run-now

# Windows
.venv_windows\Scripts\python.exe Database/backup/db_backup.py --run-now

Check next schedule:

python Database/backup/db_backup.py --status

Complete guide for rclone setup is in Database/BACKUP_SETUP.md.

Name		Name	Last commit message	Last commit date
Database		Database
analysis		analysis
backtests		backtests
docker		docker
scripts		scripts
services		services
tests		tests
.env.example		.env.example
.gitattributes		.gitattributes
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
dashboard.py		dashboard.py
docker-compose.pgcron.yml		docker-compose.pgcron.yml
docker-compose.timescale.yml		docker-compose.timescale.yml
docker-compose.yml		docker-compose.yml
health_check.py		health_check.py
pytest.ini		pytest.ini
schema_ddl.sql		schema_ddl.sql
schema_validator.py		schema_validator.py

Latest commit

History

Folders and files

Repository files navigation

Polyweather

Table of Contents

System Architecture

Complete Data Flow

Redis Keys

Database Structure (Medallion Architecture)

System Requirements

Python Dependencies (Per Service)

First Time Setup

Linux / WSL / VPS

Windows

Mandatory Configuration Before Deploy

1. Create .env file

2. Setup Xiaomi MiMo API

3. Setup CheckWX API

4. Setup Polygon Wallet

1. Create a New Wallet (recommended exclusively for trading)

2. Activate Deposit Wallet (Polymarket V2)

3. Configuration in .env file

4. Market Token ID

5. Select Decision Engine and Trading Mode

Step-by-Step Guide Based on Selected Mode

A. dry-run Mode (Simulation / Paper Trading)

B. amoy Mode (Polygon Testnet)

C. market Mode (Live Mainnet / Real Money)

6. Risk Management Configuration

Additional Features & Protections in Code (clob_client_v2.py):

Running the Pipeline

Linux / WSL

Windows PowerShell

Step-by-Step Deploy Flow

LLM Decision Service

How It Works

One Agent Cycle Flow

Available Tools for the Agent

Mandatory Tools per Trigger

Prompt Template

llm_decisions Table

All Environment Variables Reference

Database

Redis

Trading

LLM Decision Service

Redemption Monitor (CTF Smart Contract)

Google Drive Backup

Discord Notifier Bot

Redemption Monitor and Current Limitation

How It Works

Running

Automated Database Backup

About

Resources

Stars

Watchers

Forks

Releases

Packages

Used by

Contributors

Languages

3. Configuration in `.env` file

A. `dry-run` Mode (Simulation / Paper Trading)

B. `amoy` Mode (Polygon Testnet)

C. `market` Mode (Live Mainnet / Real Money)

Additional Features & Protections in Code (`clob_client_v2.py`):