Market creator (ml/create_markets.py):
- Generates Polymarket-compliant BinaryOption question specs
- Temperature threshold markets (30°C, 33°C, 35°C) per day
- Rain probability and heavy rain (10mm/25mm) markets per day
- Typhoon T8 weekly + monthly markets
- Wind gust threshold markets (>50 km/h at Chek Lap Kok)
- Calendar spread markets (rainy days per week)
- Resolution criteria tied to verifiable HKO public API sources
- Submission-ready JSON output with dispute windows + metadata
- Human-readable summary with probability bars + edge estimates
Web dashboard:
- /api/ml endpoint: per-model raw/calibrated probabilities
- ML Predictions card: color-coded, sorted by confidence
- Typhoon Probabilities card: T1/T3/T8 now + 72h/120h
- Auto-refreshes independently (avoids blocking chart refresh)
Usage:
python ml/create_markets.py # View proposals
python ml/create_markets.py --submit # JSON for Polymarket
- /api/ml endpoint: per-model raw vs calibrated, typhoon, spatial features
- ML Predictions card in web dashboard (color-coded, sorted by confidence)
- Typhoon Probabilities card (T1/T3/T8 now/72h/120h)
- PaperTrader: simulated trading with portfolio Kelly, P&L tracking,
trade history persistence, auto-resolution after 24h
- TimeDecayModel: theta decay for binary options
sigma(t) = sigma_0 * (T-t)^beta (beta=0.4 for weather)
Fair price convergence from 50%→model_prob as expiry approaches
- Paper trader CLI: --track (monitor), --report, --simulate-days
Run dashboard: python web_dashboard.py # see ML panel
Run paper: python ml/paper_trader.py --simulate-days 30
Calibration overhaul:
- Logistic regression mode for synthetic/bootstrap data (prevents LightGBM overfit)
- 3-layer calibration stack: raw LR → Platt scaling → isotonic regression
- Extreme probability smoothing: blend toward 0.5 when raw>0.95 or raw<0.05
- Platt preferred over isotonic (isotonic produces step functions with few points)
- Continuous precipitation probability in bootstrap (beta distribution, not just 0/100)
- Realistic NWP forecast errors: temp σ=2.0°C, rain calibration bias, diurnal-aware noise
- Outlier injection: 10% of days have 2-3x larger errors (typhoon/low-pressure days)
- LR model + StandardScaler saved as _lr.pkl alongside .lgb marker
Results:
- temp_gt_30c: AUC=0.987, Brier=0.049, predictions vary 20-85% per day
- rain_gt_0mm: AUC=0.979, Brier=0.042, predictions vary 15-85% per day
- temp_gt_35c: AUC=0.713 (realistic — extreme heat is hard to predict)
Tier 2 enhancements:
- SpatialWeatherClient: multi-station Open-Meteo fetcher for all HK locations
Extracts urban heat island delta, coastal-inland gradients, wind convergence,
precipitation spatial heterogeneity, composite instability index
- TyphoonModel: data-driven signal probability for T1/T3/T8/T10
Climatological base rates + conditional transition probabilities
Currently active T1 signal → 25% T3/24h, 10% T8/72h, 22% T8/120h
ENSO modulation, active storm proximity boost, month-specific seasonality
- ERA5 download/process pipeline via CDS API
Downloads hourly reanalysis for HK region, processes to daily training format
Output schema matches Open-Meteo for seamless feature compatibility
- PortfolioKelly: correlation-aware simultaneous Kelly sizing
Covariance matrix from historical outcome correlations
Prevents over-betting on correlated rain/temp/wind markets
Σ⁻¹ μ vector formulation, regularized inversion, independent fallback
- MLPredictor updated: integrates spatial + typhoon + portfolio Kelly
record_outcome feeds both calibration AND portfolio correlation matrix
- Flask web server with interactive charts (Chart.js)
- Temperature, rain probability, wind speed charts with HKO vs Open-Meteo comparison
- Current conditions card with typhoon signal indicator
- Kelly criterion sizing simulator panel
- Trading signals panel with model-implied probabilities
- Raw forecast data JSON viewer
- Github-dark theme with responsive card layout
- Fixed Open-Meteo SDK compatibility and numpy bool serialization
Run: python web_dashboard.py # http://localhost:5000
- Open-Meteo WeatherNext API client for HK forecasts
- HKO public data client (current conditions, 9-day forecast, typhoon warnings)
- HK-specific weather extraction and calibration
- Polymarket market scanning, price discovery, and market creation proposals
- Trading strategy engine: edge detection, Kelly criterion sizing, probability calibration
- End-to-end pipeline with dry-run mode and scheduled runner
- Interactive dashboard with live HK weather + forecasts + trading signals
Dependencies: Python 3.10+, openmeteo-requests, pandas
No API keys needed for dry-run mode.
Polymarket trading requires private key in .env.
ramseshk