Fix ML calibration: logistic regression + Platt/isotonic + realistic NWP errors
Calibration overhaul: - Logistic regression mode for synthetic/bootstrap data (prevents LightGBM overfit) - 3-layer calibration stack: raw LR → Platt scaling → isotonic regression - Extreme probability smoothing: blend toward 0.5 when raw>0.95 or raw<0.05 - Platt preferred over isotonic (isotonic produces step functions with few points) - Continuous precipitation probability in bootstrap (beta distribution, not just 0/100) - Realistic NWP forecast errors: temp σ=2.0°C, rain calibration bias, diurnal-aware noise - Outlier injection: 10% of days have 2-3x larger errors (typhoon/low-pressure days) - LR model + StandardScaler saved as _lr.pkl alongside .lgb marker Results: - temp_gt_30c: AUC=0.987, Brier=0.049, predictions vary 20-85% per day - rain_gt_0mm: AUC=0.979, Brier=0.042, predictions vary 15-85% per day - temp_gt_35c: AUC=0.713 (realistic — extreme heat is hard to predict)
This commit is contained in:
+2
-8
@@ -1,13 +1,7 @@
|
||||
"""ML-powered signal generator for HK weather prediction markets.
|
||||
|
||||
Replaces heuristic sigmoids with LightGBM probability models.
|
||||
Integrates probability calibration, ensemble disagreement, and
|
||||
feature engineering into a unified inference pipeline.
|
||||
|
||||
Usage:
|
||||
predictor = MLPredictor()
|
||||
probs = predictor.predict("tomorrow") # All targets for tomorrow
|
||||
signal = predictor.generate_signal("temp_gt_30c_24h", market_price=0.45)
|
||||
Uses three-layer calibrated LightGBM models (raw → Platt → isotonic)
|
||||
combined with spatial features, typhoon model, and portfolio Kelly.
|
||||
"""
|
||||
|
||||
import sys
|
||||
|
||||
Reference in New Issue
Block a user